2024-05-09T04:26:06.620 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-09T04:26:06.620 DEBUG:teuthology.run:Teuthology command: teuthology --name pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/fsync-tester}} --verbose --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699065 -- /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699065/orig.config.yaml 2024-05-09T04:26:06.690 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T04:26:06.781 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699065 branch: wip-pdonnell-testing-20240508.183908-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/fsync-tester}} email: pdonnell@redhat.com first_in_suite: false job_id: '7699065' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240508.183908-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt dry_run: true install: ceph: flavor: default sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240508.183908-debug sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1722 sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240508.183908-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi006.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOAmouKGcSeSaEFABhScVpIiSBHogD79e1A6T8qaTu+SpZLnpd4wBmQwYlEM9EWffbWjpZL0owz41MxnTf9HOfI= smithi169.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLIRm57YlYJCpjMUN/OXsNFahvf3hI6CRKaZayvOsNwJVzJDpnfHqzFRFrtMnYNSXNjCx5uJlSEaOBrfizA1hrI= smithi186.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCS4PAPp1XCK8MJkwwyrrueEOnwS9ke8Nwt7rU7BSO5ZRdn/l/WHHR12LViCXU7MDeYG7MoA2imsbYEJ9Khh3ww= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - quiescer: null - workunit: clients: all: - suites/fsync-tester.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_22:06:20 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-09T04:26:06.781 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa; will attempt to use it 2024-05-09T04:26:06.782 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa/tasks 2024-05-09T04:26:06.782 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-09T04:26:06.784 INFO:teuthology.task.internal:Checking packages... 2024-05-09T04:26:06.813 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'c9da4049f638c0f84bdd9b4e33c42e86046ac554' 2024-05-09T04:26:06.813 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-09T04:26:06.813 INFO:teuthology.packaging:ref: None 2024-05-09T04:26:06.813 INFO:teuthology.packaging:tag: None 2024-05-09T04:26:06.813 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240508.183908-debug 2024-05-09T04:26:06.813 INFO:teuthology.packaging:sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T04:26:06.813 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240508.183908-debug 2024-05-09T04:26:07.044 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3594.gc9da4049 2024-05-09T04:26:07.047 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-09T04:26:07.054 INFO:teuthology.task.internal:no buildpackages task found 2024-05-09T04:26:07.054 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-09T04:26:07.087 INFO:teuthology.task.internal:Saving configuration 2024-05-09T04:26:07.108 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-09T04:26:07.113 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-09T04:26:07.139 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi006.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699065', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 04:20:42.016734', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOAmouKGcSeSaEFABhScVpIiSBHogD79e1A6T8qaTu+SpZLnpd4wBmQwYlEM9EWffbWjpZL0owz41MxnTf9HOfI='} 2024-05-09T04:26:07.163 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi169.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699065', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 04:20:42.019869', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLIRm57YlYJCpjMUN/OXsNFahvf3hI6CRKaZayvOsNwJVzJDpnfHqzFRFrtMnYNSXNjCx5uJlSEaOBrfizA1hrI='} 2024-05-09T04:26:07.184 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi186.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699065', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 04:20:42.018415', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCS4PAPp1XCK8MJkwwyrrueEOnwS9ke8Nwt7rU7BSO5ZRdn/l/WHHR12LViCXU7MDeYG7MoA2imsbYEJ9Khh3ww='} 2024-05-09T04:26:07.185 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-09T04:26:07.191 INFO:teuthology.task.internal:roles: ubuntu@smithi006.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-09T04:26:07.191 INFO:teuthology.task.internal:roles: ubuntu@smithi169.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-09T04:26:07.191 INFO:teuthology.task.internal:roles: ubuntu@smithi186.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-09T04:26:07.191 INFO:teuthology.run_tasks:Running task console_log... 2024-05-09T04:26:07.331 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fba8675cf70>, signals=[15]) 2024-05-09T04:26:07.331 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-09T04:26:07.337 INFO:teuthology.task.internal:Opening connections... 2024-05-09T04:26:07.338 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi006.front.sepia.ceph.com 2024-05-09T04:26:07.339 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T04:26:07.437 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi169.front.sepia.ceph.com 2024-05-09T04:26:07.438 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T04:26:07.519 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi186.front.sepia.ceph.com 2024-05-09T04:26:07.520 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T04:26:07.603 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-09T04:26:07.610 DEBUG:teuthology.orchestra.run.smithi006:> uname -m 2024-05-09T04:26:07.630 INFO:teuthology.orchestra.run.smithi006.stdout:x86_64 2024-05-09T04:26:07.630 DEBUG:teuthology.orchestra.run.smithi006:> cat /etc/os-release 2024-05-09T04:26:07.687 INFO:teuthology.orchestra.run.smithi006.stdout:NAME="CentOS Stream" 2024-05-09T04:26:07.687 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION="9" 2024-05-09T04:26:07.687 INFO:teuthology.orchestra.run.smithi006.stdout:ID="centos" 2024-05-09T04:26:07.687 INFO:teuthology.orchestra.run.smithi006.stdout:ID_LIKE="rhel fedora" 2024-05-09T04:26:07.687 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION_ID="9" 2024-05-09T04:26:07.687 INFO:teuthology.orchestra.run.smithi006.stdout:PLATFORM_ID="platform:el9" 2024-05-09T04:26:07.687 INFO:teuthology.orchestra.run.smithi006.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T04:26:07.687 INFO:teuthology.orchestra.run.smithi006.stdout:ANSI_COLOR="0;31" 2024-05-09T04:26:07.688 INFO:teuthology.orchestra.run.smithi006.stdout:LOGO="fedora-logo-icon" 2024-05-09T04:26:07.688 INFO:teuthology.orchestra.run.smithi006.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T04:26:07.688 INFO:teuthology.orchestra.run.smithi006.stdout:HOME_URL="https://centos.org/" 2024-05-09T04:26:07.688 INFO:teuthology.orchestra.run.smithi006.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T04:26:07.688 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T04:26:07.688 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T04:26:07.688 INFO:teuthology.lock.ops:Updating smithi006.front.sepia.ceph.com on lock server 2024-05-09T04:26:07.708 DEBUG:teuthology.orchestra.run.smithi169:> uname -m 2024-05-09T04:26:07.725 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64 2024-05-09T04:26:07.726 DEBUG:teuthology.orchestra.run.smithi169:> cat /etc/os-release 2024-05-09T04:26:07.782 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="CentOS Stream" 2024-05-09T04:26:07.783 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="9" 2024-05-09T04:26:07.783 INFO:teuthology.orchestra.run.smithi169.stdout:ID="centos" 2024-05-09T04:26:07.783 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE="rhel fedora" 2024-05-09T04:26:07.783 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="9" 2024-05-09T04:26:07.783 INFO:teuthology.orchestra.run.smithi169.stdout:PLATFORM_ID="platform:el9" 2024-05-09T04:26:07.783 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T04:26:07.783 INFO:teuthology.orchestra.run.smithi169.stdout:ANSI_COLOR="0;31" 2024-05-09T04:26:07.783 INFO:teuthology.orchestra.run.smithi169.stdout:LOGO="fedora-logo-icon" 2024-05-09T04:26:07.783 INFO:teuthology.orchestra.run.smithi169.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T04:26:07.783 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://centos.org/" 2024-05-09T04:26:07.783 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T04:26:07.783 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T04:26:07.784 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T04:26:07.784 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2024-05-09T04:26:07.808 DEBUG:teuthology.orchestra.run.smithi186:> uname -m 2024-05-09T04:26:07.826 INFO:teuthology.orchestra.run.smithi186.stdout:x86_64 2024-05-09T04:26:07.826 DEBUG:teuthology.orchestra.run.smithi186:> cat /etc/os-release 2024-05-09T04:26:07.883 INFO:teuthology.orchestra.run.smithi186.stdout:NAME="CentOS Stream" 2024-05-09T04:26:07.883 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION="9" 2024-05-09T04:26:07.883 INFO:teuthology.orchestra.run.smithi186.stdout:ID="centos" 2024-05-09T04:26:07.883 INFO:teuthology.orchestra.run.smithi186.stdout:ID_LIKE="rhel fedora" 2024-05-09T04:26:07.883 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION_ID="9" 2024-05-09T04:26:07.884 INFO:teuthology.orchestra.run.smithi186.stdout:PLATFORM_ID="platform:el9" 2024-05-09T04:26:07.884 INFO:teuthology.orchestra.run.smithi186.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T04:26:07.884 INFO:teuthology.orchestra.run.smithi186.stdout:ANSI_COLOR="0;31" 2024-05-09T04:26:07.884 INFO:teuthology.orchestra.run.smithi186.stdout:LOGO="fedora-logo-icon" 2024-05-09T04:26:07.884 INFO:teuthology.orchestra.run.smithi186.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T04:26:07.884 INFO:teuthology.orchestra.run.smithi186.stdout:HOME_URL="https://centos.org/" 2024-05-09T04:26:07.884 INFO:teuthology.orchestra.run.smithi186.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T04:26:07.884 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T04:26:07.884 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T04:26:07.884 INFO:teuthology.lock.ops:Updating smithi186.front.sepia.ceph.com on lock server 2024-05-09T04:26:07.908 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-09T04:26:07.918 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-09T04:26:07.953 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-09T04:26:07.953 DEBUG:teuthology.orchestra.run.smithi006:> test '!' -e /home/ubuntu/cephtest 2024-05-09T04:26:07.956 DEBUG:teuthology.orchestra.run.smithi169:> test '!' -e /home/ubuntu/cephtest 2024-05-09T04:26:07.958 DEBUG:teuthology.orchestra.run.smithi186:> test '!' -e /home/ubuntu/cephtest 2024-05-09T04:26:07.974 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-09T04:26:07.980 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-09T04:26:07.980 DEBUG:teuthology.orchestra.run.smithi006:> test -z $(ls -A /var/lib/ceph) 2024-05-09T04:26:08.013 DEBUG:teuthology.orchestra.run.smithi169:> test -z $(ls -A /var/lib/ceph) 2024-05-09T04:26:08.015 DEBUG:teuthology.orchestra.run.smithi186:> test -z $(ls -A /var/lib/ceph) 2024-05-09T04:26:08.084 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-09T04:26:08.169 INFO:teuthology.run_tasks:Running task kernel... 2024-05-09T04:26:08.183 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-05-09T04:26:08.183 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-05-09T04:26:08.183 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-05-09T04:26:08.183 DEBUG:teuthology.orchestra.run.smithi006:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T04:26:08.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T04:26:08.201 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-05-09T04:26:08.356 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-05-09T04:26:08.357 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-05-09T04:26:08.357 DEBUG:teuthology.orchestra.run.smithi006:> uname -r 2024-05-09T04:26:08.375 INFO:teuthology.orchestra.run.smithi006.stdout:5.14.0-437.el9.x86_64 2024-05-09T04:26:08.375 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-05-09T04:26:08.375 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-09T04:26:08.376 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum upgrade -y linux-firmware 2024-05-09T04:26:13.071 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - BaseOS 2.7 MB/s | 8.1 MB 00:02 2024-05-09T04:26:15.513 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-05-09T04:26:20.091 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - CRB 7.1 MB/s | 6.2 MB 00:00 2024-05-09T04:26:21.548 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - Extras packages 72 kB/s | 16 kB 00:00 2024-05-09T04:26:23.393 INFO:teuthology.orchestra.run.smithi006.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-05-09T04:26:28.841 INFO:teuthology.orchestra.run.smithi006.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-05-09T04:26:31.022 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-05-09T04:26:31.023 INFO:teuthology.orchestra.run.smithi006.stdout:Nothing to do. 2024-05-09T04:26:31.023 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-05-09T04:26:31.133 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-05-09T04:26:31.133 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-05-09T04:26:31.385 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-05-09T04:26:31.385 DEBUG:teuthology.orchestra.run.smithi006:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-09T04:26:31.388 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi006.front.sepia.ceph.com to complete... 2024-05-09T04:26:48.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09 04:26:48 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-09T04:26:48.527 INFO:teuthology.orchestra.run.smithi006.stderr:FINISHED --2024-05-09 04:26:48-- 2024-05-09T04:26:48.527 INFO:teuthology.orchestra.run.smithi006.stderr:Total wall clock time: 17s 2024-05-09T04:26:48.527 INFO:teuthology.orchestra.run.smithi006.stderr:Downloaded: 1 files, 341M in 17s (20.2 MB/s) 2024-05-09T04:26:48.533 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-05-09T04:26:48.533 DEBUG:teuthology.orchestra.run.smithi006:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-09T04:26:50.725 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying... ######################################## 2024-05-09T04:26:50.843 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing... ######################################## 2024-05-09T04:26:50.965 INFO:teuthology.orchestra.run.smithi006.stdout:Updating / installing... 2024-05-09T04:26:59.271 INFO:teuthology.orchestra.run.smithi006.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-05-09T04:27:39.557 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi006.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-09T04:27:39.558 DEBUG:teuthology.orchestra.run.smithi006:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-09T04:27:39.644 INFO:teuthology.orchestra.run.smithi006.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-05-09T04:27:39.644 INFO:teuthology.orchestra.run.smithi006.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-05-09T04:27:39.644 INFO:teuthology.orchestra.run.smithi006.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-05-09T04:27:39.644 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-05-09T04:27:39.644 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-05-09T04:27:39.644 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-05-09T04:27:39.644 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-05-09T04:27:39.644 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-05-09T04:27:39.644 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-05-09T04:27:39.644 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-09T04:27:39.644 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-09T04:27:39.645 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-09T04:27:39.645 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-09T04:27:39.645 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-09T04:27:39.645 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-09T04:27:39.645 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-09T04:27:39.645 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-09T04:27:39.645 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-09T04:27:39.645 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-09T04:27:39.645 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-09T04:27:39.645 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-09T04:27:39.645 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-09T04:27:39.645 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-09T04:27:39.645 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-09T04:27:39.645 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-09T04:27:39.646 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-09T04:27:39.646 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-09T04:27:39.646 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-09T04:27:39.646 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-09T04:27:39.646 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-05-09T04:27:39.646 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-05-09T04:27:39.646 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-05-09T04:27:39.646 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-09T04:27:39.646 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-05-09T04:27:39.646 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-09T04:27:39.646 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-09T04:27:39.646 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-05-09T04:27:39.646 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-05-09T04:27:39.647 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-05-09T04:27:39.647 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-05-09T04:27:39.647 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-05-09T04:27:39.647 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-05-09T04:27:39.647 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-05-09T04:27:39.647 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-05-09T04:27:39.647 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-09T04:27:39.647 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-05-09T04:27:39.647 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-05-09T04:27:39.647 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-05-09T04:27:39.647 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-05-09T04:27:39.647 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-05-09T04:27:39.647 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-05-09T04:27:39.647 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-05-09T04:27:39.647 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-05-09T04:27:39.648 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-05-09T04:27:39.648 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-05-09T04:27:39.648 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-05-09T04:27:39.648 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-05-09T04:27:39.648 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-05-09T04:27:39.648 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-05-09T04:27:39.648 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-05-09T04:27:39.648 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-05-09T04:27:39.648 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-05-09T04:27:39.648 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-05-09T04:27:39.648 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-05-09T04:27:39.648 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-05-09T04:27:39.648 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-05-09T04:27:39.648 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-05-09T04:27:39.649 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-05-09T04:27:39.649 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-05-09T04:27:39.649 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-05-09T04:27:39.649 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-05-09T04:27:39.649 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-05-09T04:27:39.649 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-05-09T04:27:39.649 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-05-09T04:27:39.649 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-05-09T04:27:39.649 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-05-09T04:27:39.649 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-05-09T04:27:39.649 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-05-09T04:27:39.649 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-05-09T04:27:39.649 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-05-09T04:27:39.649 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-05-09T04:27:39.650 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-05-09T04:27:39.650 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-05-09T04:27:39.650 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-05-09T04:27:39.650 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-05-09T04:27:39.650 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-05-09T04:27:39.650 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-05-09T04:27:39.650 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-05-09T04:27:39.650 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-05-09T04:27:39.650 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-09T04:27:39.650 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-05-09T04:27:39.650 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-05-09T04:27:39.650 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-05-09T04:27:39.650 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-05-09T04:27:39.650 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-05-09T04:27:39.651 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-05-09T04:27:39.651 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-05-09T04:27:39.651 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-05-09T04:27:39.651 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-05-09T04:27:39.651 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-05-09T04:27:39.651 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-05-09T04:27:39.651 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-05-09T04:27:39.651 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-05-09T04:27:39.651 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-05-09T04:27:39.651 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-05-09T04:27:39.651 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-05-09T04:27:39.651 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-05-09T04:27:39.651 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-05-09T04:27:39.651 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-05-09T04:27:39.651 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-05-09T04:27:39.652 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-05-09T04:27:39.652 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-05-09T04:27:39.652 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-05-09T04:27:39.652 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-09T04:27:39.652 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-09T04:27:39.652 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-05-09T04:27:39.652 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-05-09T04:27:39.652 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-05-09T04:27:39.652 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-05-09T04:27:39.652 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-05-09T04:27:39.652 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-05-09T04:27:39.652 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-05-09T04:27:39.652 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-05-09T04:27:39.652 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-05-09T04:27:39.652 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-05-09T04:27:39.653 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-05-09T04:27:39.653 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-05-09T04:27:39.653 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-05-09T04:27:39.653 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-09T04:27:39.653 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-09T04:27:39.653 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-05-09T04:27:39.653 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-05-09T04:27:39.653 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-05-09T04:27:39.653 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-05-09T04:27:39.653 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-05-09T04:27:39.653 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-05-09T04:27:39.653 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-05-09T04:27:39.653 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-05-09T04:27:39.653 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-05-09T04:27:39.654 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-05-09T04:27:39.654 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-05-09T04:27:39.654 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-05-09T04:27:39.654 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-05-09T04:27:39.654 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-05-09T04:27:39.654 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-05-09T04:27:39.654 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-05-09T04:27:39.654 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-05-09T04:27:39.654 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-05-09T04:27:39.654 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-05-09T04:27:39.654 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-05-09T04:27:39.654 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-05-09T04:27:39.654 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-05-09T04:27:39.654 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-05-09T04:27:39.655 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-05-09T04:27:39.655 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-05-09T04:27:39.655 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-05-09T04:27:39.655 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-05-09T04:27:39.655 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-09T04:27:39.655 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-09T04:27:39.655 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-05-09T04:27:39.655 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-05-09T04:27:39.655 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-05-09T04:27:39.655 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-05-09T04:27:39.655 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-05-09T04:27:39.655 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-05-09T04:27:39.655 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-05-09T04:27:39.655 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-05-09T04:27:39.655 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-05-09T04:27:39.656 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-05-09T04:27:39.656 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-05-09T04:27:39.656 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-09T04:27:39.656 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-05-09T04:27:39.656 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-05-09T04:27:39.656 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-05-09T04:27:39.656 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-05-09T04:27:39.656 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-05-09T04:27:39.656 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-05-09T04:27:39.656 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-05-09T04:27:39.656 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-09T04:27:39.656 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-05-09T04:27:39.656 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-05-09T04:27:39.656 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-05-09T04:27:39.657 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-05-09T04:27:39.657 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-05-09T04:27:39.657 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-05-09T04:27:39.657 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-09T04:27:39.657 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-09T04:27:39.657 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-09T04:27:39.657 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-05-09T04:27:39.657 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-09T04:27:39.657 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-05-09T04:27:39.657 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-09T04:27:39.657 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-09T04:27:39.657 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-09T04:27:39.657 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-09T04:27:39.657 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-09T04:27:39.657 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-09T04:27:39.658 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-05-09T04:27:39.658 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-05-09T04:27:39.658 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-05-09T04:27:39.658 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-05-09T04:27:39.658 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-05-09T04:27:39.658 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-05-09T04:27:39.658 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-05-09T04:27:39.658 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-09T04:27:39.658 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-09T04:27:39.658 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-09T04:27:39.658 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-09T04:27:39.658 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-09T04:27:39.658 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-05-09T04:27:39.658 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-05-09T04:27:39.659 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-05-09T04:27:39.659 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-09T04:27:39.659 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-09T04:27:39.659 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-09T04:27:39.659 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-09T04:27:39.659 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-09T04:27:39.659 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-09T04:27:39.659 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-05-09T04:27:39.659 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-05-09T04:27:39.659 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-09T04:27:39.659 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-05-09T04:27:39.659 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-05-09T04:27:39.659 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-05-09T04:27:39.659 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-05-09T04:27:39.659 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-09T04:27:39.660 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-09T04:27:39.660 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-09T04:27:39.660 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-05-09T04:27:39.660 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-05-09T04:27:39.660 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-05-09T04:27:39.660 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-09T04:27:39.660 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-05-09T04:27:39.660 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-05-09T04:27:39.660 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-05-09T04:27:39.660 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-09T04:27:39.660 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-05-09T04:27:39.660 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-05-09T04:27:39.660 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-09T04:27:39.660 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-05-09T04:27:39.661 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-09T04:27:39.661 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-05-09T04:27:39.661 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-05-09T04:27:39.661 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-05-09T04:27:39.661 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-05-09T04:27:39.661 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-05-09T04:27:39.661 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-05-09T04:27:39.661 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-05-09T04:27:39.661 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-05-09T04:27:39.661 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-05-09T04:27:39.661 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-05-09T04:27:39.661 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-05-09T04:27:39.661 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-05-09T04:27:39.661 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-05-09T04:27:39.662 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-05-09T04:27:39.662 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-05-09T04:27:39.662 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-05-09T04:27:39.662 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-09T04:27:39.662 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-05-09T04:27:39.662 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-05-09T04:27:39.662 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-09T04:27:39.662 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-05-09T04:27:39.662 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-05-09T04:27:39.662 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-05-09T04:27:39.662 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-05-09T04:27:39.662 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-05-09T04:27:39.662 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-05-09T04:27:39.662 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-05-09T04:27:39.663 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-05-09T04:27:39.663 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-05-09T04:27:39.663 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-05-09T04:27:39.663 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-05-09T04:27:39.663 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-09T04:27:39.663 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-09T04:27:39.663 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-05-09T04:27:39.663 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-05-09T04:27:39.663 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-05-09T04:27:39.663 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-05-09T04:27:39.663 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-05-09T04:27:39.663 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-05-09T04:27:39.663 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-05-09T04:27:39.663 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-05-09T04:27:39.664 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-09T04:27:39.664 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-09T04:27:39.664 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-05-09T04:27:39.664 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-05-09T04:27:39.664 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-05-09T04:27:39.664 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-05-09T04:27:39.664 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-05-09T04:27:39.664 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-05-09T04:27:39.664 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-05-09T04:27:39.664 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-05-09T04:27:39.664 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-05-09T04:27:39.664 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-09T04:27:39.664 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-05-09T04:27:39.664 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-09T04:27:39.665 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-09T04:27:39.665 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-09T04:27:39.665 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-09T04:27:39.665 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-09T04:27:39.665 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-09T04:27:39.665 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-09T04:27:39.665 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-09T04:27:39.665 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-09T04:27:39.665 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-05-09T04:27:39.665 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-05-09T04:27:39.665 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-05-09T04:27:39.665 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-09T04:27:39.665 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-05-09T04:27:39.665 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-05-09T04:27:39.665 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-05-09T04:27:39.666 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-05-09T04:27:39.666 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-05-09T04:27:39.666 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-09T04:27:39.666 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-05-09T04:27:39.666 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-05-09T04:27:39.666 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-05-09T04:27:39.666 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-05-09T04:27:39.666 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-05-09T04:27:39.666 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-05-09T04:27:39.666 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-05-09T04:27:39.666 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-05-09T04:27:39.666 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-05-09T04:27:39.666 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-05-09T04:27:39.666 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-05-09T04:27:39.667 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-05-09T04:27:39.667 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-05-09T04:27:39.667 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-05-09T04:27:39.667 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-05-09T04:27:39.667 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-05-09T04:27:39.667 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-05-09T04:27:39.667 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-05-09T04:27:39.667 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-05-09T04:27:39.667 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-09T04:27:39.667 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-05-09T04:27:39.667 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-05-09T04:27:39.667 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-05-09T04:27:39.667 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-05-09T04:27:39.667 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-05-09T04:27:39.668 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-05-09T04:27:39.668 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-05-09T04:27:39.668 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-05-09T04:27:39.668 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-05-09T04:27:39.668 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-05-09T04:27:39.668 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-05-09T04:27:39.668 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-05-09T04:27:39.668 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-05-09T04:27:39.668 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-05-09T04:27:39.668 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-05-09T04:27:39.668 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-05-09T04:27:39.668 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-05-09T04:27:39.668 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-05-09T04:27:39.668 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-05-09T04:27:39.668 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-09T04:27:39.669 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-05-09T04:27:39.669 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-05-09T04:27:39.669 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-09T04:27:39.669 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-05-09T04:27:39.669 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-05-09T04:27:39.669 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-05-09T04:27:39.669 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-05-09T04:27:39.669 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-05-09T04:27:39.669 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-05-09T04:27:39.669 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-05-09T04:27:39.669 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-05-09T04:27:39.669 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-05-09T04:27:39.669 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-05-09T04:27:39.669 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-05-09T04:27:39.670 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-05-09T04:27:39.670 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-05-09T04:27:39.670 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-05-09T04:27:39.670 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-09T04:27:39.670 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-05-09T04:27:39.670 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-05-09T04:27:39.670 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-05-09T04:27:39.670 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-05-09T04:27:39.670 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-05-09T04:27:39.670 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-05-09T04:27:39.670 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-05-09T04:27:39.670 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-05-09T04:27:39.670 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-05-09T04:27:39.671 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-05-09T04:27:39.671 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-05-09T04:27:39.671 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-05-09T04:27:39.671 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-05-09T04:27:39.671 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-05-09T04:27:39.671 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-05-09T04:27:39.671 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-05-09T04:27:39.671 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-05-09T04:27:39.671 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-05-09T04:27:39.671 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-05-09T04:27:39.671 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-05-09T04:27:39.671 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-05-09T04:27:39.671 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-05-09T04:27:39.671 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-05-09T04:27:39.671 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-05-09T04:27:39.672 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-05-09T04:27:39.672 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-05-09T04:27:39.672 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-05-09T04:27:39.672 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-05-09T04:27:39.672 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-05-09T04:27:39.672 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-05-09T04:27:39.672 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-05-09T04:27:39.672 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-05-09T04:27:39.672 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-05-09T04:27:39.672 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-05-09T04:27:39.672 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-05-09T04:27:39.672 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-05-09T04:27:39.672 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-05-09T04:27:39.672 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-05-09T04:27:39.673 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-05-09T04:27:39.673 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-05-09T04:27:39.673 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-05-09T04:27:39.673 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-05-09T04:27:39.673 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-05-09T04:27:39.673 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-05-09T04:27:39.673 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-05-09T04:27:39.673 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-05-09T04:27:39.673 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-05-09T04:27:39.673 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-05-09T04:27:39.673 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-09T04:27:39.673 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-09T04:27:39.673 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-09T04:27:39.673 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-09T04:27:39.673 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-09T04:27:39.674 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-09T04:27:39.674 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-09T04:27:39.674 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-09T04:27:39.674 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-09T04:27:39.674 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-09T04:27:39.674 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-09T04:27:39.674 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-09T04:27:39.674 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-05-09T04:27:39.674 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-05-09T04:27:39.674 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-05-09T04:27:39.674 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-09T04:27:39.674 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-05-09T04:27:39.674 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-05-09T04:27:39.674 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-05-09T04:27:39.675 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-05-09T04:27:39.675 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-05-09T04:27:39.675 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-05-09T04:27:39.675 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-05-09T04:27:39.675 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-05-09T04:27:39.675 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-05-09T04:27:39.675 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-05-09T04:27:39.675 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-05-09T04:27:39.675 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-05-09T04:27:39.675 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-05-09T04:27:39.675 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-05-09T04:27:39.675 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-05-09T04:27:39.675 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-05-09T04:27:39.675 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-05-09T04:27:39.675 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-05-09T04:27:39.676 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-05-09T04:27:39.676 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-05-09T04:27:39.676 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-05-09T04:27:39.676 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-05-09T04:27:39.676 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-05-09T04:27:39.676 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-05-09T04:27:39.676 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-05-09T04:27:39.676 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-05-09T04:27:39.676 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-05-09T04:27:39.676 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-09T04:27:39.676 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-09T04:27:39.676 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-05-09T04:27:39.676 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-09T04:27:39.676 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-09T04:27:39.677 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-09T04:27:39.677 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-09T04:27:39.677 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-09T04:27:39.677 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-09T04:27:39.677 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-09T04:27:39.677 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-09T04:27:39.677 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-09T04:27:39.677 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-09T04:27:39.677 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-09T04:27:39.677 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-09T04:27:39.677 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-09T04:27:39.677 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-09T04:27:39.677 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-09T04:27:39.678 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-09T04:27:39.678 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-09T04:27:39.678 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-09T04:27:39.678 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-09T04:27:39.678 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-09T04:27:39.678 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-09T04:27:39.678 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-09T04:27:39.678 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-09T04:27:39.678 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-05-09T04:27:39.678 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-05-09T04:27:39.678 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-09T04:27:39.678 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-05-09T04:27:39.678 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-05-09T04:27:39.678 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-05-09T04:27:39.679 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-05-09T04:27:39.679 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-09T04:27:39.679 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-05-09T04:27:39.679 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-09T04:27:39.679 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-09T04:27:39.679 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-09T04:27:39.679 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-09T04:27:39.679 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-09T04:27:39.679 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-05-09T04:27:39.679 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-05-09T04:27:39.679 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-09T04:27:39.679 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-05-09T04:27:39.679 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-09T04:27:39.679 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-05-09T04:27:39.680 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-09T04:27:39.680 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-05-09T04:27:39.680 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-09T04:27:39.680 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-05-09T04:27:39.680 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-05-09T04:27:39.680 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-09T04:27:39.680 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-05-09T04:27:39.680 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-09T04:27:39.680 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-05-09T04:27:39.680 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-09T04:27:39.680 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-05-09T04:27:39.680 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-09T04:27:39.680 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-05-09T04:27:39.680 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-09T04:27:39.680 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-05-09T04:27:39.681 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-05-09T04:27:39.681 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-09T04:27:39.681 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-09T04:27:39.681 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-05-09T04:27:39.681 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-05-09T04:27:39.681 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-05-09T04:27:39.681 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-09T04:27:39.681 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-05-09T04:27:39.681 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-05-09T04:27:39.681 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-05-09T04:27:39.681 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-05-09T04:27:39.681 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-09T04:27:39.681 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-05-09T04:27:39.681 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-09T04:27:39.681 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-09T04:27:39.682 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-09T04:27:39.682 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-05-09T04:27:39.682 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-09T04:27:39.682 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-05-09T04:27:39.682 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-05-09T04:27:39.682 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-05-09T04:27:39.682 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-09T04:27:39.682 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-05-09T04:27:39.682 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-05-09T04:27:39.682 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-05-09T04:27:39.682 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-05-09T04:27:39.682 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-05-09T04:27:39.682 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-05-09T04:27:39.683 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-05-09T04:27:39.683 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-05-09T04:27:39.683 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-09T04:27:39.683 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-09T04:27:39.683 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-09T04:27:39.683 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-09T04:27:39.683 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-09T04:27:39.683 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-09T04:27:39.683 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-09T04:27:39.683 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-09T04:27:39.683 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-09T04:27:39.683 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-05-09T04:27:39.683 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-05-09T04:27:39.683 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-05-09T04:27:39.683 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-05-09T04:27:39.684 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-05-09T04:27:39.684 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-05-09T04:27:39.684 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-05-09T04:27:39.684 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-05-09T04:27:39.684 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-05-09T04:27:39.684 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-05-09T04:27:39.684 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-05-09T04:27:39.684 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-05-09T04:27:39.684 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-05-09T04:27:39.684 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-05-09T04:27:39.684 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-05-09T04:27:39.684 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-05-09T04:27:39.684 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-05-09T04:27:39.684 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-05-09T04:27:39.685 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-05-09T04:27:39.685 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-09T04:27:39.685 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-05-09T04:27:39.685 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-05-09T04:27:39.685 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-05-09T04:27:39.685 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-05-09T04:27:39.685 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-05-09T04:27:39.685 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-05-09T04:27:39.685 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-05-09T04:27:39.685 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-05-09T04:27:39.685 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-05-09T04:27:39.685 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-05-09T04:27:39.685 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-05-09T04:27:39.685 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-05-09T04:27:39.686 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-05-09T04:27:39.686 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-05-09T04:27:39.686 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-05-09T04:27:39.686 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-05-09T04:27:39.686 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-05-09T04:27:39.686 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-05-09T04:27:39.686 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-05-09T04:27:39.686 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-05-09T04:27:39.686 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-05-09T04:27:39.686 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-05-09T04:27:39.686 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-05-09T04:27:39.686 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-09T04:27:39.686 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-05-09T04:27:39.686 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-05-09T04:27:39.687 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-05-09T04:27:39.687 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-05-09T04:27:39.687 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-05-09T04:27:39.687 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-05-09T04:27:39.687 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-05-09T04:27:39.687 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-09T04:27:39.687 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-05-09T04:27:39.687 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-05-09T04:27:39.687 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-05-09T04:27:39.687 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-05-09T04:27:39.687 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-05-09T04:27:39.687 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-05-09T04:27:39.687 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-09T04:27:39.687 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-09T04:27:39.688 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-05-09T04:27:39.688 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-05-09T04:27:39.688 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-05-09T04:27:39.688 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-05-09T04:27:39.688 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-05-09T04:27:39.688 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-09T04:27:39.688 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-05-09T04:27:39.688 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-09T04:27:39.688 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-09T04:27:39.688 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-09T04:27:39.688 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-05-09T04:27:39.688 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-05-09T04:27:39.688 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-05-09T04:27:39.688 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-05-09T04:27:39.688 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-05-09T04:27:39.689 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-09T04:27:39.689 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-05-09T04:27:39.689 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-05-09T04:27:39.689 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-05-09T04:27:39.689 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-05-09T04:27:39.689 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-05-09T04:27:39.689 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-09T04:27:39.689 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-09T04:27:39.689 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-09T04:27:39.689 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-05-09T04:27:39.689 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-09T04:27:39.689 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-05-09T04:27:39.689 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-05-09T04:27:39.689 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-05-09T04:27:39.689 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-05-09T04:27:39.690 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-09T04:27:39.690 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-09T04:27:39.690 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-09T04:27:39.690 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-09T04:27:39.690 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-05-09T04:27:39.690 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-09T04:27:39.690 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-09T04:27:39.690 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-09T04:27:39.690 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-09T04:27:39.690 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-05-09T04:27:39.690 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-09T04:27:39.690 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-05-09T04:27:39.690 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-09T04:27:39.690 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-05-09T04:27:39.691 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-09T04:27:39.691 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-05-09T04:27:39.691 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-09T04:27:39.691 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-05-09T04:27:39.691 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-09T04:27:39.691 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-09T04:27:39.691 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-09T04:27:39.691 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-05-09T04:27:39.691 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-05-09T04:27:39.691 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-09T04:27:39.691 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-09T04:27:39.691 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-09T04:27:39.691 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-09T04:27:39.691 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-09T04:27:39.692 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-05-09T04:27:39.692 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-09T04:27:39.692 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-09T04:27:39.692 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-05-09T04:27:39.692 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-09T04:27:39.692 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-09T04:27:39.692 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-09T04:27:39.692 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-09T04:27:39.692 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-09T04:27:39.692 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-05-09T04:27:39.692 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-09T04:27:39.692 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-09T04:27:39.692 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-05-09T04:27:39.692 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-05-09T04:27:39.693 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-09T04:27:39.693 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-05-09T04:27:39.693 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-09T04:27:39.693 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-09T04:27:39.693 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-09T04:27:39.693 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-09T04:27:39.693 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-09T04:27:39.693 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-09T04:27:39.693 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-09T04:27:39.693 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-09T04:27:39.693 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-09T04:27:39.693 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-09T04:27:39.693 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-05-09T04:27:39.693 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-05-09T04:27:39.694 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-09T04:27:39.694 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-05-09T04:27:39.694 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-05-09T04:27:39.694 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-09T04:27:39.694 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-09T04:27:39.694 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-09T04:27:39.694 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-09T04:27:39.694 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-09T04:27:39.694 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-09T04:27:39.694 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-05-09T04:27:39.694 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-09T04:27:39.694 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-09T04:27:39.694 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-05-09T04:27:39.694 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-05-09T04:27:39.695 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-05-09T04:27:39.695 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-09T04:27:39.695 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-05-09T04:27:39.695 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-05-09T04:27:39.695 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-05-09T04:27:39.695 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-09T04:27:39.695 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-05-09T04:27:39.695 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-09T04:27:39.695 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-05-09T04:27:39.695 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-09T04:27:39.695 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-05-09T04:27:39.695 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-09T04:27:39.695 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-05-09T04:27:39.695 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-09T04:27:39.695 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-05-09T04:27:39.696 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-09T04:27:39.696 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-05-09T04:27:39.696 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-09T04:27:39.696 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-05-09T04:27:39.696 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-09T04:27:39.696 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-09T04:27:39.696 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-09T04:27:39.696 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-09T04:27:39.696 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-09T04:27:39.696 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-05-09T04:27:39.696 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-05-09T04:27:39.696 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-09T04:27:39.696 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-09T04:27:39.696 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-09T04:27:39.697 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-05-09T04:27:39.697 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-09T04:27:39.697 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-09T04:27:39.697 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-09T04:27:39.697 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-09T04:27:39.697 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-09T04:27:39.697 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-09T04:27:39.697 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-05-09T04:27:39.697 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-09T04:27:39.697 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-05-09T04:27:39.697 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-09T04:27:39.697 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-09T04:27:39.697 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-05-09T04:27:39.697 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-09T04:27:39.697 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-09T04:27:39.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-05-09T04:27:39.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-09T04:27:39.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-05-09T04:27:39.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-09T04:27:39.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-05-09T04:27:39.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-09T04:27:39.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-09T04:27:39.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-05-09T04:27:39.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-09T04:27:39.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-09T04:27:39.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-09T04:27:39.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-09T04:27:39.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-09T04:27:39.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-05-09T04:27:39.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-09T04:27:39.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-05-09T04:27:39.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-09T04:27:39.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-09T04:27:39.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-09T04:27:39.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-09T04:27:39.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-05-09T04:27:39.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-09T04:27:39.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-05-09T04:27:39.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-09T04:27:39.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-05-09T04:27:39.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-09T04:27:39.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-05-09T04:27:39.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-09T04:27:39.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-09T04:27:39.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-05-09T04:27:39.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-09T04:27:39.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-09T04:27:39.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-09T04:27:39.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-09T04:27:39.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-05-09T04:27:39.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-09T04:27:39.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-09T04:27:39.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-09T04:27:39.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-05-09T04:27:39.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-09T04:27:39.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-09T04:27:39.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-09T04:27:39.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-09T04:27:39.701 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-05-09T04:27:39.701 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-09T04:27:39.701 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-05-09T04:27:39.701 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-09T04:27:39.701 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-05-09T04:27:39.701 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-09T04:27:39.701 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-09T04:27:39.701 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-05-09T04:27:39.701 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-09T04:27:39.701 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-09T04:27:39.701 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-05-09T04:27:39.701 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-09T04:27:39.701 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-05-09T04:27:39.701 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-05-09T04:27:39.702 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-09T04:27:39.702 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-05-09T04:27:39.702 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-09T04:27:39.702 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-05-09T04:27:39.702 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-05-09T04:27:39.702 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-09T04:27:39.702 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-05-09T04:27:39.702 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-05-09T04:27:39.702 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-05-09T04:27:39.702 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-05-09T04:27:39.702 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-09T04:27:39.702 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-09T04:27:39.702 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-09T04:27:39.702 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-05-09T04:27:39.703 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-05-09T04:27:39.703 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-05-09T04:27:39.703 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-05-09T04:27:39.703 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-05-09T04:27:39.703 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-09T04:27:39.703 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-09T04:27:39.703 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-05-09T04:27:39.703 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-05-09T04:27:39.703 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-09T04:27:39.703 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-09T04:27:39.703 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-09T04:27:39.703 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-05-09T04:27:39.703 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-05-09T04:27:39.703 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-05-09T04:27:39.704 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-05-09T04:27:39.704 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-05-09T04:27:39.704 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-05-09T04:27:39.704 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-05-09T04:27:39.704 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-05-09T04:27:39.704 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-05-09T04:27:39.704 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-05-09T04:27:39.704 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-05-09T04:27:39.704 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-05-09T04:27:39.704 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-05-09T04:27:39.704 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-05-09T04:27:39.704 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-05-09T04:27:39.704 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-05-09T04:27:39.704 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-05-09T04:27:39.704 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-05-09T04:27:39.705 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-09T04:27:39.705 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-05-09T04:27:39.705 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-09T04:27:39.705 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-09T04:27:39.705 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-09T04:27:39.705 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-09T04:27:39.705 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-05-09T04:27:39.705 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-05-09T04:27:39.705 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-05-09T04:27:39.705 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-05-09T04:27:39.705 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-05-09T04:27:39.705 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-05-09T04:27:39.705 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-05-09T04:27:39.705 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-05-09T04:27:39.705 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-05-09T04:27:39.706 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-05-09T04:27:39.706 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-09T04:27:39.706 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-09T04:27:39.706 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-09T04:27:39.706 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-05-09T04:27:39.706 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-09T04:27:39.706 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-05-09T04:27:39.706 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-05-09T04:27:39.706 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-05-09T04:27:39.706 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-05-09T04:27:39.706 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-09T04:27:39.706 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-05-09T04:27:39.706 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-05-09T04:27:39.706 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-05-09T04:27:39.707 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-09T04:27:39.707 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-05-09T04:27:39.707 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-05-09T04:27:39.707 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-05-09T04:27:39.707 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-05-09T04:27:39.707 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-05-09T04:27:39.707 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-05-09T04:27:39.707 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-05-09T04:27:39.707 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-05-09T04:27:39.707 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-05-09T04:27:39.707 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-05-09T04:27:39.707 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-05-09T04:27:39.707 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-05-09T04:27:39.707 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-05-09T04:27:39.707 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-05-09T04:27:39.708 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-09T04:27:39.708 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-05-09T04:27:39.708 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-09T04:27:39.708 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-05-09T04:27:39.708 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-05-09T04:27:39.708 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-05-09T04:27:39.708 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-05-09T04:27:39.708 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-09T04:27:39.708 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-05-09T04:27:39.708 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-05-09T04:27:39.708 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-09T04:27:39.708 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-05-09T04:27:39.708 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-05-09T04:27:39.708 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-09T04:27:39.709 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-05-09T04:27:39.709 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-09T04:27:39.709 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-09T04:27:39.709 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-09T04:27:39.709 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-09T04:27:39.709 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-09T04:27:39.709 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-09T04:27:39.709 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-09T04:27:39.709 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-09T04:27:39.709 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-09T04:27:39.709 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-05-09T04:27:39.709 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-05-09T04:27:39.709 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-09T04:27:39.709 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-05-09T04:27:39.710 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-09T04:27:39.710 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-09T04:27:39.710 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-09T04:27:39.710 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-09T04:27:39.710 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-09T04:27:39.710 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-09T04:27:39.710 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-09T04:27:39.710 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-09T04:27:39.710 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-09T04:27:39.710 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-05-09T04:27:39.710 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-05-09T04:27:39.710 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-09T04:27:39.710 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-09T04:27:39.710 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-05-09T04:27:39.711 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-05-09T04:27:39.711 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-09T04:27:39.711 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-09T04:27:39.711 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-05-09T04:27:39.711 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-05-09T04:27:39.711 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-05-09T04:27:39.711 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-05-09T04:27:39.711 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-05-09T04:27:39.711 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-05-09T04:27:39.711 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-05-09T04:27:39.711 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-05-09T04:27:39.711 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-09T04:27:39.711 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-05-09T04:27:39.711 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-05-09T04:27:39.711 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-09T04:27:39.712 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-09T04:27:39.712 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-09T04:27:39.712 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-09T04:27:39.712 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-09T04:27:39.712 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-09T04:27:39.712 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-09T04:27:39.712 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-09T04:27:39.712 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-09T04:27:39.712 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-09T04:27:39.712 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-05-09T04:27:39.712 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-09T04:27:39.712 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-09T04:27:39.712 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-09T04:27:39.712 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-09T04:27:39.712 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-09T04:27:39.713 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-09T04:27:39.713 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-05-09T04:27:39.713 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-09T04:27:39.713 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-09T04:27:39.713 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-09T04:27:39.713 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-09T04:27:39.713 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-09T04:27:39.713 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-09T04:27:39.713 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-09T04:27:39.713 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-09T04:27:39.713 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-09T04:27:39.713 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-09T04:27:39.713 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-09T04:27:39.713 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-09T04:27:39.713 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-05-09T04:27:39.714 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-05-09T04:27:39.714 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-05-09T04:27:39.714 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-05-09T04:27:39.714 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-05-09T04:27:39.714 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-05-09T04:27:39.714 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-05-09T04:27:39.714 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-05-09T04:27:39.714 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-09T04:27:39.714 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-05-09T04:27:39.714 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-05-09T04:27:39.714 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-09T04:27:39.714 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-09T04:27:39.714 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-05-09T04:27:39.714 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-05-09T04:27:39.715 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-09T04:27:39.715 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-05-09T04:27:39.715 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-05-09T04:27:39.715 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-09T04:27:39.715 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-05-09T04:27:39.715 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-05-09T04:27:39.715 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-05-09T04:27:39.715 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-09T04:27:39.715 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-05-09T04:27:39.715 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-09T04:27:39.715 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-05-09T04:27:39.715 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-05-09T04:27:39.715 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-09T04:27:39.715 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-05-09T04:27:39.715 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-09T04:27:39.716 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-09T04:27:39.716 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-09T04:27:39.716 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-09T04:27:39.716 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-09T04:27:39.716 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-05-09T04:27:39.716 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-05-09T04:27:39.716 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-09T04:27:39.716 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-09T04:27:39.716 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-09T04:27:39.716 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-09T04:27:39.716 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-05-09T04:27:39.716 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-05-09T04:27:39.716 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-09T04:27:39.716 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-05-09T04:27:39.716 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-05-09T04:27:39.717 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-09T04:27:39.717 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-09T04:27:39.717 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-05-09T04:27:39.717 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-05-09T04:27:39.717 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-05-09T04:27:39.717 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-05-09T04:27:39.717 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-05-09T04:27:39.717 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-05-09T04:27:39.717 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-05-09T04:27:39.717 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-05-09T04:27:39.717 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-05-09T04:27:39.717 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-05-09T04:27:39.717 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-09T04:27:39.717 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-09T04:27:39.717 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-05-09T04:27:39.718 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-05-09T04:27:39.718 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-05-09T04:27:39.718 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-09T04:27:39.718 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-05-09T04:27:39.718 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-05-09T04:27:39.718 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-05-09T04:27:39.718 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-09T04:27:39.718 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-05-09T04:27:39.718 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-05-09T04:27:39.718 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-09T04:27:39.718 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-09T04:27:39.718 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-09T04:27:39.718 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-05-09T04:27:39.719 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-09T04:27:39.719 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-05-09T04:27:39.719 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-09T04:27:39.719 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-05-09T04:27:39.719 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-05-09T04:27:39.719 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-09T04:27:39.719 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-05-09T04:27:39.719 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-05-09T04:27:39.719 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-05-09T04:27:39.719 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-09T04:27:39.719 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-09T04:27:39.719 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-05-09T04:27:39.719 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-09T04:27:39.719 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-05-09T04:27:39.719 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-05-09T04:27:39.719 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-09T04:27:39.720 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-09T04:27:39.720 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-09T04:27:39.720 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-09T04:27:39.720 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-09T04:27:39.720 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-05-09T04:27:39.720 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-05-09T04:27:39.720 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-05-09T04:27:39.720 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-05-09T04:27:39.720 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-09T04:27:39.720 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-05-09T04:27:39.720 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-09T04:27:39.720 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-05-09T04:27:39.720 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-05-09T04:27:39.720 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-05-09T04:27:39.720 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-05-09T04:27:39.721 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-05-09T04:27:39.721 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-09T04:27:39.721 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-05-09T04:27:39.721 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-09T04:27:39.721 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-05-09T04:27:39.721 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-05-09T04:27:39.721 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-05-09T04:27:39.721 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-05-09T04:27:39.721 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-05-09T04:27:39.721 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-05-09T04:27:39.721 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-09T04:27:39.721 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-05-09T04:27:39.721 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-05-09T04:27:39.721 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-05-09T04:27:39.722 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-09T04:27:39.722 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-09T04:27:39.722 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-09T04:27:39.722 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-09T04:27:39.722 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-09T04:27:39.722 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-09T04:27:39.722 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-09T04:27:39.722 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-09T04:27:39.722 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-09T04:27:39.722 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-09T04:27:39.722 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-09T04:27:39.722 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-09T04:27:39.722 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-09T04:27:39.722 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-09T04:27:39.723 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-09T04:27:39.723 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-09T04:27:39.723 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-09T04:27:39.723 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-09T04:27:39.723 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-05-09T04:27:39.723 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-05-09T04:27:39.723 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-09T04:27:39.723 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-05-09T04:27:39.723 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-05-09T04:27:39.723 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-05-09T04:27:39.723 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-05-09T04:27:39.723 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-05-09T04:27:39.723 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-09T04:27:39.723 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-05-09T04:27:39.724 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-05-09T04:27:39.724 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-09T04:27:39.724 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-05-09T04:27:39.724 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-05-09T04:27:39.724 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-09T04:27:39.724 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-05-09T04:27:39.724 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-09T04:27:39.724 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-05-09T04:27:39.724 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-05-09T04:27:39.724 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-05-09T04:27:39.724 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-05-09T04:27:39.724 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-05-09T04:27:39.724 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-05-09T04:27:39.724 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-05-09T04:27:39.725 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-05-09T04:27:39.725 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-05-09T04:27:39.725 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-09T04:27:39.725 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-05-09T04:27:39.725 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-05-09T04:27:39.725 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-05-09T04:27:39.725 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-09T04:27:39.725 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-05-09T04:27:39.725 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-05-09T04:27:39.725 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-05-09T04:27:39.725 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-05-09T04:27:39.725 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-05-09T04:27:39.725 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-05-09T04:27:39.725 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-05-09T04:27:39.725 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-05-09T04:27:39.726 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-05-09T04:27:39.726 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-05-09T04:27:39.726 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-05-09T04:27:39.726 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-09T04:27:39.726 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-05-09T04:27:39.726 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-05-09T04:27:39.726 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-05-09T04:27:39.726 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-09T04:27:39.726 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-09T04:27:39.726 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-05-09T04:27:39.726 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-05-09T04:27:39.726 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-09T04:27:39.726 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-05-09T04:27:39.726 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-05-09T04:27:39.727 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-05-09T04:27:39.727 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-05-09T04:27:39.727 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-05-09T04:27:39.727 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-05-09T04:27:39.727 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-05-09T04:27:39.727 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-09T04:27:39.727 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-05-09T04:27:39.727 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-05-09T04:27:39.727 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-05-09T04:27:39.727 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-09T04:27:39.727 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-09T04:27:39.727 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-05-09T04:27:39.727 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-09T04:27:39.727 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-09T04:27:39.727 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-09T04:27:39.727 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-05-09T04:27:39.728 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-05-09T04:27:39.728 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-05-09T04:27:39.728 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-05-09T04:27:39.728 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-09T04:27:39.728 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-05-09T04:27:39.728 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-09T04:27:39.728 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-09T04:27:39.728 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-09T04:27:39.728 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-05-09T04:27:39.728 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-05-09T04:27:39.728 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-05-09T04:27:39.728 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-05-09T04:27:39.728 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-05-09T04:27:39.728 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-05-09T04:27:39.728 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-05-09T04:27:39.728 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-05-09T04:27:39.729 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-05-09T04:27:39.729 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-05-09T04:27:39.729 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-05-09T04:27:39.729 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-05-09T04:27:39.729 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-05-09T04:27:39.729 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-05-09T04:27:39.729 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-09T04:27:39.729 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-05-09T04:27:39.729 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-05-09T04:27:39.729 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-09T04:27:39.729 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-09T04:27:39.729 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-05-09T04:27:39.729 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-09T04:27:39.729 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-05-09T04:27:39.729 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-05-09T04:27:39.729 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-09T04:27:39.730 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-05-09T04:27:39.730 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-05-09T04:27:39.730 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-09T04:27:39.730 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-09T04:27:39.730 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-09T04:27:39.730 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-09T04:27:39.730 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-09T04:27:39.730 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-09T04:27:39.730 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-09T04:27:39.730 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-05-09T04:27:39.730 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-09T04:27:39.730 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-09T04:27:39.730 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-09T04:27:39.730 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-09T04:27:39.731 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-09T04:27:39.731 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-05-09T04:27:39.731 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-05-09T04:27:39.731 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-09T04:27:39.731 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-09T04:27:39.731 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-09T04:27:39.731 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-05-09T04:27:39.731 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-05-09T04:27:39.731 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-09T04:27:39.731 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-09T04:27:39.731 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-05-09T04:27:39.731 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-09T04:27:39.731 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-05-09T04:27:39.731 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-05-09T04:27:39.732 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-09T04:27:39.732 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-05-09T04:27:39.732 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-05-09T04:27:39.732 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-05-09T04:27:39.732 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-05-09T04:27:39.732 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-05-09T04:27:39.732 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-05-09T04:27:39.732 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-09T04:27:39.732 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-05-09T04:27:39.732 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-05-09T04:27:39.732 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-05-09T04:27:39.732 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-05-09T04:27:39.732 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-05-09T04:27:39.732 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-09T04:27:39.733 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-09T04:27:39.733 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-05-09T04:27:39.733 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-05-09T04:27:39.733 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-05-09T04:27:39.733 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-05-09T04:27:39.733 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-05-09T04:27:39.733 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-05-09T04:27:39.733 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-09T04:27:39.733 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-09T04:27:39.733 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-09T04:27:39.733 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-09T04:27:39.733 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-09T04:27:39.733 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-09T04:27:39.733 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-09T04:27:39.733 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-09T04:27:39.734 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-09T04:27:39.734 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-09T04:27:39.734 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-09T04:27:39.734 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-05-09T04:27:39.734 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-05-09T04:27:39.734 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-09T04:27:39.734 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-09T04:27:39.734 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-09T04:27:39.734 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-05-09T04:27:39.734 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-05-09T04:27:39.734 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-09T04:27:39.734 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-09T04:27:39.734 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-05-09T04:27:39.734 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-05-09T04:27:39.735 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-09T04:27:39.735 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-09T04:27:39.735 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-05-09T04:27:39.735 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-05-09T04:27:39.735 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-05-09T04:27:39.735 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-09T04:27:39.735 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-09T04:27:39.735 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-09T04:27:39.735 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-09T04:27:39.735 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-05-09T04:27:39.735 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-09T04:27:39.735 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-05-09T04:27:39.735 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-09T04:27:39.735 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-09T04:27:39.735 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-09T04:27:39.736 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-05-09T04:27:39.736 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-05-09T04:27:39.736 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-05-09T04:27:39.736 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-09T04:27:39.736 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-05-09T04:27:39.736 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-05-09T04:27:39.736 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-05-09T04:27:39.736 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-05-09T04:27:39.736 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-09T04:27:39.736 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-05-09T04:27:39.736 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-05-09T04:27:39.736 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-09T04:27:39.736 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-05-09T04:27:39.736 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-05-09T04:27:39.736 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-05-09T04:27:39.737 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-05-09T04:27:39.737 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-05-09T04:27:39.737 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-05-09T04:27:39.737 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-05-09T04:27:39.737 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-05-09T04:27:39.737 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-05-09T04:27:39.737 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-05-09T04:27:39.737 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-05-09T04:27:39.737 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-05-09T04:27:39.737 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-05-09T04:27:39.737 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-05-09T04:27:39.737 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-05-09T04:27:39.737 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-05-09T04:27:39.737 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-05-09T04:27:39.737 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-05-09T04:27:39.738 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-05-09T04:27:39.738 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-05-09T04:27:39.738 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-05-09T04:27:39.738 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-09T04:27:39.738 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-05-09T04:27:39.738 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-05-09T04:27:39.738 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-05-09T04:27:39.738 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-05-09T04:27:39.738 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-05-09T04:27:39.738 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-05-09T04:27:39.738 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-05-09T04:27:39.738 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-05-09T04:27:39.738 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-05-09T04:27:39.738 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-05-09T04:27:39.739 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-05-09T04:27:39.739 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-05-09T04:27:39.739 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-05-09T04:27:39.739 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-05-09T04:27:39.739 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-05-09T04:27:39.739 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-05-09T04:27:39.739 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-05-09T04:27:39.739 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-05-09T04:27:39.739 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-05-09T04:27:39.739 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-05-09T04:27:39.739 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-05-09T04:27:39.739 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-05-09T04:27:39.739 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-05-09T04:27:39.739 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-05-09T04:27:39.739 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-05-09T04:27:39.740 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-05-09T04:27:39.740 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-05-09T04:27:39.740 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-09T04:27:39.740 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-05-09T04:27:39.740 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-09T04:27:39.740 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-05-09T04:27:39.740 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-09T04:27:39.740 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-05-09T04:27:39.740 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-05-09T04:27:39.740 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-05-09T04:27:39.740 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-05-09T04:27:39.740 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-05-09T04:27:39.740 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-05-09T04:27:39.740 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-05-09T04:27:39.741 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-05-09T04:27:39.741 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-05-09T04:27:39.741 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-05-09T04:27:39.741 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-05-09T04:27:39.741 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-05-09T04:27:39.741 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-05-09T04:27:39.741 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-05-09T04:27:39.741 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-05-09T04:27:39.741 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-05-09T04:27:39.741 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-05-09T04:27:39.741 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-05-09T04:27:39.741 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-05-09T04:27:39.741 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-05-09T04:27:39.741 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-05-09T04:27:39.742 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-05-09T04:27:39.742 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-05-09T04:27:39.742 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-05-09T04:27:39.742 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-05-09T04:27:39.742 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-05-09T04:27:39.742 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-05-09T04:27:39.742 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-05-09T04:27:39.742 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-05-09T04:27:39.742 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-05-09T04:27:39.742 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-05-09T04:27:39.742 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-05-09T04:27:39.742 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-05-09T04:27:39.742 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-05-09T04:27:39.742 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-05-09T04:27:39.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-05-09T04:27:39.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-05-09T04:27:39.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-05-09T04:27:39.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-05-09T04:27:39.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-05-09T04:27:39.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-05-09T04:27:39.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-05-09T04:27:39.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-05-09T04:27:39.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-05-09T04:27:39.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-05-09T04:27:39.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-05-09T04:27:39.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-05-09T04:27:39.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-05-09T04:27:39.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-05-09T04:27:39.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-05-09T04:27:39.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-05-09T04:27:39.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-05-09T04:27:39.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-05-09T04:27:39.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-05-09T04:27:39.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-05-09T04:27:39.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-05-09T04:27:39.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-05-09T04:27:39.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-05-09T04:27:39.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-05-09T04:27:39.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-09T04:27:39.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-05-09T04:27:39.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-05-09T04:27:39.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-05-09T04:27:39.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-09T04:27:39.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-05-09T04:27:39.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-09T04:27:39.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-05-09T04:27:39.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-09T04:27:39.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-05-09T04:27:39.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-09T04:27:39.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-09T04:27:39.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-09T04:27:39.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-05-09T04:27:39.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-05-09T04:27:39.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-05-09T04:27:39.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-05-09T04:27:39.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-05-09T04:27:39.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-05-09T04:27:39.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-05-09T04:27:39.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-05-09T04:27:39.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-05-09T04:27:39.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-05-09T04:27:39.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-05-09T04:27:39.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-05-09T04:27:39.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-05-09T04:27:39.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-05-09T04:27:39.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-05-09T04:27:39.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-05-09T04:27:39.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-05-09T04:27:39.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-05-09T04:27:39.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-05-09T04:27:39.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-05-09T04:27:39.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-05-09T04:27:39.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-05-09T04:27:39.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-05-09T04:27:39.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-05-09T04:27:39.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-05-09T04:27:39.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-05-09T04:27:39.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-05-09T04:27:39.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-05-09T04:27:39.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-05-09T04:27:39.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-09T04:27:39.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-05-09T04:27:39.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-05-09T04:27:39.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-05-09T04:27:39.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-05-09T04:27:39.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-05-09T04:27:39.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-05-09T04:27:39.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-05-09T04:27:39.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-05-09T04:27:39.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-05-09T04:27:39.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-05-09T04:27:39.750 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-05-09T04:27:39.750 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-05-09T04:27:39.750 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-05-09T04:27:39.750 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-05-09T04:27:39.750 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-05-09T04:27:39.750 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-05-09T04:27:39.750 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-05-09T04:27:39.750 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-09T04:27:39.750 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-05-09T04:27:39.750 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-05-09T04:27:39.750 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-05-09T04:27:39.750 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-05-09T04:27:39.750 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-05-09T04:27:39.750 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-05-09T04:27:39.750 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-05-09T04:27:39.751 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-05-09T04:27:39.751 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-05-09T04:27:39.751 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-05-09T04:27:39.751 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-05-09T04:27:39.751 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-05-09T04:27:39.751 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-05-09T04:27:39.751 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-09T04:27:39.751 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-05-09T04:27:39.751 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-05-09T04:27:39.751 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-05-09T04:27:39.751 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-05-09T04:27:39.751 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-05-09T04:27:39.751 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-05-09T04:27:39.751 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-05-09T04:27:39.752 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-05-09T04:27:39.752 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-05-09T04:27:39.752 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-05-09T04:27:39.752 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-05-09T04:27:39.752 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-05-09T04:27:39.752 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-05-09T04:27:39.752 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-05-09T04:27:39.752 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-05-09T04:27:39.752 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-05-09T04:27:39.752 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-05-09T04:27:39.752 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-05-09T04:27:39.752 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-05-09T04:27:39.752 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-05-09T04:27:39.752 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-05-09T04:27:39.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-05-09T04:27:39.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-09T04:27:39.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-09T04:27:39.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-09T04:27:39.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-09T04:27:39.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-09T04:27:39.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-09T04:27:39.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-09T04:27:39.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-09T04:27:39.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-09T04:27:39.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-09T04:27:39.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-09T04:27:39.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-09T04:27:39.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-09T04:27:39.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-09T04:27:39.754 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-09T04:27:39.754 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-09T04:27:39.754 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-09T04:27:39.754 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-09T04:27:39.754 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-09T04:27:39.754 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-09T04:27:39.754 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-05-09T04:27:39.754 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-09T04:27:39.754 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-09T04:27:39.754 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-09T04:27:39.754 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-05-09T04:27:39.754 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-05-09T04:27:39.754 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-05-09T04:27:39.754 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-05-09T04:27:39.755 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-05-09T04:27:39.755 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-05-09T04:27:39.755 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-05-09T04:27:39.755 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-05-09T04:27:39.755 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-05-09T04:27:39.755 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-05-09T04:27:39.755 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-05-09T04:27:39.755 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-05-09T04:27:39.755 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-05-09T04:27:39.755 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-05-09T04:27:39.755 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-05-09T04:27:39.755 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-05-09T04:27:39.755 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-05-09T04:27:39.755 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-05-09T04:27:39.755 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-05-09T04:27:39.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-05-09T04:27:39.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-09T04:27:39.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-05-09T04:27:39.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-09T04:27:39.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-05-09T04:27:39.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-05-09T04:27:39.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-05-09T04:27:39.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-05-09T04:27:39.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-05-09T04:27:39.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-05-09T04:27:39.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-05-09T04:27:39.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-05-09T04:27:39.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-05-09T04:27:39.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-05-09T04:27:39.757 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-05-09T04:27:39.757 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-05-09T04:27:39.757 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-09T04:27:39.757 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-09T04:27:39.757 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-09T04:27:39.757 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-09T04:27:39.757 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-09T04:27:39.757 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-09T04:27:39.757 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-09T04:27:39.757 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-09T04:27:39.757 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-09T04:27:39.757 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-09T04:27:39.757 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-09T04:27:39.757 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-09T04:27:39.757 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-09T04:27:39.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-09T04:27:39.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-09T04:27:39.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-09T04:27:39.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-09T04:27:39.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-09T04:27:39.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-09T04:27:39.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-09T04:27:39.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-09T04:27:39.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-05-09T04:27:39.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-05-09T04:27:39.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-05-09T04:27:39.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-05-09T04:27:39.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-05-09T04:27:39.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-05-09T04:27:39.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-05-09T04:27:39.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-05-09T04:27:39.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-05-09T04:27:39.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-05-09T04:27:39.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-05-09T04:27:39.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-05-09T04:27:39.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-05-09T04:27:39.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-05-09T04:27:39.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-05-09T04:27:39.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-05-09T04:27:39.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-09T04:27:39.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-05-09T04:27:39.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-05-09T04:27:39.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-05-09T04:27:39.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-05-09T04:27:39.760 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-05-09T04:27:39.760 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-09T04:27:39.760 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-05-09T04:27:39.760 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-05-09T04:27:39.760 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-05-09T04:27:39.760 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-09T04:27:39.760 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-09T04:27:39.760 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-09T04:27:39.760 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-09T04:27:39.760 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-09T04:27:39.760 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-09T04:27:39.760 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-09T04:27:39.760 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-09T04:27:39.760 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-09T04:27:39.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-09T04:27:39.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-09T04:27:39.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-09T04:27:39.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-09T04:27:39.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-09T04:27:39.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-09T04:27:39.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-09T04:27:39.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-09T04:27:39.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-09T04:27:39.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-09T04:27:39.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-09T04:27:39.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-09T04:27:39.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-09T04:27:39.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-09T04:27:39.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-09T04:27:39.762 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-05-09T04:27:39.762 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-05-09T04:27:39.762 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-09T04:27:39.762 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-05-09T04:27:39.762 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-05-09T04:27:39.762 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-05-09T04:27:39.762 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-05-09T04:27:39.762 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-09T04:27:39.762 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-05-09T04:27:39.762 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-05-09T04:27:39.762 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-05-09T04:27:39.762 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-05-09T04:27:39.762 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-05-09T04:27:39.762 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-05-09T04:27:39.763 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-05-09T04:27:39.763 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-05-09T04:27:39.763 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-05-09T04:27:39.763 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-05-09T04:27:39.763 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-05-09T04:27:39.763 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-05-09T04:27:39.763 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-05-09T04:27:39.763 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-05-09T04:27:39.763 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-09T04:27:39.763 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-09T04:27:39.763 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-09T04:27:39.763 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-09T04:27:39.763 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-09T04:27:39.763 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-09T04:27:39.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-09T04:27:39.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-09T04:27:39.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-09T04:27:39.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-09T04:27:39.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-09T04:27:39.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-09T04:27:39.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-09T04:27:39.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-09T04:27:39.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-09T04:27:39.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-05-09T04:27:39.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-09T04:27:39.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-09T04:27:39.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-09T04:27:39.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-09T04:27:39.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-09T04:27:39.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-09T04:27:39.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-09T04:27:39.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-09T04:27:39.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-09T04:27:39.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-09T04:27:39.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-09T04:27:39.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-09T04:27:39.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-09T04:27:39.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-09T04:27:39.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-05-09T04:27:39.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-05-09T04:27:39.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-09T04:27:39.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-09T04:27:39.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-09T04:27:39.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-09T04:27:39.766 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-09T04:27:39.766 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-09T04:27:39.766 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-09T04:27:39.766 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-09T04:27:39.766 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-09T04:27:39.766 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-09T04:27:39.766 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-09T04:27:39.766 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-09T04:27:39.766 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-09T04:27:39.766 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-05-09T04:27:39.766 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-09T04:27:39.766 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-05-09T04:27:39.766 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-05-09T04:27:39.766 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-09T04:27:39.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-09T04:27:39.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-05-09T04:27:39.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-05-09T04:27:39.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-09T04:27:39.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-09T04:27:39.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-05-09T04:27:39.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-05-09T04:27:39.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-09T04:27:39.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-05-09T04:27:39.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-09T04:27:39.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-09T04:27:39.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-05-09T04:27:39.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-05-09T04:27:39.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-05-09T04:27:39.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-09T04:27:39.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-05-09T04:27:39.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-09T04:27:39.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-05-09T04:27:39.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-05-09T04:27:39.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-05-09T04:27:39.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-05-09T04:27:39.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-05-09T04:27:39.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-05-09T04:27:39.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-05-09T04:27:39.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-05-09T04:27:39.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-05-09T04:27:39.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-05-09T04:27:39.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-05-09T04:27:39.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-05-09T04:27:39.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-05-09T04:27:39.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-05-09T04:27:39.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-09T04:27:39.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-09T04:27:39.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-09T04:27:39.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-05-09T04:27:39.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-05-09T04:27:39.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-05-09T04:27:39.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-05-09T04:27:39.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-09T04:27:39.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-05-09T04:27:39.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-05-09T04:27:39.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-09T04:27:39.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-05-09T04:27:39.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-05-09T04:27:39.770 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-09T04:27:39.770 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-05-09T04:27:39.770 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-09T04:27:39.770 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-05-09T04:27:39.770 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-09T04:27:39.770 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-09T04:27:39.770 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-05-09T04:27:39.770 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-05-09T04:27:39.770 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-05-09T04:27:39.770 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-05-09T04:27:39.770 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-05-09T04:27:39.770 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-05-09T04:27:39.770 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-05-09T04:27:39.770 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-05-09T04:27:39.770 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-05-09T04:27:39.771 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-05-09T04:27:39.771 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-05-09T04:27:39.771 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-05-09T04:27:39.771 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-05-09T04:27:39.771 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-05-09T04:27:39.771 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-05-09T04:27:39.771 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-05-09T04:27:39.771 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-05-09T04:27:39.771 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-05-09T04:27:39.771 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-05-09T04:27:39.771 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-05-09T04:27:39.771 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-05-09T04:27:39.771 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-05-09T04:27:39.771 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-05-09T04:27:39.771 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-05-09T04:27:39.772 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-05-09T04:27:39.772 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-05-09T04:27:39.772 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-05-09T04:27:39.772 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-05-09T04:27:39.772 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-05-09T04:27:39.772 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-05-09T04:27:39.772 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-05-09T04:27:39.772 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-05-09T04:27:39.772 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-05-09T04:27:39.772 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-05-09T04:27:39.772 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-05-09T04:27:39.772 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-05-09T04:27:39.772 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-05-09T04:27:39.772 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-05-09T04:27:39.773 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-05-09T04:27:39.773 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-05-09T04:27:39.773 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-05-09T04:27:39.773 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-05-09T04:27:39.773 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-05-09T04:27:39.773 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-05-09T04:27:39.773 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-05-09T04:27:39.773 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-05-09T04:27:39.773 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-05-09T04:27:39.773 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-05-09T04:27:39.773 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-05-09T04:27:39.773 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-05-09T04:27:39.773 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-05-09T04:27:39.773 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-05-09T04:27:39.773 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-05-09T04:27:39.774 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-05-09T04:27:39.774 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-05-09T04:27:39.774 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-05-09T04:27:39.774 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-05-09T04:27:39.774 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-05-09T04:27:39.774 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-05-09T04:27:39.774 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-05-09T04:27:39.774 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-05-09T04:27:39.774 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-05-09T04:27:39.774 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-05-09T04:27:39.774 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-05-09T04:27:39.774 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-05-09T04:27:39.774 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-05-09T04:27:39.774 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-05-09T04:27:39.775 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-05-09T04:27:39.775 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-05-09T04:27:39.775 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-05-09T04:27:39.775 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-05-09T04:27:39.775 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-05-09T04:27:39.775 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-05-09T04:27:39.775 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-05-09T04:27:39.775 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-05-09T04:27:39.775 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-05-09T04:27:39.775 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-05-09T04:27:39.775 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-05-09T04:27:39.775 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-05-09T04:27:39.775 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-05-09T04:27:39.775 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-05-09T04:27:39.776 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-05-09T04:27:39.776 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-05-09T04:27:39.776 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-05-09T04:27:39.776 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-05-09T04:27:39.776 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-05-09T04:27:39.776 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-05-09T04:27:39.776 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-05-09T04:27:39.776 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-05-09T04:27:39.776 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-05-09T04:27:39.776 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-05-09T04:27:39.776 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-05-09T04:27:39.776 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-05-09T04:27:39.776 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-05-09T04:27:39.776 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-05-09T04:27:39.776 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-05-09T04:27:39.776 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-05-09T04:27:39.777 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-05-09T04:27:39.777 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-05-09T04:27:39.777 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-05-09T04:27:39.777 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-05-09T04:27:39.777 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-05-09T04:27:39.777 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-05-09T04:27:39.777 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-05-09T04:27:39.777 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-05-09T04:27:39.777 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-05-09T04:27:39.777 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-05-09T04:27:39.777 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-05-09T04:27:39.777 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-05-09T04:27:39.777 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-05-09T04:27:39.777 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-05-09T04:27:39.778 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-05-09T04:27:39.778 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-05-09T04:27:39.778 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-05-09T04:27:39.778 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-05-09T04:27:39.778 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-09T04:27:39.778 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-09T04:27:39.778 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-05-09T04:27:39.778 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-05-09T04:27:39.778 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-09T04:27:39.778 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-05-09T04:27:39.778 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-05-09T04:27:39.778 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-05-09T04:27:39.778 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-05-09T04:27:39.779 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-05-09T04:27:39.779 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-05-09T04:27:39.779 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-09T04:27:39.779 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-09T04:27:39.779 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-05-09T04:27:39.779 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-09T04:27:39.779 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-09T04:27:39.779 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-05-09T04:27:39.779 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-09T04:27:39.779 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-05-09T04:27:39.779 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-05-09T04:27:39.779 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-05-09T04:27:39.779 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-09T04:27:39.779 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-05-09T04:27:39.779 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-05-09T04:27:39.779 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-05-09T04:27:39.780 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-05-09T04:27:39.780 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-05-09T04:27:39.780 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-05-09T04:27:39.780 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-05-09T04:27:39.780 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-05-09T04:27:39.780 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-05-09T04:27:39.780 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-05-09T04:27:39.780 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-05-09T04:27:39.780 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-05-09T04:27:39.780 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-05-09T04:27:39.780 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-05-09T04:27:39.780 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-05-09T04:27:39.780 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-05-09T04:27:39.784 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-05-09T04:27:39.785 DEBUG:teuthology.orchestra.run.smithi006:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-09T04:27:39.809 INFO:teuthology.orchestra.run.smithi006.stdout:postinstall scriptlet (using /bin/sh): 2024-05-09T04:27:39.809 INFO:teuthology.orchestra.run.smithi006.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-09T04:27:39.809 INFO:teuthology.orchestra.run.smithi006.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-05-09T04:27:39.809 INFO:teuthology.orchestra.run.smithi006.stdout:fi 2024-05-09T04:27:39.809 INFO:teuthology.orchestra.run.smithi006.stdout:for file in vmlinuz System.map config; do 2024-05-09T04:27:39.810 INFO:teuthology.orchestra.run.smithi006.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-05-09T04:27:39.810 INFO:teuthology.orchestra.run.smithi006.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-05-09T04:27:39.810 INFO:teuthology.orchestra.run.smithi006.stdout: fi 2024-05-09T04:27:39.810 INFO:teuthology.orchestra.run.smithi006.stdout:done 2024-05-09T04:27:39.810 INFO:teuthology.orchestra.run.smithi006.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-05-09T04:27:39.810 INFO:teuthology.orchestra.run.smithi006.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-05-09T04:27:39.810 INFO:teuthology.orchestra.run.smithi006.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-05-09T04:27:39.810 INFO:teuthology.orchestra.run.smithi006.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-05-09T04:27:39.810 INFO:teuthology.orchestra.run.smithi006.stdout:fi 2024-05-09T04:27:39.810 INFO:teuthology.orchestra.run.smithi006.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-05-09T04:27:39.810 INFO:teuthology.orchestra.run.smithi006.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-05-09T04:27:39.810 INFO:teuthology.orchestra.run.smithi006.stdout:fi 2024-05-09T04:27:39.810 INFO:teuthology.orchestra.run.smithi006.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-09T04:27:39.811 INFO:teuthology.orchestra.run.smithi006.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-09T04:27:39.811 INFO:teuthology.orchestra.run.smithi006.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-05-09T04:27:39.811 INFO:teuthology.orchestra.run.smithi006.stdout:fi 2024-05-09T04:27:39.812 DEBUG:teuthology.orchestra.run.smithi006:> sudo rpm -qi grub2-tools 2024-05-09T04:27:39.896 INFO:teuthology.orchestra.run.smithi006.stdout:Name : grub2-tools 2024-05-09T04:27:39.896 INFO:teuthology.orchestra.run.smithi006.stdout:Epoch : 1 2024-05-09T04:27:39.896 INFO:teuthology.orchestra.run.smithi006.stdout:Version : 2.06 2024-05-09T04:27:39.896 INFO:teuthology.orchestra.run.smithi006.stdout:Release : 68.el9 2024-05-09T04:27:39.896 INFO:teuthology.orchestra.run.smithi006.stdout:Architecture: x86_64 2024-05-09T04:27:39.896 INFO:teuthology.orchestra.run.smithi006.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-09T04:27:39.896 INFO:teuthology.orchestra.run.smithi006.stdout:Group : Unspecified 2024-05-09T04:27:39.897 INFO:teuthology.orchestra.run.smithi006.stdout:Size : 8264254 2024-05-09T04:27:39.897 INFO:teuthology.orchestra.run.smithi006.stdout:License : GPLv3+ 2024-05-09T04:27:39.897 INFO:teuthology.orchestra.run.smithi006.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-09T04:27:39.897 INFO:teuthology.orchestra.run.smithi006.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-09T04:27:39.897 INFO:teuthology.orchestra.run.smithi006.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-09T04:27:39.897 INFO:teuthology.orchestra.run.smithi006.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-09T04:27:39.897 INFO:teuthology.orchestra.run.smithi006.stdout:Packager : builder@centos.org 2024-05-09T04:27:39.897 INFO:teuthology.orchestra.run.smithi006.stdout:Vendor : CentOS 2024-05-09T04:27:39.897 INFO:teuthology.orchestra.run.smithi006.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-09T04:27:39.897 INFO:teuthology.orchestra.run.smithi006.stdout:Summary : Support tools for GRUB. 2024-05-09T04:27:39.897 INFO:teuthology.orchestra.run.smithi006.stdout:Description : 2024-05-09T04:27:39.897 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:27:39.897 INFO:teuthology.orchestra.run.smithi006.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-09T04:27:39.897 INFO:teuthology.orchestra.run.smithi006.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-09T04:27:39.897 INFO:teuthology.orchestra.run.smithi006.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-09T04:27:39.898 INFO:teuthology.orchestra.run.smithi006.stdout:hardware devices. 2024-05-09T04:27:39.898 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:27:39.898 INFO:teuthology.orchestra.run.smithi006.stdout:This subpackage provides tools for support of all platforms. 2024-05-09T04:27:39.899 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-09T04:27:39.899 INFO:teuthology.task.kernel:Updating grub on smithi006 to boot 6.9.0-rc3-g806ec36b2112 2024-05-09T04:27:39.899 DEBUG:teuthology.orchestra.run.smithi006:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-09T04:27:40.485 INFO:teuthology.orchestra.run.smithi006.stderr:Generating grub configuration file ... 2024-05-09T04:27:41.953 INFO:teuthology.orchestra.run.smithi006.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-09T04:27:41.989 INFO:teuthology.orchestra.run.smithi006.stderr:done 2024-05-09T04:27:41.991 DEBUG:teuthology.orchestra.run.smithi006:> mktemp 2024-05-09T04:27:42.019 INFO:teuthology.orchestra.run.smithi006.stdout:/tmp/tmp.TPp5v1CURq 2024-05-09T04:27:42.020 DEBUG:teuthology.orchestra.run.smithi006:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.TPp5v1CURq 2024-05-09T04:27:42.088 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 0666 /tmp/tmp.TPp5v1CURq 2024-05-09T04:27:42.221 DEBUG:teuthology.orchestra.remote:smithi006:/tmp/tmp.TPp5v1CURq is 6KB 2024-05-09T04:27:42.242 DEBUG:teuthology.orchestra.run.smithi006:> rm -fr /tmp/tmp.TPp5v1CURq 2024-05-09T04:27:42.258 DEBUG:teuthology.orchestra.run.smithi006:> sudo /bin/ls /boot/loader/entries || true 2024-05-09T04:27:42.324 INFO:teuthology.orchestra.run.smithi006.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-09T04:27:42.325 INFO:teuthology.orchestra.run.smithi006.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-09T04:27:42.325 INFO:teuthology.orchestra.run.smithi006.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-09T04:27:42.325 INFO:teuthology.orchestra.run.smithi006.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-05-09T04:27:42.326 DEBUG:teuthology.orchestra.run.smithi006:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-05-09T04:27:42.491 DEBUG:teuthology.orchestra.run.smithi006:> sudo shutdown -r now 2024-05-09T04:28:12.494 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-09T04:28:12.494 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-05-09T04:28:12.495 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T04:28:30.878 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.6 2024-05-09T04:28:39.879 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-05-09T04:28:39.880 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T04:29:39.883 DEBUG:teuthology.orchestra.remote:timed out 2024-05-09T04:29:51.885 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-05-09T04:29:51.886 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T04:29:52.304 DEBUG:teuthology.orchestra.run.smithi006:> true 2024-05-09T04:29:52.927 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-05-09T04:29:52.927 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-05-09T04:29:52.927 DEBUG:teuthology.orchestra.run.smithi006:> uname -r 2024-05-09T04:29:52.983 INFO:teuthology.orchestra.run.smithi006.stdout:6.9.0-rc3-g806ec36b2112 2024-05-09T04:29:52.983 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-05-09T04:29:52.983 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-09T04:29:52.983 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-09T04:29:53.985 DEBUG:teuthology.parallel:result is None 2024-05-09T04:29:53.985 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-09T04:29:53.993 INFO:teuthology.task.internal:Creating test directory... 2024-05-09T04:29:53.993 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T04:29:54.001 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T04:29:54.003 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T04:29:54.053 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-09T04:29:54.074 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-09T04:29:54.130 INFO:teuthology.task.internal:Creating archive directory... 2024-05-09T04:29:54.131 DEBUG:teuthology.orchestra.run.smithi006:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T04:29:54.138 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T04:29:54.141 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T04:29:54.265 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-09T04:29:54.271 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-09T04:29:54.271 DEBUG:teuthology.orchestra.run.smithi169:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T04:29:54.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T04:29:54.289 DEBUG:teuthology.orchestra.run.smithi186:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T04:29:54.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T04:29:54.306 DEBUG:teuthology.orchestra.run.smithi006:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T04:29:54.313 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T04:29:54.331 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T04:29:54.371 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T04:29:54.384 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T04:29:54.399 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T04:29:54.402 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T04:29:54.437 INFO:teuthology.orchestra.run.smithi006.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T04:29:54.496 INFO:teuthology.orchestra.run.smithi006.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T04:29:54.501 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-09T04:29:54.508 INFO:teuthology.task.internal:Configuring sudo... 2024-05-09T04:29:54.508 DEBUG:teuthology.orchestra.run.smithi006:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T04:29:54.512 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T04:29:54.515 DEBUG:teuthology.orchestra.run.smithi186:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T04:29:54.598 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-09T04:29:54.607 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-09T04:29:54.608 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T04:29:54.614 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T04:29:54.616 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T04:29:54.664 DEBUG:teuthology.orchestra.run.smithi006:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T04:29:54.746 DEBUG:teuthology.orchestra.run.smithi006:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T04:29:54.850 DEBUG:teuthology.orchestra.run.smithi006:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T04:29:54.938 DEBUG:teuthology.orchestra.run.smithi006:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T04:29:55.035 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T04:29:55.035 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T04:29:55.124 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T04:29:55.165 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T04:29:55.227 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T04:29:55.311 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T04:29:55.353 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-09T04:29:55.353 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T04:29:55.418 DEBUG:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T04:29:55.456 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T04:29:55.526 DEBUG:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T04:29:55.569 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T04:29:55.612 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-05-09T04:29:55.612 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T04:29:55.677 DEBUG:teuthology.orchestra.run.smithi006:> sudo service rsyslog restart 2024-05-09T04:29:55.685 DEBUG:teuthology.orchestra.run.smithi169:> sudo service rsyslog restart 2024-05-09T04:29:55.687 DEBUG:teuthology.orchestra.run.smithi186:> sudo service rsyslog restart 2024-05-09T04:29:55.721 INFO:teuthology.orchestra.run.smithi169.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T04:29:55.753 INFO:teuthology.orchestra.run.smithi186.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T04:29:55.799 INFO:teuthology.orchestra.run.smithi006.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T04:29:56.280 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-09T04:29:56.288 INFO:teuthology.task.internal:Starting timer... 2024-05-09T04:29:56.288 INFO:teuthology.run_tasks:Running task pcp... 2024-05-09T04:29:56.334 INFO:teuthology.run_tasks:Running task selinux... 2024-05-09T04:29:56.382 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-09T04:29:56.382 DEBUG:teuthology.orchestra.run.smithi006:> sudo service auditd rotate 2024-05-09T04:29:56.511 INFO:teuthology.orchestra.run.smithi006.stdout:Rotating logs: 2024-05-09T04:29:56.515 DEBUG:teuthology.orchestra.run.smithi169:> sudo service auditd rotate 2024-05-09T04:29:56.569 INFO:teuthology.orchestra.run.smithi169.stdout:Rotating logs: 2024-05-09T04:29:56.571 DEBUG:teuthology.orchestra.run.smithi186:> sudo service auditd rotate 2024-05-09T04:29:56.638 INFO:teuthology.orchestra.run.smithi186.stdout:Rotating logs: 2024-05-09T04:29:56.640 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-09T04:29:56.640 DEBUG:teuthology.orchestra.run.smithi006:> /usr/sbin/getenforce 2024-05-09T04:29:56.705 INFO:teuthology.orchestra.run.smithi006.stdout:Permissive 2024-05-09T04:29:56.706 DEBUG:teuthology.orchestra.run.smithi169:> /usr/sbin/getenforce 2024-05-09T04:29:56.727 INFO:teuthology.orchestra.run.smithi169.stdout:Permissive 2024-05-09T04:29:56.727 DEBUG:teuthology.orchestra.run.smithi186:> /usr/sbin/getenforce 2024-05-09T04:29:56.775 INFO:teuthology.orchestra.run.smithi186.stdout:Permissive 2024-05-09T04:29:56.775 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi006.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi169.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi186.front.sepia.ceph.com': 'permissive'} 2024-05-09T04:29:56.775 DEBUG:teuthology.orchestra.run.smithi006:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T04:29:56.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T04:29:56.850 DEBUG:teuthology.orchestra.run.smithi169:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T04:29:56.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T04:29:56.876 DEBUG:teuthology.orchestra.run.smithi186:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T04:29:56.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T04:29:56.900 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-09T04:29:56.901 DEBUG:teuthology.orchestra.run.smithi006:> sudo /usr/sbin/setenforce permissive 2024-05-09T04:29:56.975 DEBUG:teuthology.orchestra.run.smithi169:> sudo /usr/sbin/setenforce permissive 2024-05-09T04:29:57.004 DEBUG:teuthology.orchestra.run.smithi186:> sudo /usr/sbin/setenforce permissive 2024-05-09T04:29:57.044 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-09T04:29:57.054 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-09T04:29:57.071 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-09T04:29:57.073 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi006.front.sepia.ceph.com,smithi169.front.sepia.ceph.com,smithi186.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-09T04:35:51.514 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi006.front.sepia.ceph.com'), Remote(name='ubuntu@smithi169.front.sepia.ceph.com'), Remote(name='ubuntu@smithi186.front.sepia.ceph.com')] 2024-05-09T04:35:51.515 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-05-09T04:35:51.516 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T04:35:51.629 DEBUG:teuthology.orchestra.run.smithi006:> true 2024-05-09T04:35:51.782 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-05-09T04:35:51.783 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-05-09T04:35:51.783 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T04:35:51.858 DEBUG:teuthology.orchestra.run.smithi169:> true 2024-05-09T04:35:51.941 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-05-09T04:35:51.941 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com' 2024-05-09T04:35:51.942 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T04:35:52.016 DEBUG:teuthology.orchestra.run.smithi186:> true 2024-05-09T04:35:52.101 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi186.front.sepia.ceph.com' 2024-05-09T04:35:52.101 INFO:teuthology.run_tasks:Running task clock... 2024-05-09T04:35:52.114 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-09T04:35:52.114 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T04:35:52.114 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T04:35:52.121 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T04:35:52.122 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T04:35:52.124 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T04:35:52.124 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T04:35:52.151 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T04:35:52.165 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T04:35:52.170 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T04:35:52.184 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T04:35:52.189 INFO:teuthology.orchestra.run.smithi169.stderr:sudo: ntpd: command not found 2024-05-09T04:35:52.200 INFO:teuthology.orchestra.run.smithi169.stdout:506 Cannot talk to daemon 2024-05-09T04:35:52.207 INFO:teuthology.orchestra.run.smithi186.stderr:sudo: ntpd: command not found 2024-05-09T04:35:52.213 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T04:35:52.213 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T04:35:52.218 INFO:teuthology.orchestra.run.smithi186.stdout:506 Cannot talk to daemon 2024-05-09T04:35:52.227 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T04:35:52.231 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T04:35:52.244 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T04:35:52.259 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T04:35:52.282 INFO:teuthology.orchestra.run.smithi169.stderr:bash: line 1: ntpq: command not found 2024-05-09T04:35:52.286 INFO:teuthology.orchestra.run.smithi169.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T04:35:52.286 INFO:teuthology.orchestra.run.smithi169.stdout:=============================================================================== 2024-05-09T04:35:52.286 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T04:35:52.286 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T04:35:52.286 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T04:35:52.286 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T04:35:52.303 INFO:teuthology.orchestra.run.smithi186.stderr:bash: line 1: ntpq: command not found 2024-05-09T04:35:52.306 INFO:teuthology.orchestra.run.smithi186.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T04:35:52.306 INFO:teuthology.orchestra.run.smithi186.stdout:=============================================================================== 2024-05-09T04:35:52.306 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T04:35:52.306 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T04:35:52.306 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T04:35:52.306 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T04:35:52.338 INFO:teuthology.orchestra.run.smithi006.stderr:sudo: ntpd: command not found 2024-05-09T04:35:52.374 INFO:teuthology.orchestra.run.smithi006.stdout:506 Cannot talk to daemon 2024-05-09T04:35:52.420 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T04:35:52.464 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T04:35:52.612 INFO:teuthology.orchestra.run.smithi006.stderr:bash: line 1: ntpq: command not found 2024-05-09T04:35:52.623 INFO:teuthology.orchestra.run.smithi006.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T04:35:52.623 INFO:teuthology.orchestra.run.smithi006.stdout:=============================================================================== 2024-05-09T04:35:52.623 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T04:35:52.623 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T04:35:52.623 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T04:35:52.623 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T04:35:52.625 INFO:teuthology.run_tasks:Running task install... 2024-05-09T04:35:52.633 DEBUG:teuthology.task.install:project ceph 2024-05-09T04:35:52.633 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554'}} 2024-05-09T04:35:52.633 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554'} 2024-05-09T04:35:52.634 INFO:teuthology.task.install:Using flavor: default 2024-05-09T04:35:52.639 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-09T04:35:52.639 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-09T04:35:52.640 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'tag': None, 'wait_for_package': False} 2024-05-09T04:35:52.640 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T04:35:52.642 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'tag': None, 'wait_for_package': False} 2024-05-09T04:35:52.642 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T04:35:52.643 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'tag': None, 'wait_for_package': False} 2024-05-09T04:35:52.644 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T04:35:52.794 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/ 2024-05-09T04:35:52.795 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3594.gc9da4049 2024-05-09T04:35:52.798 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/ 2024-05-09T04:35:52.798 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3594.gc9da4049 2024-05-09T04:35:52.896 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/ 2024-05-09T04:35:52.896 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3594.gc9da4049 2024-05-09T04:35:52.909 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T04:35:52.909 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T04:35:52.909 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T04:35:52.969 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T04:35:52.969 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-09T04:35:52.969 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T04:35:52.995 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T04:35:52.995 DEBUG:teuthology.orchestra.run.smithi169:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c9da4049f638c0f84bdd9b4e33c42e86046ac554/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T04:35:53.005 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T04:35:53.006 DEBUG:teuthology.orchestra.run.smithi006:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c9da4049f638c0f84bdd9b4e33c42e86046ac554/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T04:35:53.060 DEBUG:teuthology.orchestra.run.smithi169:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T04:35:53.135 DEBUG:teuthology.orchestra.run.smithi169:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T04:35:53.139 DEBUG:teuthology.orchestra.run.smithi006:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T04:35:53.198 INFO:teuthology.orchestra.run.smithi169.stdout:check_obsoletes = 1 2024-05-09T04:35:53.200 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum clean all 2024-05-09T04:35:53.250 DEBUG:teuthology.orchestra.run.smithi006:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T04:35:53.333 INFO:teuthology.orchestra.run.smithi006.stdout:check_obsoletes = 1 2024-05-09T04:35:53.338 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum clean all 2024-05-09T04:35:53.427 INFO:teuthology.orchestra.run.smithi169.stdout:54 files removed 2024-05-09T04:35:53.455 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T04:35:53.678 INFO:teuthology.orchestra.run.smithi006.stdout:54 files removed 2024-05-09T04:35:53.718 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T04:35:54.128 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T04:35:54.128 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-05-09T04:35:54.128 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T04:35:54.158 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T04:35:54.159 DEBUG:teuthology.orchestra.run.smithi186:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c9da4049f638c0f84bdd9b4e33c42e86046ac554/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T04:35:54.227 DEBUG:teuthology.orchestra.run.smithi186:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T04:35:54.305 DEBUG:teuthology.orchestra.run.smithi186:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T04:35:54.370 INFO:teuthology.orchestra.run.smithi186.stdout:check_obsoletes = 1 2024-05-09T04:35:54.372 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum clean all 2024-05-09T04:35:54.389 INFO:teuthology.orchestra.run.smithi169.stdout:ceph packages for x86_64 125 kB/s | 84 kB 00:00 2024-05-09T04:35:54.593 INFO:teuthology.orchestra.run.smithi006.stdout:ceph packages for x86_64 198 kB/s | 84 kB 00:00 2024-05-09T04:35:54.603 INFO:teuthology.orchestra.run.smithi186.stdout:54 files removed 2024-05-09T04:35:54.631 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T04:35:54.731 INFO:teuthology.orchestra.run.smithi169.stdout:ceph noarch packages 39 kB/s | 12 kB 00:00 2024-05-09T04:35:55.070 INFO:teuthology.orchestra.run.smithi006.stdout:ceph noarch packages 32 kB/s | 12 kB 00:00 2024-05-09T04:35:55.165 INFO:teuthology.orchestra.run.smithi169.stdout:ceph source packages 4.8 kB/s | 1.9 kB 00:00 2024-05-09T04:35:55.507 INFO:teuthology.orchestra.run.smithi186.stdout:ceph packages for x86_64 136 kB/s | 84 kB 00:00 2024-05-09T04:35:55.527 INFO:teuthology.orchestra.run.smithi006.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-05-09T04:35:56.716 INFO:teuthology.orchestra.run.smithi186.stdout:ceph noarch packages 10 kB/s | 12 kB 00:01 2024-05-09T04:35:57.284 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.1 MB 00:01 2024-05-09T04:35:58.376 INFO:teuthology.orchestra.run.smithi186.stdout:ceph source packages 1.2 kB/s | 1.9 kB 00:01 2024-05-09T04:35:58.607 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - BaseOS 2.4 MB/s | 8.1 MB 00:03 2024-05-09T04:36:00.700 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - BaseOS 3.5 MB/s | 8.1 MB 00:02 2024-05-09T04:36:02.341 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - AppStream 7.2 MB/s | 19 MB 00:02 2024-05-09T04:36:03.061 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - AppStream 4.2 MB/s | 19 MB 00:04 2024-05-09T04:36:04.035 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - AppStream 8.6 MB/s | 19 MB 00:02 2024-05-09T04:36:07.800 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.2 MB 00:01 2024-05-09T04:36:10.217 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 16 kB 00:01 2024-05-09T04:36:11.253 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - CRB 1.5 MB/s | 6.2 MB 00:04 2024-05-09T04:36:11.505 INFO:teuthology.orchestra.run.smithi169.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-05-09T04:36:11.702 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - CRB 1.5 MB/s | 6.2 MB 00:04 2024-05-09T04:36:13.245 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - Extras packages 29 kB/s | 16 kB 00:00 2024-05-09T04:36:13.477 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 16 kB 00:00 2024-05-09T04:36:15.340 INFO:teuthology.orchestra.run.smithi186.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-05-09T04:36:16.677 INFO:teuthology.orchestra.run.smithi169.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-09T04:36:18.180 INFO:teuthology.orchestra.run.smithi006.stdout:Extra Packages for Enterprise Linux 4.5 MB/s | 21 MB 00:04 2024-05-09T04:36:18.970 INFO:teuthology.orchestra.run.smithi169.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T04:36:18.971 INFO:teuthology.orchestra.run.smithi169.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T04:36:18.982 INFO:teuthology.orchestra.run.smithi169.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T04:36:18.982 INFO:teuthology.orchestra.run.smithi169.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T04:36:18.983 INFO:teuthology.orchestra.run.smithi169.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T04:36:18.984 INFO:teuthology.orchestra.run.smithi169.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T04:36:18.985 INFO:teuthology.orchestra.run.smithi169.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T04:36:18.987 INFO:teuthology.orchestra.run.smithi169.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T04:36:18.988 INFO:teuthology.orchestra.run.smithi169.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T04:36:19.049 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-05-09T04:36:19.069 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-05-09T04:36:19.069 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-05-09T04:36:19.070 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-05-09T04:36:19.070 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-05-09T04:36:19.070 INFO:teuthology.orchestra.run.smithi169.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T04:36:19.070 INFO:teuthology.orchestra.run.smithi169.stdout: ceph x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 6.6 k 2024-05-09T04:36:19.070 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.4 M 2024-05-09T04:36:19.070 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 863 k 2024-05-09T04:36:19.070 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 147 k 2024-05-09T04:36:19.070 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 1.5 M 2024-05-09T04:36:19.070 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 142 k 2024-05-09T04:36:19.070 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 3.5 M 2024-05-09T04:36:19.070 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 7.4 M 2024-05-09T04:36:19.071 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-rook noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 49 k 2024-05-09T04:36:19.071 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-radosgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 10 M 2024-05-09T04:36:19.071 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-test x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 43 M 2024-05-09T04:36:19.071 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-volume noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 273 k 2024-05-09T04:36:19.071 INFO:teuthology.orchestra.run.smithi169.stdout: cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 763 k 2024-05-09T04:36:19.071 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 224 k 2024-05-09T04:36:19.071 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-shell x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 27 k 2024-05-09T04:36:19.071 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-top noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 18 k 2024-05-09T04:36:19.071 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T04:36:19.071 INFO:teuthology.orchestra.run.smithi169.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T04:36:19.071 INFO:teuthology.orchestra.run.smithi169.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T04:36:19.071 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 32 k 2024-05-09T04:36:19.071 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 756 k 2024-05-09T04:36:19.071 INFO:teuthology.orchestra.run.smithi169.stdout: librados-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 127 k 2024-05-09T04:36:19.072 INFO:teuthology.orchestra.run.smithi169.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T04:36:19.072 INFO:teuthology.orchestra.run.smithi169.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T04:36:19.072 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T04:36:19.072 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T04:36:19.072 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T04:36:19.072 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T04:36:19.072 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cephfs x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 162 k 2024-05-09T04:36:19.072 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 321 k 2024-05-09T04:36:19.072 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 301 k 2024-05-09T04:36:19.072 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 99 k 2024-05-09T04:36:19.072 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 86 k 2024-05-09T04:36:19.072 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.0 M 2024-05-09T04:36:19.072 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-nbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 169 k 2024-05-09T04:36:19.072 INFO:teuthology.orchestra.run.smithi169.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T04:36:19.073 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrading: 2024-05-09T04:36:19.073 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T04:36:19.073 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T04:36:19.073 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T04:36:19.073 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T04:36:19.073 INFO:teuthology.orchestra.run.smithi169.stdout: librados2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.3 M 2024-05-09T04:36:19.073 INFO:teuthology.orchestra.run.smithi169.stdout: librbd1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.1 M 2024-05-09T04:36:19.073 INFO:teuthology.orchestra.run.smithi169.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T04:36:19.073 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T04:36:19.073 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-05-09T04:36:19.073 INFO:teuthology.orchestra.run.smithi169.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T04:36:19.073 INFO:teuthology.orchestra.run.smithi169.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T04:36:19.073 INFO:teuthology.orchestra.run.smithi169.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T04:36:19.073 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 20 M 2024-05-09T04:36:19.074 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 34 k 2024-05-09T04:36:19.074 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mds x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 2.3 M 2024-05-09T04:36:19.074 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 274 k 2024-05-09T04:36:19.074 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mon x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 4.9 M 2024-05-09T04:36:19.074 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-osd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 18 M 2024-05-09T04:36:19.074 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 16 k 2024-05-09T04:36:19.074 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-selinux x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 25 k 2024-05-09T04:36:19.074 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T04:36:19.074 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T04:36:19.074 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T04:36:19.074 INFO:teuthology.orchestra.run.smithi169.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T04:36:19.074 INFO:teuthology.orchestra.run.smithi169.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T04:36:19.074 INFO:teuthology.orchestra.run.smithi169.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T04:36:19.074 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T04:36:19.075 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T04:36:19.075 INFO:teuthology.orchestra.run.smithi169.stdout: libcephsqlite x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 158 k 2024-05-09T04:36:19.075 INFO:teuthology.orchestra.run.smithi169.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T04:36:19.075 INFO:teuthology.orchestra.run.smithi169.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T04:36:19.075 INFO:teuthology.orchestra.run.smithi169.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T04:36:19.075 INFO:teuthology.orchestra.run.smithi169.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T04:36:19.075 INFO:teuthology.orchestra.run.smithi169.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T04:36:19.075 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 497 k 2024-05-09T04:36:19.075 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T04:36:19.075 INFO:teuthology.orchestra.run.smithi169.stdout: librgw2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.2 M 2024-05-09T04:36:19.075 INFO:teuthology.orchestra.run.smithi169.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T04:36:19.075 INFO:teuthology.orchestra.run.smithi169.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T04:36:19.075 INFO:teuthology.orchestra.run.smithi169.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T04:36:19.075 INFO:teuthology.orchestra.run.smithi169.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T04:36:19.076 INFO:teuthology.orchestra.run.smithi169.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T04:36:19.076 INFO:teuthology.orchestra.run.smithi169.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T04:36:19.076 INFO:teuthology.orchestra.run.smithi169.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T04:36:19.076 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T04:36:19.076 INFO:teuthology.orchestra.run.smithi169.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T04:36:19.076 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T04:36:19.076 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T04:36:19.076 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T04:36:19.076 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T04:36:19.076 INFO:teuthology.orchestra.run.smithi169.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T04:36:19.076 INFO:teuthology.orchestra.run.smithi169.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T04:36:19.076 INFO:teuthology.orchestra.run.smithi169.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T04:36:19.076 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T04:36:19.077 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 45 k 2024-05-09T04:36:19.077 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 135 k 2024-05-09T04:36:19.077 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T04:36:19.077 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T04:36:19.077 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T04:36:19.077 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T04:36:19.077 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T04:36:19.077 INFO:teuthology.orchestra.run.smithi169.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T04:36:19.077 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T04:36:19.077 INFO:teuthology.orchestra.run.smithi169.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T04:36:19.077 INFO:teuthology.orchestra.run.smithi169.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T04:36:19.077 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T04:36:19.077 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T04:36:19.077 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T04:36:19.078 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T04:36:19.078 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T04:36:19.078 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T04:36:19.078 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T04:36:19.078 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T04:36:19.078 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T04:36:19.078 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T04:36:19.078 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T04:36:19.078 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T04:36:19.078 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T04:36:19.078 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T04:36:19.078 INFO:teuthology.orchestra.run.smithi169.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T04:36:19.078 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T04:36:19.078 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T04:36:19.079 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T04:36:19.079 INFO:teuthology.orchestra.run.smithi169.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T04:36:19.079 INFO:teuthology.orchestra.run.smithi169.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T04:36:19.079 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T04:36:19.079 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T04:36:19.079 INFO:teuthology.orchestra.run.smithi169.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T04:36:19.079 INFO:teuthology.orchestra.run.smithi169.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T04:36:19.079 INFO:teuthology.orchestra.run.smithi169.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T04:36:19.079 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T04:36:19.079 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T04:36:19.079 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T04:36:19.079 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T04:36:19.079 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T04:36:19.079 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T04:36:19.079 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T04:36:19.080 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T04:36:19.080 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T04:36:19.080 INFO:teuthology.orchestra.run.smithi169.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T04:36:19.080 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T04:36:19.080 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T04:36:19.080 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T04:36:19.080 INFO:teuthology.orchestra.run.smithi169.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T04:36:19.080 INFO:teuthology.orchestra.run.smithi169.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T04:36:19.080 INFO:teuthology.orchestra.run.smithi169.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T04:36:19.080 INFO:teuthology.orchestra.run.smithi169.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T04:36:19.080 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T04:36:19.080 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T04:36:19.080 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T04:36:19.080 INFO:teuthology.orchestra.run.smithi169.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T04:36:19.080 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T04:36:19.081 INFO:teuthology.orchestra.run.smithi169.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T04:36:19.081 INFO:teuthology.orchestra.run.smithi169.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T04:36:19.081 INFO:teuthology.orchestra.run.smithi169.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T04:36:19.081 INFO:teuthology.orchestra.run.smithi169.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T04:36:19.081 INFO:teuthology.orchestra.run.smithi169.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T04:36:19.081 INFO:teuthology.orchestra.run.smithi169.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T04:36:19.081 INFO:teuthology.orchestra.run.smithi169.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T04:36:19.081 INFO:teuthology.orchestra.run.smithi169.stdout:Installing weak dependencies: 2024-05-09T04:36:19.081 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T04:36:19.081 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-09T04:36:19.081 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-05-09T04:36:19.081 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-05-09T04:36:19.081 INFO:teuthology.orchestra.run.smithi169.stdout:Install 148 Packages 2024-05-09T04:36:19.081 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrade 8 Packages 2024-05-09T04:36:19.082 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-09T04:36:19.082 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 219 M 2024-05-09T04:36:19.082 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-05-09T04:36:20.070 INFO:teuthology.orchestra.run.smithi169.stdout:(1/156): ceph-19.0.0-3594.gc9da4049.el9.x86_64. 30 kB/s | 6.6 kB 00:00 2024-05-09T04:36:20.545 INFO:teuthology.orchestra.run.smithi186.stdout:lab-extras 22 kB/s | 1.7 kB 00:00 2024-05-09T04:36:21.879 INFO:teuthology.orchestra.run.smithi169.stdout:(2/156): ceph-fuse-19.0.0-3594.gc9da4049.el9.x8 477 kB/s | 863 kB 00:01 2024-05-09T04:36:22.429 INFO:teuthology.orchestra.run.smithi169.stdout:(3/156): ceph-immutable-object-cache-19.0.0-359 267 kB/s | 147 kB 00:00 2024-05-09T04:36:22.830 INFO:teuthology.orchestra.run.smithi186.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T04:36:22.830 INFO:teuthology.orchestra.run.smithi186.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T04:36:22.841 INFO:teuthology.orchestra.run.smithi186.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T04:36:22.842 INFO:teuthology.orchestra.run.smithi186.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T04:36:22.842 INFO:teuthology.orchestra.run.smithi186.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T04:36:22.843 INFO:teuthology.orchestra.run.smithi186.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T04:36:22.845 INFO:teuthology.orchestra.run.smithi186.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T04:36:22.847 INFO:teuthology.orchestra.run.smithi186.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T04:36:22.848 INFO:teuthology.orchestra.run.smithi186.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T04:36:22.908 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-05-09T04:36:22.926 INFO:teuthology.orchestra.run.smithi186.stdout:======================================================================================= 2024-05-09T04:36:22.926 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2024-05-09T04:36:22.926 INFO:teuthology.orchestra.run.smithi186.stdout:======================================================================================= 2024-05-09T04:36:22.926 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2024-05-09T04:36:22.926 INFO:teuthology.orchestra.run.smithi186.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T04:36:22.926 INFO:teuthology.orchestra.run.smithi186.stdout: ceph x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 6.6 k 2024-05-09T04:36:22.926 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.4 M 2024-05-09T04:36:22.926 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 863 k 2024-05-09T04:36:22.926 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 147 k 2024-05-09T04:36:22.927 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 1.5 M 2024-05-09T04:36:22.927 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 142 k 2024-05-09T04:36:22.927 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 3.5 M 2024-05-09T04:36:22.927 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 7.4 M 2024-05-09T04:36:22.927 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-rook noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 49 k 2024-05-09T04:36:22.927 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-radosgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 10 M 2024-05-09T04:36:22.927 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-test x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 43 M 2024-05-09T04:36:22.927 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-volume noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 273 k 2024-05-09T04:36:22.927 INFO:teuthology.orchestra.run.smithi186.stdout: cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 763 k 2024-05-09T04:36:22.927 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 224 k 2024-05-09T04:36:22.927 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-shell x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 27 k 2024-05-09T04:36:22.927 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-top noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 18 k 2024-05-09T04:36:22.927 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T04:36:22.928 INFO:teuthology.orchestra.run.smithi186.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T04:36:22.928 INFO:teuthology.orchestra.run.smithi186.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T04:36:22.928 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 32 k 2024-05-09T04:36:22.928 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 756 k 2024-05-09T04:36:22.928 INFO:teuthology.orchestra.run.smithi186.stdout: librados-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 127 k 2024-05-09T04:36:22.928 INFO:teuthology.orchestra.run.smithi186.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T04:36:22.928 INFO:teuthology.orchestra.run.smithi186.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T04:36:22.928 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T04:36:22.928 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T04:36:22.928 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T04:36:22.928 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T04:36:22.928 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cephfs x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 162 k 2024-05-09T04:36:22.928 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rados x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 321 k 2024-05-09T04:36:22.929 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 301 k 2024-05-09T04:36:22.929 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 99 k 2024-05-09T04:36:22.929 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 86 k 2024-05-09T04:36:22.929 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.0 M 2024-05-09T04:36:22.929 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-nbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 169 k 2024-05-09T04:36:22.929 INFO:teuthology.orchestra.run.smithi186.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T04:36:22.929 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrading: 2024-05-09T04:36:22.929 INFO:teuthology.orchestra.run.smithi186.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T04:36:22.929 INFO:teuthology.orchestra.run.smithi186.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T04:36:22.929 INFO:teuthology.orchestra.run.smithi186.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T04:36:22.929 INFO:teuthology.orchestra.run.smithi186.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T04:36:22.929 INFO:teuthology.orchestra.run.smithi186.stdout: librados2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.3 M 2024-05-09T04:36:22.929 INFO:teuthology.orchestra.run.smithi186.stdout: librbd1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.1 M 2024-05-09T04:36:22.929 INFO:teuthology.orchestra.run.smithi186.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T04:36:22.930 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T04:36:22.930 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2024-05-09T04:36:22.930 INFO:teuthology.orchestra.run.smithi186.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T04:36:22.930 INFO:teuthology.orchestra.run.smithi186.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T04:36:22.930 INFO:teuthology.orchestra.run.smithi186.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T04:36:22.930 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 20 M 2024-05-09T04:36:22.930 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 34 k 2024-05-09T04:36:22.930 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mds x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 2.3 M 2024-05-09T04:36:22.930 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 274 k 2024-05-09T04:36:22.930 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mon x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 4.9 M 2024-05-09T04:36:22.930 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-osd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 18 M 2024-05-09T04:36:22.930 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 16 k 2024-05-09T04:36:22.930 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-selinux x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 25 k 2024-05-09T04:36:22.931 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T04:36:22.931 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T04:36:22.931 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T04:36:22.931 INFO:teuthology.orchestra.run.smithi186.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T04:36:22.931 INFO:teuthology.orchestra.run.smithi186.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T04:36:22.931 INFO:teuthology.orchestra.run.smithi186.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T04:36:22.931 INFO:teuthology.orchestra.run.smithi186.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T04:36:22.931 INFO:teuthology.orchestra.run.smithi186.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T04:36:22.931 INFO:teuthology.orchestra.run.smithi186.stdout: libcephsqlite x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 158 k 2024-05-09T04:36:22.931 INFO:teuthology.orchestra.run.smithi186.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T04:36:22.931 INFO:teuthology.orchestra.run.smithi186.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T04:36:22.931 INFO:teuthology.orchestra.run.smithi186.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T04:36:22.931 INFO:teuthology.orchestra.run.smithi186.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T04:36:22.931 INFO:teuthology.orchestra.run.smithi186.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T04:36:22.931 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 497 k 2024-05-09T04:36:22.932 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T04:36:22.932 INFO:teuthology.orchestra.run.smithi186.stdout: librgw2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.2 M 2024-05-09T04:36:22.932 INFO:teuthology.orchestra.run.smithi186.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T04:36:22.932 INFO:teuthology.orchestra.run.smithi186.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T04:36:22.932 INFO:teuthology.orchestra.run.smithi186.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T04:36:22.932 INFO:teuthology.orchestra.run.smithi186.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T04:36:22.932 INFO:teuthology.orchestra.run.smithi186.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T04:36:22.932 INFO:teuthology.orchestra.run.smithi186.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T04:36:22.932 INFO:teuthology.orchestra.run.smithi186.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T04:36:22.932 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T04:36:22.932 INFO:teuthology.orchestra.run.smithi186.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T04:36:22.932 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T04:36:22.932 INFO:teuthology.orchestra.run.smithi186.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T04:36:22.933 INFO:teuthology.orchestra.run.smithi186.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T04:36:22.933 INFO:teuthology.orchestra.run.smithi186.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T04:36:22.933 INFO:teuthology.orchestra.run.smithi186.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T04:36:22.933 INFO:teuthology.orchestra.run.smithi186.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T04:36:22.933 INFO:teuthology.orchestra.run.smithi186.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T04:36:22.933 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T04:36:22.933 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-argparse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 45 k 2024-05-09T04:36:22.933 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 135 k 2024-05-09T04:36:22.933 INFO:teuthology.orchestra.run.smithi186.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T04:36:22.933 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T04:36:22.933 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T04:36:22.933 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T04:36:22.933 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T04:36:22.933 INFO:teuthology.orchestra.run.smithi186.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T04:36:22.933 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T04:36:22.934 INFO:teuthology.orchestra.run.smithi186.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T04:36:22.934 INFO:teuthology.orchestra.run.smithi186.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T04:36:22.934 INFO:teuthology.orchestra.run.smithi186.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T04:36:22.934 INFO:teuthology.orchestra.run.smithi186.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T04:36:22.934 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T04:36:22.934 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T04:36:22.934 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T04:36:22.934 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T04:36:22.934 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T04:36:22.934 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T04:36:22.934 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T04:36:22.934 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T04:36:22.934 INFO:teuthology.orchestra.run.smithi186.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T04:36:22.934 INFO:teuthology.orchestra.run.smithi186.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T04:36:22.935 INFO:teuthology.orchestra.run.smithi186.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T04:36:22.935 INFO:teuthology.orchestra.run.smithi186.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T04:36:22.935 INFO:teuthology.orchestra.run.smithi186.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T04:36:22.935 INFO:teuthology.orchestra.run.smithi186.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T04:36:22.935 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T04:36:22.935 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T04:36:22.935 INFO:teuthology.orchestra.run.smithi186.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T04:36:22.935 INFO:teuthology.orchestra.run.smithi186.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T04:36:22.935 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T04:36:22.935 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T04:36:22.935 INFO:teuthology.orchestra.run.smithi186.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T04:36:22.935 INFO:teuthology.orchestra.run.smithi186.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T04:36:22.935 INFO:teuthology.orchestra.run.smithi186.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T04:36:22.935 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T04:36:22.935 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T04:36:22.936 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T04:36:22.936 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T04:36:22.936 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T04:36:22.936 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T04:36:22.936 INFO:teuthology.orchestra.run.smithi186.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T04:36:22.936 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T04:36:22.936 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T04:36:22.936 INFO:teuthology.orchestra.run.smithi186.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T04:36:22.936 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T04:36:22.936 INFO:teuthology.orchestra.run.smithi186.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T04:36:22.936 INFO:teuthology.orchestra.run.smithi186.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T04:36:22.936 INFO:teuthology.orchestra.run.smithi186.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T04:36:22.936 INFO:teuthology.orchestra.run.smithi186.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T04:36:22.936 INFO:teuthology.orchestra.run.smithi186.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T04:36:22.936 INFO:teuthology.orchestra.run.smithi186.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T04:36:22.936 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T04:36:22.937 INFO:teuthology.orchestra.run.smithi186.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T04:36:22.937 INFO:teuthology.orchestra.run.smithi186.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T04:36:22.937 INFO:teuthology.orchestra.run.smithi186.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T04:36:22.937 INFO:teuthology.orchestra.run.smithi186.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T04:36:22.937 INFO:teuthology.orchestra.run.smithi186.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T04:36:22.937 INFO:teuthology.orchestra.run.smithi186.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T04:36:22.937 INFO:teuthology.orchestra.run.smithi186.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T04:36:22.937 INFO:teuthology.orchestra.run.smithi186.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T04:36:22.937 INFO:teuthology.orchestra.run.smithi186.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T04:36:22.937 INFO:teuthology.orchestra.run.smithi186.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T04:36:22.937 INFO:teuthology.orchestra.run.smithi186.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T04:36:22.937 INFO:teuthology.orchestra.run.smithi186.stdout:Installing weak dependencies: 2024-05-09T04:36:22.937 INFO:teuthology.orchestra.run.smithi186.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T04:36:22.937 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-05-09T04:36:22.937 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2024-05-09T04:36:22.937 INFO:teuthology.orchestra.run.smithi186.stdout:======================================================================================= 2024-05-09T04:36:22.938 INFO:teuthology.orchestra.run.smithi186.stdout:Install 148 Packages 2024-05-09T04:36:22.938 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrade 8 Packages 2024-05-09T04:36:22.938 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-05-09T04:36:22.938 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 219 M 2024-05-09T04:36:22.938 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2024-05-09T04:36:23.664 INFO:teuthology.orchestra.run.smithi186.stdout:(1/156): ceph-19.0.0-3594.gc9da4049.el9.x86_64. 13 kB/s | 6.6 kB 00:00 2024-05-09T04:36:23.838 INFO:teuthology.orchestra.run.smithi169.stdout:(4/156): ceph-base-19.0.0-3594.gc9da4049.el9.x8 1.4 MB/s | 5.4 MB 00:03 2024-05-09T04:36:24.113 INFO:teuthology.orchestra.run.smithi169.stdout:(5/156): ceph-mds-19.0.0-3594.gc9da4049.el9.x86 1.3 MB/s | 2.3 MB 00:01 2024-05-09T04:36:24.438 INFO:teuthology.orchestra.run.smithi169.stdout:(6/156): ceph-mgr-19.0.0-3594.gc9da4049.el9.x86 2.5 MB/s | 1.5 MB 00:00 2024-05-09T04:36:24.613 INFO:teuthology.orchestra.run.smithi006.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-05-09T04:36:25.772 INFO:teuthology.orchestra.run.smithi186.stdout:(2/156): ceph-fuse-19.0.0-3594.gc9da4049.el9.x8 409 kB/s | 863 kB 00:02 2024-05-09T04:36:25.981 INFO:teuthology.orchestra.run.smithi186.stdout:(3/156): ceph-immutable-object-cache-19.0.0-359 706 kB/s | 147 kB 00:00 2024-05-09T04:36:26.855 INFO:teuthology.orchestra.run.smithi169.stdout:(7/156): ceph-mon-19.0.0-3594.gc9da4049.el9.x86 1.8 MB/s | 4.9 MB 00:02 2024-05-09T04:36:27.292 INFO:teuthology.orchestra.run.smithi006.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T04:36:27.293 INFO:teuthology.orchestra.run.smithi006.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T04:36:27.315 INFO:teuthology.orchestra.run.smithi006.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T04:36:27.317 INFO:teuthology.orchestra.run.smithi006.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T04:36:27.318 INFO:teuthology.orchestra.run.smithi006.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T04:36:27.320 INFO:teuthology.orchestra.run.smithi006.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T04:36:27.322 INFO:teuthology.orchestra.run.smithi006.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T04:36:27.326 INFO:teuthology.orchestra.run.smithi006.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T04:36:27.327 INFO:teuthology.orchestra.run.smithi006.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T04:36:27.404 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-05-09T04:36:27.423 INFO:teuthology.orchestra.run.smithi006.stdout:======================================================================================= 2024-05-09T04:36:27.423 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2024-05-09T04:36:27.424 INFO:teuthology.orchestra.run.smithi006.stdout:======================================================================================= 2024-05-09T04:36:27.424 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-05-09T04:36:27.424 INFO:teuthology.orchestra.run.smithi006.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T04:36:27.424 INFO:teuthology.orchestra.run.smithi006.stdout: ceph x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 6.6 k 2024-05-09T04:36:27.424 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-base x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.4 M 2024-05-09T04:36:27.424 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 863 k 2024-05-09T04:36:27.424 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 147 k 2024-05-09T04:36:27.424 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 1.5 M 2024-05-09T04:36:27.424 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 142 k 2024-05-09T04:36:27.424 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 3.5 M 2024-05-09T04:36:27.424 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 7.4 M 2024-05-09T04:36:27.424 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-rook noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 49 k 2024-05-09T04:36:27.424 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-radosgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 10 M 2024-05-09T04:36:27.424 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-test x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 43 M 2024-05-09T04:36:27.425 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-volume noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 273 k 2024-05-09T04:36:27.425 INFO:teuthology.orchestra.run.smithi006.stdout: cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 763 k 2024-05-09T04:36:27.425 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 224 k 2024-05-09T04:36:27.425 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-shell x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 27 k 2024-05-09T04:36:27.425 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-top noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 18 k 2024-05-09T04:36:27.425 INFO:teuthology.orchestra.run.smithi006.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T04:36:27.425 INFO:teuthology.orchestra.run.smithi006.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T04:36:27.425 INFO:teuthology.orchestra.run.smithi006.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T04:36:27.425 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 32 k 2024-05-09T04:36:27.425 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 756 k 2024-05-09T04:36:27.425 INFO:teuthology.orchestra.run.smithi006.stdout: librados-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 127 k 2024-05-09T04:36:27.425 INFO:teuthology.orchestra.run.smithi006.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T04:36:27.425 INFO:teuthology.orchestra.run.smithi006.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T04:36:27.425 INFO:teuthology.orchestra.run.smithi006.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T04:36:27.425 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T04:36:27.425 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T04:36:27.426 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T04:36:27.426 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cephfs x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 162 k 2024-05-09T04:36:27.426 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rados x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 321 k 2024-05-09T04:36:27.426 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 301 k 2024-05-09T04:36:27.426 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 99 k 2024-05-09T04:36:27.426 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 86 k 2024-05-09T04:36:27.426 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.0 M 2024-05-09T04:36:27.426 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-nbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 169 k 2024-05-09T04:36:27.426 INFO:teuthology.orchestra.run.smithi006.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T04:36:27.426 INFO:teuthology.orchestra.run.smithi006.stdout:Upgrading: 2024-05-09T04:36:27.426 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T04:36:27.426 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T04:36:27.426 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T04:36:27.426 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T04:36:27.427 INFO:teuthology.orchestra.run.smithi006.stdout: librados2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.3 M 2024-05-09T04:36:27.427 INFO:teuthology.orchestra.run.smithi006.stdout: librbd1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.1 M 2024-05-09T04:36:27.427 INFO:teuthology.orchestra.run.smithi006.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T04:36:27.427 INFO:teuthology.orchestra.run.smithi006.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T04:36:27.427 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2024-05-09T04:36:27.427 INFO:teuthology.orchestra.run.smithi006.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T04:36:27.427 INFO:teuthology.orchestra.run.smithi006.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T04:36:27.427 INFO:teuthology.orchestra.run.smithi006.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T04:36:27.427 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 20 M 2024-05-09T04:36:27.427 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 34 k 2024-05-09T04:36:27.427 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mds x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 2.3 M 2024-05-09T04:36:27.427 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 274 k 2024-05-09T04:36:27.427 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mon x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 4.9 M 2024-05-09T04:36:27.427 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-osd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 18 M 2024-05-09T04:36:27.427 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 16 k 2024-05-09T04:36:27.427 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-selinux x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 25 k 2024-05-09T04:36:27.428 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T04:36:27.428 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T04:36:27.428 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T04:36:27.428 INFO:teuthology.orchestra.run.smithi006.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T04:36:27.428 INFO:teuthology.orchestra.run.smithi006.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T04:36:27.428 INFO:teuthology.orchestra.run.smithi006.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T04:36:27.428 INFO:teuthology.orchestra.run.smithi006.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T04:36:27.428 INFO:teuthology.orchestra.run.smithi006.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T04:36:27.428 INFO:teuthology.orchestra.run.smithi006.stdout: libcephsqlite x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 158 k 2024-05-09T04:36:27.428 INFO:teuthology.orchestra.run.smithi006.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T04:36:27.428 INFO:teuthology.orchestra.run.smithi006.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T04:36:27.428 INFO:teuthology.orchestra.run.smithi006.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T04:36:27.428 INFO:teuthology.orchestra.run.smithi006.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T04:36:27.429 INFO:teuthology.orchestra.run.smithi006.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T04:36:27.429 INFO:teuthology.orchestra.run.smithi006.stdout: libradosstriper1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 497 k 2024-05-09T04:36:27.429 INFO:teuthology.orchestra.run.smithi006.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T04:36:27.429 INFO:teuthology.orchestra.run.smithi006.stdout: librgw2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.2 M 2024-05-09T04:36:27.429 INFO:teuthology.orchestra.run.smithi006.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T04:36:27.429 INFO:teuthology.orchestra.run.smithi006.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T04:36:27.429 INFO:teuthology.orchestra.run.smithi006.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T04:36:27.429 INFO:teuthology.orchestra.run.smithi006.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T04:36:27.429 INFO:teuthology.orchestra.run.smithi006.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T04:36:27.429 INFO:teuthology.orchestra.run.smithi006.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T04:36:27.429 INFO:teuthology.orchestra.run.smithi006.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T04:36:27.429 INFO:teuthology.orchestra.run.smithi006.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T04:36:27.429 INFO:teuthology.orchestra.run.smithi006.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T04:36:27.429 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T04:36:27.429 INFO:teuthology.orchestra.run.smithi006.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T04:36:27.429 INFO:teuthology.orchestra.run.smithi006.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T04:36:27.430 INFO:teuthology.orchestra.run.smithi006.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T04:36:27.430 INFO:teuthology.orchestra.run.smithi006.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T04:36:27.430 INFO:teuthology.orchestra.run.smithi006.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T04:36:27.430 INFO:teuthology.orchestra.run.smithi006.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T04:36:27.430 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T04:36:27.430 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-argparse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 45 k 2024-05-09T04:36:27.430 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 135 k 2024-05-09T04:36:27.430 INFO:teuthology.orchestra.run.smithi006.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T04:36:27.430 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T04:36:27.430 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T04:36:27.430 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T04:36:27.430 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T04:36:27.430 INFO:teuthology.orchestra.run.smithi006.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T04:36:27.430 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T04:36:27.431 INFO:teuthology.orchestra.run.smithi006.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T04:36:27.431 INFO:teuthology.orchestra.run.smithi006.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T04:36:27.431 INFO:teuthology.orchestra.run.smithi006.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T04:36:27.431 INFO:teuthology.orchestra.run.smithi006.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T04:36:27.431 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T04:36:27.431 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T04:36:27.431 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T04:36:27.431 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T04:36:27.431 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T04:36:27.431 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T04:36:27.431 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T04:36:27.431 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T04:36:27.431 INFO:teuthology.orchestra.run.smithi006.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T04:36:27.431 INFO:teuthology.orchestra.run.smithi006.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T04:36:27.432 INFO:teuthology.orchestra.run.smithi006.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T04:36:27.432 INFO:teuthology.orchestra.run.smithi006.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T04:36:27.432 INFO:teuthology.orchestra.run.smithi006.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T04:36:27.432 INFO:teuthology.orchestra.run.smithi006.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T04:36:27.432 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T04:36:27.432 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T04:36:27.432 INFO:teuthology.orchestra.run.smithi006.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T04:36:27.432 INFO:teuthology.orchestra.run.smithi006.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T04:36:27.432 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T04:36:27.432 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T04:36:27.432 INFO:teuthology.orchestra.run.smithi006.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T04:36:27.432 INFO:teuthology.orchestra.run.smithi006.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T04:36:27.432 INFO:teuthology.orchestra.run.smithi006.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T04:36:27.432 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T04:36:27.433 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T04:36:27.433 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T04:36:27.433 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T04:36:27.433 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T04:36:27.433 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T04:36:27.433 INFO:teuthology.orchestra.run.smithi006.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T04:36:27.433 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T04:36:27.433 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T04:36:27.433 INFO:teuthology.orchestra.run.smithi006.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T04:36:27.433 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T04:36:27.433 INFO:teuthology.orchestra.run.smithi006.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T04:36:27.433 INFO:teuthology.orchestra.run.smithi006.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T04:36:27.433 INFO:teuthology.orchestra.run.smithi006.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T04:36:27.434 INFO:teuthology.orchestra.run.smithi006.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T04:36:27.434 INFO:teuthology.orchestra.run.smithi006.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T04:36:27.434 INFO:teuthology.orchestra.run.smithi006.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T04:36:27.434 INFO:teuthology.orchestra.run.smithi006.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T04:36:27.434 INFO:teuthology.orchestra.run.smithi006.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T04:36:27.434 INFO:teuthology.orchestra.run.smithi006.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T04:36:27.434 INFO:teuthology.orchestra.run.smithi006.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T04:36:27.434 INFO:teuthology.orchestra.run.smithi006.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T04:36:27.434 INFO:teuthology.orchestra.run.smithi006.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T04:36:27.434 INFO:teuthology.orchestra.run.smithi006.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T04:36:27.434 INFO:teuthology.orchestra.run.smithi006.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T04:36:27.434 INFO:teuthology.orchestra.run.smithi006.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T04:36:27.434 INFO:teuthology.orchestra.run.smithi006.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T04:36:27.434 INFO:teuthology.orchestra.run.smithi006.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T04:36:27.435 INFO:teuthology.orchestra.run.smithi006.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T04:36:27.435 INFO:teuthology.orchestra.run.smithi006.stdout:Installing weak dependencies: 2024-05-09T04:36:27.435 INFO:teuthology.orchestra.run.smithi006.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T04:36:27.435 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:36:27.435 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-05-09T04:36:27.435 INFO:teuthology.orchestra.run.smithi006.stdout:======================================================================================= 2024-05-09T04:36:27.435 INFO:teuthology.orchestra.run.smithi006.stdout:Install 148 Packages 2024-05-09T04:36:27.435 INFO:teuthology.orchestra.run.smithi006.stdout:Upgrade 8 Packages 2024-05-09T04:36:27.435 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:36:27.436 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 219 M 2024-05-09T04:36:27.436 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-05-09T04:36:27.539 INFO:teuthology.orchestra.run.smithi186.stdout:(4/156): ceph-mds-19.0.0-3594.gc9da4049.el9.x86 1.5 MB/s | 2.3 MB 00:01 2024-05-09T04:36:27.639 INFO:teuthology.orchestra.run.smithi186.stdout:(5/156): ceph-base-19.0.0-3594.gc9da4049.el9.x8 1.2 MB/s | 5.4 MB 00:04 2024-05-09T04:36:28.148 INFO:teuthology.orchestra.run.smithi186.stdout:(6/156): ceph-mgr-19.0.0-3594.gc9da4049.el9.x86 2.5 MB/s | 1.5 MB 00:00 2024-05-09T04:36:28.675 INFO:teuthology.orchestra.run.smithi006.stdout:(1/156): ceph-19.0.0-3594.gc9da4049.el9.x86_64. 12 kB/s | 6.6 kB 00:00 2024-05-09T04:36:28.781 INFO:teuthology.orchestra.run.smithi186.stdout:(7/156): ceph-mon-19.0.0-3594.gc9da4049.el9.x86 4.3 MB/s | 4.9 MB 00:01 2024-05-09T04:36:31.040 INFO:teuthology.orchestra.run.smithi186.stdout:(8/156): ceph-radosgw-19.0.0-3594.gc9da4049.el9 4.5 MB/s | 10 MB 00:02 2024-05-09T04:36:31.140 INFO:teuthology.orchestra.run.smithi186.stdout:(9/156): ceph-selinux-19.0.0-3594.gc9da4049.el9 252 kB/s | 25 kB 00:00 2024-05-09T04:36:31.746 INFO:teuthology.orchestra.run.smithi169.stdout:(8/156): ceph-radosgw-19.0.0-3594.gc9da4049.el9 2.1 MB/s | 10 MB 00:04 2024-05-09T04:36:31.871 INFO:teuthology.orchestra.run.smithi169.stdout:(9/156): ceph-selinux-19.0.0-3594.gc9da4049.el9 202 kB/s | 25 kB 00:00 2024-05-09T04:36:32.124 INFO:teuthology.orchestra.run.smithi006.stdout:(2/156): ceph-fuse-19.0.0-3594.gc9da4049.el9.x8 250 kB/s | 863 kB 00:03 2024-05-09T04:36:32.483 INFO:teuthology.orchestra.run.smithi006.stdout:(3/156): ceph-immutable-object-cache-19.0.0-359 411 kB/s | 147 kB 00:00 2024-05-09T04:36:32.615 INFO:teuthology.orchestra.run.smithi186.stdout:(10/156): ceph-osd-19.0.0-3594.gc9da4049.el9.x8 4.0 MB/s | 18 MB 00:04 2024-05-09T04:36:33.174 INFO:teuthology.orchestra.run.smithi186.stdout:(11/156): cephfs-mirror-19.0.0-3594.gc9da4049.e 401 kB/s | 224 kB 00:00 2024-05-09T04:36:33.192 INFO:teuthology.orchestra.run.smithi006.stdout:(4/156): ceph-base-19.0.0-3594.gc9da4049.el9.x8 1.1 MB/s | 5.4 MB 00:05 2024-05-09T04:36:33.265 INFO:teuthology.orchestra.run.smithi186.stdout:(12/156): cephfs-shell-19.0.0-3594.gc9da4049.el 295 kB/s | 27 kB 00:00 2024-05-09T04:36:33.330 INFO:teuthology.orchestra.run.smithi169.stdout:(10/156): ceph-osd-19.0.0-3594.gc9da4049.el9.x8 2.0 MB/s | 18 MB 00:08 2024-05-09T04:36:33.580 INFO:teuthology.orchestra.run.smithi169.stdout:(11/156): cephfs-mirror-19.0.0-3594.gc9da4049.e 897 kB/s | 224 kB 00:00 2024-05-09T04:36:33.675 INFO:teuthology.orchestra.run.smithi006.stdout:(5/156): ceph-mgr-19.0.0-3594.gc9da4049.el9.x86 3.1 MB/s | 1.5 MB 00:00 2024-05-09T04:36:33.705 INFO:teuthology.orchestra.run.smithi169.stdout:(12/156): cephfs-shell-19.0.0-3594.gc9da4049.el 216 kB/s | 27 kB 00:00 2024-05-09T04:36:33.797 INFO:teuthology.orchestra.run.smithi169.stdout:(13/156): libcephfs-devel-19.0.0-3594.gc9da4049 347 kB/s | 32 kB 00:00 2024-05-09T04:36:34.166 INFO:teuthology.orchestra.run.smithi186.stdout:(13/156): libcephfs-devel-19.0.0-3594.gc9da4049 35 kB/s | 32 kB 00:00 2024-05-09T04:36:34.606 INFO:teuthology.orchestra.run.smithi169.stdout:(14/156): libcephfs2-19.0.0-3594.gc9da4049.el9. 936 kB/s | 756 kB 00:00 2024-05-09T04:36:34.716 INFO:teuthology.orchestra.run.smithi186.stdout:(14/156): libcephfs2-19.0.0-3594.gc9da4049.el9. 1.3 MB/s | 756 kB 00:00 2024-05-09T04:36:34.756 INFO:teuthology.orchestra.run.smithi169.stdout:(15/156): libcephsqlite-19.0.0-3594.gc9da4049.e 1.0 MB/s | 158 kB 00:00 2024-05-09T04:36:34.850 INFO:teuthology.orchestra.run.smithi186.stdout:(15/156): libcephsqlite-19.0.0-3594.gc9da4049.e 1.2 MB/s | 158 kB 00:00 2024-05-09T04:36:34.873 INFO:teuthology.orchestra.run.smithi169.stdout:(16/156): librados-devel-19.0.0-3594.gc9da4049. 1.1 MB/s | 127 kB 00:00 2024-05-09T04:36:34.916 INFO:teuthology.orchestra.run.smithi186.stdout:(16/156): librados-devel-19.0.0-3594.gc9da4049. 1.9 MB/s | 127 kB 00:00 2024-05-09T04:36:34.917 INFO:teuthology.orchestra.run.smithi006.stdout:(6/156): ceph-mds-19.0.0-3594.gc9da4049.el9.x86 951 kB/s | 2.3 MB 00:02 2024-05-09T04:36:35.175 INFO:teuthology.orchestra.run.smithi186.stdout:(17/156): libradosstriper1-19.0.0-3594.gc9da404 1.9 MB/s | 497 kB 00:00 2024-05-09T04:36:35.190 INFO:teuthology.orchestra.run.smithi169.stdout:(17/156): libradosstriper1-19.0.0-3594.gc9da404 1.5 MB/s | 497 kB 00:00 2024-05-09T04:36:35.568 INFO:teuthology.orchestra.run.smithi006.stdout:(7/156): ceph-mon-19.0.0-3594.gc9da4049.el9.x86 2.6 MB/s | 4.9 MB 00:01 2024-05-09T04:36:36.709 INFO:teuthology.orchestra.run.smithi186.stdout:(18/156): librgw2-19.0.0-3594.gc9da4049.el9.x86 3.4 MB/s | 5.2 MB 00:01 2024-05-09T04:36:36.784 INFO:teuthology.orchestra.run.smithi186.stdout:(19/156): python3-ceph-argparse-19.0.0-3594.gc9 603 kB/s | 45 kB 00:00 2024-05-09T04:36:36.884 INFO:teuthology.orchestra.run.smithi186.stdout:(20/156): python3-ceph-common-19.0.0-3594.gc9da 1.3 MB/s | 135 kB 00:00 2024-05-09T04:36:36.951 INFO:teuthology.orchestra.run.smithi186.stdout:(21/156): python3-cephfs-19.0.0-3594.gc9da4049. 2.4 MB/s | 162 kB 00:00 2024-05-09T04:36:37.060 INFO:teuthology.orchestra.run.smithi186.stdout:(22/156): python3-rados-19.0.0-3594.gc9da4049.e 2.9 MB/s | 321 kB 00:00 2024-05-09T04:36:37.082 INFO:teuthology.orchestra.run.smithi169.stdout:(18/156): ceph-common-19.0.0-3594.gc9da4049.el9 1.1 MB/s | 20 MB 00:17 2024-05-09T04:36:37.218 INFO:teuthology.orchestra.run.smithi186.stdout:(23/156): python3-rbd-19.0.0-3594.gc9da4049.el9 1.9 MB/s | 301 kB 00:00 2024-05-09T04:36:37.241 INFO:teuthology.orchestra.run.smithi169.stdout:(19/156): python3-ceph-argparse-19.0.0-3594.gc9 286 kB/s | 45 kB 00:00 2024-05-09T04:36:37.302 INFO:teuthology.orchestra.run.smithi186.stdout:(24/156): python3-rgw-19.0.0-3594.gc9da4049.el9 1.2 MB/s | 99 kB 00:00 2024-05-09T04:36:37.394 INFO:teuthology.orchestra.run.smithi186.stdout:(25/156): rbd-fuse-19.0.0-3594.gc9da4049.el9.x8 935 kB/s | 86 kB 00:00 2024-05-09T04:36:37.524 INFO:teuthology.orchestra.run.smithi169.stdout:(20/156): python3-ceph-common-19.0.0-3594.gc9da 479 kB/s | 135 kB 00:00 2024-05-09T04:36:37.766 INFO:teuthology.orchestra.run.smithi169.stdout:(21/156): python3-cephfs-19.0.0-3594.gc9da4049. 670 kB/s | 162 kB 00:00 2024-05-09T04:36:38.244 INFO:teuthology.orchestra.run.smithi186.stdout:(26/156): rbd-mirror-19.0.0-3594.gc9da4049.el9. 3.6 MB/s | 3.0 MB 00:00 2024-05-09T04:36:38.311 INFO:teuthology.orchestra.run.smithi186.stdout:(27/156): rbd-nbd-19.0.0-3594.gc9da4049.el9.x86 2.5 MB/s | 169 kB 00:00 2024-05-09T04:36:38.378 INFO:teuthology.orchestra.run.smithi186.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3594.g 514 kB/s | 34 kB 00:00 2024-05-09T04:36:38.433 INFO:teuthology.orchestra.run.smithi169.stdout:(22/156): librgw2-19.0.0-3594.gc9da4049.el9.x86 1.6 MB/s | 5.2 MB 00:03 2024-05-09T04:36:38.445 INFO:teuthology.orchestra.run.smithi186.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3594.gc9da404 2.1 MB/s | 142 kB 00:00 2024-05-09T04:36:38.633 INFO:teuthology.orchestra.run.smithi169.stdout:(23/156): python3-rados-19.0.0-3594.gc9da4049.e 371 kB/s | 321 kB 00:00 2024-05-09T04:36:38.658 INFO:teuthology.orchestra.run.smithi169.stdout:(24/156): python3-rbd-19.0.0-3594.gc9da4049.el9 1.3 MB/s | 301 kB 00:00 2024-05-09T04:36:38.709 INFO:teuthology.orchestra.run.smithi169.stdout:(25/156): rbd-fuse-19.0.0-3594.gc9da4049.el9.x8 1.7 MB/s | 86 kB 00:00 2024-05-09T04:36:38.792 INFO:teuthology.orchestra.run.smithi006.stdout:(8/156): ceph-radosgw-19.0.0-3594.gc9da4049.el9 3.1 MB/s | 10 MB 00:03 2024-05-09T04:36:38.817 INFO:teuthology.orchestra.run.smithi169.stdout:(26/156): python3-rgw-19.0.0-3594.gc9da4049.el9 539 kB/s | 99 kB 00:00 2024-05-09T04:36:38.851 INFO:teuthology.orchestra.run.smithi006.stdout:(9/156): ceph-selinux-19.0.0-3594.gc9da4049.el9 440 kB/s | 25 kB 00:00 2024-05-09T04:36:39.109 INFO:teuthology.orchestra.run.smithi169.stdout:(27/156): rbd-nbd-19.0.0-3594.gc9da4049.el9.x86 579 kB/s | 169 kB 00:00 2024-05-09T04:36:39.145 INFO:teuthology.orchestra.run.smithi186.stdout:(30/156): ceph-mgr-dashboard-19.0.0-3594.gc9da4 5.1 MB/s | 3.5 MB 00:00 2024-05-09T04:36:39.218 INFO:teuthology.orchestra.run.smithi169.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3594.g 315 kB/s | 34 kB 00:00 2024-05-09T04:36:39.760 INFO:teuthology.orchestra.run.smithi169.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3594.gc9da404 263 kB/s | 142 kB 00:00 2024-05-09T04:36:40.118 INFO:teuthology.orchestra.run.smithi169.stdout:(30/156): rbd-mirror-19.0.0-3594.gc9da4049.el9. 2.2 MB/s | 3.0 MB 00:01 2024-05-09T04:36:40.812 INFO:teuthology.orchestra.run.smithi186.stdout:(31/156): ceph-mgr-diskprediction-local-19.0.0- 4.4 MB/s | 7.4 MB 00:01 2024-05-09T04:36:40.937 INFO:teuthology.orchestra.run.smithi186.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3594.gc9 2.1 MB/s | 274 kB 00:00 2024-05-09T04:36:41.013 INFO:teuthology.orchestra.run.smithi186.stdout:(33/156): ceph-mgr-rook-19.0.0-3594.gc9da4049.e 656 kB/s | 49 kB 00:00 2024-05-09T04:36:41.405 INFO:teuthology.orchestra.run.smithi186.stdout:(34/156): ceph-test-19.0.0-3594.gc9da4049.el9.x 4.2 MB/s | 43 MB 00:10 2024-05-09T04:36:41.430 INFO:teuthology.orchestra.run.smithi186.stdout:(35/156): ceph-prometheus-alerts-19.0.0-3594.gc 39 kB/s | 16 kB 00:00 2024-05-09T04:36:41.522 INFO:teuthology.orchestra.run.smithi186.stdout:(36/156): ceph-volume-19.0.0-3594.gc9da4049.el9 2.3 MB/s | 273 kB 00:00 2024-05-09T04:36:41.589 INFO:teuthology.orchestra.run.smithi186.stdout:(37/156): cephfs-top-19.0.0-3594.gc9da4049.el9. 274 kB/s | 18 kB 00:00 2024-05-09T04:36:41.664 INFO:teuthology.orchestra.run.smithi186.stdout:(38/156): cephadm-19.0.0-3594.gc9da4049.el9.noa 3.2 MB/s | 763 kB 00:00 2024-05-09T04:36:42.056 INFO:teuthology.orchestra.run.smithi186.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 172 kB/s | 80 kB 00:00 2024-05-09T04:36:42.332 INFO:teuthology.orchestra.run.smithi186.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 710 kB/s | 195 kB 00:00 2024-05-09T04:36:42.557 INFO:teuthology.orchestra.run.smithi186.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-05-09T04:36:42.678 INFO:teuthology.orchestra.run.smithi169.stdout:(31/156): ceph-mgr-diskprediction-local-19.0.0- 2.9 MB/s | 7.4 MB 00:02 2024-05-09T04:36:42.709 INFO:teuthology.orchestra.run.smithi006.stdout:(10/156): ceph-common-19.0.0-3594.gc9da4049.el9 1.3 MB/s | 20 MB 00:14 2024-05-09T04:36:42.732 INFO:teuthology.orchestra.run.smithi186.stdout:(42/156): libgfortran-11.4.1-3.el9.x86_64.rpm 752 kB/s | 803 kB 00:01 2024-05-09T04:36:42.735 INFO:teuthology.orchestra.run.smithi169.stdout:(32/156): ceph-mgr-dashboard-19.0.0-3594.gc9da4 1.2 MB/s | 3.5 MB 00:02 2024-05-09T04:36:42.794 INFO:teuthology.orchestra.run.smithi169.stdout:(33/156): ceph-mgr-modules-core-19.0.0-3594.gc9 2.3 MB/s | 274 kB 00:00 2024-05-09T04:36:42.811 INFO:teuthology.orchestra.run.smithi169.stdout:(34/156): ceph-mgr-rook-19.0.0-3594.gc9da4049.e 656 kB/s | 49 kB 00:00 2024-05-09T04:36:42.818 INFO:teuthology.orchestra.run.smithi006.stdout:(11/156): cephfs-mirror-19.0.0-3594.gc9da4049.e 2.0 MB/s | 224 kB 00:00 2024-05-09T04:36:42.828 INFO:teuthology.orchestra.run.smithi169.stdout:(35/156): ceph-prometheus-alerts-19.0.0-3594.gc 490 kB/s | 16 kB 00:00 2024-05-09T04:36:42.849 INFO:teuthology.orchestra.run.smithi186.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 914 kB/s | 106 kB 00:00 2024-05-09T04:36:42.974 INFO:teuthology.orchestra.run.smithi186.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 2.9 MB/s | 1.2 MB 00:00 2024-05-09T04:36:42.999 INFO:teuthology.orchestra.run.smithi186.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 900 kB/s | 135 kB 00:00 2024-05-09T04:36:43.028 INFO:teuthology.orchestra.run.smithi169.stdout:(36/156): cephadm-19.0.0-3594.gc9da4049.el9.noa 3.7 MB/s | 763 kB 00:00 2024-05-09T04:36:43.054 INFO:teuthology.orchestra.run.smithi169.stdout:(37/156): ceph-volume-19.0.0-3594.gc9da4049.el9 1.1 MB/s | 273 kB 00:00 2024-05-09T04:36:43.070 INFO:teuthology.orchestra.run.smithi169.stdout:(38/156): cephfs-top-19.0.0-3594.gc9da4049.el9. 438 kB/s | 18 kB 00:00 2024-05-09T04:36:43.100 INFO:teuthology.orchestra.run.smithi186.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 998 kB/s | 125 kB 00:00 2024-05-09T04:36:43.133 INFO:teuthology.orchestra.run.smithi186.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.6 MB/s | 215 kB 00:00 2024-05-09T04:36:43.234 INFO:teuthology.orchestra.run.smithi186.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 5.0 MB/s | 689 kB 00:00 2024-05-09T04:36:43.284 INFO:teuthology.orchestra.run.smithi186.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 6.1 MB/s | 943 kB 00:00 2024-05-09T04:36:43.293 INFO:teuthology.orchestra.run.smithi006.stdout:(12/156): ceph-osd-19.0.0-3594.gc9da4049.el9.x8 2.1 MB/s | 18 MB 00:08 2024-05-09T04:36:43.318 INFO:teuthology.orchestra.run.smithi186.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.2 MB/s | 107 kB 00:00 2024-05-09T04:36:43.318 INFO:teuthology.orchestra.run.smithi006.stdout:(13/156): cephfs-shell-19.0.0-3594.gc9da4049.el 54 kB/s | 27 kB 00:00 2024-05-09T04:36:43.359 INFO:teuthology.orchestra.run.smithi186.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 286 kB/s | 21 kB 00:00 2024-05-09T04:36:43.360 INFO:teuthology.orchestra.run.smithi006.stdout:(14/156): libcephfs-devel-19.0.0-3594.gc9da4049 483 kB/s | 32 kB 00:00 2024-05-09T04:36:43.410 INFO:teuthology.orchestra.run.smithi186.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 655 kB/s | 33 kB 00:00 2024-05-09T04:36:43.435 INFO:teuthology.orchestra.run.smithi186.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-05-09T04:36:43.485 INFO:teuthology.orchestra.run.smithi186.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-05-09T04:36:43.486 INFO:teuthology.orchestra.run.smithi006.stdout:(15/156): libcephsqlite-19.0.0-3594.gc9da4049.e 1.2 MB/s | 158 kB 00:00 2024-05-09T04:36:43.519 INFO:teuthology.orchestra.run.smithi006.stdout:(16/156): libcephfs2-19.0.0-3594.gc9da4049.el9. 3.7 MB/s | 756 kB 00:00 2024-05-09T04:36:43.521 INFO:teuthology.orchestra.run.smithi169.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 172 kB/s | 80 kB 00:00 2024-05-09T04:36:43.527 INFO:teuthology.orchestra.run.smithi186.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 268 kB/s | 11 kB 00:00 2024-05-09T04:36:43.561 INFO:teuthology.orchestra.run.smithi006.stdout:(17/156): librados-devel-19.0.0-3594.gc9da4049. 1.7 MB/s | 127 kB 00:00 2024-05-09T04:36:43.594 INFO:teuthology.orchestra.run.smithi186.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.4 MB/s | 160 kB 00:00 2024-05-09T04:36:43.670 INFO:teuthology.orchestra.run.smithi186.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-05-09T04:36:43.695 INFO:teuthology.orchestra.run.smithi186.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 451 kB/s | 45 kB 00:00 2024-05-09T04:36:43.712 INFO:teuthology.orchestra.run.smithi006.stdout:(18/156): libradosstriper1-19.0.0-3594.gc9da404 2.5 MB/s | 497 kB 00:00 2024-05-09T04:36:43.778 INFO:teuthology.orchestra.run.smithi006.stdout:(19/156): python3-ceph-argparse-19.0.0-3594.gc9 696 kB/s | 45 kB 00:00 2024-05-09T04:36:43.787 INFO:teuthology.orchestra.run.smithi186.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-05-09T04:36:43.804 INFO:teuthology.orchestra.run.smithi169.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 690 kB/s | 195 kB 00:00 2024-05-09T04:36:43.820 INFO:teuthology.orchestra.run.smithi186.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 4.5 MB/s | 582 kB 00:00 2024-05-09T04:36:43.845 INFO:teuthology.orchestra.run.smithi006.stdout:(20/156): python3-ceph-common-19.0.0-3594.gc9da 2.0 MB/s | 135 kB 00:00 2024-05-09T04:36:43.862 INFO:teuthology.orchestra.run.smithi186.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 339 kB/s | 25 kB 00:00 2024-05-09T04:36:43.871 INFO:teuthology.orchestra.run.smithi169.stdout:(41/156): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 803 kB 00:00 2024-05-09T04:36:43.888 INFO:teuthology.orchestra.run.smithi186.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 707 kB/s | 47 kB 00:00 2024-05-09T04:36:43.955 INFO:teuthology.orchestra.run.smithi186.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-05-09T04:36:43.980 INFO:teuthology.orchestra.run.smithi186.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-05-09T04:36:44.005 INFO:teuthology.orchestra.run.smithi169.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-05-09T04:36:44.022 INFO:teuthology.orchestra.run.smithi186.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 528 kB/s | 35 kB 00:00 2024-05-09T04:36:44.122 INFO:teuthology.orchestra.run.smithi169.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 914 kB/s | 106 kB 00:00 2024-05-09T04:36:44.172 INFO:teuthology.orchestra.run.smithi169.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 4.1 MB/s | 1.2 MB 00:00 2024-05-09T04:36:44.204 INFO:teuthology.orchestra.run.smithi006.stdout:(21/156): python3-cephfs-19.0.0-3594.gc9da4049. 454 kB/s | 162 kB 00:00 2024-05-09T04:36:44.231 INFO:teuthology.orchestra.run.smithi169.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-05-09T04:36:44.264 INFO:teuthology.orchestra.run.smithi186.stdout:(66/156): openblas-openmp-0.3.21-2.el9.x86_64.r 17 MB/s | 4.7 MB 00:00 2024-05-09T04:36:44.273 INFO:teuthology.orchestra.run.smithi169.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-05-09T04:36:44.329 INFO:teuthology.orchestra.run.smithi006.stdout:(22/156): python3-rados-19.0.0-3594.gc9da4049.e 2.5 MB/s | 321 kB 00:00 2024-05-09T04:36:44.331 INFO:teuthology.orchestra.run.smithi186.stdout:(67/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-05-09T04:36:44.407 INFO:teuthology.orchestra.run.smithi169.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.2 MB/s | 215 kB 00:00 2024-05-09T04:36:44.464 INFO:teuthology.orchestra.run.smithi186.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-05-09T04:36:44.513 INFO:teuthology.orchestra.run.smithi006.stdout:(23/156): python3-rbd-19.0.0-3594.gc9da4049.el9 1.6 MB/s | 301 kB 00:00 2024-05-09T04:36:44.523 INFO:teuthology.orchestra.run.smithi186.stdout:(69/156): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-05-09T04:36:44.557 INFO:teuthology.orchestra.run.smithi186.stdout:(70/156): postgresql-contrib-13.14-1.el9.x86_64 4.0 MB/s | 911 kB 00:00 2024-05-09T04:36:44.572 INFO:teuthology.orchestra.run.smithi006.stdout:(24/156): python3-rgw-19.0.0-3594.gc9da4049.el9 1.7 MB/s | 99 kB 00:00 2024-05-09T04:36:44.639 INFO:teuthology.orchestra.run.smithi006.stdout:(25/156): rbd-fuse-19.0.0-3594.gc9da4049.el9.x8 1.3 MB/s | 86 kB 00:00 2024-05-09T04:36:44.674 INFO:teuthology.orchestra.run.smithi186.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 8.6 MB/s | 1.0 MB 00:00 2024-05-09T04:36:44.682 INFO:teuthology.orchestra.run.smithi169.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 1.6 MB/s | 689 kB 00:00 2024-05-09T04:36:44.732 INFO:teuthology.orchestra.run.smithi186.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-05-09T04:36:44.740 INFO:teuthology.orchestra.run.smithi169.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-05-09T04:36:44.807 INFO:teuthology.orchestra.run.smithi169.stdout:(50/156): elfutils-libelf-devel-0.190-2.el9.x86 323 kB/s | 21 kB 00:00 2024-05-09T04:36:44.841 INFO:teuthology.orchestra.run.smithi169.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 2.1 MB/s | 943 kB 00:00 2024-05-09T04:36:44.883 INFO:teuthology.orchestra.run.smithi186.stdout:(73/156): postgresql-server-13.14-1.el9.x86_64. 16 MB/s | 5.8 MB 00:00 2024-05-09T04:36:44.908 INFO:teuthology.orchestra.run.smithi169.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 491 kB/s | 33 kB 00:00 2024-05-09T04:36:44.958 INFO:teuthology.orchestra.run.smithi186.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 3.2 MB/s | 244 kB 00:00 2024-05-09T04:36:45.033 INFO:teuthology.orchestra.run.smithi186.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-05-09T04:36:45.133 INFO:teuthology.orchestra.run.smithi186.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-05-09T04:36:45.159 INFO:teuthology.orchestra.run.smithi186.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 205 kB/s | 26 kB 00:00 2024-05-09T04:36:45.201 INFO:teuthology.orchestra.run.smithi186.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 286 kB/s | 19 kB 00:00 2024-05-09T04:36:45.243 INFO:teuthology.orchestra.run.smithi186.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-05-09T04:36:45.247 INFO:teuthology.orchestra.run.smithi006.stdout:(26/156): rbd-mirror-19.0.0-3594.gc9da4049.el9. 5.0 MB/s | 3.0 MB 00:00 2024-05-09T04:36:45.268 INFO:teuthology.orchestra.run.smithi186.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 520 kB/s | 35 kB 00:00 2024-05-09T04:36:45.317 INFO:teuthology.orchestra.run.smithi169.stdout:(53/156): ceph-test-19.0.0-3594.gc9da4049.el9.x 3.2 MB/s | 43 MB 00:13 2024-05-09T04:36:45.342 INFO:teuthology.orchestra.run.smithi169.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 587 kB/s | 313 kB 00:00 2024-05-09T04:36:45.343 INFO:teuthology.orchestra.run.smithi186.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-05-09T04:36:45.348 INFO:teuthology.orchestra.run.smithi006.stdout:(27/156): librgw2-19.0.0-3594.gc9da4049.el9.x86 2.9 MB/s | 5.2 MB 00:01 2024-05-09T04:36:45.381 INFO:teuthology.orchestra.run.smithi006.stdout:(28/156): rbd-nbd-19.0.0-3594.gc9da4049.el9.x86 1.2 MB/s | 169 kB 00:00 2024-05-09T04:36:45.417 INFO:teuthology.orchestra.run.smithi169.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 148 kB/s | 11 kB 00:00 2024-05-09T04:36:45.432 INFO:teuthology.orchestra.run.smithi006.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3594.gc9da404 2.8 MB/s | 142 kB 00:00 2024-05-09T04:36:45.442 INFO:teuthology.orchestra.run.smithi169.stdout:(56/156): flexiblas-openblas-openmp-3.0.4-8.el9 145 kB/s | 18 kB 00:00 2024-05-09T04:36:45.457 INFO:teuthology.orchestra.run.smithi006.stdout:(30/156): ceph-grafana-dashboards-19.0.0-3594.g 316 kB/s | 34 kB 00:00 2024-05-09T04:36:45.510 INFO:teuthology.orchestra.run.smithi186.stdout:(82/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 19 MB/s | 5.1 MB 00:00 2024-05-09T04:36:45.543 INFO:teuthology.orchestra.run.smithi169.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-05-09T04:36:45.568 INFO:teuthology.orchestra.run.smithi169.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 361 kB/s | 45 kB 00:00 2024-05-09T04:36:45.569 INFO:teuthology.orchestra.run.smithi186.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-05-09T04:36:45.627 INFO:teuthology.orchestra.run.smithi186.stdout:(84/156): python3-prettytable-0.7.2-27.el9.noar 726 kB/s | 42 kB 00:00 2024-05-09T04:36:45.661 INFO:teuthology.orchestra.run.smithi186.stdout:(85/156): python3-oauthlib-3.1.1-5.el9.noarch.r 700 kB/s | 222 kB 00:00 2024-05-09T04:36:45.728 INFO:teuthology.orchestra.run.smithi186.stdout:(86/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-05-09T04:36:45.753 INFO:teuthology.orchestra.run.smithi186.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 2.1 MB/s | 269 kB 00:00 2024-05-09T04:36:45.811 INFO:teuthology.orchestra.run.smithi186.stdout:(88/156): python3-pytz-2021.1-5.el9.noarch.rpm 873 kB/s | 51 kB 00:00 2024-05-09T04:36:45.818 INFO:teuthology.orchestra.run.smithi169.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 2.3 MB/s | 662 kB 00:00 2024-05-09T04:36:45.854 INFO:teuthology.orchestra.run.smithi186.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 279 kB 00:00 2024-05-09T04:36:45.904 INFO:teuthology.orchestra.run.smithi186.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 582 kB/s | 54 kB 00:00 2024-05-09T04:36:45.910 INFO:teuthology.orchestra.run.smithi169.stdout:(60/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 277 kB/s | 25 kB 00:00 2024-05-09T04:36:45.953 INFO:teuthology.orchestra.run.smithi169.stdout:(61/156): libtool-2.4.6-45.el9.x86_64.rpm 1.5 MB/s | 582 kB 00:00 2024-05-09T04:36:45.963 INFO:teuthology.orchestra.run.smithi186.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 713 kB/s | 42 kB 00:00 2024-05-09T04:36:45.966 INFO:teuthology.orchestra.run.smithi006.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3594.gc9da4 6.6 MB/s | 3.5 MB 00:00 2024-05-09T04:36:45.978 INFO:teuthology.orchestra.run.smithi169.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 709 kB/s | 47 kB 00:00 2024-05-09T04:36:46.021 INFO:teuthology.orchestra.run.smithi186.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 756 kB/s | 44 kB 00:00 2024-05-09T04:36:46.049 INFO:teuthology.orchestra.run.smithi006.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3594.gc9 3.3 MB/s | 274 kB 00:00 2024-05-09T04:36:46.103 INFO:teuthology.orchestra.run.smithi169.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-05-09T04:36:46.105 INFO:teuthology.orchestra.run.smithi186.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-05-09T04:36:46.153 INFO:teuthology.orchestra.run.smithi169.stdout:(64/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.4 MB/s | 3.0 MB 00:01 2024-05-09T04:36:46.155 INFO:teuthology.orchestra.run.smithi186.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-05-09T04:36:46.170 INFO:teuthology.orchestra.run.smithi169.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 527 kB/s | 35 kB 00:00 2024-05-09T04:36:46.196 INFO:teuthology.orchestra.run.smithi169.stdout:(66/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.2 MB/s | 292 kB 00:00 2024-05-09T04:36:46.234 INFO:teuthology.orchestra.run.smithi186.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 602 kB/s | 47 kB 00:00 2024-05-09T04:36:46.331 INFO:teuthology.orchestra.run.smithi186.stdout:(96/156): socat-1.7.4.1-5.el9.x86_64.rpm 3.1 MB/s | 305 kB 00:00 2024-05-09T04:36:46.390 INFO:teuthology.orchestra.run.smithi186.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 991 kB/s | 58 kB 00:00 2024-05-09T04:36:46.457 INFO:teuthology.orchestra.run.smithi186.stdout:(98/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-05-09T04:36:46.525 INFO:teuthology.orchestra.run.smithi186.stdout:(99/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 957 kB/s | 64 kB 00:00 2024-05-09T04:36:46.558 INFO:teuthology.orchestra.run.smithi006.stdout:(33/156): ceph-test-19.0.0-3594.gc9da4049.el9.x 5.6 MB/s | 43 MB 00:07 2024-05-09T04:36:46.583 INFO:teuthology.orchestra.run.smithi006.stdout:(34/156): ceph-mgr-rook-19.0.0-3594.gc9da4049.e 92 kB/s | 49 kB 00:00 2024-05-09T04:36:46.610 INFO:teuthology.orchestra.run.smithi006.stdout:(35/156): ceph-prometheus-alerts-19.0.0-3594.gc 335 kB/s | 16 kB 00:00 2024-05-09T04:36:46.715 INFO:teuthology.orchestra.run.smithi186.stdout:(100/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-05-09T04:36:46.734 INFO:teuthology.orchestra.run.smithi006.stdout:(36/156): cephadm-19.0.0-3594.gc9da4049.el9.noa 6.0 MB/s | 763 kB 00:00 2024-05-09T04:36:46.741 INFO:teuthology.orchestra.run.smithi186.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 206 kB/s | 45 kB 00:00 2024-05-09T04:36:46.786 INFO:teuthology.orchestra.run.smithi006.stdout:(37/156): cephfs-top-19.0.0-3594.gc9da4049.el9. 371 kB/s | 18 kB 00:00 2024-05-09T04:36:46.818 INFO:teuthology.orchestra.run.smithi006.stdout:(38/156): ceph-volume-19.0.0-3594.gc9da4049.el9 1.1 MB/s | 273 kB 00:00 2024-05-09T04:36:46.882 INFO:teuthology.orchestra.run.smithi186.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 134 kB/s | 22 kB 00:00 2024-05-09T04:36:46.988 INFO:teuthology.orchestra.run.smithi169.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 2.0 MB/s | 1.6 MB 00:00 2024-05-09T04:36:47.066 INFO:teuthology.orchestra.run.smithi186.stdout:(103/156): abseil-cpp-20211102.0-3.el9.x86_64.r 2.9 MB/s | 548 kB 00:00 2024-05-09T04:36:47.125 INFO:teuthology.orchestra.run.smithi186.stdout:(104/156): gperftools-libs-2.9.1-2.el9.x86_64.r 5.2 MB/s | 309 kB 00:00 2024-05-09T04:36:47.167 INFO:teuthology.orchestra.run.smithi186.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 467 kB/s | 19 kB 00:00 2024-05-09T04:36:47.288 INFO:teuthology.orchestra.run.smithi169.stdout:(68/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 4.0 MB/s | 4.4 MB 00:01 2024-05-09T04:36:47.317 INFO:teuthology.orchestra.run.smithi186.stdout:(106/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.5 MB/s | 863 kB 00:00 2024-05-09T04:36:47.384 INFO:teuthology.orchestra.run.smithi186.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 391 kB/s | 26 kB 00:00 2024-05-09T04:36:47.451 INFO:teuthology.orchestra.run.smithi186.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 730 kB/s | 49 kB 00:00 2024-05-09T04:36:47.488 INFO:teuthology.orchestra.run.smithi169.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 1.8 MB/s | 911 kB 00:00 2024-05-09T04:36:47.510 INFO:teuthology.orchestra.run.smithi186.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-05-09T04:36:47.560 INFO:teuthology.orchestra.run.smithi186.stdout:(110/156): luarocks-3.9.2-1.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-05-09T04:36:47.655 INFO:teuthology.orchestra.run.smithi169.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 363 kB/s | 133 kB 00:00 2024-05-09T04:36:47.727 INFO:teuthology.orchestra.run.smithi186.stdout:(111/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.9 MB/s | 839 kB 00:00 2024-05-09T04:36:47.835 INFO:teuthology.orchestra.run.smithi006.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 77 kB/s | 80 kB 00:01 2024-05-09T04:36:47.836 INFO:teuthology.orchestra.run.smithi186.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 4.9 MB/s | 548 kB 00:00 2024-05-09T04:36:47.878 INFO:teuthology.orchestra.run.smithi186.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-05-09T04:36:47.936 INFO:teuthology.orchestra.run.smithi186.stdout:(114/156): python3-cachetools-4.2.4-1.el9.noarc 554 kB/s | 32 kB 00:00 2024-05-09T04:36:47.978 INFO:teuthology.orchestra.run.smithi186.stdout:(115/156): python3-certifi-2021.10.8-2.el9.noar 360 kB/s | 15 kB 00:00 2024-05-09T04:36:47.997 INFO:teuthology.orchestra.run.smithi169.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 3.0 MB/s | 1.0 MB 00:00 2024-05-09T04:36:48.046 INFO:teuthology.orchestra.run.smithi186.stdout:(116/156): libarrow-9.0.0-11.el9.x86_64.rpm 5.0 MB/s | 4.4 MB 00:00 2024-05-09T04:36:48.070 INFO:teuthology.orchestra.run.smithi186.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-05-09T04:36:48.097 INFO:teuthology.orchestra.run.smithi169.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 875 kB/s | 87 kB 00:00 2024-05-09T04:36:48.137 INFO:teuthology.orchestra.run.smithi186.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-05-09T04:36:48.163 INFO:teuthology.orchestra.run.smithi186.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-05-09T04:36:48.172 INFO:teuthology.orchestra.run.smithi169.stdout:(73/156): openblas-openmp-0.3.21-2.el9.x86_64.r 2.3 MB/s | 4.7 MB 00:02 2024-05-09T04:36:48.188 INFO:teuthology.orchestra.run.smithi186.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 700 kB/s | 35 kB 00:00 2024-05-09T04:36:48.276 INFO:teuthology.orchestra.run.smithi006.stdout:(40/156): libgfortran-11.4.1-3.el9.x86_64.rpm 551 kB/s | 803 kB 00:01 2024-05-09T04:36:48.314 INFO:teuthology.orchestra.run.smithi169.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 1.7 MB/s | 244 kB 00:00 2024-05-09T04:36:48.422 INFO:teuthology.orchestra.run.smithi186.stdout:(121/156): ceph-common-19.0.0-3594.gc9da4049.el 793 kB/s | 20 MB 00:25 2024-05-09T04:36:48.447 INFO:teuthology.orchestra.run.smithi186.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 724 kB/s | 206 kB 00:00 2024-05-09T04:36:48.448 INFO:teuthology.orchestra.run.smithi169.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 1.8 MB/s | 248 kB 00:00 2024-05-09T04:36:48.489 INFO:teuthology.orchestra.run.smithi186.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 257 kB/s | 11 kB 00:00 2024-05-09T04:36:48.523 INFO:teuthology.orchestra.run.smithi169.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 342 kB/s | 26 kB 00:00 2024-05-09T04:36:48.561 INFO:teuthology.orchestra.run.smithi006.stdout:(41/156): ceph-mgr-diskprediction-local-19.0.0- 2.4 MB/s | 7.4 MB 00:03 2024-05-09T04:36:48.572 INFO:teuthology.orchestra.run.smithi186.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 212 kB/s | 18 kB 00:00 2024-05-09T04:36:48.607 INFO:teuthology.orchestra.run.smithi169.stdout:(77/156): python3-jsonpointer-2.0-4.el9.noarch. 230 kB/s | 19 kB 00:00 2024-05-09T04:36:48.614 INFO:teuthology.orchestra.run.smithi186.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 556 kB/s | 23 kB 00:00 2024-05-09T04:36:48.681 INFO:teuthology.orchestra.run.smithi186.stdout:(126/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.1 MB/s | 2.0 MB 00:00 2024-05-09T04:36:48.690 INFO:teuthology.orchestra.run.smithi169.stdout:(78/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-05-09T04:36:48.723 INFO:teuthology.orchestra.run.smithi186.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 472 kB/s | 20 kB 00:00 2024-05-09T04:36:48.748 INFO:teuthology.orchestra.run.smithi186.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 145 kB/s | 19 kB 00:00 2024-05-09T04:36:48.774 INFO:teuthology.orchestra.run.smithi186.stdout:(129/156): python3-grpcio-tools-1.46.7-10.el9.x 411 kB/s | 144 kB 00:00 2024-05-09T04:36:48.774 INFO:teuthology.orchestra.run.smithi169.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 417 kB/s | 35 kB 00:00 2024-05-09T04:36:48.799 INFO:teuthology.orchestra.run.smithi186.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 926 kB/s | 46 kB 00:00 2024-05-09T04:36:48.849 INFO:teuthology.orchestra.run.smithi186.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-05-09T04:36:48.875 INFO:teuthology.orchestra.run.smithi186.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 785 kB/s | 79 kB 00:00 2024-05-09T04:36:48.950 INFO:teuthology.orchestra.run.smithi186.stdout:(133/156): python3-kubernetes-26.1.0-2.el9.noar 4.5 MB/s | 1.0 MB 00:00 2024-05-09T04:36:48.975 INFO:teuthology.orchestra.run.smithi186.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 164 kB/s | 16 kB 00:00 2024-05-09T04:36:49.001 INFO:teuthology.orchestra.run.smithi186.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-05-09T04:36:49.026 INFO:teuthology.orchestra.run.smithi186.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-05-09T04:36:49.051 INFO:teuthology.orchestra.run.smithi186.stdout:(137/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 893 kB/s | 90 kB 00:00 2024-05-09T04:36:49.085 INFO:teuthology.orchestra.run.smithi186.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 367 kB/s | 31 kB 00:00 2024-05-09T04:36:49.110 INFO:teuthology.orchestra.run.smithi186.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-05-09T04:36:49.135 INFO:teuthology.orchestra.run.smithi186.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 707 kB/s | 59 kB 00:00 2024-05-09T04:36:49.152 INFO:teuthology.orchestra.run.smithi006.stdout:(42/156): libquadmath-11.4.1-3.el9.x86_64.rpm 148 kB/s | 195 kB 00:01 2024-05-09T04:36:49.177 INFO:teuthology.orchestra.run.smithi186.stdout:(141/156): python3-typing-extensions-4.4.0-2.el 1.2 MB/s | 51 kB 00:00 2024-05-09T04:36:49.202 INFO:teuthology.orchestra.run.smithi186.stdout:(142/156): python3-tempora-5.0.0-2.el9.noarch.r 305 kB/s | 36 kB 00:00 2024-05-09T04:36:49.253 INFO:teuthology.orchestra.run.smithi186.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-05-09T04:36:49.278 INFO:teuthology.orchestra.run.smithi186.stdout:(144/156): python3-webob-1.8.7-6.el9.noarch.rpm 1.6 MB/s | 230 kB 00:00 2024-05-09T04:36:49.303 INFO:teuthology.orchestra.run.smithi186.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-05-09T04:36:49.345 INFO:teuthology.orchestra.run.smithi186.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-05-09T04:36:49.495 INFO:teuthology.orchestra.run.smithi186.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 1.4 MB/s | 427 kB 00:00 2024-05-09T04:36:49.533 INFO:teuthology.orchestra.run.smithi169.stdout:(80/156): postgresql-server-13.14-1.el9.x86_64. 2.8 MB/s | 5.8 MB 00:02 2024-05-09T04:36:49.562 INFO:teuthology.orchestra.run.smithi186.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 6.1 MB/s | 1.6 MB 00:00 2024-05-09T04:36:49.602 INFO:teuthology.orchestra.run.smithi006.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 191 kB/s | 253 kB 00:01 2024-05-09T04:36:49.650 INFO:teuthology.orchestra.run.smithi169.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-05-09T04:36:49.742 INFO:teuthology.orchestra.run.smithi169.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-05-09T04:36:49.787 INFO:teuthology.orchestra.run.smithi186.stdout:(149/156): librados2-19.0.0-3594.gc9da4049.el9. 7.5 MB/s | 3.3 MB 00:00 2024-05-09T04:36:49.850 INFO:teuthology.orchestra.run.smithi169.stdout:(83/156): python3-babel-2.9.1-2.el9.noarch.rpm 3.4 MB/s | 6.0 MB 00:01 2024-05-09T04:36:49.876 INFO:teuthology.orchestra.run.smithi169.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 577 kB/s | 77 kB 00:00 2024-05-09T04:36:49.901 INFO:teuthology.orchestra.run.smithi169.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 839 kB/s | 42 kB 00:00 2024-05-09T04:36:49.929 INFO:teuthology.orchestra.run.smithi186.stdout:(150/156): librbd1-19.0.0-3594.gc9da4049.el9.x8 7.1 MB/s | 3.1 MB 00:00 2024-05-09T04:36:49.960 INFO:teuthology.orchestra.run.smithi169.stdout:(86/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-05-09T04:36:49.985 INFO:teuthology.orchestra.run.smithi169.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-05-09T04:36:50.019 INFO:teuthology.orchestra.run.smithi006.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 862 kB/s | 1.2 MB 00:01 2024-05-09T04:36:50.044 INFO:teuthology.orchestra.run.smithi169.stdout:(88/156): python3-pytz-2021.1-5.el9.noarch.rpm 873 kB/s | 51 kB 00:00 2024-05-09T04:36:50.069 INFO:teuthology.orchestra.run.smithi169.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-05-09T04:36:50.078 INFO:teuthology.orchestra.run.smithi006.stdout:(45/156): python3-ply-3.11-14.el9.noarch.rpm 115 kB/s | 106 kB 00:00 2024-05-09T04:36:50.103 INFO:teuthology.orchestra.run.smithi169.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 917 kB/s | 54 kB 00:00 2024-05-09T04:36:50.186 INFO:teuthology.orchestra.run.smithi169.stdout:(91/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.6 MB/s | 5.1 MB 00:01 2024-05-09T04:36:50.203 INFO:teuthology.orchestra.run.smithi169.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 416 kB/s | 42 kB 00:00 2024-05-09T04:36:50.237 INFO:teuthology.orchestra.run.smithi169.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 881 kB/s | 44 kB 00:00 2024-05-09T04:36:50.296 INFO:teuthology.orchestra.run.smithi169.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 615 kB/s | 36 kB 00:00 2024-05-09T04:36:50.313 INFO:teuthology.orchestra.run.smithi186.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 109 kB/s | 42 kB 00:00 2024-05-09T04:36:50.321 INFO:teuthology.orchestra.run.smithi169.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.8 MB/s | 221 kB 00:00 2024-05-09T04:36:50.355 INFO:teuthology.orchestra.run.smithi169.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 809 kB/s | 47 kB 00:00 2024-05-09T04:36:50.422 INFO:teuthology.orchestra.run.smithi169.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 867 kB/s | 58 kB 00:00 2024-05-09T04:36:50.447 INFO:teuthology.orchestra.run.smithi169.stdout:(98/156): socat-1.7.4.1-5.el9.x86_64.rpm 2.4 MB/s | 305 kB 00:00 2024-05-09T04:36:50.489 INFO:teuthology.orchestra.run.smithi169.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-05-09T04:36:50.505 INFO:teuthology.orchestra.run.smithi169.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-05-09T04:36:50.547 INFO:teuthology.orchestra.run.smithi169.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 766 kB/s | 45 kB 00:00 2024-05-09T04:36:50.570 INFO:teuthology.orchestra.run.smithi006.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 140 kB/s | 135 kB 00:00 2024-05-09T04:36:50.581 INFO:teuthology.orchestra.run.smithi169.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 296 kB/s | 22 kB 00:00 2024-05-09T04:36:50.722 INFO:teuthology.orchestra.run.smithi186.stdout:(152/156): NetworkManager-tui-1.47.5-1.el9.x86_ 609 kB/s | 248 kB 00:00 2024-05-09T04:36:50.773 INFO:teuthology.orchestra.run.smithi169.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.7 MB/s | 863 kB 00:00 2024-05-09T04:36:50.822 INFO:teuthology.orchestra.run.smithi186.stdout:(153/156): NetworkManager-libnm-1.47.5-1.el9.x8 1.8 MB/s | 1.8 MB 00:01 2024-05-09T04:36:50.872 INFO:teuthology.orchestra.run.smithi186.stdout:(154/156): NetworkManager-1.47.5-1.el9.x86_64.r 1.8 MB/s | 2.3 MB 00:01 2024-05-09T04:36:50.945 INFO:teuthology.orchestra.run.smithi006.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 136 kB/s | 125 kB 00:00 2024-05-09T04:36:51.079 INFO:teuthology.orchestra.run.smithi006.stdout:(48/156): python3-urllib3-1.26.5-5.el9.noarch.r 215 kB/s | 215 kB 00:00 2024-05-09T04:36:51.181 INFO:teuthology.orchestra.run.smithi186.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.7 MB/s | 2.4 MB 00:00 2024-05-09T04:36:51.196 INFO:teuthology.orchestra.run.smithi006.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 1.1 MB/s | 689 kB 00:00 2024-05-09T04:36:51.207 INFO:teuthology.orchestra.run.smithi169.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 877 kB/s | 548 kB 00:00 2024-05-09T04:36:51.247 INFO:teuthology.orchestra.run.smithi186.stdout:(156/156): openssl-3.2.1-1.el9.x86_64.rpm 2.6 MB/s | 1.3 MB 00:00 2024-05-09T04:36:51.251 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2024-05-09T04:36:51.251 INFO:teuthology.orchestra.run.smithi186.stdout:Total 7.7 MB/s | 219 MB 00:28 2024-05-09T04:36:51.271 INFO:teuthology.orchestra.run.smithi006.stdout:(50/156): elfutils-libelf-devel-0.190-2.el9.x86 291 kB/s | 21 kB 00:00 2024-05-09T04:36:51.307 INFO:teuthology.orchestra.run.smithi169.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 195 kB/s | 19 kB 00:00 2024-05-09T04:36:51.357 INFO:teuthology.orchestra.run.smithi169.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 530 kB/s | 309 kB 00:00 2024-05-09T04:36:51.438 INFO:teuthology.orchestra.run.smithi006.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 299 kB/s | 107 kB 00:00 2024-05-09T04:36:51.449 INFO:teuthology.orchestra.run.smithi169.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 283 kB/s | 26 kB 00:00 2024-05-09T04:36:51.480 INFO:teuthology.orchestra.run.smithi006.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 1.5 MB/s | 313 kB 00:00 2024-05-09T04:36:51.522 INFO:teuthology.orchestra.run.smithi006.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 394 kB/s | 33 kB 00:00 2024-05-09T04:36:51.566 INFO:teuthology.orchestra.run.smithi169.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 417 kB/s | 49 kB 00:00 2024-05-09T04:36:51.606 INFO:teuthology.orchestra.run.smithi006.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 220 kB/s | 18 kB 00:00 2024-05-09T04:36:51.706 INFO:teuthology.orchestra.run.smithi006.stdout:(55/156): bison-3.7.4-5.el9.x86_64.rpm 1.2 MB/s | 943 kB 00:00 2024-05-09T04:36:51.708 INFO:teuthology.orchestra.run.smithi169.stdout:(109/156): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-05-09T04:36:51.723 INFO:teuthology.orchestra.run.smithi006.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 96 kB/s | 11 kB 00:00 2024-05-09T04:36:51.734 INFO:teuthology.orchestra.run.smithi169.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 403 kB/s | 67 kB 00:00 2024-05-09T04:36:51.801 INFO:teuthology.orchestra.run.smithi169.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-05-09T04:36:51.807 INFO:teuthology.orchestra.run.smithi006.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 543 kB/s | 45 kB 00:00 2024-05-09T04:36:51.849 INFO:teuthology.orchestra.run.smithi006.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 160 kB 00:00 2024-05-09T04:36:51.910 INFO:teuthology.orchestra.run.smithi169.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 4.9 MB/s | 548 kB 00:00 2024-05-09T04:36:51.985 INFO:teuthology.orchestra.run.smithi169.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.3 MB/s | 839 kB 00:00 2024-05-09T04:36:52.018 INFO:teuthology.orchestra.run.smithi169.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 400 kB/s | 43 kB 00:00 2024-05-09T04:36:52.082 INFO:teuthology.orchestra.run.smithi006.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 2.4 MB/s | 662 kB 00:00 2024-05-09T04:36:52.085 INFO:teuthology.orchestra.run.smithi169.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 321 kB/s | 32 kB 00:00 2024-05-09T04:36:52.111 INFO:teuthology.orchestra.run.smithi169.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-05-09T04:36:52.124 INFO:teuthology.orchestra.run.smithi006.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 2.1 MB/s | 582 kB 00:00 2024-05-09T04:36:52.174 INFO:teuthology.orchestra.run.smithi006.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 280 kB/s | 25 kB 00:00 2024-05-09T04:36:52.286 INFO:teuthology.orchestra.run.smithi169.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 857 kB/s | 172 kB 00:00 2024-05-09T04:36:52.328 INFO:teuthology.orchestra.run.smithi169.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 1.6 MB/s | 358 kB 00:00 2024-05-09T04:36:52.333 INFO:teuthology.orchestra.run.smithi006.stdout:(62/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.8 MB/s | 292 kB 00:00 2024-05-09T04:36:52.403 INFO:teuthology.orchestra.run.smithi169.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-05-09T04:36:52.417 INFO:teuthology.orchestra.run.smithi006.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-05-09T04:36:52.420 INFO:teuthology.orchestra.run.smithi169.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 383 kB/s | 35 kB 00:00 2024-05-09T04:36:52.429 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2024-05-09T04:36:52.509 INFO:teuthology.orchestra.run.smithi006.stdout:(64/156): openblas-0.3.21-2.el9.x86_64.rpm 389 kB/s | 35 kB 00:00 2024-05-09T04:36:52.520 INFO:teuthology.orchestra.run.smithi169.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-05-09T04:36:52.559 INFO:teuthology.orchestra.run.smithi006.stdout:(65/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 109 kB/s | 47 kB 00:00 2024-05-09T04:36:52.583 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2024-05-09T04:36:52.583 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2024-05-09T04:36:52.755 INFO:teuthology.orchestra.run.smithi169.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 618 kB/s | 144 kB 00:00 2024-05-09T04:36:52.805 INFO:teuthology.orchestra.run.smithi169.stdout:(123/156): python3-grpcio-1.46.7-10.el9.x86_64. 5.3 MB/s | 2.0 MB 00:00 2024-05-09T04:36:52.847 INFO:teuthology.orchestra.run.smithi169.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-05-09T04:36:52.859 INFO:teuthology.orchestra.run.smithi006.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.2 MB/s | 3.0 MB 00:01 2024-05-09T04:36:52.872 INFO:teuthology.orchestra.run.smithi169.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 264 kB/s | 18 kB 00:00 2024-05-09T04:36:52.923 INFO:teuthology.orchestra.run.smithi169.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 308 kB/s | 23 kB 00:00 2024-05-09T04:36:52.951 INFO:teuthology.orchestra.run.smithi169.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 257 kB/s | 19 kB 00:00 2024-05-09T04:36:53.006 INFO:teuthology.orchestra.run.smithi169.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 236 kB/s | 20 kB 00:00 2024-05-09T04:36:53.057 INFO:teuthology.orchestra.run.smithi169.stdout:(129/156): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-05-09T04:36:53.099 INFO:teuthology.orchestra.run.smithi169.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 503 kB/s | 46 kB 00:00 2024-05-09T04:36:53.141 INFO:teuthology.orchestra.run.smithi169.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 941 kB/s | 79 kB 00:00 2024-05-09T04:36:53.191 INFO:teuthology.orchestra.run.smithi169.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 627 kB/s | 58 kB 00:00 2024-05-09T04:36:53.225 INFO:teuthology.orchestra.run.smithi169.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-05-09T04:36:53.283 INFO:teuthology.orchestra.run.smithi169.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 179 kB/s | 16 kB 00:00 2024-05-09T04:36:53.308 INFO:teuthology.orchestra.run.smithi169.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-05-09T04:36:53.376 INFO:teuthology.orchestra.run.smithi169.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 307 kB/s | 28 kB 00:00 2024-05-09T04:36:53.401 INFO:teuthology.orchestra.run.smithi169.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 334 kB/s | 31 kB 00:00 2024-05-09T04:36:53.476 INFO:teuthology.orchestra.run.smithi169.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-05-09T04:36:53.501 INFO:teuthology.orchestra.run.smithi169.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 588 kB/s | 59 kB 00:00 2024-05-09T04:36:53.551 INFO:teuthology.orchestra.run.smithi006.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 4.5 MB/s | 4.7 MB 00:01 2024-05-09T04:36:53.585 INFO:teuthology.orchestra.run.smithi169.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 329 kB/s | 36 kB 00:00 2024-05-09T04:36:53.601 INFO:teuthology.orchestra.run.smithi006.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 2.1 MB/s | 1.6 MB 00:00 2024-05-09T04:36:53.610 INFO:teuthology.orchestra.run.smithi169.stdout:(141/156): python3-typing-extensions-4.4.0-2.el 470 kB/s | 51 kB 00:00 2024-05-09T04:36:53.711 INFO:teuthology.orchestra.run.smithi169.stdout:(142/156): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-05-09T04:36:53.736 INFO:teuthology.orchestra.run.smithi169.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 715 kB/s | 90 kB 00:00 2024-05-09T04:36:53.752 INFO:teuthology.orchestra.run.smithi006.stdout:(69/156): postgresql-private-libs-13.14-1.el9.x 894 kB/s | 133 kB 00:00 2024-05-09T04:36:53.820 INFO:teuthology.orchestra.run.smithi169.stdout:(144/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-05-09T04:36:53.835 INFO:teuthology.orchestra.run.smithi006.stdout:(70/156): postgresql-contrib-13.14-1.el9.x86_64 3.1 MB/s | 911 kB 00:00 2024-05-09T04:36:53.845 INFO:teuthology.orchestra.run.smithi169.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 184 kB/s | 20 kB 00:00 2024-05-09T04:36:53.929 INFO:teuthology.orchestra.run.smithi169.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-05-09T04:36:54.194 INFO:teuthology.orchestra.run.smithi006.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 2.8 MB/s | 1.0 MB 00:00 2024-05-09T04:36:54.229 INFO:teuthology.orchestra.run.smithi169.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 4.1 MB/s | 1.6 MB 00:00 2024-05-09T04:36:54.294 INFO:teuthology.orchestra.run.smithi006.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 883 kB/s | 87 kB 00:00 2024-05-09T04:36:54.363 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2024-05-09T04:36:54.364 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2024-05-09T04:36:54.413 INFO:teuthology.orchestra.run.smithi169.stdout:(148/156): librados2-19.0.0-3594.gc9da4049.el9. 6.9 MB/s | 3.3 MB 00:00 2024-05-09T04:36:54.779 INFO:teuthology.orchestra.run.smithi169.stdout:(149/156): librbd1-19.0.0-3594.gc9da4049.el9.x8 5.6 MB/s | 3.1 MB 00:00 2024-05-09T04:36:54.945 INFO:teuthology.orchestra.run.smithi006.stdout:(73/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 1.9 MB/s | 4.4 MB 00:02 2024-05-09T04:36:55.153 INFO:teuthology.orchestra.run.smithi006.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 1.2 MB/s | 244 kB 00:00 2024-05-09T04:36:55.312 INFO:teuthology.orchestra.run.smithi006.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 1.5 MB/s | 248 kB 00:00 2024-05-09T04:36:55.371 INFO:teuthology.orchestra.run.smithi006.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 446 kB/s | 26 kB 00:00 2024-05-09T04:36:55.380 INFO:teuthology.orchestra.run.smithi169.stdout:(150/156): NetworkManager-1.47.5-1.el9.x86_64.r 2.4 MB/s | 2.3 MB 00:00 2024-05-09T04:36:55.430 INFO:teuthology.orchestra.run.smithi006.stdout:(77/156): python3-jsonpointer-2.0-4.el9.noarch. 332 kB/s | 19 kB 00:00 2024-05-09T04:36:55.488 INFO:teuthology.orchestra.run.smithi169.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 386 kB/s | 42 kB 00:00 2024-05-09T04:36:55.555 INFO:teuthology.orchestra.run.smithi006.stdout:(78/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-05-09T04:36:55.605 INFO:teuthology.orchestra.run.smithi169.stdout:(152/156): NetworkManager-tui-1.47.5-1.el9.x86_ 2.1 MB/s | 248 kB 00:00 2024-05-09T04:36:55.613 INFO:teuthology.orchestra.run.smithi006.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 599 kB/s | 35 kB 00:00 2024-05-09T04:36:55.730 INFO:teuthology.orchestra.run.smithi169.stdout:(153/156): openssl-3.2.1-1.el9.x86_64.rpm 11 MB/s | 1.3 MB 00:00 2024-05-09T04:36:55.772 INFO:teuthology.orchestra.run.smithi169.stdout:(154/156): NetworkManager-libnm-1.47.5-1.el9.x8 1.8 MB/s | 1.8 MB 00:00 2024-05-09T04:36:55.814 INFO:teuthology.orchestra.run.smithi006.stdout:(80/156): python3-babel-2.9.1-2.el9.noarch.rpm 3.9 MB/s | 6.0 MB 00:01 2024-05-09T04:36:55.906 INFO:teuthology.orchestra.run.smithi006.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-05-09T04:36:55.973 INFO:teuthology.orchestra.run.smithi006.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-05-09T04:36:56.031 INFO:teuthology.orchestra.run.smithi006.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-05-09T04:36:56.090 INFO:teuthology.orchestra.run.smithi006.stdout:(84/156): python3-prettytable-0.7.2-27.el9.noar 724 kB/s | 42 kB 00:00 2024-05-09T04:36:56.114 INFO:teuthology.orchestra.run.smithi169.stdout:(155/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 2.7 MB/s | 16 MB 00:06 2024-05-09T04:36:56.204 INFO:teuthology.orchestra.run.smithi006.stdout:(85/156): python3-protobuf-3.14.0-13.el9.noarch 2.3 MB/s | 269 kB 00:00 2024-05-09T04:36:56.266 INFO:teuthology.orchestra.run.smithi006.stdout:(86/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-05-09T04:36:56.358 INFO:teuthology.orchestra.run.smithi006.stdout:(87/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-05-09T04:36:56.424 INFO:teuthology.orchestra.run.smithi006.stdout:(88/156): python3-pytz-2021.1-5.el9.noarch.rpm 775 kB/s | 51 kB 00:00 2024-05-09T04:36:56.481 INFO:teuthology.orchestra.run.smithi169.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 3.2 MB/s | 2.4 MB 00:00 2024-05-09T04:36:56.483 INFO:teuthology.orchestra.run.smithi006.stdout:(89/156): python3-requests-oauthlib-1.3.0-12.el 935 kB/s | 54 kB 00:00 2024-05-09T04:36:56.487 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-05-09T04:36:56.487 INFO:teuthology.orchestra.run.smithi169.stdout:Total 5.8 MB/s | 219 MB 00:37 2024-05-09T04:36:56.917 INFO:teuthology.orchestra.run.smithi006.stdout:(90/156): postgresql-server-13.14-1.el9.x86_64. 1.8 MB/s | 5.8 MB 00:03 2024-05-09T04:36:56.976 INFO:teuthology.orchestra.run.smithi006.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 727 kB/s | 42 kB 00:00 2024-05-09T04:36:57.034 INFO:teuthology.orchestra.run.smithi006.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 769 kB/s | 44 kB 00:00 2024-05-09T04:36:57.168 INFO:teuthology.orchestra.run.smithi006.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.6 MB/s | 221 kB 00:00 2024-05-09T04:36:57.343 INFO:teuthology.orchestra.run.smithi006.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 206 kB/s | 36 kB 00:00 2024-05-09T04:36:57.402 INFO:teuthology.orchestra.run.smithi006.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 832 kB/s | 47 kB 00:00 2024-05-09T04:36:57.543 INFO:teuthology.orchestra.run.smithi006.stdout:(96/156): socat-1.7.4.1-5.el9.x86_64.rpm 2.1 MB/s | 305 kB 00:00 2024-05-09T04:36:57.611 INFO:teuthology.orchestra.run.smithi006.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 875 kB/s | 58 kB 00:00 2024-05-09T04:36:57.678 INFO:teuthology.orchestra.run.smithi006.stdout:(98/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-05-09T04:36:57.683 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-05-09T04:36:57.745 INFO:teuthology.orchestra.run.smithi006.stdout:(99/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 967 kB/s | 64 kB 00:00 2024-05-09T04:36:57.834 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-05-09T04:36:57.834 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-05-09T04:36:57.845 INFO:teuthology.orchestra.run.smithi006.stdout:(100/156): python3-numpy-1.20.1-5.el9.x86_64.rp 2.3 MB/s | 5.1 MB 00:02 2024-05-09T04:36:57.870 INFO:teuthology.orchestra.run.smithi006.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 360 kB/s | 45 kB 00:00 2024-05-09T04:36:57.954 INFO:teuthology.orchestra.run.smithi006.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 207 kB/s | 22 kB 00:00 2024-05-09T04:36:58.071 INFO:teuthology.orchestra.run.smithi006.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 4.2 MB/s | 863 kB 00:00 2024-05-09T04:36:58.355 INFO:teuthology.orchestra.run.smithi006.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.3 MB/s | 548 kB 00:00 2024-05-09T04:36:58.455 INFO:teuthology.orchestra.run.smithi006.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 806 kB/s | 309 kB 00:00 2024-05-09T04:36:58.472 INFO:teuthology.orchestra.run.smithi006.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 167 kB/s | 19 kB 00:00 2024-05-09T04:36:58.539 INFO:teuthology.orchestra.run.smithi006.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 393 kB/s | 26 kB 00:00 2024-05-09T04:36:58.689 INFO:teuthology.orchestra.run.smithi006.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 325 kB/s | 49 kB 00:00 2024-05-09T04:36:58.790 INFO:teuthology.orchestra.run.smithi006.stdout:(109/156): libarrow-9.0.0-11.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-05-09T04:36:58.815 INFO:teuthology.orchestra.run.smithi006.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 542 kB/s | 67 kB 00:00 2024-05-09T04:36:58.873 INFO:teuthology.orchestra.run.smithi006.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-05-09T04:36:58.982 INFO:teuthology.orchestra.run.smithi006.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.9 MB/s | 839 kB 00:00 2024-05-09T04:36:59.024 INFO:teuthology.orchestra.run.smithi006.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 3.6 MB/s | 548 kB 00:00 2024-05-09T04:36:59.099 INFO:teuthology.orchestra.run.smithi006.stdout:(114/156): python3-cachetools-4.2.4-1.el9.noarc 433 kB/s | 32 kB 00:00 2024-05-09T04:36:59.125 INFO:teuthology.orchestra.run.smithi006.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 307 kB/s | 43 kB 00:00 2024-05-09T04:36:59.175 INFO:teuthology.orchestra.run.smithi006.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 201 kB/s | 15 kB 00:00 2024-05-09T04:36:59.208 INFO:teuthology.orchestra.run.smithi006.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-05-09T04:36:59.426 INFO:teuthology.orchestra.run.smithi006.stdout:(118/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.5 MB/s | 16 MB 00:02 2024-05-09T04:36:59.459 INFO:teuthology.orchestra.run.smithi006.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.2 MB/s | 303 kB 00:00 2024-05-09T04:36:59.493 INFO:teuthology.orchestra.run.smithi006.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 1.1 MB/s | 358 kB 00:00 2024-05-09T04:36:59.535 INFO:teuthology.orchestra.run.smithi006.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-05-09T04:36:59.602 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-05-09T04:36:59.603 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-05-09T04:36:59.610 INFO:teuthology.orchestra.run.smithi006.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-05-09T04:36:59.635 INFO:teuthology.orchestra.run.smithi006.stdout:(123/156): python3-colorama-0.4.4-12.el9.noarch 169 kB/s | 35 kB 00:00 2024-05-09T04:36:59.721 INFO:teuthology.orchestra.run.smithi006.stdout:(124/156): python3-grpcio-1.46.7-10.el9.x86_64. 9.0 MB/s | 2.0 MB 00:00 2024-05-09T04:36:59.752 INFO:teuthology.orchestra.run.smithi006.stdout:(125/156): python3-jaraco-8.2.1-3.el9.noarch.rp 75 kB/s | 11 kB 00:00 2024-05-09T04:36:59.778 INFO:teuthology.orchestra.run.smithi006.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 125 kB/s | 18 kB 00:00 2024-05-09T04:36:59.803 INFO:teuthology.orchestra.run.smithi006.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 287 kB/s | 23 kB 00:00 2024-05-09T04:36:59.829 INFO:teuthology.orchestra.run.smithi006.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 259 kB/s | 19 kB 00:00 2024-05-09T04:36:59.854 INFO:teuthology.orchestra.run.smithi006.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 262 kB/s | 20 kB 00:00 2024-05-09T04:36:59.912 INFO:teuthology.orchestra.run.smithi006.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-05-09T04:36:59.938 INFO:teuthology.orchestra.run.smithi006.stdout:(131/156): python3-logutils-0.3.5-21.el9.noarch 427 kB/s | 46 kB 00:00 2024-05-09T04:36:59.972 INFO:teuthology.orchestra.run.smithi006.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 674 kB/s | 79 kB 00:00 2024-05-09T04:36:59.997 INFO:teuthology.orchestra.run.smithi006.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 694 kB/s | 58 kB 00:00 2024-05-09T04:37:00.030 INFO:teuthology.orchestra.run.smithi006.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-05-09T04:37:00.056 INFO:teuthology.orchestra.run.smithi006.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 197 kB/s | 16 kB 00:00 2024-05-09T04:37:00.081 INFO:teuthology.orchestra.run.smithi006.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-05-09T04:37:00.106 INFO:teuthology.orchestra.run.smithi006.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 377 kB/s | 28 kB 00:00 2024-05-09T04:37:00.132 INFO:teuthology.orchestra.run.smithi006.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 411 kB/s | 31 kB 00:00 2024-05-09T04:37:00.165 INFO:teuthology.orchestra.run.smithi006.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-05-09T04:37:00.190 INFO:teuthology.orchestra.run.smithi006.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 710 kB/s | 59 kB 00:00 2024-05-09T04:37:00.216 INFO:teuthology.orchestra.run.smithi006.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 430 kB/s | 36 kB 00:00 2024-05-09T04:37:00.241 INFO:teuthology.orchestra.run.smithi006.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 679 kB/s | 51 kB 00:00 2024-05-09T04:37:00.274 INFO:teuthology.orchestra.run.smithi006.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-05-09T04:37:00.300 INFO:teuthology.orchestra.run.smithi006.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2024-05-09T04:37:00.366 INFO:teuthology.orchestra.run.smithi006.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.4 MB/s | 427 kB 00:00 2024-05-09T04:37:00.400 INFO:teuthology.orchestra.run.smithi006.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 159 kB/s | 20 kB 00:00 2024-05-09T04:37:00.438 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2024-05-09T04:37:00.476 INFO:teuthology.orchestra.run.smithi006.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 1.1 MB/s | 192 kB 00:00 2024-05-09T04:37:00.518 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T04:37:00.557 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T04:37:00.559 INFO:teuthology.orchestra.run.smithi006.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 8.2 MB/s | 1.6 MB 00:00 2024-05-09T04:37:00.610 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T04:37:00.645 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T04:37:00.670 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T04:37:00.720 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T04:37:00.876 INFO:teuthology.orchestra.run.smithi006.stdout:(149/156): librbd1-19.0.0-3594.gc9da4049.el9.x8 7.7 MB/s | 3.1 MB 00:00 2024-05-09T04:37:01.051 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T04:37:01.163 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T04:37:01.288 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T04:37:01.314 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T04:37:01.405 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T04:37:01.434 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T04:37:01.493 INFO:teuthology.orchestra.run.smithi006.stdout:(150/156): librados2-19.0.0-3594.gc9da4049.el9. 3.0 MB/s | 3.3 MB 00:01 2024-05-09T04:37:01.584 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T04:37:01.640 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T04:37:01.668 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T04:37:01.773 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T04:37:01.802 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T04:37:01.865 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T04:37:01.895 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T04:37:01.902 INFO:teuthology.orchestra.run.smithi006.stdout:(151/156): NetworkManager-1.47.5-1.el9.x86_64.r 1.7 MB/s | 2.3 MB 00:01 2024-05-09T04:37:01.919 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T04:37:02.103 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T04:37:02.230 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T04:37:02.244 INFO:teuthology.orchestra.run.smithi006.stdout:(152/156): NetworkManager-team-1.47.5-1.el9.x86 56 kB/s | 42 kB 00:00 2024-05-09T04:37:02.404 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T04:37:02.461 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T04:37:02.555 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T04:37:02.662 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T04:37:02.847 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T04:37:02.927 INFO:teuthology.orchestra.run.smithi006.stdout:(153/156): NetworkManager-tui-1.47.5-1.el9.x86_ 243 kB/s | 248 kB 00:01 2024-05-09T04:37:02.958 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T04:37:02.991 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T04:37:03.026 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-09T04:37:03.090 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-09T04:37:03.125 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-09T04:37:03.169 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-09T04:37:03.201 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-09T04:37:03.229 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-09T04:37:03.261 INFO:teuthology.orchestra.run.smithi006.stdout:(154/156): NetworkManager-libnm-1.47.5-1.el9.x8 809 kB/s | 1.8 MB 00:02 2024-05-09T04:37:03.297 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-09T04:37:03.359 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-09T04:37:03.390 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-09T04:37:03.426 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-09T04:37:03.502 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-09T04:37:03.528 INFO:teuthology.orchestra.run.smithi006.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 1.1 MB/s | 1.3 MB 00:01 2024-05-09T04:37:03.536 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 32/164 2024-05-09T04:37:03.580 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 33/164 2024-05-09T04:37:03.609 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 34/164 2024-05-09T04:37:03.678 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-09T04:37:03.723 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-09T04:37:03.797 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-09T04:37:03.939 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-09T04:37:03.992 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-09T04:37:04.436 INFO:teuthology.orchestra.run.smithi006.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 1.6 MB/s | 2.4 MB 00:01 2024-05-09T04:37:04.443 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-05-09T04:37:04.443 INFO:teuthology.orchestra.run.smithi006.stdout:Total 5.9 MB/s | 219 MB 00:37 2024-05-09T04:37:04.520 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-09T04:37:04.605 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-09T04:37:04.634 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-09T04:37:04.676 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-09T04:37:04.715 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 44/164 2024-05-09T04:37:04.756 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-09T04:37:04.904 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-09T04:37:04.968 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-09T04:37:05.058 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-09T04:37:05.123 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-09T04:37:05.196 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T04:37:05.227 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T04:37:05.263 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T04:37:05.311 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T04:37:05.346 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T04:37:05.397 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T04:37:05.439 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T04:37:05.576 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T04:37:05.620 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T04:37:05.650 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T04:37:05.670 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-05-09T04:37:05.691 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T04:37:05.752 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T04:37:05.786 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T04:37:05.799 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T04:37:05.816 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T04:37:05.857 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T04:37:05.861 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T04:37:05.882 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T04:37:05.895 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T04:37:05.907 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T04:37:05.920 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T04:37:05.963 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T04:37:05.981 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T04:37:06.109 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T04:37:06.156 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T04:37:06.184 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T04:37:06.218 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T04:37:06.257 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T04:37:06.289 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T04:37:06.291 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T04:37:06.318 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T04:37:06.353 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T04:37:06.378 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T04:37:06.396 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T04:37:06.419 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T04:37:06.454 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T04:37:06.520 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T04:37:06.541 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T04:37:06.555 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T04:37:06.584 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T04:37:06.615 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T04:37:06.647 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T04:37:06.662 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T04:37:06.677 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T04:37:06.761 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T04:37:06.817 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T04:37:06.827 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T04:37:06.874 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T04:37:06.902 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T04:37:06.990 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T04:37:07.019 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T04:37:07.065 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T04:37:07.079 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T04:37:07.104 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T04:37:07.247 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T04:37:07.376 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T04:37:07.527 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T04:37:07.528 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-05-09T04:37:07.544 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T04:37:07.587 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T04:37:07.589 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T04:37:07.623 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T04:37:07.690 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T04:37:07.721 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T04:37:07.780 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T04:37:07.788 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T04:37:07.792 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-05-09T04:37:07.792 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-05-09T04:37:07.876 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T04:37:07.908 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T04:37:07.975 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T04:37:08.088 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T04:37:08.103 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T04:37:08.113 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T04:37:08.134 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T04:37:08.168 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-09T04:37:08.192 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T04:37:08.243 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-09T04:37:08.277 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-09T04:37:08.320 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-09T04:37:08.353 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-09T04:37:08.381 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-09T04:37:08.451 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-09T04:37:08.512 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-09T04:37:08.542 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-09T04:37:08.586 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-09T04:37:08.662 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-09T04:37:08.696 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 32/164 2024-05-09T04:37:08.740 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 33/164 2024-05-09T04:37:08.769 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 34/164 2024-05-09T04:37:08.831 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-09T04:37:08.840 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T04:37:08.884 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-09T04:37:08.958 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-09T04:37:08.961 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T04:37:09.093 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-09T04:37:09.145 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-09T04:37:09.680 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-09T04:37:09.757 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-09T04:37:09.786 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-09T04:37:09.837 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-09T04:37:09.875 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 44/164 2024-05-09T04:37:09.916 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-09T04:37:10.078 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-09T04:37:10.144 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-09T04:37:10.198 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T04:37:10.237 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-09T04:37:10.298 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T04:37:10.309 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-09T04:37:10.320 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T04:37:10.344 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T04:37:10.382 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T04:37:10.412 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T04:37:10.433 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-05-09T04:37:10.434 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-05-09T04:37:10.449 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T04:37:10.488 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T04:37:10.524 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T04:37:10.566 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T04:37:10.609 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T04:37:10.666 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T04:37:10.708 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T04:37:10.746 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T04:37:10.798 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T04:37:10.806 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T04:37:10.829 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 102/164 2024-05-09T04:37:10.836 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T04:37:10.865 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T04:37:10.877 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T04:37:10.973 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T04:37:11.020 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T04:37:11.059 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T04:37:11.085 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T04:37:11.109 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T04:37:11.186 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T04:37:11.313 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T04:37:11.342 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T04:37:11.353 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T04:37:11.370 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T04:37:11.386 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T04:37:11.405 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T04:37:11.435 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T04:37:11.457 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T04:37:11.467 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T04:37:11.487 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 106/164 2024-05-09T04:37:11.496 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T04:37:11.531 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T04:37:11.565 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T04:37:11.606 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T04:37:11.641 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T04:37:11.728 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T04:37:11.771 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T04:37:11.811 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T04:37:11.858 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T04:37:11.957 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T04:37:12.004 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T04:37:12.741 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T04:37:12.793 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T04:37:12.826 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T04:37:12.926 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T04:37:12.976 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T04:37:13.064 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T04:37:13.304 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T04:37:13.569 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T04:37:13.601 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T04:37:13.679 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T04:37:13.693 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T04:37:13.735 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T04:37:14.295 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T04:37:14.335 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T04:37:14.338 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T04:37:14.430 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T04:37:14.430 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T04:37:14.431 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T04:37:14.431 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-05-09T04:37:14.457 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T04:37:14.459 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T04:37:15.713 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T04:37:15.799 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T04:37:15.818 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T04:37:15.844 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T04:37:16.163 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T04:37:16.206 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T04:37:16.296 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T04:37:16.328 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 102/164 2024-05-09T04:37:16.364 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T04:37:16.853 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T04:37:16.884 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T04:37:16.942 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-05-09T04:37:16.955 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T04:37:16.977 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 106/164 2024-05-09T04:37:17.103 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T04:37:17.223 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T04:37:17.347 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T04:37:17.410 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T04:37:17.447 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T04:37:17.545 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T04:37:17.987 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T04:37:18.110 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T04:37:18.292 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T04:37:18.336 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T04:37:18.468 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T04:37:18.511 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T04:37:18.838 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T04:37:18.916 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T04:37:18.974 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T04:37:19.145 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T04:37:19.192 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T04:37:19.220 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T04:37:19.257 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T04:37:19.279 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T04:37:19.310 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T04:37:19.386 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T04:37:19.729 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T04:37:19.819 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T04:37:19.853 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T04:37:19.945 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T04:37:19.945 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T04:37:19.945 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T04:37:19.945 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-09T04:37:19.979 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T04:37:20.072 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T04:37:20.249 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T04:37:20.330 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T04:37:20.538 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T04:37:20.661 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T04:37:21.006 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T04:37:21.235 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T04:37:21.276 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T04:37:21.335 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-09T04:37:21.456 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-09T04:37:21.536 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-09T04:37:21.623 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-09T04:37:21.671 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-09T04:37:21.709 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-09T04:37:21.830 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-09T04:37:21.904 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-09T04:37:21.942 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-09T04:37:22.005 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-09T04:37:22.124 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-09T04:37:22.201 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 32/164 2024-05-09T04:37:22.264 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 33/164 2024-05-09T04:37:22.305 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 34/164 2024-05-09T04:37:22.420 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-09T04:37:22.485 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-09T04:37:22.598 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-09T04:37:22.790 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-09T04:37:22.877 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-09T04:37:23.594 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-09T04:37:23.716 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-09T04:37:23.752 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-09T04:37:23.797 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-09T04:37:23.838 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 44/164 2024-05-09T04:37:23.899 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-09T04:37:24.115 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-09T04:37:24.188 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-09T04:37:24.318 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-09T04:37:24.394 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-09T04:37:24.481 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T04:37:24.539 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T04:37:24.578 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T04:37:24.646 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T04:37:24.686 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T04:37:24.741 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T04:37:24.804 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T04:37:25.009 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T04:37:25.059 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T04:37:25.097 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T04:37:25.164 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T04:37:25.317 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T04:37:25.354 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T04:37:25.428 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T04:37:25.463 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T04:37:25.512 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T04:37:25.722 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T04:37:25.903 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T04:37:25.942 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T04:37:25.981 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T04:37:26.053 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T04:37:26.122 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T04:37:26.190 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T04:37:26.202 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T04:37:26.202 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /sys 2024-05-09T04:37:26.202 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /proc 2024-05-09T04:37:26.203 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /mnt 2024-05-09T04:37:26.203 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /var/tmp 2024-05-09T04:37:26.203 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /home 2024-05-09T04:37:26.203 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /root 2024-05-09T04:37:26.203 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /tmp 2024-05-09T04:37:26.203 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-05-09T04:37:26.242 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T04:37:26.295 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T04:37:26.344 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T04:37:26.380 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T04:37:26.395 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T04:37:26.395 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:26.395 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T04:37:26.395 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-05-09T04:37:26.423 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T04:37:26.466 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T04:37:26.612 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T04:37:26.699 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T04:37:26.750 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T04:37:26.836 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T04:37:27.013 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T04:37:27.078 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T04:37:27.983 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T04:37:28.008 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T04:37:28.009 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:28.009 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T04:37:28.009 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T04:37:28.009 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T04:37:28.009 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-05-09T04:37:28.066 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T04:37:28.127 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T04:37:28.161 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T04:37:28.233 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T04:37:28.256 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T04:37:28.256 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:28.256 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T04:37:28.256 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T04:37:28.256 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T04:37:28.257 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-05-09T04:37:28.289 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T04:37:28.357 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T04:37:28.467 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T04:37:28.504 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T04:37:28.717 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T04:37:28.739 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T04:37:28.740 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:28.740 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T04:37:28.740 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T04:37:28.740 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T04:37:28.740 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-05-09T04:37:28.747 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T04:37:28.784 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T04:37:28.839 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T04:37:28.893 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T04:37:28.897 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T04:37:28.953 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T04:37:29.139 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T04:37:29.185 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T04:37:29.263 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T04:37:29.351 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T04:37:29.491 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T04:37:29.536 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T04:37:29.567 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T04:37:29.594 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T04:37:29.777 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T04:37:29.948 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T04:37:30.039 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T04:37:30.068 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T04:37:30.101 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 126/164 2024-05-09T04:37:30.162 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 127/164 2024-05-09T04:37:30.191 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T04:37:31.237 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T04:37:31.279 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T04:37:31.613 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T04:37:31.614 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /sys 2024-05-09T04:37:31.614 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /proc 2024-05-09T04:37:31.614 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /mnt 2024-05-09T04:37:31.614 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /var/tmp 2024-05-09T04:37:31.614 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /home 2024-05-09T04:37:31.614 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /root 2024-05-09T04:37:31.614 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /tmp 2024-05-09T04:37:31.614 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-09T04:37:31.629 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T04:37:31.723 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T04:37:31.760 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T04:37:31.796 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T04:37:31.811 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T04:37:31.811 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:31.812 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T04:37:31.812 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-09T04:37:31.821 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T04:37:32.221 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T04:37:32.254 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T04:37:32.270 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T04:37:32.277 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T04:37:32.388 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T04:37:32.395 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T04:37:32.452 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 102/164 2024-05-09T04:37:32.513 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T04:37:32.551 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 131/164 2024-05-09T04:37:32.587 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T04:37:32.610 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T04:37:32.610 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:32.610 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T04:37:32.611 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T04:37:32.611 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T04:37:32.611 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-05-09T04:37:32.654 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T04:37:32.665 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T04:37:32.764 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 134/164 2024-05-09T04:37:32.811 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 135/164 2024-05-09T04:37:33.127 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T04:37:33.173 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T04:37:33.325 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T04:37:33.351 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 106/164 2024-05-09T04:37:33.382 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T04:37:33.408 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T04:37:33.408 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:33.408 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T04:37:33.408 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T04:37:33.409 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T04:37:33.409 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-09T04:37:33.633 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T04:37:33.655 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T04:37:33.655 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:33.655 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T04:37:33.655 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T04:37:33.656 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T04:37:33.656 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-09T04:37:33.748 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T04:37:33.774 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T04:37:33.797 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T04:37:33.797 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:33.797 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T04:37:33.797 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T04:37:33.797 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T04:37:33.797 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-05-09T04:37:33.841 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T04:37:33.863 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T04:37:33.863 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:33.863 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T04:37:33.863 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-05-09T04:37:33.908 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T04:37:33.930 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T04:37:33.931 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:33.931 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T04:37:33.931 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T04:37:33.931 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T04:37:33.931 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-05-09T04:37:34.117 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T04:37:34.140 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T04:37:34.140 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:34.140 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T04:37:34.140 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T04:37:34.140 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T04:37:34.140 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-09T04:37:34.243 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T04:37:34.255 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T04:37:34.265 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T04:37:34.265 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:34.265 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T04:37:34.265 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T04:37:34.265 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T04:37:34.265 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-05-09T04:37:34.313 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T04:37:34.368 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T04:37:34.555 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T04:37:34.601 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T04:37:34.688 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T04:37:34.777 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T04:37:34.917 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T04:37:34.968 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T04:37:35.000 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T04:37:35.028 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T04:37:35.389 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T04:37:35.418 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T04:37:35.451 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 126/164 2024-05-09T04:37:35.513 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 127/164 2024-05-09T04:37:35.541 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T04:37:36.198 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T04:37:36.277 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T04:37:36.585 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T04:37:36.618 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T04:37:37.005 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T04:37:37.052 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T04:37:37.209 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T04:37:37.209 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T04:37:37.209 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T04:37:37.210 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:37:37.261 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T04:37:37.593 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T04:37:37.627 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T04:37:37.739 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T04:37:37.903 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 131/164 2024-05-09T04:37:37.930 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T04:37:37.953 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T04:37:37.953 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:37.953 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T04:37:37.953 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T04:37:37.953 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T04:37:37.953 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-09T04:37:37.997 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T04:37:38.008 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T04:37:38.106 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 134/164 2024-05-09T04:37:38.154 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 135/164 2024-05-09T04:37:38.563 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 141/164 2024-05-09T04:37:38.604 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 142/164 2024-05-09T04:37:38.645 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 143/164 2024-05-09T04:37:38.677 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T04:37:38.704 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T04:37:38.887 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 146/164 2024-05-09T04:37:38.933 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T04:37:38.964 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 148/164 2024-05-09T04:37:38.993 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T04:37:39.039 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T04:37:39.089 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T04:37:39.116 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T04:37:39.139 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T04:37:39.139 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:39.139 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T04:37:39.139 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T04:37:39.139 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T04:37:39.139 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-09T04:37:39.183 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T04:37:39.205 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T04:37:39.205 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:39.206 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T04:37:39.206 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-09T04:37:39.259 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T04:37:39.281 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T04:37:39.281 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:39.282 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T04:37:39.282 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T04:37:39.282 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T04:37:39.282 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-09T04:37:39.593 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T04:37:39.616 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T04:37:39.616 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:39.616 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T04:37:39.616 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T04:37:39.616 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T04:37:39.616 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-09T04:37:39.732 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T04:37:39.771 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T04:37:39.790 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T04:37:40.631 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 152/164 2024-05-09T04:37:40.718 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T04:37:40.759 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T04:37:40.840 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T04:37:40.879 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T04:37:40.905 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T04:37:40.905 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T04:37:40.989 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T04:37:41.014 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T04:37:41.014 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T04:37:41.082 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T04:37:41.107 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T04:37:41.112 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T04:37:41.112 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T04:37:41.292 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T04:37:41.292 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T04:37:41.376 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T04:37:41.439 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T04:37:43.889 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 141/164 2024-05-09T04:37:43.930 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 142/164 2024-05-09T04:37:43.970 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 143/164 2024-05-09T04:37:44.003 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T04:37:44.030 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T04:37:44.217 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 146/164 2024-05-09T04:37:44.260 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T04:37:44.290 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 148/164 2024-05-09T04:37:44.320 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T04:37:44.366 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T04:37:44.968 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T04:37:45.006 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T04:37:45.026 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T04:37:45.877 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 152/164 2024-05-09T04:37:45.978 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T04:37:46.018 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T04:37:46.109 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T04:37:46.148 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T04:37:46.174 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T04:37:46.174 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T04:37:46.258 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T04:37:46.283 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T04:37:46.283 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T04:37:46.368 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T04:37:46.393 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T04:37:46.398 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T04:37:46.398 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T04:37:46.586 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T04:37:46.586 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T04:37:46.671 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T04:37:46.735 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T04:37:46.963 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T04:37:46.963 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 1/164 2024-05-09T04:37:46.963 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2/164 2024-05-09T04:37:46.964 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 3/164 2024-05-09T04:37:46.964 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 4/164 2024-05-09T04:37:46.964 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 5/164 2024-05-09T04:37:46.964 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 6/164 2024-05-09T04:37:46.964 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 7/164 2024-05-09T04:37:46.964 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T04:37:46.964 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T04:37:46.964 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T04:37:46.964 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 11/164 2024-05-09T04:37:46.964 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 12/164 2024-05-09T04:37:46.964 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T04:37:46.964 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 14/164 2024-05-09T04:37:46.964 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 15/164 2024-05-09T04:37:46.964 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 16/164 2024-05-09T04:37:46.965 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 17/164 2024-05-09T04:37:46.965 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 18/164 2024-05-09T04:37:46.965 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 19/164 2024-05-09T04:37:46.965 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 20/164 2024-05-09T04:37:46.965 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 21/164 2024-05-09T04:37:46.965 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 22/164 2024-05-09T04:37:46.965 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 23/164 2024-05-09T04:37:46.965 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 24/164 2024-05-09T04:37:46.965 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 25/164 2024-05-09T04:37:46.965 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 26/164 2024-05-09T04:37:46.965 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 27/164 2024-05-09T04:37:46.965 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 28/164 2024-05-09T04:37:46.965 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 29/164 2024-05-09T04:37:46.966 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 30/164 2024-05-09T04:37:46.966 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 31/164 2024-05-09T04:37:46.966 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 32/164 2024-05-09T04:37:46.966 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 33/164 2024-05-09T04:37:46.966 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 34/164 2024-05-09T04:37:46.966 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 35/164 2024-05-09T04:37:46.966 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 36/164 2024-05-09T04:37:46.966 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 37/164 2024-05-09T04:37:46.966 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 38/164 2024-05-09T04:37:46.966 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 39/164 2024-05-09T04:37:46.967 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T04:37:46.967 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T04:37:46.967 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T04:37:46.967 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T04:37:46.967 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T04:37:46.967 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T04:37:46.967 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T04:37:46.967 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T04:37:46.967 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T04:37:46.967 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T04:37:46.967 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T04:37:46.967 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T04:37:46.967 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T04:37:46.967 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T04:37:46.968 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T04:37:46.968 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T04:37:46.968 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T04:37:46.968 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T04:37:46.968 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T04:37:46.968 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T04:37:46.968 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T04:37:46.968 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T04:37:46.968 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T04:37:46.968 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T04:37:46.968 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T04:37:46.968 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T04:37:46.968 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T04:37:46.968 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T04:37:46.969 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T04:37:46.969 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T04:37:46.969 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T04:37:46.969 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T04:37:46.969 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T04:37:46.969 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T04:37:46.969 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T04:37:46.969 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T04:37:46.969 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T04:37:46.969 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T04:37:46.969 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T04:37:46.969 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T04:37:46.969 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T04:37:46.969 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T04:37:46.970 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T04:37:46.970 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T04:37:46.970 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T04:37:46.970 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T04:37:46.970 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T04:37:46.970 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T04:37:46.970 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T04:37:46.970 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T04:37:46.970 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T04:37:46.970 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T04:37:46.970 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T04:37:46.970 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T04:37:46.970 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T04:37:46.971 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T04:37:46.971 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T04:37:46.971 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T04:37:46.971 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T04:37:46.971 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T04:37:46.971 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T04:37:46.971 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T04:37:46.971 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T04:37:46.971 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T04:37:46.971 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T04:37:46.971 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T04:37:46.971 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T04:37:46.971 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T04:37:46.971 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T04:37:46.972 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T04:37:46.972 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T04:37:46.972 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T04:37:46.972 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T04:37:46.972 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T04:37:46.972 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T04:37:46.972 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T04:37:46.972 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T04:37:46.972 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T04:37:46.972 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T04:37:46.972 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T04:37:46.972 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T04:37:46.972 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T04:37:46.973 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T04:37:46.973 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T04:37:46.973 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T04:37:46.973 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T04:37:46.973 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T04:37:46.973 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T04:37:46.973 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T04:37:46.973 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T04:37:46.973 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T04:37:46.973 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T04:37:46.973 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T04:37:46.973 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T04:37:46.973 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T04:37:46.973 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T04:37:46.974 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T04:37:46.974 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T04:37:46.974 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T04:37:46.974 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T04:37:46.974 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T04:37:46.974 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T04:37:46.974 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T04:37:46.974 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T04:37:46.974 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T04:37:46.974 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T04:37:46.974 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T04:37:46.974 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T04:37:46.974 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T04:37:46.974 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 149/164 2024-05-09T04:37:46.975 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T04:37:46.975 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 151/164 2024-05-09T04:37:46.975 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T04:37:46.975 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T04:37:46.975 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T04:37:46.975 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T04:37:46.975 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T04:37:46.975 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T04:37:46.975 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T04:37:46.975 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T04:37:46.975 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T04:37:46.975 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T04:37:46.975 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T04:37:46.975 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T04:37:49.614 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T04:37:49.614 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-05-09T04:37:49.614 INFO:teuthology.orchestra.run.smithi186.stdout:Upgraded: 2024-05-09T04:37:49.614 INFO:teuthology.orchestra.run.smithi186.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T04:37:49.614 INFO:teuthology.orchestra.run.smithi186.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T04:37:49.614 INFO:teuthology.orchestra.run.smithi186.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T04:37:49.614 INFO:teuthology.orchestra.run.smithi186.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T04:37:49.614 INFO:teuthology.orchestra.run.smithi186.stdout: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:49.614 INFO:teuthology.orchestra.run.smithi186.stdout: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:49.614 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T04:37:49.615 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T04:37:49.615 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2024-05-09T04:37:49.615 INFO:teuthology.orchestra.run.smithi186.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T04:37:49.615 INFO:teuthology.orchestra.run.smithi186.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T04:37:49.615 INFO:teuthology.orchestra.run.smithi186.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T04:37:49.615 INFO:teuthology.orchestra.run.smithi186.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T04:37:49.615 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:49.615 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:49.615 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:49.615 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:49.615 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:49.615 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:49.615 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:49.615 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:49.616 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:49.616 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:49.616 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:49.616 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:49.616 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:49.616 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:49.616 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:49.616 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:49.616 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:49.616 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:49.616 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:49.616 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:49.616 INFO:teuthology.orchestra.run.smithi186.stdout: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:49.616 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:49.617 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:49.617 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:49.617 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T04:37:49.617 INFO:teuthology.orchestra.run.smithi186.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T04:37:49.617 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T04:37:49.617 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T04:37:49.617 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T04:37:49.618 INFO:teuthology.orchestra.run.smithi186.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T04:37:49.618 INFO:teuthology.orchestra.run.smithi186.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T04:37:49.618 INFO:teuthology.orchestra.run.smithi186.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T04:37:49.618 INFO:teuthology.orchestra.run.smithi186.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T04:37:49.618 INFO:teuthology.orchestra.run.smithi186.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T04:37:49.618 INFO:teuthology.orchestra.run.smithi186.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T04:37:49.618 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:49.618 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:49.618 INFO:teuthology.orchestra.run.smithi186.stdout: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:49.618 INFO:teuthology.orchestra.run.smithi186.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T04:37:49.619 INFO:teuthology.orchestra.run.smithi186.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T04:37:49.619 INFO:teuthology.orchestra.run.smithi186.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T04:37:49.619 INFO:teuthology.orchestra.run.smithi186.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T04:37:49.619 INFO:teuthology.orchestra.run.smithi186.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T04:37:49.619 INFO:teuthology.orchestra.run.smithi186.stdout: librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:49.619 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:49.619 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T04:37:49.620 INFO:teuthology.orchestra.run.smithi186.stdout: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:49.620 INFO:teuthology.orchestra.run.smithi186.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T04:37:49.620 INFO:teuthology.orchestra.run.smithi186.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T04:37:49.620 INFO:teuthology.orchestra.run.smithi186.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T04:37:49.620 INFO:teuthology.orchestra.run.smithi186.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T04:37:49.620 INFO:teuthology.orchestra.run.smithi186.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T04:37:49.620 INFO:teuthology.orchestra.run.smithi186.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T04:37:49.620 INFO:teuthology.orchestra.run.smithi186.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T04:37:49.620 INFO:teuthology.orchestra.run.smithi186.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T04:37:49.620 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T04:37:49.620 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T04:37:49.620 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T04:37:49.620 INFO:teuthology.orchestra.run.smithi186.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T04:37:49.621 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T04:37:49.621 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T04:37:49.621 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T04:37:49.621 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T04:37:49.621 INFO:teuthology.orchestra.run.smithi186.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T04:37:49.621 INFO:teuthology.orchestra.run.smithi186.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T04:37:49.621 INFO:teuthology.orchestra.run.smithi186.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T04:37:49.621 INFO:teuthology.orchestra.run.smithi186.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T04:37:49.621 INFO:teuthology.orchestra.run.smithi186.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T04:37:49.621 INFO:teuthology.orchestra.run.smithi186.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T04:37:49.621 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T04:37:49.621 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:49.621 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:49.621 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:49.622 INFO:teuthology.orchestra.run.smithi186.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T04:37:49.622 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T04:37:49.622 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T04:37:49.622 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T04:37:49.622 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T04:37:49.622 INFO:teuthology.orchestra.run.smithi186.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T04:37:49.622 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T04:37:49.622 INFO:teuthology.orchestra.run.smithi186.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T04:37:49.622 INFO:teuthology.orchestra.run.smithi186.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T04:37:49.622 INFO:teuthology.orchestra.run.smithi186.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T04:37:49.622 INFO:teuthology.orchestra.run.smithi186.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T04:37:49.622 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T04:37:49.622 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T04:37:49.623 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T04:37:49.623 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T04:37:49.623 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T04:37:49.623 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T04:37:49.623 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T04:37:49.623 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T04:37:49.623 INFO:teuthology.orchestra.run.smithi186.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T04:37:49.623 INFO:teuthology.orchestra.run.smithi186.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T04:37:49.623 INFO:teuthology.orchestra.run.smithi186.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T04:37:49.623 INFO:teuthology.orchestra.run.smithi186.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T04:37:49.623 INFO:teuthology.orchestra.run.smithi186.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T04:37:49.623 INFO:teuthology.orchestra.run.smithi186.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T04:37:49.623 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T04:37:49.623 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T04:37:49.624 INFO:teuthology.orchestra.run.smithi186.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T04:37:49.624 INFO:teuthology.orchestra.run.smithi186.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T04:37:49.624 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T04:37:49.624 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T04:37:49.624 INFO:teuthology.orchestra.run.smithi186.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T04:37:49.624 INFO:teuthology.orchestra.run.smithi186.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T04:37:49.624 INFO:teuthology.orchestra.run.smithi186.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T04:37:49.624 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T04:37:49.624 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T04:37:49.624 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T04:37:49.624 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T04:37:49.624 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T04:37:49.624 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T04:37:49.624 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:49.625 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:49.625 INFO:teuthology.orchestra.run.smithi186.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T04:37:49.625 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T04:37:49.625 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T04:37:49.625 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:49.625 INFO:teuthology.orchestra.run.smithi186.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T04:37:49.625 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T04:37:49.625 INFO:teuthology.orchestra.run.smithi186.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T04:37:49.625 INFO:teuthology.orchestra.run.smithi186.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T04:37:49.625 INFO:teuthology.orchestra.run.smithi186.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T04:37:49.625 INFO:teuthology.orchestra.run.smithi186.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T04:37:49.625 INFO:teuthology.orchestra.run.smithi186.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T04:37:49.625 INFO:teuthology.orchestra.run.smithi186.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T04:37:49.625 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T04:37:49.626 INFO:teuthology.orchestra.run.smithi186.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T04:37:49.626 INFO:teuthology.orchestra.run.smithi186.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T04:37:49.626 INFO:teuthology.orchestra.run.smithi186.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T04:37:49.626 INFO:teuthology.orchestra.run.smithi186.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T04:37:49.626 INFO:teuthology.orchestra.run.smithi186.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T04:37:49.626 INFO:teuthology.orchestra.run.smithi186.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T04:37:49.626 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:49.626 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:49.626 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:49.626 INFO:teuthology.orchestra.run.smithi186.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T04:37:49.626 INFO:teuthology.orchestra.run.smithi186.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T04:37:49.626 INFO:teuthology.orchestra.run.smithi186.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T04:37:49.626 INFO:teuthology.orchestra.run.smithi186.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T04:37:49.627 INFO:teuthology.orchestra.run.smithi186.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T04:37:49.627 INFO:teuthology.orchestra.run.smithi186.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T04:37:49.627 INFO:teuthology.orchestra.run.smithi186.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T04:37:49.627 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-05-09T04:37:49.627 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-05-09T04:37:50.041 DEBUG:teuthology.parallel:result is None 2024-05-09T04:37:50.475 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T04:37:50.476 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /sys 2024-05-09T04:37:50.476 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /proc 2024-05-09T04:37:50.476 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /mnt 2024-05-09T04:37:50.476 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /var/tmp 2024-05-09T04:37:50.476 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /home 2024-05-09T04:37:50.476 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /root 2024-05-09T04:37:50.476 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /tmp 2024-05-09T04:37:50.476 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:37:50.729 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T04:37:50.783 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T04:37:50.783 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:50.783 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T04:37:50.783 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:37:52.354 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T04:37:52.354 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 1/164 2024-05-09T04:37:52.354 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2/164 2024-05-09T04:37:52.354 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 3/164 2024-05-09T04:37:52.354 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 4/164 2024-05-09T04:37:52.354 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 5/164 2024-05-09T04:37:52.354 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 6/164 2024-05-09T04:37:52.354 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 7/164 2024-05-09T04:37:52.354 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T04:37:52.355 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T04:37:52.355 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T04:37:52.355 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 11/164 2024-05-09T04:37:52.355 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 12/164 2024-05-09T04:37:52.355 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T04:37:52.355 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 14/164 2024-05-09T04:37:52.355 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 15/164 2024-05-09T04:37:52.355 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 16/164 2024-05-09T04:37:52.355 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 17/164 2024-05-09T04:37:52.355 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 18/164 2024-05-09T04:37:52.355 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 19/164 2024-05-09T04:37:52.355 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 20/164 2024-05-09T04:37:52.355 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 21/164 2024-05-09T04:37:52.356 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 22/164 2024-05-09T04:37:52.356 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 23/164 2024-05-09T04:37:52.356 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 24/164 2024-05-09T04:37:52.356 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 25/164 2024-05-09T04:37:52.356 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 26/164 2024-05-09T04:37:52.356 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 27/164 2024-05-09T04:37:52.356 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 28/164 2024-05-09T04:37:52.356 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 29/164 2024-05-09T04:37:52.356 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 30/164 2024-05-09T04:37:52.356 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 31/164 2024-05-09T04:37:52.356 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 32/164 2024-05-09T04:37:52.356 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 33/164 2024-05-09T04:37:52.356 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 34/164 2024-05-09T04:37:52.356 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 35/164 2024-05-09T04:37:52.356 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 36/164 2024-05-09T04:37:52.357 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 37/164 2024-05-09T04:37:52.357 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 38/164 2024-05-09T04:37:52.357 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 39/164 2024-05-09T04:37:52.357 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T04:37:52.357 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T04:37:52.357 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T04:37:52.357 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T04:37:52.357 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T04:37:52.357 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T04:37:52.357 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T04:37:52.357 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T04:37:52.357 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T04:37:52.357 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T04:37:52.357 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T04:37:52.358 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T04:37:52.358 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T04:37:52.358 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T04:37:52.358 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T04:37:52.358 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T04:37:52.358 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T04:37:52.358 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T04:37:52.358 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T04:37:52.358 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T04:37:52.358 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T04:37:52.358 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T04:37:52.358 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T04:37:52.358 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T04:37:52.359 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T04:37:52.359 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T04:37:52.359 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T04:37:52.359 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T04:37:52.359 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T04:37:52.359 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T04:37:52.359 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T04:37:52.359 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T04:37:52.359 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T04:37:52.359 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T04:37:52.359 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T04:37:52.359 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T04:37:52.359 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T04:37:52.359 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T04:37:52.360 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T04:37:52.360 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T04:37:52.360 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T04:37:52.360 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T04:37:52.360 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T04:37:52.360 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T04:37:52.360 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T04:37:52.360 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T04:37:52.360 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T04:37:52.360 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T04:37:52.360 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T04:37:52.360 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T04:37:52.360 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T04:37:52.360 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T04:37:52.361 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T04:37:52.361 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T04:37:52.361 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T04:37:52.361 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T04:37:52.361 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T04:37:52.361 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T04:37:52.361 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T04:37:52.361 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T04:37:52.361 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T04:37:52.361 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T04:37:52.361 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T04:37:52.361 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T04:37:52.361 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T04:37:52.361 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T04:37:52.362 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T04:37:52.362 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T04:37:52.362 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T04:37:52.362 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T04:37:52.362 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T04:37:52.362 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T04:37:52.362 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T04:37:52.362 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T04:37:52.362 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T04:37:52.362 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T04:37:52.362 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T04:37:52.362 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T04:37:52.362 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T04:37:52.363 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T04:37:52.363 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T04:37:52.363 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T04:37:52.363 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T04:37:52.363 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T04:37:52.363 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T04:37:52.363 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T04:37:52.363 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T04:37:52.363 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T04:37:52.363 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T04:37:52.363 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T04:37:52.363 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T04:37:52.363 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T04:37:52.363 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T04:37:52.363 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T04:37:52.364 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T04:37:52.364 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T04:37:52.364 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T04:37:52.364 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T04:37:52.364 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T04:37:52.364 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T04:37:52.364 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T04:37:52.364 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T04:37:52.364 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T04:37:52.364 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T04:37:52.364 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T04:37:52.364 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T04:37:52.364 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T04:37:52.364 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T04:37:52.365 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T04:37:52.365 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 149/164 2024-05-09T04:37:52.365 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T04:37:52.365 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 151/164 2024-05-09T04:37:52.365 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T04:37:52.365 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T04:37:52.365 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T04:37:52.365 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T04:37:52.365 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T04:37:52.365 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T04:37:52.365 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T04:37:52.365 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T04:37:52.365 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T04:37:52.365 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T04:37:52.365 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T04:37:52.365 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T04:37:52.606 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T04:37:52.694 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T04:37:52.694 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:52.694 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T04:37:52.694 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T04:37:52.694 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T04:37:52.694 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:37:52.973 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T04:37:53.070 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T04:37:53.070 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:53.070 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T04:37:53.070 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T04:37:53.070 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T04:37:53.071 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:37:53.615 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T04:37:53.709 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T04:37:53.709 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:53.709 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T04:37:53.709 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T04:37:53.709 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T04:37:53.709 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:37:53.884 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T04:37:53.964 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T04:37:54.055 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T04:37:54.318 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T04:37:54.380 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T04:37:54.472 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T04:37:54.607 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T04:37:54.790 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T04:37:54.844 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T04:37:54.845 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-09T04:37:54.845 INFO:teuthology.orchestra.run.smithi169.stdout:Upgraded: 2024-05-09T04:37:54.845 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T04:37:54.845 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T04:37:54.845 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T04:37:54.845 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T04:37:54.845 INFO:teuthology.orchestra.run.smithi169.stdout: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.845 INFO:teuthology.orchestra.run.smithi169.stdout: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.845 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T04:37:54.845 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T04:37:54.845 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-05-09T04:37:54.845 INFO:teuthology.orchestra.run.smithi169.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T04:37:54.845 INFO:teuthology.orchestra.run.smithi169.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T04:37:54.846 INFO:teuthology.orchestra.run.smithi169.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T04:37:54.846 INFO:teuthology.orchestra.run.smithi169.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T04:37:54.846 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.846 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.846 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.846 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.846 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:54.846 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.846 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.846 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.846 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:54.846 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:54.846 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:54.846 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:54.847 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:54.847 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.847 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.847 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:54.847 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.847 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.847 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.847 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:54.847 INFO:teuthology.orchestra.run.smithi169.stdout: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:54.847 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.847 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.847 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:54.847 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T04:37:54.847 INFO:teuthology.orchestra.run.smithi169.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T04:37:54.847 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T04:37:54.848 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T04:37:54.848 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T04:37:54.848 INFO:teuthology.orchestra.run.smithi169.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T04:37:54.848 INFO:teuthology.orchestra.run.smithi169.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T04:37:54.848 INFO:teuthology.orchestra.run.smithi169.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T04:37:54.848 INFO:teuthology.orchestra.run.smithi169.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T04:37:54.848 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T04:37:54.848 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T04:37:54.848 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.848 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.848 INFO:teuthology.orchestra.run.smithi169.stdout: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.848 INFO:teuthology.orchestra.run.smithi169.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T04:37:54.848 INFO:teuthology.orchestra.run.smithi169.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T04:37:54.849 INFO:teuthology.orchestra.run.smithi169.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T04:37:54.849 INFO:teuthology.orchestra.run.smithi169.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T04:37:54.849 INFO:teuthology.orchestra.run.smithi169.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T04:37:54.849 INFO:teuthology.orchestra.run.smithi169.stdout: librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.849 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.849 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T04:37:54.849 INFO:teuthology.orchestra.run.smithi169.stdout: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.849 INFO:teuthology.orchestra.run.smithi169.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T04:37:54.849 INFO:teuthology.orchestra.run.smithi169.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T04:37:54.849 INFO:teuthology.orchestra.run.smithi169.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T04:37:54.849 INFO:teuthology.orchestra.run.smithi169.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T04:37:54.849 INFO:teuthology.orchestra.run.smithi169.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T04:37:54.849 INFO:teuthology.orchestra.run.smithi169.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T04:37:54.849 INFO:teuthology.orchestra.run.smithi169.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T04:37:54.850 INFO:teuthology.orchestra.run.smithi169.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T04:37:54.850 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T04:37:54.850 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T04:37:54.850 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T04:37:54.850 INFO:teuthology.orchestra.run.smithi169.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T04:37:54.850 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T04:37:54.850 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T04:37:54.850 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T04:37:54.850 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T04:37:54.850 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T04:37:54.850 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T04:37:54.850 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T04:37:54.850 INFO:teuthology.orchestra.run.smithi169.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T04:37:54.850 INFO:teuthology.orchestra.run.smithi169.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T04:37:54.851 INFO:teuthology.orchestra.run.smithi169.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T04:37:54.851 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T04:37:54.851 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.851 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.851 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.851 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T04:37:54.851 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T04:37:54.851 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T04:37:54.851 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T04:37:54.851 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T04:37:54.851 INFO:teuthology.orchestra.run.smithi169.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T04:37:54.852 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T04:37:54.852 INFO:teuthology.orchestra.run.smithi169.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T04:37:54.852 INFO:teuthology.orchestra.run.smithi169.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T04:37:54.852 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T04:37:54.852 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T04:37:54.852 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T04:37:54.852 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T04:37:54.852 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T04:37:54.852 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T04:37:54.852 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T04:37:54.852 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T04:37:54.852 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T04:37:54.852 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T04:37:54.852 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T04:37:54.853 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T04:37:54.853 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T04:37:54.853 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T04:37:54.853 INFO:teuthology.orchestra.run.smithi169.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T04:37:54.853 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T04:37:54.853 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T04:37:54.853 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T04:37:54.853 INFO:teuthology.orchestra.run.smithi169.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T04:37:54.853 INFO:teuthology.orchestra.run.smithi169.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T04:37:54.853 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T04:37:54.853 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T04:37:54.853 INFO:teuthology.orchestra.run.smithi169.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T04:37:54.853 INFO:teuthology.orchestra.run.smithi169.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T04:37:54.853 INFO:teuthology.orchestra.run.smithi169.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T04:37:54.853 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T04:37:54.854 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T04:37:54.854 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T04:37:54.854 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T04:37:54.854 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T04:37:54.854 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T04:37:54.854 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.854 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.854 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T04:37:54.854 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T04:37:54.854 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T04:37:54.854 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.854 INFO:teuthology.orchestra.run.smithi169.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T04:37:54.854 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T04:37:54.855 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T04:37:54.855 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T04:37:54.855 INFO:teuthology.orchestra.run.smithi169.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T04:37:54.855 INFO:teuthology.orchestra.run.smithi169.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T04:37:54.855 INFO:teuthology.orchestra.run.smithi169.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T04:37:54.855 INFO:teuthology.orchestra.run.smithi169.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T04:37:54.855 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T04:37:54.855 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T04:37:54.855 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T04:37:54.855 INFO:teuthology.orchestra.run.smithi169.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T04:37:54.855 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T04:37:54.855 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T04:37:54.855 INFO:teuthology.orchestra.run.smithi169.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T04:37:54.855 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.856 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.856 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.856 INFO:teuthology.orchestra.run.smithi169.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T04:37:54.856 INFO:teuthology.orchestra.run.smithi169.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T04:37:54.856 INFO:teuthology.orchestra.run.smithi169.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T04:37:54.856 INFO:teuthology.orchestra.run.smithi169.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T04:37:54.856 INFO:teuthology.orchestra.run.smithi169.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T04:37:54.856 INFO:teuthology.orchestra.run.smithi169.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T04:37:54.857 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T04:37:54.857 INFO:teuthology.orchestra.run.smithi169.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T04:37:54.857 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-09T04:37:54.857 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-05-09T04:37:54.888 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T04:37:54.919 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T04:37:55.249 DEBUG:teuthology.parallel:result is None 2024-05-09T04:37:55.582 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T04:37:55.614 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T04:37:55.670 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 126/164 2024-05-09T04:37:55.796 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 127/164 2024-05-09T04:37:55.840 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T04:37:57.214 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T04:37:57.259 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T04:37:58.426 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T04:37:58.459 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T04:37:58.624 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T04:37:58.813 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 131/164 2024-05-09T04:37:58.956 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T04:37:58.956 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T04:37:58.956 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:58.956 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T04:37:58.956 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T04:37:58.956 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T04:37:58.956 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:37:59.004 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T04:37:59.043 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T04:37:59.160 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 134/164 2024-05-09T04:37:59.236 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 135/164 2024-05-09T04:38:00.321 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T04:38:00.365 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T04:38:00.465 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T04:38:00.465 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:38:00.465 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T04:38:00.465 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T04:38:00.465 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T04:38:00.465 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:38:00.515 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T04:38:00.588 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T04:38:00.589 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:38:00.589 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T04:38:00.589 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:38:00.658 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T04:38:00.733 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T04:38:00.733 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:38:00.733 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T04:38:00.733 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T04:38:00.733 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T04:38:00.733 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:38:01.134 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T04:38:01.208 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T04:38:01.208 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:38:01.208 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T04:38:01.208 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T04:38:01.208 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T04:38:01.208 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:38:06.114 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 141/164 2024-05-09T04:38:06.165 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 142/164 2024-05-09T04:38:06.221 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 143/164 2024-05-09T04:38:06.263 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T04:38:06.309 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T04:38:06.630 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 146/164 2024-05-09T04:38:06.694 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T04:38:06.722 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 148/164 2024-05-09T04:38:06.759 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T04:38:06.808 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T04:38:07.765 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T04:38:07.813 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T04:38:07.885 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T04:38:09.307 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 152/164 2024-05-09T04:38:09.429 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T04:38:09.503 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T04:38:09.680 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T04:38:09.722 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T04:38:09.760 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T04:38:09.760 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T04:38:09.896 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T04:38:09.942 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T04:38:09.943 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T04:38:10.089 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T04:38:10.127 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T04:38:10.164 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T04:38:10.164 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T04:38:10.540 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T04:38:10.540 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T04:38:10.668 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T04:38:10.803 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T04:38:11.084 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T04:38:11.084 INFO:teuthology.orchestra.run.smithi006.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-09T04:38:11.085 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:38:16.999 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 1/164 2024-05-09T04:38:16.999 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2/164 2024-05-09T04:38:16.999 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 3/164 2024-05-09T04:38:16.999 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 4/164 2024-05-09T04:38:16.999 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 5/164 2024-05-09T04:38:17.000 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 6/164 2024-05-09T04:38:17.000 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 7/164 2024-05-09T04:38:17.000 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T04:38:17.000 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T04:38:17.000 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T04:38:17.000 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 11/164 2024-05-09T04:38:17.000 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 12/164 2024-05-09T04:38:17.000 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T04:38:17.000 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 14/164 2024-05-09T04:38:17.000 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 15/164 2024-05-09T04:38:17.000 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 16/164 2024-05-09T04:38:17.000 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 17/164 2024-05-09T04:38:17.000 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 18/164 2024-05-09T04:38:17.001 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 19/164 2024-05-09T04:38:17.001 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 20/164 2024-05-09T04:38:17.002 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 21/164 2024-05-09T04:38:17.002 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 22/164 2024-05-09T04:38:17.002 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 23/164 2024-05-09T04:38:17.002 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 24/164 2024-05-09T04:38:17.002 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 25/164 2024-05-09T04:38:17.002 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 26/164 2024-05-09T04:38:17.002 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 27/164 2024-05-09T04:38:17.002 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 28/164 2024-05-09T04:38:17.002 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 29/164 2024-05-09T04:38:17.002 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 30/164 2024-05-09T04:38:17.002 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 31/164 2024-05-09T04:38:17.003 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 32/164 2024-05-09T04:38:17.003 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 33/164 2024-05-09T04:38:17.003 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 34/164 2024-05-09T04:38:17.003 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 35/164 2024-05-09T04:38:17.003 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 36/164 2024-05-09T04:38:17.003 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 37/164 2024-05-09T04:38:17.003 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 38/164 2024-05-09T04:38:17.003 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 39/164 2024-05-09T04:38:17.003 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T04:38:17.003 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T04:38:17.003 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T04:38:17.003 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T04:38:17.003 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T04:38:17.004 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T04:38:17.004 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T04:38:17.004 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T04:38:17.004 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T04:38:17.004 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T04:38:17.004 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T04:38:17.004 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T04:38:17.004 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T04:38:17.004 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T04:38:17.004 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T04:38:17.004 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T04:38:17.004 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T04:38:17.004 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T04:38:17.004 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T04:38:17.005 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T04:38:17.005 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T04:38:17.005 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T04:38:17.005 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T04:38:17.005 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T04:38:17.005 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T04:38:17.005 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T04:38:17.005 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T04:38:17.005 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T04:38:17.005 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T04:38:17.005 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T04:38:17.005 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T04:38:17.005 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T04:38:17.006 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T04:38:17.006 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T04:38:17.006 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T04:38:17.006 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T04:38:17.006 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T04:38:17.006 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T04:38:17.006 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T04:38:17.006 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T04:38:17.006 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T04:38:17.006 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T04:38:17.006 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T04:38:17.006 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T04:38:17.007 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T04:38:17.007 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T04:38:17.007 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T04:38:17.007 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T04:38:17.007 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T04:38:17.007 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T04:38:17.007 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T04:38:17.007 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T04:38:17.007 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T04:38:17.007 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T04:38:17.007 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T04:38:17.007 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T04:38:17.007 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T04:38:17.007 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T04:38:17.008 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T04:38:17.008 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T04:38:17.008 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T04:38:17.008 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T04:38:17.008 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T04:38:17.008 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T04:38:17.008 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T04:38:17.008 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T04:38:17.008 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T04:38:17.008 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T04:38:17.008 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T04:38:17.008 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T04:38:17.008 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T04:38:17.009 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T04:38:17.009 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T04:38:17.009 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T04:38:17.009 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T04:38:17.009 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T04:38:17.009 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T04:38:17.009 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T04:38:17.009 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T04:38:17.009 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T04:38:17.009 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T04:38:17.009 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T04:38:17.009 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T04:38:17.009 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T04:38:17.009 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T04:38:17.010 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T04:38:17.010 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T04:38:17.010 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T04:38:17.010 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T04:38:17.010 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T04:38:17.010 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T04:38:17.010 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T04:38:17.010 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T04:38:17.010 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T04:38:17.010 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T04:38:17.010 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T04:38:17.010 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T04:38:17.010 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T04:38:17.010 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T04:38:17.011 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T04:38:17.011 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T04:38:17.011 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T04:38:17.011 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T04:38:17.011 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T04:38:17.011 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T04:38:17.011 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T04:38:17.011 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T04:38:17.011 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T04:38:17.011 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T04:38:17.011 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 149/164 2024-05-09T04:38:17.011 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T04:38:17.011 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 151/164 2024-05-09T04:38:17.011 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T04:38:17.012 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T04:38:17.012 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T04:38:17.012 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T04:38:17.012 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T04:38:17.012 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T04:38:17.012 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T04:38:17.012 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T04:38:17.012 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T04:38:17.012 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T04:38:17.012 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T04:38:17.012 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T04:38:19.371 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T04:38:19.371 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:38:19.372 INFO:teuthology.orchestra.run.smithi006.stdout:Upgraded: 2024-05-09T04:38:19.372 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T04:38:19.372 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T04:38:19.372 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T04:38:19.372 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T04:38:19.372 INFO:teuthology.orchestra.run.smithi006.stdout: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:19.372 INFO:teuthology.orchestra.run.smithi006.stdout: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:19.372 INFO:teuthology.orchestra.run.smithi006.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T04:38:19.372 INFO:teuthology.orchestra.run.smithi006.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T04:38:19.372 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-05-09T04:38:19.372 INFO:teuthology.orchestra.run.smithi006.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T04:38:19.372 INFO:teuthology.orchestra.run.smithi006.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T04:38:19.372 INFO:teuthology.orchestra.run.smithi006.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T04:38:19.373 INFO:teuthology.orchestra.run.smithi006.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T04:38:19.373 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:19.373 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:19.373 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:19.373 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:19.373 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:38:19.373 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:19.373 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:19.373 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:19.373 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:38:19.373 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:38:19.373 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:38:19.373 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:38:19.374 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:38:19.374 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:19.374 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:19.374 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:38:19.374 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:19.374 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:19.374 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:19.374 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:38:19.374 INFO:teuthology.orchestra.run.smithi006.stdout: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:38:19.374 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:19.374 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:19.374 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:38:19.374 INFO:teuthology.orchestra.run.smithi006.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T04:38:19.374 INFO:teuthology.orchestra.run.smithi006.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T04:38:19.375 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T04:38:19.375 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T04:38:19.375 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T04:38:19.375 INFO:teuthology.orchestra.run.smithi006.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T04:38:19.375 INFO:teuthology.orchestra.run.smithi006.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T04:38:19.375 INFO:teuthology.orchestra.run.smithi006.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T04:38:19.375 INFO:teuthology.orchestra.run.smithi006.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T04:38:19.375 INFO:teuthology.orchestra.run.smithi006.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T04:38:19.375 INFO:teuthology.orchestra.run.smithi006.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T04:38:19.375 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:19.375 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:19.375 INFO:teuthology.orchestra.run.smithi006.stdout: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:19.375 INFO:teuthology.orchestra.run.smithi006.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T04:38:19.375 INFO:teuthology.orchestra.run.smithi006.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T04:38:19.376 INFO:teuthology.orchestra.run.smithi006.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T04:38:19.376 INFO:teuthology.orchestra.run.smithi006.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T04:38:19.376 INFO:teuthology.orchestra.run.smithi006.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T04:38:19.376 INFO:teuthology.orchestra.run.smithi006.stdout: librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:19.376 INFO:teuthology.orchestra.run.smithi006.stdout: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:19.376 INFO:teuthology.orchestra.run.smithi006.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T04:38:19.376 INFO:teuthology.orchestra.run.smithi006.stdout: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:19.376 INFO:teuthology.orchestra.run.smithi006.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T04:38:19.376 INFO:teuthology.orchestra.run.smithi006.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T04:38:19.376 INFO:teuthology.orchestra.run.smithi006.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T04:38:19.376 INFO:teuthology.orchestra.run.smithi006.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T04:38:19.376 INFO:teuthology.orchestra.run.smithi006.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T04:38:19.376 INFO:teuthology.orchestra.run.smithi006.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T04:38:19.376 INFO:teuthology.orchestra.run.smithi006.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T04:38:19.377 INFO:teuthology.orchestra.run.smithi006.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T04:38:19.377 INFO:teuthology.orchestra.run.smithi006.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T04:38:19.377 INFO:teuthology.orchestra.run.smithi006.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T04:38:19.377 INFO:teuthology.orchestra.run.smithi006.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T04:38:19.377 INFO:teuthology.orchestra.run.smithi006.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T04:38:19.377 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T04:38:19.377 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T04:38:19.377 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T04:38:19.377 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T04:38:19.377 INFO:teuthology.orchestra.run.smithi006.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T04:38:19.377 INFO:teuthology.orchestra.run.smithi006.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T04:38:19.377 INFO:teuthology.orchestra.run.smithi006.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T04:38:19.378 INFO:teuthology.orchestra.run.smithi006.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T04:38:19.378 INFO:teuthology.orchestra.run.smithi006.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T04:38:19.378 INFO:teuthology.orchestra.run.smithi006.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T04:38:19.378 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T04:38:19.378 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:19.378 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:19.378 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:19.378 INFO:teuthology.orchestra.run.smithi006.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T04:38:19.378 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T04:38:19.378 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T04:38:19.379 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T04:38:19.379 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T04:38:19.379 INFO:teuthology.orchestra.run.smithi006.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T04:38:19.379 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T04:38:19.379 INFO:teuthology.orchestra.run.smithi006.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T04:38:19.379 INFO:teuthology.orchestra.run.smithi006.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T04:38:19.379 INFO:teuthology.orchestra.run.smithi006.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T04:38:19.379 INFO:teuthology.orchestra.run.smithi006.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T04:38:19.379 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T04:38:19.379 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T04:38:19.379 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T04:38:19.379 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T04:38:19.379 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T04:38:19.379 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T04:38:19.380 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T04:38:19.380 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T04:38:19.380 INFO:teuthology.orchestra.run.smithi006.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T04:38:19.380 INFO:teuthology.orchestra.run.smithi006.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T04:38:19.380 INFO:teuthology.orchestra.run.smithi006.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T04:38:19.380 INFO:teuthology.orchestra.run.smithi006.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T04:38:19.380 INFO:teuthology.orchestra.run.smithi006.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T04:38:19.380 INFO:teuthology.orchestra.run.smithi006.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T04:38:19.380 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T04:38:19.380 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T04:38:19.380 INFO:teuthology.orchestra.run.smithi006.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T04:38:19.380 INFO:teuthology.orchestra.run.smithi006.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T04:38:19.380 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T04:38:19.380 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T04:38:19.380 INFO:teuthology.orchestra.run.smithi006.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T04:38:19.381 INFO:teuthology.orchestra.run.smithi006.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T04:38:19.381 INFO:teuthology.orchestra.run.smithi006.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T04:38:19.381 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T04:38:19.381 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T04:38:19.381 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T04:38:19.381 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T04:38:19.381 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T04:38:19.381 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T04:38:19.382 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:19.382 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:19.382 INFO:teuthology.orchestra.run.smithi006.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T04:38:19.382 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T04:38:19.382 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T04:38:19.382 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:19.382 INFO:teuthology.orchestra.run.smithi006.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T04:38:19.382 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T04:38:19.382 INFO:teuthology.orchestra.run.smithi006.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T04:38:19.382 INFO:teuthology.orchestra.run.smithi006.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T04:38:19.382 INFO:teuthology.orchestra.run.smithi006.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T04:38:19.382 INFO:teuthology.orchestra.run.smithi006.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T04:38:19.382 INFO:teuthology.orchestra.run.smithi006.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T04:38:19.383 INFO:teuthology.orchestra.run.smithi006.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T04:38:19.383 INFO:teuthology.orchestra.run.smithi006.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T04:38:19.383 INFO:teuthology.orchestra.run.smithi006.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T04:38:19.383 INFO:teuthology.orchestra.run.smithi006.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T04:38:19.383 INFO:teuthology.orchestra.run.smithi006.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T04:38:19.383 INFO:teuthology.orchestra.run.smithi006.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T04:38:19.383 INFO:teuthology.orchestra.run.smithi006.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T04:38:19.383 INFO:teuthology.orchestra.run.smithi006.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T04:38:19.383 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:19.383 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:19.383 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:19.383 INFO:teuthology.orchestra.run.smithi006.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T04:38:19.383 INFO:teuthology.orchestra.run.smithi006.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T04:38:19.383 INFO:teuthology.orchestra.run.smithi006.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T04:38:19.384 INFO:teuthology.orchestra.run.smithi006.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T04:38:19.384 INFO:teuthology.orchestra.run.smithi006.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T04:38:19.384 INFO:teuthology.orchestra.run.smithi006.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T04:38:19.384 INFO:teuthology.orchestra.run.smithi006.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T04:38:19.384 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:38:19.384 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-05-09T04:38:20.041 DEBUG:teuthology.parallel:result is None 2024-05-09T04:38:20.042 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T04:38:20.205 DEBUG:teuthology.orchestra.run.smithi006:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T04:38:20.264 INFO:teuthology.orchestra.run.smithi006.stdout:19.0.0-3594.gc9da4049.el9 2024-05-09T04:38:20.265 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3594.gc9da4049.el9 2024-05-09T04:38:20.265 INFO:teuthology.task.install:The correct ceph version 19.0.0-3594.gc9da4049 is installed. 2024-05-09T04:38:20.267 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T04:38:20.486 DEBUG:teuthology.orchestra.run.smithi169:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T04:38:20.512 INFO:teuthology.orchestra.run.smithi169.stdout:19.0.0-3594.gc9da4049.el9 2024-05-09T04:38:20.512 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3594.gc9da4049.el9 2024-05-09T04:38:20.512 INFO:teuthology.task.install:The correct ceph version 19.0.0-3594.gc9da4049 is installed. 2024-05-09T04:38:20.514 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T04:38:20.743 DEBUG:teuthology.orchestra.run.smithi186:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T04:38:20.769 INFO:teuthology.orchestra.run.smithi186.stdout:19.0.0-3594.gc9da4049.el9 2024-05-09T04:38:20.769 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3594.gc9da4049.el9 2024-05-09T04:38:20.769 INFO:teuthology.task.install:The correct ceph version 19.0.0-3594.gc9da4049 is installed. 2024-05-09T04:38:20.771 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-09T04:38:20.771 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T04:38:20.771 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T04:38:20.845 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-09T04:38:20.845 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T04:38:20.876 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-05-09T04:38:20.876 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T04:38:20.906 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-09T04:38:20.906 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T04:38:20.906 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T04:38:20.975 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T04:38:21.086 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-09T04:38:21.086 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T04:38:21.115 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T04:38:21.183 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-05-09T04:38:21.183 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T04:38:21.207 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T04:38:21.272 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-09T04:38:21.272 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T04:38:21.272 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T04:38:21.340 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T04:38:21.448 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-09T04:38:21.449 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T04:38:21.472 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T04:38:21.538 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-05-09T04:38:21.538 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T04:38:21.563 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T04:38:21.627 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-09T04:38:21.627 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T04:38:21.627 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T04:38:21.696 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T04:38:21.801 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-09T04:38:21.801 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T04:38:21.825 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T04:38:21.889 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-05-09T04:38:21.889 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T04:38:21.912 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T04:38:21.977 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-09T04:38:22.079 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-05-09T04:38:22.080 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T04:38:22.080 INFO:tasks.cephadm:Cluster fsid is f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:38:22.080 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-09T04:38:22.081 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.6', 'mon.b': '172.21.15.169', 'mon.c': '172.21.15.186'} 2024-05-09T04:38:22.081 INFO:tasks.cephadm:First mon is mon.a on smithi006 2024-05-09T04:38:22.081 INFO:tasks.cephadm:First mgr is x 2024-05-09T04:38:22.081 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-09T04:38:22.081 DEBUG:teuthology.orchestra.run.smithi006:> sudo hostname $(hostname -s) 2024-05-09T04:38:22.155 DEBUG:teuthology.orchestra.run.smithi169:> sudo hostname $(hostname -s) 2024-05-09T04:38:22.179 DEBUG:teuthology.orchestra.run.smithi186:> sudo hostname $(hostname -s) 2024-05-09T04:38:22.202 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-09T04:38:22.203 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T04:38:22.419 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79226/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3594-gc9da4049', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3594.gc9da4049'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-09 00:57:42.125322', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240508.183908-debug', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79224/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3594-gc9da4049', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3594.gc9da4049'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-08 19:52:58.989794', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240508.183908-debug', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-05-09T04:38:22.563 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-pdonnell-testing-20240508.183908-debug, sha1 c9da4049f638c0f84bdd9b4e33c42e86046ac554 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T04:38:22.565 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/x86_64/flavors/default/cephadm 2024-05-09T04:38:22.565 DEBUG:teuthology.orchestra.run.smithi006:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-09T04:38:23.213 INFO:teuthology.orchestra.run.smithi006.stdout:-rw-r--r--. 1 ubuntu ubuntu 782864 May 9 04:38 /home/ubuntu/cephtest/cephadm 2024-05-09T04:38:23.213 DEBUG:teuthology.orchestra.run.smithi169:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-09T04:38:23.963 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r--. 1 ubuntu ubuntu 782864 May 9 04:38 /home/ubuntu/cephtest/cephadm 2024-05-09T04:38:23.963 DEBUG:teuthology.orchestra.run.smithi186:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-09T04:38:25.105 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r--. 1 ubuntu ubuntu 782864 May 9 04:38 /home/ubuntu/cephtest/cephadm 2024-05-09T04:38:25.106 DEBUG:teuthology.orchestra.run.smithi006:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-09T04:38:25.166 DEBUG:teuthology.orchestra.run.smithi169:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-09T04:38:25.185 DEBUG:teuthology.orchestra.run.smithi186:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-09T04:38:25.220 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 on all hosts... 2024-05-09T04:38:25.220 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 pull 2024-05-09T04:38:25.227 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 pull 2024-05-09T04:38:25.230 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 pull 2024-05-09T04:38:26.804 INFO:teuthology.orchestra.run.smithi169.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-09T04:38:26.807 INFO:teuthology.orchestra.run.smithi186.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-09T04:38:27.327 INFO:teuthology.orchestra.run.smithi006.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-09T04:39:18.898 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-09T04:39:18.899 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_version": "ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev)", 2024-05-09T04:39:18.899 INFO:teuthology.orchestra.run.smithi006.stdout: "image_id": "db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7", 2024-05-09T04:39:18.899 INFO:teuthology.orchestra.run.smithi006.stdout: "repo_digests": [ 2024-05-09T04:39:18.899 INFO:teuthology.orchestra.run.smithi006.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584" 2024-05-09T04:39:18.899 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2024-05-09T04:39:18.899 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-09T04:39:21.871 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-05-09T04:39:21.872 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_version": "ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev)", 2024-05-09T04:39:21.872 INFO:teuthology.orchestra.run.smithi169.stdout: "image_id": "db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7", 2024-05-09T04:39:21.872 INFO:teuthology.orchestra.run.smithi169.stdout: "repo_digests": [ 2024-05-09T04:39:21.872 INFO:teuthology.orchestra.run.smithi169.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584" 2024-05-09T04:39:21.872 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2024-05-09T04:39:21.872 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-05-09T04:39:22.052 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2024-05-09T04:39:22.053 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_version": "ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev)", 2024-05-09T04:39:22.053 INFO:teuthology.orchestra.run.smithi186.stdout: "image_id": "db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7", 2024-05-09T04:39:22.053 INFO:teuthology.orchestra.run.smithi186.stdout: "repo_digests": [ 2024-05-09T04:39:22.053 INFO:teuthology.orchestra.run.smithi186.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584" 2024-05-09T04:39:22.053 INFO:teuthology.orchestra.run.smithi186.stdout: ] 2024-05-09T04:39:22.053 INFO:teuthology.orchestra.run.smithi186.stdout:} 2024-05-09T04:39:22.080 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /etc/ceph 2024-05-09T04:39:22.179 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /etc/ceph 2024-05-09T04:39:22.205 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /etc/ceph 2024-05-09T04:39:22.233 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 777 /etc/ceph 2024-05-09T04:39:22.324 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 777 /etc/ceph 2024-05-09T04:39:22.349 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 777 /etc/ceph 2024-05-09T04:39:22.374 INFO:tasks.cephadm:Writing seed config... 2024-05-09T04:39:22.375 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-09T04:39:22.375 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-09T04:39:22.375 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-09T04:39:22.375 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-09T04:39:22.375 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-09T04:39:22.375 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-09T04:39:22.376 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-09T04:39:22.376 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-09T04:39:22.376 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-09T04:39:22.376 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-09T04:39:22.376 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-09T04:39:22.376 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-09T04:39:22.376 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-09T04:39:22.376 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-09T04:39:22.376 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-09T04:39:22.376 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-09T04:39:22.376 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-09T04:39:22.376 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-05-09T04:39:22.376 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-05-09T04:39:22.376 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-09T04:39:22.376 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-09T04:39:22.377 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-09T04:39:22.377 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-09T04:39:22.377 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-09T04:39:22.377 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-09T04:39:22.377 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-09T04:39:22.377 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-09T04:39:22.377 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-09T04:39:22.377 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-09T04:39:22.377 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-09T04:39:22.377 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-09T04:39:22.377 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-09T04:39:22.377 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-09T04:39:22.377 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-09T04:39:22.378 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-09T04:39:22.378 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-09T04:39:22.378 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-09T04:39:22.378 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-09T04:39:22.378 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-09T04:39:22.378 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-09T04:39:22.378 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-09T04:39:22.378 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-09T04:39:22.378 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-09T04:39:22.378 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-09T04:39:22.378 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-09T04:39:22.378 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-05-09T04:39:22.379 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T04:39:22.379 DEBUG:teuthology.orchestra.run.smithi006:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-09T04:39:22.439 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = f74d813c-0dbd-11ef-bc97-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-09T04:39:22.439 DEBUG:teuthology.orchestra.run.smithi006:mon.a> sudo journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mon.a.service 2024-05-09T04:39:22.488 DEBUG:teuthology.orchestra.run.smithi006:mgr.x> sudo journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mgr.x.service 2024-05-09T04:39:22.534 INFO:tasks.cephadm:Bootstrapping... 2024-05-09T04:39:22.534 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 -v bootstrap --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.6 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-09T04:39:22.812 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-05-09T04:39:22.812 INFO:teuthology.orchestra.run.smithi006.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554', '-v', 'bootstrap', '--fsid', 'f74d813c-0dbd-11ef-bc97-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.6', '--skip-admin-label'] 2024-05-09T04:39:22.813 INFO:teuthology.orchestra.run.smithi006.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-09T04:39:22.813 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying podman|docker is present... 2024-05-09T04:39:22.872 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-09T04:39:22.873 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying lvm2 is present... 2024-05-09T04:39:22.873 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying time synchronization is in place... 2024-05-09T04:39:22.889 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-09T04:39:22.889 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-09T04:39:22.904 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-09T04:39:22.904 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2024-05-09T04:39:22.922 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout enabled 2024-05-09T04:39:22.939 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout active 2024-05-09T04:39:22.939 INFO:teuthology.orchestra.run.smithi006.stdout:Unit chronyd.service is enabled and running 2024-05-09T04:39:22.939 INFO:teuthology.orchestra.run.smithi006.stdout:Repeating the final host check... 2024-05-09T04:39:22.994 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-09T04:39:22.994 INFO:teuthology.orchestra.run.smithi006.stdout:podman (/bin/podman) version 4.9.4 is present 2024-05-09T04:39:22.995 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl is present 2024-05-09T04:39:22.995 INFO:teuthology.orchestra.run.smithi006.stdout:lvcreate is present 2024-05-09T04:39:23.012 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-09T04:39:23.012 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-09T04:39:23.031 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-09T04:39:23.032 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2024-05-09T04:39:23.057 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout enabled 2024-05-09T04:39:23.080 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout active 2024-05-09T04:39:23.080 INFO:teuthology.orchestra.run.smithi006.stdout:Unit chronyd.service is enabled and running 2024-05-09T04:39:23.081 INFO:teuthology.orchestra.run.smithi006.stdout:Host looks OK 2024-05-09T04:39:23.081 INFO:teuthology.orchestra.run.smithi006.stdout:Cluster fsid: f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:39:23.082 INFO:teuthology.orchestra.run.smithi006.stdout:Acquiring lock 140148851518912 on /run/cephadm/f74d813c-0dbd-11ef-bc97-c7b262605968.lock 2024-05-09T04:39:23.082 INFO:teuthology.orchestra.run.smithi006.stdout:Lock 140148851518912 acquired on /run/cephadm/f74d813c-0dbd-11ef-bc97-c7b262605968.lock 2024-05-09T04:39:23.083 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying IP 172.21.15.6 port 3300 ... 2024-05-09T04:39:23.084 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying IP 172.21.15.6 port 6789 ... 2024-05-09T04:39:23.085 INFO:teuthology.orchestra.run.smithi006.stdout:Base mon IP(s) is [172.21.15.6:3300, 172.21.15.6:6789], mon addrv is [v2:172.21.15.6:3300,v1:172.21.15.6:6789] 2024-05-09T04:39:23.100 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.6 metric 100 2024-05-09T04:39:23.101 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.6 metric 100 2024-05-09T04:39:23.113 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-09T04:39:23.114 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-05-09T04:39:23.114 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-05-09T04:39:23.126 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-09T04:39:23.126 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-09T04:39:23.126 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-09T04:39:23.126 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-09T04:39:23.127 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15e4/64 scope link noprefixroute 2024-05-09T04:39:23.127 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-09T04:39:23.129 INFO:teuthology.orchestra.run.smithi006.stdout:Mon IP `172.21.15.6` is in CIDR network `172.21.0.0/20` 2024-05-09T04:39:23.130 INFO:teuthology.orchestra.run.smithi006.stdout:Mon IP `172.21.15.6` is in CIDR network `172.21.0.0/20` 2024-05-09T04:39:23.130 INFO:teuthology.orchestra.run.smithi006.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-09T04:39:23.133 INFO:teuthology.orchestra.run.smithi006.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-09T04:39:23.136 INFO:teuthology.orchestra.run.smithi006.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-09T04:39:23.783 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stdout db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 2024-05-09T04:39:23.783 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-09T04:39:23.783 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Getting image source signatures 2024-05-09T04:39:23.783 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Copying blob sha256:0411e0fd7a5d45f8ab31b23cec89f97c70494de39461f644c7cdc77de3b9cbca 2024-05-09T04:39:23.784 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Copying blob sha256:bf5b95bd68251da39c6cd2eb96c6ba4138bb37a62bdc39287908b25e92047d7e 2024-05-09T04:39:23.784 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Copying config sha256:db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 2024-05-09T04:39:23.784 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-09T04:39:25.179 INFO:teuthology.orchestra.run.smithi006.stdout:ceph: stdout ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev) 2024-05-09T04:39:25.180 INFO:teuthology.orchestra.run.smithi006.stdout:Ceph version: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev) 2024-05-09T04:39:25.180 INFO:teuthology.orchestra.run.smithi006.stdout:Extracting ceph user uid/gid from container image... 2024-05-09T04:39:26.342 INFO:teuthology.orchestra.run.smithi006.stdout:stat: stdout 167 167 2024-05-09T04:39:26.342 INFO:teuthology.orchestra.run.smithi006.stdout:Creating initial keys... 2024-05-09T04:39:27.899 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph-authtool: stdout AQB/Uzxm1jkPBRAA8e92p0GKwqP2v6r/cq9ENA== 2024-05-09T04:39:29.121 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph-authtool: stdout AQCAUzxm3vR1GxAADBr7HfUQINHz1eqCvZM6bQ== 2024-05-09T04:39:30.298 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph-authtool: stdout AQCBUzxmAuTUKRAArfnMFV/VMwtk23anai9KaA== 2024-05-09T04:39:30.300 INFO:teuthology.orchestra.run.smithi006.stdout:Creating initial monmap... 2024-05-09T04:39:31.610 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-09T04:39:31.610 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-09T04:39:31.610 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:39:31.610 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-09T04:39:31.611 INFO:teuthology.orchestra.run.smithi006.stdout:monmaptool for a [v2:172.21.15.6:3300,v1:172.21.15.6:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-09T04:39:31.611 INFO:teuthology.orchestra.run.smithi006.stdout:setting min_mon_release = quincy 2024-05-09T04:39:31.611 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: set fsid to f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:39:31.611 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-09T04:39:31.611 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:39:31.611 INFO:teuthology.orchestra.run.smithi006.stdout:Creating mon... 2024-05-09T04:39:32.849 INFO:teuthology.orchestra.run.smithi006.stdout:create mon.a on 2024-05-09T04:39:33.228 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-09T04:39:33.484 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-09T04:39:33.804 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-f74d813c-0dbd-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-f74d813c-0dbd-11ef-bc97-c7b262605968.target. 2024-05-09T04:39:33.804 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-f74d813c-0dbd-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-f74d813c-0dbd-11ef-bc97-c7b262605968.target. 2024-05-09T04:39:34.201 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mon.a 2024-05-09T04:39:34.201 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mon.a.service: Unit ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mon.a.service not loaded. 2024-05-09T04:39:34.496 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f74d813c-0dbd-11ef-bc97-c7b262605968.target.wants/ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mon.a.service → /etc/systemd/system/ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@.service. 2024-05-09T04:39:35.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:35 smithi006 podman[29433]: 2024-05-09 04:39:35.131647665 +0000 UTC m=+0.365556827 container start 6cf1bb868c1881be08c136ad0227b95d355dc185e5b0a5e0f179a2724b69a4af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T04:39:35.245 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T04:39:35.245 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout disabled 2024-05-09T04:39:35.263 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T04:39:35.263 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2024-05-09T04:39:35.263 INFO:teuthology.orchestra.run.smithi006.stdout:firewalld.service is not enabled 2024-05-09T04:39:35.263 INFO:teuthology.orchestra.run.smithi006.stdout:Not possible to enable service . firewalld.service is not available 2024-05-09T04:39:35.263 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mon to start... 2024-05-09T04:39:35.264 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mon... 2024-05-09T04:39:35.518 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:35 smithi006 bash[29433]: 6cf1bb868c1881be08c136ad0227b95d355dc185e5b0a5e0f179a2724b69a4af 2024-05-09T04:39:35.518 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:35 smithi006 systemd[1]: Started Ceph mon.a for f74d813c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:39:35.518 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:35 smithi006 ceph-mon[29452]: mkfs f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:39:35.518 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:35 smithi006 ceph-mon[29452]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-09T04:39:36.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:36 smithi006 ceph-mon[29452]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-09T04:39:36.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:36 smithi006 ceph-mon[29452]: monmap epoch 1 2024-05-09T04:39:36.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:36 smithi006 ceph-mon[29452]: fsid f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:39:36.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:36 smithi006 ceph-mon[29452]: last_changed 2024-05-09T04:39:30.984541+0000 2024-05-09T04:39:36.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:36 smithi006 ceph-mon[29452]: created 2024-05-09T04:39:30.984541+0000 2024-05-09T04:39:36.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:36 smithi006 ceph-mon[29452]: min_mon_release 19 (squid) 2024-05-09T04:39:36.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:36 smithi006 ceph-mon[29452]: election_strategy: 1 2024-05-09T04:39:36.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:36 smithi006 ceph-mon[29452]: 0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-09T04:39:36.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:36 smithi006 ceph-mon[29452]: fsmap 2024-05-09T04:39:36.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:36 smithi006 ceph-mon[29452]: osdmap e1: 0 total, 0 up, 0 in 2024-05-09T04:39:36.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:36 smithi006 ceph-mon[29452]: mgrmap e1: no daemons active 2024-05-09T04:39:36.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:36 smithi006 ceph-mon[29452]: from='client.? 172.21.15.6:0/3249709324' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-09T04:39:45.856 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout cluster: 2024-05-09T04:39:45.862 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout id: f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:39:45.862 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-09T04:39:45.863 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-05-09T04:39:45.863 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout services: 2024-05-09T04:39:45.863 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.846642s) 2024-05-09T04:39:45.863 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-09T04:39:45.863 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-09T04:39:45.863 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-05-09T04:39:45.863 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout data: 2024-05-09T04:39:45.863 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-09T04:39:45.863 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-09T04:39:45.863 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-09T04:39:45.863 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout pgs: 2024-05-09T04:39:45.863 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-05-09T04:39:45.863 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.037+0000 7fb971600700 1 Processor -- start 2024-05-09T04:39:45.863 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.037+0000 7fb971600700 1 -- start start 2024-05-09T04:39:45.864 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.038+0000 7fb971600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb96c109990 0x7fb96c109da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:45.864 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.038+0000 7fb971600700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c10a390 con 0x7fb96c109990 2024-05-09T04:39:45.864 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.039+0000 7fb96be00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb96c109990 0x7fb96c109da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:45.864 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.039+0000 7fb96be00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb96c109990 0x7fb96c109da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60032/0 (socket says 172.21.15.6:60032) 2024-05-09T04:39:45.864 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.039+0000 7fb96be00700 1 -- 172.21.15.6:0/694425085 learned_addr learned my addr 172.21.15.6:0/694425085 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:39:45.864 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.039+0000 7fb96be00700 1 -- 172.21.15.6:0/694425085 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb96c10ab10 con 0x7fb96c109990 2024-05-09T04:39:45.864 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.040+0000 7fb96be00700 1 --2- 172.21.15.6:0/694425085 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb96c109990 0x7fb96c109da0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb95c0098a0 tx=0x7fb95c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34347fb8f4894f53 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:45.864 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.040+0000 7fb96aa00700 1 -- 172.21.15.6:0/694425085 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb95c004160 con 0x7fb96c109990 2024-05-09T04:39:45.864 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.040+0000 7fb96aa00700 1 -- 172.21.15.6:0/694425085 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb95c004300 con 0x7fb96c109990 2024-05-09T04:39:45.864 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.040+0000 7fb96aa00700 1 -- 172.21.15.6:0/694425085 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb95c0036a0 con 0x7fb96c109990 2024-05-09T04:39:45.864 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.041+0000 7fb971600700 1 -- 172.21.15.6:0/694425085 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb96c109990 msgr2=0x7fb96c109da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:45.864 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.041+0000 7fb971600700 1 --2- 172.21.15.6:0/694425085 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb96c109990 0x7fb96c109da0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb95c0098a0 tx=0x7fb95c009bb0 comp rx=0 tx=0).stop 2024-05-09T04:39:45.864 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.041+0000 7fb971600700 1 -- 172.21.15.6:0/694425085 shutdown_connections 2024-05-09T04:39:45.864 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.041+0000 7fb971600700 1 --2- 172.21.15.6:0/694425085 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb96c109990 0x7fb96c109da0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:45.865 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.041+0000 7fb971600700 1 -- 172.21.15.6:0/694425085 >> 172.21.15.6:0/694425085 conn(0x7fb96c104ff0 msgr2=0x7fb96c107480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:39:45.865 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.042+0000 7fb971600700 1 -- 172.21.15.6:0/694425085 shutdown_connections 2024-05-09T04:39:45.865 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.042+0000 7fb971600700 1 -- 172.21.15.6:0/694425085 wait complete. 2024-05-09T04:39:45.865 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.043+0000 7fb971600700 1 Processor -- start 2024-05-09T04:39:45.865 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.043+0000 7fb971600700 1 -- start start 2024-05-09T04:39:45.865 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.043+0000 7fb971600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb96c109990 0x7fb96c19f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:45.865 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.043+0000 7fb971600700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c10b040 con 0x7fb96c109990 2024-05-09T04:39:45.865 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.044+0000 7fb96be00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb96c109990 0x7fb96c19f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:45.865 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.044+0000 7fb96be00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb96c109990 0x7fb96c19f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60034/0 (socket says 172.21.15.6:60034) 2024-05-09T04:39:45.865 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.044+0000 7fb96be00700 1 -- 172.21.15.6:0/3249709324 learned_addr learned my addr 172.21.15.6:0/3249709324 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:39:45.865 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.044+0000 7fb96be00700 1 -- 172.21.15.6:0/3249709324 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb96c19fc50 con 0x7fb96c109990 2024-05-09T04:39:45.865 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.045+0000 7fb96be00700 1 --2- 172.21.15.6:0/3249709324 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb96c109990 0x7fb96c19f710 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb95c000c00 tx=0x7fb95c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:45.865 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.045+0000 7fb968c00700 1 -- 172.21.15.6:0/3249709324 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb95c003db0 con 0x7fb96c109990 2024-05-09T04:39:45.865 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.045+0000 7fb968c00700 1 -- 172.21.15.6:0/3249709324 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb95c011630 con 0x7fb96c109990 2024-05-09T04:39:45.865 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.045+0000 7fb971600700 1 -- 172.21.15.6:0/3249709324 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb96c19fee0 con 0x7fb96c109990 2024-05-09T04:39:45.866 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.045+0000 7fb968c00700 1 -- 172.21.15.6:0/3249709324 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb95c011950 con 0x7fb96c109990 2024-05-09T04:39:45.866 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.045+0000 7fb971600700 1 -- 172.21.15.6:0/3249709324 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb96c1a2af0 con 0x7fb96c109990 2024-05-09T04:39:45.866 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.046+0000 7fb968c00700 1 -- 172.21.15.6:0/3249709324 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb95c01a460 con 0x7fb96c109990 2024-05-09T04:39:45.866 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.046+0000 7fb968c00700 1 -- 172.21.15.6:0/3249709324 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb95c01b780 con 0x7fb96c109990 2024-05-09T04:39:45.866 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.046+0000 7fb971600700 1 -- 172.21.15.6:0/3249709324 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb96c064e20 con 0x7fb96c109990 2024-05-09T04:39:45.866 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.050+0000 7fb968c00700 1 -- 172.21.15.6:0/3249709324 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fb95c031020 con 0x7fb96c109990 2024-05-09T04:39:45.866 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.131+0000 7fb971600700 1 -- 172.21.15.6:0/3249709324 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fb96c1a2cc0 con 0x7fb96c109990 2024-05-09T04:39:45.866 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.131+0000 7fb968c00700 1 -- 172.21.15.6:0/3249709324 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fb95c01b9d0 con 0x7fb96c109990 2024-05-09T04:39:45.866 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.133+0000 7fb971600700 1 -- 172.21.15.6:0/3249709324 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb96c109990 msgr2=0x7fb96c19f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:45.866 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.133+0000 7fb971600700 1 --2- 172.21.15.6:0/3249709324 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb96c109990 0x7fb96c19f710 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb95c000c00 tx=0x7fb95c004750 comp rx=0 tx=0).stop 2024-05-09T04:39:45.866 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.133+0000 7fb971600700 1 -- 172.21.15.6:0/3249709324 shutdown_connections 2024-05-09T04:39:45.866 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.133+0000 7fb971600700 1 --2- 172.21.15.6:0/3249709324 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb96c109990 0x7fb96c19f710 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:45.866 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.133+0000 7fb971600700 1 -- 172.21.15.6:0/3249709324 >> 172.21.15.6:0/3249709324 conn(0x7fb96c104ff0 msgr2=0x7fb96c07e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:39:45.866 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.133+0000 7fb971600700 1 -- 172.21.15.6:0/3249709324 shutdown_connections 2024-05-09T04:39:45.867 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.133+0000 7fb971600700 1 -- 172.21.15.6:0/3249709324 wait complete. 2024-05-09T04:39:45.867 INFO:teuthology.orchestra.run.smithi006.stdout:mon is available 2024-05-09T04:39:45.867 INFO:teuthology.orchestra.run.smithi006.stdout:Assimilating anything we can from ceph.conf... 2024-05-09T04:39:46.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:46 smithi006 ceph-mon[29452]: from='client.? 172.21.15.6:0/1934785348' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-09T04:39:46.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:46 smithi006 ceph-mon[29452]: from='client.? 172.21.15.6:0/1934785348' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-09T04:39:47.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-05-09T04:39:47.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [global] 2024-05-09T04:39:47.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout fsid = f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:39:47.385 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-09T04:39:47.385 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.6:3300,v1:172.21.15.6:6789] 2024-05-09T04:39:47.385 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-09T04:39:47.385 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-09T04:39:47.385 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-09T04:39:47.385 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-09T04:39:47.385 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-05-09T04:39:47.385 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-09T04:39:47.386 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-09T04:39:47.386 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-05-09T04:39:47.386 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [osd] 2024-05-09T04:39:47.386 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-09T04:39:47.386 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-09T04:39:47.389 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-09T04:39:47.389 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-09T04:39:47.389 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-09T04:39:47.389 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.557+0000 7fa368a00700 1 Processor -- start 2024-05-09T04:39:47.389 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.557+0000 7fa368a00700 1 -- start start 2024-05-09T04:39:47.389 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.558+0000 7fa368a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa36410bb10 0x7fa36410bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:47.389 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.558+0000 7fa368a00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa36410c510 con 0x7fa36410bb10 2024-05-09T04:39:47.390 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.558+0000 7fa363400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa36410bb10 0x7fa36410bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:47.390 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.558+0000 7fa363400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa36410bb10 0x7fa36410bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38694/0 (socket says 172.21.15.6:38694) 2024-05-09T04:39:47.390 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.558+0000 7fa363400700 1 -- 172.21.15.6:0/1801343947 learned_addr learned my addr 172.21.15.6:0/1801343947 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:39:47.390 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.558+0000 7fa363400700 1 -- 172.21.15.6:0/1801343947 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa36410cce0 con 0x7fa36410bb10 2024-05-09T04:39:47.390 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.559+0000 7fa363400700 1 --2- 172.21.15.6:0/1801343947 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa36410bb10 0x7fa36410bf20 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa3540098a0 tx=0x7fa354009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5980522389214b2a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:47.390 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.559+0000 7fa362a00700 1 -- 172.21.15.6:0/1801343947 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa354004160 con 0x7fa36410bb10 2024-05-09T04:39:47.390 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.559+0000 7fa362a00700 1 -- 172.21.15.6:0/1801343947 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa354004300 con 0x7fa36410bb10 2024-05-09T04:39:47.390 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.559+0000 7fa362a00700 1 -- 172.21.15.6:0/1801343947 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3540036a0 con 0x7fa36410bb10 2024-05-09T04:39:47.390 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.560+0000 7fa368a00700 1 -- 172.21.15.6:0/1801343947 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa36410bb10 msgr2=0x7fa36410bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:47.390 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.560+0000 7fa368a00700 1 --2- 172.21.15.6:0/1801343947 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa36410bb10 0x7fa36410bf20 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa3540098a0 tx=0x7fa354009bb0 comp rx=0 tx=0).stop 2024-05-09T04:39:47.390 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.560+0000 7fa368a00700 1 -- 172.21.15.6:0/1801343947 shutdown_connections 2024-05-09T04:39:47.390 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.560+0000 7fa368a00700 1 --2- 172.21.15.6:0/1801343947 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa36410bb10 0x7fa36410bf20 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:47.391 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.560+0000 7fa368a00700 1 -- 172.21.15.6:0/1801343947 >> 172.21.15.6:0/1801343947 conn(0x7fa36407efa0 msgr2=0x7fa36407f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:39:47.391 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.560+0000 7fa368a00700 1 -- 172.21.15.6:0/1801343947 shutdown_connections 2024-05-09T04:39:47.391 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.560+0000 7fa368a00700 1 -- 172.21.15.6:0/1801343947 wait complete. 2024-05-09T04:39:47.391 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.561+0000 7fa368a00700 1 Processor -- start 2024-05-09T04:39:47.391 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.562+0000 7fa368a00700 1 -- start start 2024-05-09T04:39:47.391 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.562+0000 7fa368a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa36410bb10 0x7fa36419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:47.391 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.562+0000 7fa368a00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa36410d210 con 0x7fa36410bb10 2024-05-09T04:39:47.391 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.562+0000 7fa363400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa36410bb10 0x7fa36419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:47.391 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.562+0000 7fa363400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa36410bb10 0x7fa36419f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38696/0 (socket says 172.21.15.6:38696) 2024-05-09T04:39:47.391 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.562+0000 7fa363400700 1 -- 172.21.15.6:0/1934785348 learned_addr learned my addr 172.21.15.6:0/1934785348 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:39:47.391 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.563+0000 7fa363400700 1 -- 172.21.15.6:0/1934785348 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa36419fbf0 con 0x7fa36410bb10 2024-05-09T04:39:47.394 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.563+0000 7fa363400700 1 --2- 172.21.15.6:0/1934785348 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa36410bb10 0x7fa36419f6b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa354000c00 tx=0x7fa354004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:47.394 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.563+0000 7fa360c00700 1 -- 172.21.15.6:0/1934785348 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa354003db0 con 0x7fa36410bb10 2024-05-09T04:39:47.394 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.563+0000 7fa360c00700 1 -- 172.21.15.6:0/1934785348 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa354011630 con 0x7fa36410bb10 2024-05-09T04:39:47.394 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.563+0000 7fa368a00700 1 -- 172.21.15.6:0/1934785348 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa36419fe80 con 0x7fa36410bb10 2024-05-09T04:39:47.394 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.563+0000 7fa360c00700 1 -- 172.21.15.6:0/1934785348 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa354011950 con 0x7fa36410bb10 2024-05-09T04:39:47.394 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.564+0000 7fa368a00700 1 -- 172.21.15.6:0/1934785348 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3641a2a90 con 0x7fa36410bb10 2024-05-09T04:39:47.395 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.564+0000 7fa360c00700 1 -- 172.21.15.6:0/1934785348 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa35401a460 con 0x7fa36410bb10 2024-05-09T04:39:47.395 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.564+0000 7fa360c00700 1 -- 172.21.15.6:0/1934785348 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa35401b780 con 0x7fa36410bb10 2024-05-09T04:39:47.395 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.565+0000 7fa368a00700 1 -- 172.21.15.6:0/1934785348 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa364064e20 con 0x7fa36410bb10 2024-05-09T04:39:47.395 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.568+0000 7fa360c00700 1 -- 172.21.15.6:0/1934785348 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa354031020 con 0x7fa36410bb10 2024-05-09T04:39:47.395 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.646+0000 7fa368a00700 1 -- 172.21.15.6:0/1934785348 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fa36402e870 con 0x7fa36410bb10 2024-05-09T04:39:47.395 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.654+0000 7fa360c00700 1 -- 172.21.15.6:0/1934785348 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+463 (secure 0 0 0) 0x7fa35401bad0 con 0x7fa36410bb10 2024-05-09T04:39:47.396 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.656+0000 7fa368a00700 1 -- 172.21.15.6:0/1934785348 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa36410bb10 msgr2=0x7fa36419f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:47.396 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.656+0000 7fa368a00700 1 --2- 172.21.15.6:0/1934785348 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa36410bb10 0x7fa36419f6b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa354000c00 tx=0x7fa354004750 comp rx=0 tx=0).stop 2024-05-09T04:39:47.396 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.656+0000 7fa368a00700 1 -- 172.21.15.6:0/1934785348 shutdown_connections 2024-05-09T04:39:47.396 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.656+0000 7fa368a00700 1 --2- 172.21.15.6:0/1934785348 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa36410bb10 0x7fa36419f6b0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:47.396 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.656+0000 7fa368a00700 1 -- 172.21.15.6:0/1934785348 >> 172.21.15.6:0/1934785348 conn(0x7fa36407efa0 msgr2=0x7fa364195d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:39:47.396 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.656+0000 7fa368a00700 1 -- 172.21.15.6:0/1934785348 shutdown_connections 2024-05-09T04:39:47.397 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.656+0000 7fa368a00700 1 -- 172.21.15.6:0/1934785348 wait complete. 2024-05-09T04:39:47.397 INFO:teuthology.orchestra.run.smithi006.stdout:Generating new minimal ceph.conf... 2024-05-09T04:39:48.446 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:48 smithi006 ceph-mon[29452]: from='client.? 172.21.15.6:0/3514079605' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf", "output-file": "/var/lib/ceph/mon/ceph-a/config"}]: dispatch 2024-05-09T04:39:48.813 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.062+0000 7fa3ddc00700 1 Processor -- start 2024-05-09T04:39:48.814 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.062+0000 7fa3ddc00700 1 -- start start 2024-05-09T04:39:48.815 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.063+0000 7fa3ddc00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3d807f6a0 0x7fa3d807dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:48.815 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.063+0000 7fa3ddc00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d807e010 con 0x7fa3d807f6a0 2024-05-09T04:39:48.815 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.063+0000 7fa3d6a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3d807f6a0 0x7fa3d807dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:48.815 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.064+0000 7fa3d6a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3d807f6a0 0x7fa3d807dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45436/0 (socket says 172.21.15.6:45436) 2024-05-09T04:39:48.815 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.064+0000 7fa3d6a00700 1 -- 172.21.15.6:0/2305811325 learned_addr learned my addr 172.21.15.6:0/2305811325 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:39:48.815 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.064+0000 7fa3d6a00700 1 -- 172.21.15.6:0/2305811325 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d807e190 con 0x7fa3d807f6a0 2024-05-09T04:39:48.815 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.065+0000 7fa3d6a00700 1 --2- 172.21.15.6:0/2305811325 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3d807f6a0 0x7fa3d807dad0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa3c80098a0 tx=0x7fa3c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31698420c71dca86 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:48.815 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.065+0000 7fa3d5600700 1 -- 172.21.15.6:0/2305811325 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3c8004160 con 0x7fa3d807f6a0 2024-05-09T04:39:48.815 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.065+0000 7fa3d5600700 1 -- 172.21.15.6:0/2305811325 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa3c8004300 con 0x7fa3d807f6a0 2024-05-09T04:39:48.815 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.065+0000 7fa3d5600700 1 -- 172.21.15.6:0/2305811325 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3c8011af0 con 0x7fa3d807f6a0 2024-05-09T04:39:48.819 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.066+0000 7fa3ddc00700 1 -- 172.21.15.6:0/2305811325 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3d807f6a0 msgr2=0x7fa3d807dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:48.819 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.066+0000 7fa3ddc00700 1 --2- 172.21.15.6:0/2305811325 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3d807f6a0 0x7fa3d807dad0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa3c80098a0 tx=0x7fa3c8009bb0 comp rx=0 tx=0).stop 2024-05-09T04:39:48.819 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.067+0000 7fa3ddc00700 1 -- 172.21.15.6:0/2305811325 shutdown_connections 2024-05-09T04:39:48.819 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.067+0000 7fa3ddc00700 1 --2- 172.21.15.6:0/2305811325 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3d807f6a0 0x7fa3d807dad0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:48.819 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.067+0000 7fa3ddc00700 1 -- 172.21.15.6:0/2305811325 >> 172.21.15.6:0/2305811325 conn(0x7fa3d81051c0 msgr2=0x7fa3d8107610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:39:48.819 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.067+0000 7fa3ddc00700 1 -- 172.21.15.6:0/2305811325 shutdown_connections 2024-05-09T04:39:48.819 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.067+0000 7fa3ddc00700 1 -- 172.21.15.6:0/2305811325 wait complete. 2024-05-09T04:39:48.819 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.068+0000 7fa3ddc00700 1 Processor -- start 2024-05-09T04:39:48.819 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.068+0000 7fa3ddc00700 1 -- start start 2024-05-09T04:39:48.819 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.068+0000 7fa3ddc00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3d807f6a0 0x7fa3d81a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:48.820 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.068+0000 7fa3ddc00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d81100a0 con 0x7fa3d807f6a0 2024-05-09T04:39:48.820 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.069+0000 7fa3d6a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3d807f6a0 0x7fa3d81a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:48.820 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.069+0000 7fa3d6a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3d807f6a0 0x7fa3d81a00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45446/0 (socket says 172.21.15.6:45446) 2024-05-09T04:39:48.820 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.069+0000 7fa3d6a00700 1 -- 172.21.15.6:0/3514079605 learned_addr learned my addr 172.21.15.6:0/3514079605 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:39:48.820 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.069+0000 7fa3d6a00700 1 -- 172.21.15.6:0/3514079605 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d81a05f0 con 0x7fa3d807f6a0 2024-05-09T04:39:48.820 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.070+0000 7fa3d6a00700 1 --2- 172.21.15.6:0/3514079605 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3d807f6a0 0x7fa3d81a00b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa3c80117b0 tx=0x7fa3c8011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:48.820 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.070+0000 7fa3cf400700 1 -- 172.21.15.6:0/3514079605 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3c8003af0 con 0x7fa3d807f6a0 2024-05-09T04:39:48.820 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.070+0000 7fa3cf400700 1 -- 172.21.15.6:0/3514079605 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa3c8020070 con 0x7fa3d807f6a0 2024-05-09T04:39:48.820 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.070+0000 7fa3ddc00700 1 -- 172.21.15.6:0/3514079605 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3d81a0880 con 0x7fa3d807f6a0 2024-05-09T04:39:48.820 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.070+0000 7fa3cf400700 1 -- 172.21.15.6:0/3514079605 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3c8019630 con 0x7fa3d807f6a0 2024-05-09T04:39:48.820 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.070+0000 7fa3ddc00700 1 -- 172.21.15.6:0/3514079605 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3d81a3490 con 0x7fa3d807f6a0 2024-05-09T04:39:48.820 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.070+0000 7fa3cf400700 1 -- 172.21.15.6:0/3514079605 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa3c8019c30 con 0x7fa3d807f6a0 2024-05-09T04:39:48.820 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.070+0000 7fa3cf400700 1 -- 172.21.15.6:0/3514079605 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa3c800b720 con 0x7fa3d807f6a0 2024-05-09T04:39:48.821 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.072+0000 7fa3ddc00700 1 -- 172.21.15.6:0/3514079605 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3d8064e20 con 0x7fa3d807f6a0 2024-05-09T04:39:48.821 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.075+0000 7fa3cf400700 1 -- 172.21.15.6:0/3514079605 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa3c80042c0 con 0x7fa3d807f6a0 2024-05-09T04:39:48.821 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.152+0000 7fa3ddc00700 1 -- 172.21.15.6:0/3514079605 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "config generate-minimal-conf", "output-file": "/var/lib/ceph/mon/ceph-a/config"} v 0) -- 0x7fa3d81a3780 con 0x7fa3d807f6a0 2024-05-09T04:39:48.821 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.153+0000 7fa3cf400700 1 -- 172.21.15.6:0/3514079605 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf", "output-file": "/var/lib/ceph/mon/ceph-a/config"}]=0 v2) ==== 126+0+173 (secure 0 0 0) 0x7fa3c801a9c0 con 0x7fa3d807f6a0 2024-05-09T04:39:48.821 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.155+0000 7fa3ddc00700 1 -- 172.21.15.6:0/3514079605 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3d807f6a0 msgr2=0x7fa3d81a00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:48.821 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.155+0000 7fa3ddc00700 1 --2- 172.21.15.6:0/3514079605 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3d807f6a0 0x7fa3d81a00b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa3c80117b0 tx=0x7fa3c8011b70 comp rx=0 tx=0).stop 2024-05-09T04:39:48.821 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.155+0000 7fa3ddc00700 1 -- 172.21.15.6:0/3514079605 shutdown_connections 2024-05-09T04:39:48.821 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.155+0000 7fa3ddc00700 1 --2- 172.21.15.6:0/3514079605 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3d807f6a0 0x7fa3d81a00b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:48.821 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.155+0000 7fa3ddc00700 1 -- 172.21.15.6:0/3514079605 >> 172.21.15.6:0/3514079605 conn(0x7fa3d81051c0 msgr2=0x7fa3d8106c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:39:48.822 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.155+0000 7fa3ddc00700 1 -- 172.21.15.6:0/3514079605 shutdown_connections 2024-05-09T04:39:48.822 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:48.155+0000 7fa3ddc00700 1 -- 172.21.15.6:0/3514079605 wait complete. 2024-05-09T04:39:48.822 INFO:teuthology.orchestra.run.smithi006.stdout:Restarting the monitor... 2024-05-09T04:39:49.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:48 smithi006 systemd[1]: Stopping Ceph mon.a for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T04:39:49.696 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:49 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mon-a[29448]: 2024-05-09T04:39:49.325+0000 7ff4bae00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T04:39:49.696 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:49 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mon-a[29448]: 2024-05-09T04:39:49.325+0000 7ff4bae00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-09T04:39:49.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:49 smithi006 podman[29706]: 2024-05-09 04:39:49.418779641 +0000 UTC m=+0.318370298 container died 6cf1bb868c1881be08c136ad0227b95d355dc185e5b0a5e0f179a2724b69a4af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux ) 2024-05-09T04:39:50.399 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:50 smithi006 podman[29706]: 2024-05-09 04:39:50.139779937 +0000 UTC m=+1.039370588 container cleanup 6cf1bb868c1881be08c136ad0227b95d355dc185e5b0a5e0f179a2724b69a4af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507) 2024-05-09T04:39:50.399 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:50 smithi006 bash[29706]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mon-a 2024-05-09T04:39:50.676 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:50 smithi006 podman[29722]: 2024-05-09 04:39:50.398797133 +0000 UTC m=+0.964941378 container remove 6cf1bb868c1881be08c136ad0227b95d355dc185e5b0a5e0f179a2724b69a4af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS) 2024-05-09T04:39:51.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:50 smithi006 systemd[1]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-09T04:39:51.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:50 smithi006 systemd[1]: Stopped Ceph mon.a for f74d813c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:39:51.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:50 smithi006 systemd[1]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mon.a.service: Consumed 1.947s CPU time. 2024-05-09T04:39:51.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:50 smithi006 systemd[1]: Starting Ceph mon.a for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T04:39:51.619 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 podman[29816]: 2024-05-09 04:39:51.254895896 +0000 UTC m=+0.039604755 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T04:39:51.620 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 podman[29816]: 2024-05-09 04:39:51.383070882 +0000 UTC m=+0.167779777 container create 34c0cb98e2cc80a5e82399c5c05feb7cf9f03b2681647ba61167705b326893ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T04:39:51.620 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 podman[29816]: 2024-05-09 04:39:51.620116629 +0000 UTC m=+0.404825531 container init 34c0cb98e2cc80a5e82399c5c05feb7cf9f03b2681647ba61167705b326893ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-05-09T04:39:51.719 INFO:teuthology.orchestra.run.smithi006.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-09T04:39:51.883 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 podman[29816]: 2024-05-09 04:39:51.631324451 +0000 UTC m=+0.416033317 container start 34c0cb98e2cc80a5e82399c5c05feb7cf9f03b2681647ba61167705b326893ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mon-a, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS) 2024-05-09T04:39:51.883 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 bash[29816]: 34c0cb98e2cc80a5e82399c5c05feb7cf9f03b2681647ba61167705b326893ca 2024-05-09T04:39:51.883 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 systemd[1]: Started Ceph mon.a for f74d813c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:39:51.883 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: set uid:gid to 167:167 (ceph:ceph) 2024-05-09T04:39:51.884 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev), process ceph-mon, pid 2 2024-05-09T04:39:51.884 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: pidfile_write: ignore empty --pid-file 2024-05-09T04:39:51.884 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: load: jerasure load: lrc 2024-05-09T04:39:51.884 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: RocksDB version: 7.9.2 2024-05-09T04:39:51.884 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Git sha 0 2024-05-09T04:39:51.885 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Compile date 2024-05-08 19:33:14 2024-05-09T04:39:51.885 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: DB SUMMARY 2024-05-09T04:39:51.885 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: DB Session ID: DZYZHYE627YUVKWWQU3R 2024-05-09T04:39:51.885 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: CURRENT file: CURRENT 2024-05-09T04:39:51.885 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: IDENTITY file: IDENTITY 2024-05-09T04:39:51.885 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-09T04:39:51.886 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-09T04:39:51.886 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104848 ; 2024-05-09T04:39:51.886 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.error_if_exists: 0 2024-05-09T04:39:51.886 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.create_if_missing: 0 2024-05-09T04:39:51.886 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.paranoid_checks: 1 2024-05-09T04:39:51.886 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-09T04:39:51.887 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-09T04:39:51.887 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-09T04:39:51.887 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.env: 0x560553c69500 2024-05-09T04:39:51.887 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.fs: PosixFileSystem 2024-05-09T04:39:51.887 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.info_log: 0x56055717c060 2024-05-09T04:39:51.887 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.max_file_opening_threads: 16 2024-05-09T04:39:51.888 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.statistics: (nil) 2024-05-09T04:39:51.888 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.use_fsync: 0 2024-05-09T04:39:51.888 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.max_log_file_size: 0 2024-05-09T04:39:51.888 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-09T04:39:51.888 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-09T04:39:51.888 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.keep_log_file_num: 1000 2024-05-09T04:39:51.891 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.recycle_log_file_num: 0 2024-05-09T04:39:51.891 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.allow_fallocate: 1 2024-05-09T04:39:51.891 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.allow_mmap_reads: 0 2024-05-09T04:39:51.891 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.allow_mmap_writes: 0 2024-05-09T04:39:51.891 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.use_direct_reads: 0 2024-05-09T04:39:51.892 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-09T04:39:51.892 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.create_missing_column_families: 0 2024-05-09T04:39:51.892 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.db_log_dir: 2024-05-09T04:39:51.892 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.wal_dir: 2024-05-09T04:39:51.892 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-09T04:39:51.892 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-09T04:39:51.893 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-09T04:39:51.893 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-09T04:39:51.893 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-09T04:39:51.893 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-09T04:39:51.893 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.advise_random_on_open: 1 2024-05-09T04:39:51.893 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.db_write_buffer_size: 0 2024-05-09T04:39:51.893 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.write_buffer_manager: 0x5605570468c0 2024-05-09T04:39:51.894 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-09T04:39:51.894 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-09T04:39:51.894 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-09T04:39:51.894 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.rate_limiter: (nil) 2024-05-09T04:39:51.894 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-09T04:39:51.894 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.wal_recovery_mode: 2 2024-05-09T04:39:51.895 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.enable_thread_tracking: 0 2024-05-09T04:39:51.895 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.enable_pipelined_write: 0 2024-05-09T04:39:51.895 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.unordered_write: 0 2024-05-09T04:39:51.895 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-09T04:39:51.895 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-09T04:39:51.895 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-09T04:39:51.896 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-09T04:39:51.896 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.row_cache: None 2024-05-09T04:39:51.896 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.wal_filter: None 2024-05-09T04:39:51.896 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-09T04:39:51.896 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.allow_ingest_behind: 0 2024-05-09T04:39:51.897 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.two_write_queues: 0 2024-05-09T04:39:51.897 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.manual_wal_flush: 0 2024-05-09T04:39:51.897 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.wal_compression: 0 2024-05-09T04:39:51.897 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.atomic_flush: 0 2024-05-09T04:39:51.898 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-09T04:39:51.898 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-09T04:39:51.898 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-09T04:39:51.898 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.log_readahead_size: 0 2024-05-09T04:39:51.898 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-09T04:39:51.898 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.best_efforts_recovery: 0 2024-05-09T04:39:51.899 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-09T04:39:51.899 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-09T04:39:51.899 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.allow_data_in_errors: 0 2024-05-09T04:39:51.899 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.db_host_id: __hostname__ 2024-05-09T04:39:51.899 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-09T04:39:51.899 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.max_background_jobs: 2 2024-05-09T04:39:51.900 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.max_background_compactions: -1 2024-05-09T04:39:51.900 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.max_subcompactions: 1 2024-05-09T04:39:51.900 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-09T04:39:51.900 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-09T04:39:51.900 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-09T04:39:51.900 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.max_total_wal_size: 0 2024-05-09T04:39:51.900 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-09T04:39:51.901 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-09T04:39:51.901 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-09T04:39:51.901 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-09T04:39:51.901 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.max_open_files: -1 2024-05-09T04:39:51.901 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.bytes_per_sync: 0 2024-05-09T04:39:51.901 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-09T04:39:51.902 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-09T04:39:51.902 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.compaction_readahead_size: 0 2024-05-09T04:39:51.902 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.max_background_flushes: -1 2024-05-09T04:39:51.902 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Compression algorithms supported: 2024-05-09T04:39:51.902 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-09T04:39:51.902 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: kZSTD supported: 0 2024-05-09T04:39:51.903 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: kXpressCompression supported: 0 2024-05-09T04:39:51.903 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: kLZ4HCCompression supported: 1 2024-05-09T04:39:51.903 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: kZlibCompression supported: 1 2024-05-09T04:39:51.903 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: kSnappyCompression supported: 1 2024-05-09T04:39:51.903 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: kLZ4Compression supported: 1 2024-05-09T04:39:51.903 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: kBZip2Compression supported: 0 2024-05-09T04:39:51.904 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-09T04:39:51.904 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-09T04:39:51.904 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-09T04:39:51.904 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-09T04:39:51.904 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-09T04:39:51.904 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.merge_operator: 2024-05-09T04:39:51.905 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.compaction_filter: None 2024-05-09T04:39:51.905 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.compaction_filter_factory: None 2024-05-09T04:39:51.905 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.sst_partitioner_factory: None 2024-05-09T04:39:51.905 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-09T04:39:51.905 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-09T04:39:51.905 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56055717c220) 2024-05-09T04:39:51.906 INFO:journalctl@ceph.mon.a.smithi006.stdout: cache_index_and_filter_blocks: 1 2024-05-09T04:39:51.906 INFO:journalctl@ceph.mon.a.smithi006.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-09T04:39:51.906 INFO:journalctl@ceph.mon.a.smithi006.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-09T04:39:51.906 INFO:journalctl@ceph.mon.a.smithi006.stdout: pin_top_level_index_and_filter: 1 2024-05-09T04:39:51.906 INFO:journalctl@ceph.mon.a.smithi006.stdout: index_type: 0 2024-05-09T04:39:51.907 INFO:journalctl@ceph.mon.a.smithi006.stdout: data_block_index_type: 0 2024-05-09T04:39:51.907 INFO:journalctl@ceph.mon.a.smithi006.stdout: index_shortening: 1 2024-05-09T04:39:51.907 INFO:journalctl@ceph.mon.a.smithi006.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-09T04:39:51.907 INFO:journalctl@ceph.mon.a.smithi006.stdout: checksum: 4 2024-05-09T04:39:51.907 INFO:journalctl@ceph.mon.a.smithi006.stdout: no_block_cache: 0 2024-05-09T04:39:51.907 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_cache: 0x5605570791f0 2024-05-09T04:39:51.908 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_cache_name: BinnedLRUCache 2024-05-09T04:39:51.908 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_cache_options: 2024-05-09T04:39:51.908 INFO:journalctl@ceph.mon.a.smithi006.stdout: capacity : 536870912 2024-05-09T04:39:51.908 INFO:journalctl@ceph.mon.a.smithi006.stdout: num_shard_bits : 4 2024-05-09T04:39:51.908 INFO:journalctl@ceph.mon.a.smithi006.stdout: strict_capacity_limit : 0 2024-05-09T04:39:51.908 INFO:journalctl@ceph.mon.a.smithi006.stdout: high_pri_pool_ratio: 0.000 2024-05-09T04:39:51.909 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_cache_compressed: (nil) 2024-05-09T04:39:51.909 INFO:journalctl@ceph.mon.a.smithi006.stdout: persistent_cache: (nil) 2024-05-09T04:39:51.909 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_size: 4096 2024-05-09T04:39:51.909 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_size_deviation: 10 2024-05-09T04:39:51.909 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_restart_interval: 16 2024-05-09T04:39:51.909 INFO:journalctl@ceph.mon.a.smithi006.stdout: index_block_restart_interval: 1 2024-05-09T04:39:51.910 INFO:journalctl@ceph.mon.a.smithi006.stdout: metadata_block_size: 4096 2024-05-09T04:39:51.910 INFO:journalctl@ceph.mon.a.smithi006.stdout: partition_filters: 0 2024-05-09T04:39:51.910 INFO:journalctl@ceph.mon.a.smithi006.stdout: use_delta_encoding: 1 2024-05-09T04:39:51.910 INFO:journalctl@ceph.mon.a.smithi006.stdout: filter_policy: bloomfilter 2024-05-09T04:39:51.910 INFO:journalctl@ceph.mon.a.smithi006.stdout: whole_key_filtering: 1 2024-05-09T04:39:51.911 INFO:journalctl@ceph.mon.a.smithi006.stdout: verify_compression: 0 2024-05-09T04:39:51.911 INFO:journalctl@ceph.mon.a.smithi006.stdout: read_amp_bytes_per_bit: 0 2024-05-09T04:39:51.911 INFO:journalctl@ceph.mon.a.smithi006.stdout: format_version: 5 2024-05-09T04:39:51.911 INFO:journalctl@ceph.mon.a.smithi006.stdout: enable_index_compression: 1 2024-05-09T04:39:51.911 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_align: 0 2024-05-09T04:39:51.911 INFO:journalctl@ceph.mon.a.smithi006.stdout: max_auto_readahead_size: 262144 2024-05-09T04:39:51.912 INFO:journalctl@ceph.mon.a.smithi006.stdout: prepopulate_block_cache: 0 2024-05-09T04:39:51.912 INFO:journalctl@ceph.mon.a.smithi006.stdout: initial_auto_readahead_size: 8192 2024-05-09T04:39:51.912 INFO:journalctl@ceph.mon.a.smithi006.stdout: num_file_reads_for_auto_readahead: 2 2024-05-09T04:39:51.912 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.write_buffer_size: 33554432 2024-05-09T04:39:51.912 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.max_write_buffer_number: 2 2024-05-09T04:39:51.912 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.compression: NoCompression 2024-05-09T04:39:51.912 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.bottommost_compression: Disabled 2024-05-09T04:39:51.913 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.prefix_extractor: nullptr 2024-05-09T04:39:51.913 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-09T04:39:51.913 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.num_levels: 7 2024-05-09T04:39:51.913 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-09T04:39:51.913 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-09T04:39:51.913 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-09T04:39:51.914 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-09T04:39:51.914 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-09T04:39:51.914 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-09T04:39:51.914 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-09T04:39:51.914 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-09T04:39:51.915 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-09T04:39:51.915 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-09T04:39:51.915 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-09T04:39:51.915 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-09T04:39:51.915 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-09T04:39:51.915 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.compression_opts.level: 32767 2024-05-09T04:39:51.915 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.compression_opts.strategy: 0 2024-05-09T04:39:51.916 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-09T04:39:51.916 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-09T04:39:51.916 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-09T04:39:51.916 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-09T04:39:51.916 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.compression_opts.enabled: false 2024-05-09T04:39:51.916 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-09T04:39:51.917 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-09T04:39:51.917 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-09T04:39:51.917 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-09T04:39:51.917 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.target_file_size_base: 67108864 2024-05-09T04:39:51.917 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-09T04:39:51.918 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-09T04:39:51.918 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-09T04:39:51.918 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-09T04:39:51.918 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-09T04:39:51.918 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-09T04:39:51.918 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-09T04:39:51.919 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-09T04:39:51.919 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-09T04:39:51.919 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-09T04:39:51.919 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-09T04:39:51.919 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-09T04:39:51.920 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-09T04:39:51.920 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-09T04:39:51.920 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.arena_block_size: 1048576 2024-05-09T04:39:51.920 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-09T04:39:51.920 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-09T04:39:51.920 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.disable_auto_compactions: 0 2024-05-09T04:39:51.920 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-09T04:39:51.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-09T04:39:51.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-09T04:39:51.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-09T04:39:51.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-09T04:39:51.921 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-09T04:39:51.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-09T04:39:51.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-09T04:39:51.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-09T04:39:51.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-09T04:39:51.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-09T04:39:51.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.inplace_update_support: 0 2024-05-09T04:39:51.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-09T04:39:51.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-09T04:39:51.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-09T04:39:51.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-09T04:39:51.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.bloom_locality: 0 2024-05-09T04:39:51.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.max_successive_merges: 0 2024-05-09T04:39:51.924 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-09T04:39:51.924 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.paranoid_file_checks: 0 2024-05-09T04:39:51.924 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.force_consistency_checks: 1 2024-05-09T04:39:51.924 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.report_bg_io_stats: 0 2024-05-09T04:39:51.924 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.ttl: 2592000 2024-05-09T04:39:51.924 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-09T04:39:51.924 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-09T04:39:51.925 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-09T04:39:51.925 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.enable_blob_files: false 2024-05-09T04:39:51.925 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.min_blob_size: 0 2024-05-09T04:39:51.925 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.blob_file_size: 268435456 2024-05-09T04:39:51.925 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-09T04:39:51.925 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-09T04:39:51.925 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-09T04:39:51.926 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-09T04:39:51.926 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-09T04:39:51.926 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.blob_file_starting_level: 0 2024-05-09T04:39:51.926 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-09T04:39:51.926 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-09T04:39:51.926 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-09T04:39:51.927 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c01bf229-4292-483d-b498-ba5c20636b15 2024-05-09T04:39:51.927 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715229591769872, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-09T04:39:51.927 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-09T04:39:51.927 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715229591772939, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100057, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97943, "index_size": 306, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14060, "raw_average_key_size": 49, "raw_value_size": 90283, "raw_average_value_size": 319, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715229591, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c01bf229-4292-483d-b498-ba5c20636b15", "db_session_id": "DZYZHYE627YUVKWWQU3R", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-09T04:39:51.927 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715229591773268, "job": 1, "event": "recovery_finished"} 2024-05-09T04:39:51.927 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-09T04:39:51.928 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-09T04:39:51.928 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56055705dc00 2024-05-09T04:39:51.928 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: DB pointer 0x56055be50000 2024-05-09T04:39:51.928 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-09T04:39:51.928 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-09T04:39:51.928 INFO:journalctl@ceph.mon.a.smithi006.stdout: ** DB Stats ** 2024-05-09T04:39:51.928 INFO:journalctl@ceph.mon.a.smithi006.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T04:39:51.929 INFO:journalctl@ceph.mon.a.smithi006.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-09T04:39:51.929 INFO:journalctl@ceph.mon.a.smithi006.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T04:39:51.929 INFO:journalctl@ceph.mon.a.smithi006.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T04:39:51.929 INFO:journalctl@ceph.mon.a.smithi006.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-09T04:39:51.929 INFO:journalctl@ceph.mon.a.smithi006.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T04:39:51.929 INFO:journalctl@ceph.mon.a.smithi006.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T04:39:51.930 INFO:journalctl@ceph.mon.a.smithi006.stdout: 2024-05-09T04:39:51.930 INFO:journalctl@ceph.mon.a.smithi006.stdout: ** Compaction Stats [default] ** 2024-05-09T04:39:51.930 INFO:journalctl@ceph.mon.a.smithi006.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T04:39:51.930 INFO:journalctl@ceph.mon.a.smithi006.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-09T04:39:51.930 INFO:journalctl@ceph.mon.a.smithi006.stdout: L0 2/0 99.57 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 57.4 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-05-09T04:39:51.931 INFO:journalctl@ceph.mon.a.smithi006.stdout: Sum 2/0 99.57 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 57.4 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-05-09T04:39:51.931 INFO:journalctl@ceph.mon.a.smithi006.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 57.4 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-05-09T04:39:51.931 INFO:journalctl@ceph.mon.a.smithi006.stdout: 2024-05-09T04:39:51.931 INFO:journalctl@ceph.mon.a.smithi006.stdout: ** Compaction Stats [default] ** 2024-05-09T04:39:51.931 INFO:journalctl@ceph.mon.a.smithi006.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T04:39:51.931 INFO:journalctl@ceph.mon.a.smithi006.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-09T04:39:51.931 INFO:journalctl@ceph.mon.a.smithi006.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 57.4 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-05-09T04:39:51.932 INFO:journalctl@ceph.mon.a.smithi006.stdout: 2024-05-09T04:39:51.932 INFO:journalctl@ceph.mon.a.smithi006.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-09T04:39:51.932 INFO:journalctl@ceph.mon.a.smithi006.stdout: 2024-05-09T04:39:51.932 INFO:journalctl@ceph.mon.a.smithi006.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T04:39:51.932 INFO:journalctl@ceph.mon.a.smithi006.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-09T04:39:51.932 INFO:journalctl@ceph.mon.a.smithi006.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-09T04:39:51.933 INFO:journalctl@ceph.mon.a.smithi006.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-09T04:39:51.933 INFO:journalctl@ceph.mon.a.smithi006.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-09T04:39:51.933 INFO:journalctl@ceph.mon.a.smithi006.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-09T04:39:51.933 INFO:journalctl@ceph.mon.a.smithi006.stdout: Cumulative compaction: 0.00 GB write, 9.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T04:39:51.933 INFO:journalctl@ceph.mon.a.smithi006.stdout: Interval compaction: 0.00 GB write, 9.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T04:39:51.933 INFO:journalctl@ceph.mon.a.smithi006.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-09T04:39:51.934 INFO:journalctl@ceph.mon.a.smithi006.stdout: Block cache BinnedLRUCache@0x5605570791f0#2 capacity: 512.00 MB usage: 1.53 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-05-09T04:39:51.934 INFO:journalctl@ceph.mon.a.smithi006.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.03 KB,0.000196695%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-09T04:39:51.934 INFO:journalctl@ceph.mon.a.smithi006.stdout: 2024-05-09T04:39:51.934 INFO:journalctl@ceph.mon.a.smithi006.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-09T04:39:51.935 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: starting mon.a rank 0 at public addrs [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] at bind addrs [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:39:51.935 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: mon.a@-1(???) e1 preinit fsid f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:39:51.935 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: mon.a@-1(???).mds e1 new map 2024-05-09T04:39:51.935 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: mon.a@-1(???).mds e1 print_map 2024-05-09T04:39:51.935 INFO:journalctl@ceph.mon.a.smithi006.stdout: e1 2024-05-09T04:39:51.935 INFO:journalctl@ceph.mon.a.smithi006.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T04:39:51.935 INFO:journalctl@ceph.mon.a.smithi006.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T04:39:51.936 INFO:journalctl@ceph.mon.a.smithi006.stdout: legacy client fscid: -1 2024-05-09T04:39:51.936 INFO:journalctl@ceph.mon.a.smithi006.stdout: 2024-05-09T04:39:51.936 INFO:journalctl@ceph.mon.a.smithi006.stdout: No filesystems configured 2024-05-09T04:39:51.936 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-09T04:39:51.937 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T04:39:51.937 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T04:39:51.937 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T04:39:51.937 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-09T04:39:51.937 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-09T04:39:51.937 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: monmap epoch 1 2024-05-09T04:39:51.938 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: fsid f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:39:51.938 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: last_changed 2024-05-09T04:39:30.984541+0000 2024-05-09T04:39:51.938 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: created 2024-05-09T04:39:30.984541+0000 2024-05-09T04:39:51.938 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: min_mon_release 19 (squid) 2024-05-09T04:39:51.938 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: election_strategy: 1 2024-05-09T04:39:51.939 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: 0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-09T04:39:51.939 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: fsmap 2024-05-09T04:39:51.939 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: osdmap e1: 0 total, 0 up, 0 in 2024-05-09T04:39:51.939 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:51 smithi006 ceph-mon[29835]: mgrmap e1: no daemons active 2024-05-09T04:39:53.245 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.529+0000 7f1a73c00700 1 Processor -- start 2024-05-09T04:39:53.245 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.529+0000 7f1a73c00700 1 -- start start 2024-05-09T04:39:53.245 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.529+0000 7f1a73c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a6c110880 0x7f1a6c110cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:53.245 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.529+0000 7f1a73c00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6c111300 con 0x7f1a6c110880 2024-05-09T04:39:53.245 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.530+0000 7f1a71200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a6c110880 0x7f1a6c110cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:53.245 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.530+0000 7f1a71200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a6c110880 0x7f1a6c110cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45450/0 (socket says 172.21.15.6:45450) 2024-05-09T04:39:53.245 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.530+0000 7f1a71200700 1 -- 172.21.15.6:0/3039098871 learned_addr learned my addr 172.21.15.6:0/3039098871 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:39:53.245 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.531+0000 7f1a71200700 1 -- 172.21.15.6:0/3039098871 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a6c113610 con 0x7f1a6c110880 2024-05-09T04:39:53.245 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.531+0000 7f1a71200700 1 --2- 172.21.15.6:0/3039098871 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a6c110880 0x7f1a6c110cf0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1a600098a0 tx=0x7f1a60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9875f3266dc2e30 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:53.245 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.531+0000 7f1a6b400700 1 -- 172.21.15.6:0/3039098871 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a60004160 con 0x7f1a6c110880 2024-05-09T04:39:53.245 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.531+0000 7f1a6b400700 1 -- 172.21.15.6:0/3039098871 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1a60004300 con 0x7f1a6c110880 2024-05-09T04:39:53.246 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.531+0000 7f1a6b400700 1 -- 172.21.15.6:0/3039098871 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a60011af0 con 0x7f1a6c110880 2024-05-09T04:39:53.246 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.532+0000 7f1a73c00700 1 -- 172.21.15.6:0/3039098871 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a6c110880 msgr2=0x7f1a6c110cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:53.246 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.532+0000 7f1a73c00700 1 --2- 172.21.15.6:0/3039098871 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a6c110880 0x7f1a6c110cf0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1a600098a0 tx=0x7f1a60009bb0 comp rx=0 tx=0).stop 2024-05-09T04:39:53.246 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.532+0000 7f1a73c00700 1 -- 172.21.15.6:0/3039098871 shutdown_connections 2024-05-09T04:39:53.246 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.532+0000 7f1a73c00700 1 --2- 172.21.15.6:0/3039098871 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a6c110880 0x7f1a6c110cf0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:53.246 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.532+0000 7f1a73c00700 1 -- 172.21.15.6:0/3039098871 >> 172.21.15.6:0/3039098871 conn(0x7f1a6c07f1f0 msgr2=0x7f1a6c081640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:39:53.246 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.533+0000 7f1a73c00700 1 -- 172.21.15.6:0/3039098871 shutdown_connections 2024-05-09T04:39:53.246 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.533+0000 7f1a73c00700 1 -- 172.21.15.6:0/3039098871 wait complete. 2024-05-09T04:39:53.246 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.534+0000 7f1a73c00700 1 Processor -- start 2024-05-09T04:39:53.246 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.534+0000 7f1a73c00700 1 -- start start 2024-05-09T04:39:53.246 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.534+0000 7f1a73c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a6c110880 0x7f1a6c1a4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:53.246 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.534+0000 7f1a73c00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6c114570 con 0x7f1a6c110880 2024-05-09T04:39:53.246 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.535+0000 7f1a71200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a6c110880 0x7f1a6c1a4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:53.246 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.535+0000 7f1a71200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a6c110880 0x7f1a6c1a4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45462/0 (socket says 172.21.15.6:45462) 2024-05-09T04:39:53.247 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.535+0000 7f1a71200700 1 -- 172.21.15.6:0/725206852 learned_addr learned my addr 172.21.15.6:0/725206852 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:39:53.249 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.535+0000 7f1a71200700 1 -- 172.21.15.6:0/725206852 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a6c1a4aa0 con 0x7f1a6c110880 2024-05-09T04:39:53.249 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.535+0000 7f1a71200700 1 --2- 172.21.15.6:0/725206852 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a6c110880 0x7f1a6c1a4560 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1a600117b0 tx=0x7f1a60011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:53.249 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.536+0000 7f1a69600700 1 -- 172.21.15.6:0/725206852 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a60003af0 con 0x7f1a6c110880 2024-05-09T04:39:53.249 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.536+0000 7f1a69600700 1 -- 172.21.15.6:0/725206852 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1a60020070 con 0x7f1a6c110880 2024-05-09T04:39:53.249 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.536+0000 7f1a73c00700 1 -- 172.21.15.6:0/725206852 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a6c1a4d30 con 0x7f1a6c110880 2024-05-09T04:39:53.249 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.536+0000 7f1a69600700 1 -- 172.21.15.6:0/725206852 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a60019630 con 0x7f1a6c110880 2024-05-09T04:39:53.249 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.536+0000 7f1a73c00700 1 -- 172.21.15.6:0/725206852 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a6c1a7940 con 0x7f1a6c110880 2024-05-09T04:39:53.249 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.536+0000 7f1a69600700 1 -- 172.21.15.6:0/725206852 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1a60019c30 con 0x7f1a6c110880 2024-05-09T04:39:53.249 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.536+0000 7f1a69600700 1 -- 172.21.15.6:0/725206852 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1a6000b720 con 0x7f1a6c110880 2024-05-09T04:39:53.249 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.537+0000 7f1a73c00700 1 -- 172.21.15.6:0/725206852 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a6c064e20 con 0x7f1a6c110880 2024-05-09T04:39:53.249 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.541+0000 7f1a69600700 1 -- 172.21.15.6:0/725206852 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f1a600042a0 con 0x7f1a6c110880 2024-05-09T04:39:53.250 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.620+0000 7f1a73c00700 1 -- 172.21.15.6:0/725206852 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f1a6c1a7e00 con 0x7f1a6c110880 2024-05-09T04:39:53.250 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.625+0000 7f1a69600700 1 -- 172.21.15.6:0/725206852 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f1a6001a9c0 con 0x7f1a6c110880 2024-05-09T04:39:53.250 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.626+0000 7f1a73c00700 1 -- 172.21.15.6:0/725206852 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a6c110880 msgr2=0x7f1a6c1a4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:53.250 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.626+0000 7f1a73c00700 1 --2- 172.21.15.6:0/725206852 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a6c110880 0x7f1a6c1a4560 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1a600117b0 tx=0x7f1a60011b70 comp rx=0 tx=0).stop 2024-05-09T04:39:53.250 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.626+0000 7f1a73c00700 1 -- 172.21.15.6:0/725206852 shutdown_connections 2024-05-09T04:39:53.250 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.626+0000 7f1a73c00700 1 --2- 172.21.15.6:0/725206852 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a6c110880 0x7f1a6c1a4560 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:53.250 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.626+0000 7f1a73c00700 1 -- 172.21.15.6:0/725206852 >> 172.21.15.6:0/725206852 conn(0x7f1a6c07f1f0 msgr2=0x7f1a6c080c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:39:53.250 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.626+0000 7f1a73c00700 1 -- 172.21.15.6:0/725206852 shutdown_connections 2024-05-09T04:39:53.250 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:52.626+0000 7f1a73c00700 1 -- 172.21.15.6:0/725206852 wait complete. 2024-05-09T04:39:53.250 INFO:teuthology.orchestra.run.smithi006.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-09T04:39:53.273 INFO:teuthology.orchestra.run.smithi006.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-09T04:39:53.274 INFO:teuthology.orchestra.run.smithi006.stdout:Creating mgr... 2024-05-09T04:39:53.275 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-09T04:39:53.276 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-09T04:39:53.716 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mgr.x 2024-05-09T04:39:53.717 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mgr.x.service: Unit ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mgr.x.service not loaded. 2024-05-09T04:39:53.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:53 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/725206852' entity='client.admin' 2024-05-09T04:39:53.993 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f74d813c-0dbd-11ef-bc97-c7b262605968.target.wants/ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@.service. 2024-05-09T04:39:54.403 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:39:54 smithi006 systemd[1]: Starting Ceph mgr.x for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T04:39:54.680 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:39:54 smithi006 podman[30048]: 2024-05-09 04:39:54.402412585 +0000 UTC m=+0.125361166 container create ede2170d178c1d12a413da01b2c3882aa847430f4e564d614a89488d7ab06074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux ) 2024-05-09T04:39:54.680 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:39:54 smithi006 podman[30048]: 2024-05-09 04:39:54.312760193 +0000 UTC m=+0.035708741 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T04:39:54.681 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:39:54 smithi006 podman[30048]: 2024-05-09 04:39:54.680397373 +0000 UTC m=+0.403345931 container init ede2170d178c1d12a413da01b2c3882aa847430f4e564d614a89488d7ab06074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-09T04:39:54.813 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T04:39:54.813 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout disabled 2024-05-09T04:39:54.834 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T04:39:54.835 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2024-05-09T04:39:54.835 INFO:teuthology.orchestra.run.smithi006.stdout:firewalld.service is not enabled 2024-05-09T04:39:54.835 INFO:teuthology.orchestra.run.smithi006.stdout:Not possible to enable service . firewalld.service is not available 2024-05-09T04:39:54.855 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T04:39:54.855 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout disabled 2024-05-09T04:39:54.873 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T04:39:54.873 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2024-05-09T04:39:54.874 INFO:teuthology.orchestra.run.smithi006.stdout:firewalld.service is not enabled 2024-05-09T04:39:54.874 INFO:teuthology.orchestra.run.smithi006.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-09T04:39:54.874 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mgr to start... 2024-05-09T04:39:54.874 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mgr... 2024-05-09T04:39:54.946 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:39:54 smithi006 podman[30048]: 2024-05-09 04:39:54.691306767 +0000 UTC m=+0.414255351 container start ede2170d178c1d12a413da01b2c3882aa847430f4e564d614a89488d7ab06074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-09T04:39:54.946 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:39:54 smithi006 bash[30048]: ede2170d178c1d12a413da01b2c3882aa847430f4e564d614a89488d7ab06074 2024-05-09T04:39:54.947 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:39:54 smithi006 systemd[1]: Started Ceph mgr.x for f74d813c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:39:55.288 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:39:55 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:39:55.025+0000 7fbd32de3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T04:39:55.573 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:39:55 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:39:55.287+0000 7fbd32de3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T04:39:55.573 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:39:55 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:39:55.427+0000 7fbd32de3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T04:39:56.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:55 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/3805696850' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T04:39:56.196 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:39:55 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:39:55.885+0000 7fbd32de3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T04:39:56.196 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:39:56 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:39:56.015+0000 7fbd32de3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T04:39:56.642 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-05-09T04:39:56.642 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-05-09T04:39:56.642 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsid": "f74d813c-0dbd-11ef-bc97-c7b262605968", 2024-05-09T04:39:56.642 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T04:39:56.642 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T04:39:56.642 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T04:39:56.642 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T04:39:56.643 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:56.643 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T04:39:56.643 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T04:39:56.643 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 0 2024-05-09T04:39:56.643 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-05-09T04:39:56.643 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T04:39:56.643 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T04:39:56.643 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-05-09T04:39:56.643 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-05-09T04:39:56.643 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T04:39:56.643 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T04:39:56.643 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T04:39:56.643 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T04:39:56.643 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:56.644 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T04:39:56.644 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T04:39:56.644 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T04:39:56.644 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T04:39:56.644 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T04:39:56.644 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T04:39:56.644 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T04:39:56.644 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T04:39:56.644 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:56.644 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T04:39:56.648 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T04:39:56.648 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T04:39:56.648 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T04:39:56.648 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T04:39:56.648 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T04:39:56.648 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T04:39:56.648 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T04:39:56.648 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T04:39:56.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:56.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T04:39:56.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T04:39:56.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T04:39:56.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T04:39:56.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:56.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T04:39:56.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-09T04:39:56.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T04:39:56.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T04:39:56.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T04:39:56.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T04:39:56.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T04:39:56.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-05-09T04:39:56.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T04:39:56.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:56.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T04:39:56.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T04:39:56.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T04:39:35.288282+0000", 2024-05-09T04:39:56.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T04:39:56.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:56.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T04:39:56.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-05-09T04:39:56.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.661+0000 7f6c9e800700 1 Processor -- start 2024-05-09T04:39:56.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.661+0000 7f6c9e800700 1 -- start start 2024-05-09T04:39:56.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.662+0000 7f6c9e800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c9810a610 0x7f6c9810ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:56.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.662+0000 7f6c9e800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9807cdc0 con 0x7f6c9810a610 2024-05-09T04:39:56.651 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.662+0000 7f6c97400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c9810a610 0x7f6c9810ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:56.651 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.663+0000 7f6c97400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c9810a610 0x7f6c9810ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45472/0 (socket says 172.21.15.6:45472) 2024-05-09T04:39:56.651 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.663+0000 7f6c97400700 1 -- 172.21.15.6:0/3843221016 learned_addr learned my addr 172.21.15.6:0/3843221016 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:39:56.651 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.663+0000 7f6c97400700 1 -- 172.21.15.6:0/3843221016 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c9810cf70 con 0x7f6c9810a610 2024-05-09T04:39:56.652 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.663+0000 7f6c97400700 1 --2- 172.21.15.6:0/3843221016 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c9810a610 0x7f6c9810ca30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6c880098a0 tx=0x7f6c88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b005fa1970b8199e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:56.652 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.664+0000 7f6c96000700 1 -- 172.21.15.6:0/3843221016 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c88004160 con 0x7f6c9810a610 2024-05-09T04:39:56.652 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.664+0000 7f6c96000700 1 -- 172.21.15.6:0/3843221016 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6c88004300 con 0x7f6c9810a610 2024-05-09T04:39:56.653 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.664+0000 7f6c96000700 1 -- 172.21.15.6:0/3843221016 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c88011af0 con 0x7f6c9810a610 2024-05-09T04:39:56.653 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.665+0000 7f6c9e800700 1 -- 172.21.15.6:0/3843221016 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c9810a610 msgr2=0x7f6c9810ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:56.653 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.665+0000 7f6c9e800700 1 --2- 172.21.15.6:0/3843221016 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c9810a610 0x7f6c9810ca30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6c880098a0 tx=0x7f6c88009bb0 comp rx=0 tx=0).stop 2024-05-09T04:39:56.653 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.665+0000 7f6c9e800700 1 -- 172.21.15.6:0/3843221016 shutdown_connections 2024-05-09T04:39:56.653 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.665+0000 7f6c9e800700 1 --2- 172.21.15.6:0/3843221016 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c9810a610 0x7f6c9810ca30 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:56.653 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.665+0000 7f6c9e800700 1 -- 172.21.15.6:0/3843221016 >> 172.21.15.6:0/3843221016 conn(0x7f6c98104070 msgr2=0x7f6c981064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:39:56.653 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.665+0000 7f6c9e800700 1 -- 172.21.15.6:0/3843221016 shutdown_connections 2024-05-09T04:39:56.653 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.665+0000 7f6c9e800700 1 -- 172.21.15.6:0/3843221016 wait complete. 2024-05-09T04:39:56.653 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.666+0000 7f6c9e800700 1 Processor -- start 2024-05-09T04:39:56.653 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.667+0000 7f6c9e800700 1 -- start start 2024-05-09T04:39:56.653 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.667+0000 7f6c9e800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c9810a610 0x7f6c981a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:56.654 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.667+0000 7f6c9e800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9810de70 con 0x7f6c9810a610 2024-05-09T04:39:56.654 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.667+0000 7f6c97400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c9810a610 0x7f6c981a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:56.654 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.667+0000 7f6c97400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c9810a610 0x7f6c981a0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45482/0 (socket says 172.21.15.6:45482) 2024-05-09T04:39:56.654 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.667+0000 7f6c97400700 1 -- 172.21.15.6:0/3805696850 learned_addr learned my addr 172.21.15.6:0/3805696850 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:39:56.654 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.668+0000 7f6c97400700 1 -- 172.21.15.6:0/3805696850 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c981a05d0 con 0x7f6c9810a610 2024-05-09T04:39:56.654 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.668+0000 7f6c97400700 1 --2- 172.21.15.6:0/3805696850 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c9810a610 0x7f6c981a0090 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6c880117b0 tx=0x7f6c88011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:56.654 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.668+0000 7f6c8fe00700 1 -- 172.21.15.6:0/3805696850 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c88003af0 con 0x7f6c9810a610 2024-05-09T04:39:56.654 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.668+0000 7f6c8fe00700 1 -- 172.21.15.6:0/3805696850 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6c88020070 con 0x7f6c9810a610 2024-05-09T04:39:56.654 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.668+0000 7f6c9e800700 1 -- 172.21.15.6:0/3805696850 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c981a0860 con 0x7f6c9810a610 2024-05-09T04:39:56.654 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.668+0000 7f6c8fe00700 1 -- 172.21.15.6:0/3805696850 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c88019630 con 0x7f6c9810a610 2024-05-09T04:39:56.654 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.668+0000 7f6c9e800700 1 -- 172.21.15.6:0/3805696850 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c981a3470 con 0x7f6c9810a610 2024-05-09T04:39:56.654 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.669+0000 7f6c8fe00700 1 -- 172.21.15.6:0/3805696850 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6c88019c80 con 0x7f6c9810a610 2024-05-09T04:39:56.654 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.669+0000 7f6c8fe00700 1 -- 172.21.15.6:0/3805696850 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6c8800b720 con 0x7f6c9810a610 2024-05-09T04:39:56.655 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.670+0000 7f6c9e800700 1 -- 172.21.15.6:0/3805696850 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c98064e20 con 0x7f6c9810a610 2024-05-09T04:39:56.655 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.673+0000 7f6c8fe00700 1 -- 172.21.15.6:0/3805696850 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f6c880042a0 con 0x7f6c9810a610 2024-05-09T04:39:56.655 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.775+0000 7f6c9e800700 1 -- 172.21.15.6:0/3805696850 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f6c981a3760 con 0x7f6c9810a610 2024-05-09T04:39:56.655 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.776+0000 7f6c8fe00700 1 -- 172.21.15.6:0/3805696850 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f6c8801a9c0 con 0x7f6c9810a610 2024-05-09T04:39:56.655 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.777+0000 7f6c9e800700 1 -- 172.21.15.6:0/3805696850 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c9810a610 msgr2=0x7f6c981a0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:56.655 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.777+0000 7f6c9e800700 1 --2- 172.21.15.6:0/3805696850 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c9810a610 0x7f6c981a0090 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6c880117b0 tx=0x7f6c88011b70 comp rx=0 tx=0).stop 2024-05-09T04:39:56.655 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.777+0000 7f6c9e800700 1 -- 172.21.15.6:0/3805696850 shutdown_connections 2024-05-09T04:39:56.655 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.777+0000 7f6c9e800700 1 --2- 172.21.15.6:0/3805696850 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c9810a610 0x7f6c981a0090 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:56.655 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.777+0000 7f6c9e800700 1 -- 172.21.15.6:0/3805696850 >> 172.21.15.6:0/3805696850 conn(0x7f6c98104070 msgr2=0x7f6c98105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:39:56.655 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.777+0000 7f6c9e800700 1 -- 172.21.15.6:0/3805696850 shutdown_connections 2024-05-09T04:39:56.655 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.777+0000 7f6c9e800700 1 -- 172.21.15.6:0/3805696850 wait complete. 2024-05-09T04:39:56.656 INFO:teuthology.orchestra.run.smithi006.stdout:mgr not available, waiting (1/15)... 2024-05-09T04:39:57.946 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:39:57 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:39:57.683+0000 7fbd32de3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T04:39:57.946 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:39:57 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:39:57.922+0000 7fbd32de3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T04:39:58.446 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:39:58 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:39:58.144+0000 7fbd32de3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T04:39:58.810 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:39:58 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:39:58.543+0000 7fbd32de3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T04:39:58.810 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:39:58 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:39:58.672+0000 7fbd32de3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T04:39:59.123 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:39:58 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:39:58.934+0000 7fbd32de3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T04:39:59.446 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:39:59 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:39:59.202+0000 7fbd32de3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T04:39:59.819 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:39:59 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1843765786' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T04:40:00.135 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-05-09T04:40:00.135 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-05-09T04:40:00.135 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsid": "f74d813c-0dbd-11ef-bc97-c7b262605968", 2024-05-09T04:40:00.138 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T04:40:00.138 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T04:40:00.138 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T04:40:00.138 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T04:40:00.138 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:40:00.138 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T04:40:00.138 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T04:40:00.138 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 0 2024-05-09T04:40:00.139 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-05-09T04:40:00.139 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T04:40:00.139 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T04:40:00.139 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-05-09T04:40:00.139 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-05-09T04:40:00.139 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T04:40:00.139 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T04:40:00.139 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T04:40:00.139 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T04:40:00.139 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:40:00.139 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T04:40:00.139 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T04:40:00.139 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T04:40:00.140 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T04:40:00.140 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T04:40:00.141 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T04:40:00.141 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T04:40:00.141 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T04:40:00.141 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:40:00.141 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T04:40:00.141 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T04:40:00.141 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T04:40:00.141 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T04:40:00.149 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T04:40:00.149 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T04:40:00.149 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T04:40:00.149 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T04:40:00.149 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T04:40:00.150 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:40:00.150 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T04:40:00.150 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T04:40:00.150 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T04:40:00.150 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T04:40:00.150 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:40:00.150 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T04:40:00.150 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-09T04:40:00.150 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T04:40:00.150 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T04:40:00.150 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T04:40:00.150 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T04:40:00.150 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T04:40:00.151 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-05-09T04:40:00.151 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T04:40:00.151 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:40:00.151 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T04:40:00.151 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T04:40:00.151 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T04:39:35.288282+0000", 2024-05-09T04:40:00.151 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T04:40:00.151 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:40:00.151 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T04:40:00.151 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-05-09T04:40:00.151 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.356+0000 7fd4bbe00700 1 Processor -- start 2024-05-09T04:40:00.151 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.357+0000 7fd4bbe00700 1 -- start start 2024-05-09T04:40:00.151 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.357+0000 7fd4bbe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4b4109b10 0x7fd4b4109f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:00.151 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.357+0000 7fd4bbe00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b410a510 con 0x7fd4b4109b10 2024-05-09T04:40:00.151 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.358+0000 7fd4b9400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4b4109b10 0x7fd4b4109f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:00.152 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.358+0000 7fd4b9400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4b4109b10 0x7fd4b4109f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35314/0 (socket says 172.21.15.6:35314) 2024-05-09T04:40:00.152 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.358+0000 7fd4b9400700 1 -- 172.21.15.6:0/101692797 learned_addr learned my addr 172.21.15.6:0/101692797 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:00.152 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.358+0000 7fd4b9400700 1 -- 172.21.15.6:0/101692797 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b410ac90 con 0x7fd4b4109b10 2024-05-09T04:40:00.152 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.358+0000 7fd4b9400700 1 --2- 172.21.15.6:0/101692797 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4b4109b10 0x7fd4b4109f20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd4a80098a0 tx=0x7fd4a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26e60310f24abf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:00.152 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.359+0000 7fd4b3e00700 1 -- 172.21.15.6:0/101692797 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4a8004160 con 0x7fd4b4109b10 2024-05-09T04:40:00.152 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.359+0000 7fd4b3e00700 1 -- 172.21.15.6:0/101692797 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd4a8004300 con 0x7fd4b4109b10 2024-05-09T04:40:00.152 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.359+0000 7fd4b3e00700 1 -- 172.21.15.6:0/101692797 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4a8011af0 con 0x7fd4b4109b10 2024-05-09T04:40:00.152 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.360+0000 7fd4bbe00700 1 -- 172.21.15.6:0/101692797 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4b4109b10 msgr2=0x7fd4b4109f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:00.152 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.360+0000 7fd4bbe00700 1 --2- 172.21.15.6:0/101692797 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4b4109b10 0x7fd4b4109f20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd4a80098a0 tx=0x7fd4a8009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:00.152 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.360+0000 7fd4bbe00700 1 -- 172.21.15.6:0/101692797 shutdown_connections 2024-05-09T04:40:00.152 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.360+0000 7fd4bbe00700 1 --2- 172.21.15.6:0/101692797 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4b4109b10 0x7fd4b4109f20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:00.152 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.360+0000 7fd4bbe00700 1 -- 172.21.15.6:0/101692797 >> 172.21.15.6:0/101692797 conn(0x7fd4b4105230 msgr2=0x7fd4b4107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:00.152 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.360+0000 7fd4bbe00700 1 -- 172.21.15.6:0/101692797 shutdown_connections 2024-05-09T04:40:00.152 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.360+0000 7fd4bbe00700 1 -- 172.21.15.6:0/101692797 wait complete. 2024-05-09T04:40:00.153 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.361+0000 7fd4bbe00700 1 Processor -- start 2024-05-09T04:40:00.153 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.361+0000 7fd4bbe00700 1 -- start start 2024-05-09T04:40:00.153 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.362+0000 7fd4bbe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4b4109b10 0x7fd4b419bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:00.153 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.362+0000 7fd4bbe00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b407daf0 con 0x7fd4b4109b10 2024-05-09T04:40:00.153 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.362+0000 7fd4b9400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4b4109b10 0x7fd4b419bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:00.153 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.362+0000 7fd4b9400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4b4109b10 0x7fd4b419bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35320/0 (socket says 172.21.15.6:35320) 2024-05-09T04:40:00.153 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.362+0000 7fd4b9400700 1 -- 172.21.15.6:0/1843765786 learned_addr learned my addr 172.21.15.6:0/1843765786 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:00.153 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.362+0000 7fd4b9400700 1 -- 172.21.15.6:0/1843765786 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b419c210 con 0x7fd4b4109b10 2024-05-09T04:40:00.153 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.363+0000 7fd4b9400700 1 --2- 172.21.15.6:0/1843765786 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4b4109b10 0x7fd4b419bcd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8011890 tx=0x7fd4a8011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:00.153 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.363+0000 7fd4b2000700 1 -- 172.21.15.6:0/1843765786 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4a80039e0 con 0x7fd4b4109b10 2024-05-09T04:40:00.153 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.363+0000 7fd4b2000700 1 -- 172.21.15.6:0/1843765786 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd4a8003b80 con 0x7fd4b4109b10 2024-05-09T04:40:00.153 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.363+0000 7fd4b2000700 1 -- 172.21.15.6:0/1843765786 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4a8024750 con 0x7fd4b4109b10 2024-05-09T04:40:00.153 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.363+0000 7fd4bbe00700 1 -- 172.21.15.6:0/1843765786 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4b419c4a0 con 0x7fd4b4109b10 2024-05-09T04:40:00.153 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.363+0000 7fd4bbe00700 1 -- 172.21.15.6:0/1843765786 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4b419f0b0 con 0x7fd4b4109b10 2024-05-09T04:40:00.153 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.364+0000 7fd4b2000700 1 -- 172.21.15.6:0/1843765786 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd4a8022020 con 0x7fd4b4109b10 2024-05-09T04:40:00.154 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.364+0000 7fd4b2000700 1 -- 172.21.15.6:0/1843765786 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd4a802dcf0 con 0x7fd4b4109b10 2024-05-09T04:40:00.154 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.364+0000 7fd4bbe00700 1 -- 172.21.15.6:0/1843765786 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4b4064e20 con 0x7fd4b4109b10 2024-05-09T04:40:00.154 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.369+0000 7fd4b2000700 1 -- 172.21.15.6:0/1843765786 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fd4a8043ec0 con 0x7fd4b4109b10 2024-05-09T04:40:00.154 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.450+0000 7fd4bbe00700 1 -- 172.21.15.6:0/1843765786 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd4b402e8f0 con 0x7fd4b4109b10 2024-05-09T04:40:00.154 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.451+0000 7fd4b2000700 1 -- 172.21.15.6:0/1843765786 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fd4a8032020 con 0x7fd4b4109b10 2024-05-09T04:40:00.154 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.452+0000 7fd4bbe00700 1 -- 172.21.15.6:0/1843765786 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4b4109b10 msgr2=0x7fd4b419bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:00.154 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.452+0000 7fd4bbe00700 1 --2- 172.21.15.6:0/1843765786 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4b4109b10 0x7fd4b419bcd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8011890 tx=0x7fd4a8011970 comp rx=0 tx=0).stop 2024-05-09T04:40:00.154 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.453+0000 7fd4bbe00700 1 -- 172.21.15.6:0/1843765786 shutdown_connections 2024-05-09T04:40:00.154 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.453+0000 7fd4bbe00700 1 --2- 172.21.15.6:0/1843765786 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4b4109b10 0x7fd4b419bcd0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:00.154 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.453+0000 7fd4bbe00700 1 -- 172.21.15.6:0/1843765786 >> 172.21.15.6:0/1843765786 conn(0x7fd4b4105230 msgr2=0x7fd4b41073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:00.154 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.453+0000 7fd4bbe00700 1 -- 172.21.15.6:0/1843765786 shutdown_connections 2024-05-09T04:40:00.154 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.453+0000 7fd4bbe00700 1 -- 172.21.15.6:0/1843765786 wait complete. 2024-05-09T04:40:00.155 INFO:teuthology.orchestra.run.smithi006.stdout:mgr not available, waiting (2/15)... 2024-05-09T04:40:00.382 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:00 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:00.073+0000 7fbd32de3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T04:40:00.382 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:00 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:00.381+0000 7fbd32de3200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T04:40:01.446 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:00 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:00.961+0000 7fbd32de3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T04:40:01.446 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:01 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:01.096+0000 7fbd32de3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T04:40:01.946 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:01 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:01.596+0000 7fbd32de3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T04:40:02.326 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:02 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:02.074+0000 7fbd32de3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T04:40:02.327 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:02 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:02.210+0000 7fbd32de3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T04:40:02.581 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:02 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:02.564+0000 7fbd32de3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T04:40:02.893 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:02 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:02.759+0000 7fbd32de3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T04:40:02.893 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:02 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:02.894+0000 7fbd32de3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T04:40:03.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:02 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/220364463' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T04:40:03.489 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:03 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:03.313+0000 7fbd32de3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T04:40:03.489 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:03 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:03.448+0000 7fbd32de3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T04:40:03.507 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-05-09T04:40:03.507 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-05-09T04:40:03.509 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsid": "f74d813c-0dbd-11ef-bc97-c7b262605968", 2024-05-09T04:40:03.509 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T04:40:03.510 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T04:40:03.510 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T04:40:03.510 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T04:40:03.510 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:40:03.510 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T04:40:03.510 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T04:40:03.510 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 0 2024-05-09T04:40:03.510 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-05-09T04:40:03.510 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T04:40:03.510 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T04:40:03.510 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-05-09T04:40:03.510 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-05-09T04:40:03.510 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T04:40:03.510 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T04:40:03.511 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T04:40:03.511 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T04:40:03.511 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:40:03.511 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T04:40:03.511 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T04:40:03.511 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T04:40:03.511 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T04:40:03.519 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T04:40:03.520 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T04:40:03.520 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T04:40:03.520 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T04:40:03.520 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:40:03.520 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T04:40:03.520 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T04:40:03.520 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T04:40:03.520 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T04:40:03.520 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T04:40:03.520 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T04:40:03.521 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T04:40:03.521 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T04:40:03.521 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T04:40:03.521 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:40:03.521 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T04:40:03.521 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T04:40:03.521 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T04:40:03.521 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T04:40:03.521 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:40:03.521 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T04:40:03.521 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-09T04:40:03.521 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T04:40:03.521 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T04:40:03.521 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T04:40:03.522 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T04:40:03.522 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T04:40:03.522 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-05-09T04:40:03.522 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T04:40:03.522 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:40:03.522 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T04:40:03.522 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T04:40:03.522 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T04:39:35.288282+0000", 2024-05-09T04:40:03.523 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T04:40:03.523 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:40:03.523 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T04:40:03.523 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-05-09T04:40:03.523 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.804+0000 7fc35d200700 1 Processor -- start 2024-05-09T04:40:03.523 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.804+0000 7fc35d200700 1 -- start start 2024-05-09T04:40:03.523 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.804+0000 7fc35d200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc358109b00 0x7fc358109f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:03.523 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.804+0000 7fc35d200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc35810a500 con 0x7fc358109b00 2024-05-09T04:40:03.523 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.805+0000 7fc355e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc358109b00 0x7fc358109f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:03.523 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.805+0000 7fc355e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc358109b00 0x7fc358109f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35324/0 (socket says 172.21.15.6:35324) 2024-05-09T04:40:03.523 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.805+0000 7fc355e00700 1 -- 172.21.15.6:0/4199314481 learned_addr learned my addr 172.21.15.6:0/4199314481 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:03.523 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.805+0000 7fc355e00700 1 -- 172.21.15.6:0/4199314481 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc35810ac80 con 0x7fc358109b00 2024-05-09T04:40:03.523 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.806+0000 7fc355e00700 1 --2- 172.21.15.6:0/4199314481 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc358109b00 0x7fc358109f10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc3480098a0 tx=0x7fc348009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=957200d809652b46 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:03.524 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.806+0000 7fc354a00700 1 -- 172.21.15.6:0/4199314481 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc348004160 con 0x7fc358109b00 2024-05-09T04:40:03.524 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.806+0000 7fc354a00700 1 -- 172.21.15.6:0/4199314481 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc348004300 con 0x7fc358109b00 2024-05-09T04:40:03.524 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.806+0000 7fc354a00700 1 -- 172.21.15.6:0/4199314481 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc348011af0 con 0x7fc358109b00 2024-05-09T04:40:03.524 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.807+0000 7fc35d200700 1 -- 172.21.15.6:0/4199314481 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc358109b00 msgr2=0x7fc358109f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:03.524 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.807+0000 7fc35d200700 1 --2- 172.21.15.6:0/4199314481 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc358109b00 0x7fc358109f10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc3480098a0 tx=0x7fc348009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:03.524 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.807+0000 7fc35d200700 1 -- 172.21.15.6:0/4199314481 shutdown_connections 2024-05-09T04:40:03.524 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.807+0000 7fc35d200700 1 --2- 172.21.15.6:0/4199314481 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc358109b00 0x7fc358109f10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:03.524 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.807+0000 7fc35d200700 1 -- 172.21.15.6:0/4199314481 >> 172.21.15.6:0/4199314481 conn(0x7fc358105120 msgr2=0x7fc3581075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:03.524 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.808+0000 7fc35d200700 1 -- 172.21.15.6:0/4199314481 shutdown_connections 2024-05-09T04:40:03.524 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.808+0000 7fc35d200700 1 -- 172.21.15.6:0/4199314481 wait complete. 2024-05-09T04:40:03.524 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.809+0000 7fc35d200700 1 Processor -- start 2024-05-09T04:40:03.524 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.809+0000 7fc35d200700 1 -- start start 2024-05-09T04:40:03.524 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.809+0000 7fc35d200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc358109b00 0x7fc3581a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:03.525 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.809+0000 7fc35d200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc35810e030 con 0x7fc358109b00 2024-05-09T04:40:03.525 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.810+0000 7fc355e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc358109b00 0x7fc3581a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:03.525 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.810+0000 7fc355e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc358109b00 0x7fc3581a01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35330/0 (socket says 172.21.15.6:35330) 2024-05-09T04:40:03.525 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.810+0000 7fc355e00700 1 -- 172.21.15.6:0/220364463 learned_addr learned my addr 172.21.15.6:0/220364463 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:03.525 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.810+0000 7fc355e00700 1 -- 172.21.15.6:0/220364463 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3581a06f0 con 0x7fc358109b00 2024-05-09T04:40:03.525 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.810+0000 7fc355e00700 1 --2- 172.21.15.6:0/220364463 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc358109b00 0x7fc3581a01b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc348011890 tx=0x7fc348011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:03.525 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.811+0000 7fc34ea00700 1 -- 172.21.15.6:0/220364463 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3480039e0 con 0x7fc358109b00 2024-05-09T04:40:03.525 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.811+0000 7fc34ea00700 1 -- 172.21.15.6:0/220364463 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc348003b80 con 0x7fc358109b00 2024-05-09T04:40:03.525 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.811+0000 7fc34ea00700 1 -- 172.21.15.6:0/220364463 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3480246e0 con 0x7fc358109b00 2024-05-09T04:40:03.525 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.811+0000 7fc35d200700 1 -- 172.21.15.6:0/220364463 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3581a0980 con 0x7fc358109b00 2024-05-09T04:40:03.525 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.811+0000 7fc35d200700 1 -- 172.21.15.6:0/220364463 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3581a3590 con 0x7fc358109b00 2024-05-09T04:40:03.525 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.811+0000 7fc34ea00700 1 -- 172.21.15.6:0/220364463 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc348022020 con 0x7fc358109b00 2024-05-09T04:40:03.525 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.811+0000 7fc34ea00700 1 -- 172.21.15.6:0/220364463 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc34802dc90 con 0x7fc358109b00 2024-05-09T04:40:03.525 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.812+0000 7fc35d200700 1 -- 172.21.15.6:0/220364463 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc358064e20 con 0x7fc358109b00 2024-05-09T04:40:03.526 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.815+0000 7fc34ea00700 1 -- 172.21.15.6:0/220364463 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fc34801aa70 con 0x7fc358109b00 2024-05-09T04:40:03.526 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.895+0000 7fc35d200700 1 -- 172.21.15.6:0/220364463 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc35802e5f0 con 0x7fc358109b00 2024-05-09T04:40:03.526 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.896+0000 7fc34ea00700 1 -- 172.21.15.6:0/220364463 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7fc35802e5f0 con 0x7fc358109b00 2024-05-09T04:40:03.526 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.898+0000 7fc35d200700 1 -- 172.21.15.6:0/220364463 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc358109b00 msgr2=0x7fc3581a01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:03.526 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.898+0000 7fc35d200700 1 --2- 172.21.15.6:0/220364463 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc358109b00 0x7fc3581a01b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc348011890 tx=0x7fc348011970 comp rx=0 tx=0).stop 2024-05-09T04:40:03.526 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.898+0000 7fc35d200700 1 -- 172.21.15.6:0/220364463 shutdown_connections 2024-05-09T04:40:03.526 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.898+0000 7fc35d200700 1 --2- 172.21.15.6:0/220364463 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc358109b00 0x7fc3581a01b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:03.526 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.898+0000 7fc35d200700 1 -- 172.21.15.6:0/220364463 >> 172.21.15.6:0/220364463 conn(0x7fc358105120 msgr2=0x7fc35807d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:03.526 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.898+0000 7fc35d200700 1 -- 172.21.15.6:0/220364463 shutdown_connections 2024-05-09T04:40:03.526 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.899+0000 7fc35d200700 1 -- 172.21.15.6:0/220364463 wait complete. 2024-05-09T04:40:03.526 INFO:teuthology.orchestra.run.smithi006.stdout:mgr not available, waiting (3/15)... 2024-05-09T04:40:04.552 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:04 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:04.253+0000 7fbd32de3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T04:40:04.946 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:04 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:04.552+0000 7fbd32de3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T04:40:04.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:04 smithi006 ceph-mon[29835]: Activating manager daemon x 2024-05-09T04:40:04.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:04 smithi006 ceph-mon[29835]: mgrmap e2: x(active, starting, since 0.0119773s) 2024-05-09T04:40:04.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:04 smithi006 ceph-mon[29835]: from='mgr.14100 172.21.15.6:0/2556401518' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T04:40:04.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:04 smithi006 ceph-mon[29835]: from='mgr.14100 172.21.15.6:0/2556401518' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T04:40:04.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:04 smithi006 ceph-mon[29835]: from='mgr.14100 172.21.15.6:0/2556401518' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T04:40:04.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:04 smithi006 ceph-mon[29835]: from='mgr.14100 172.21.15.6:0/2556401518' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:40:04.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:04 smithi006 ceph-mon[29835]: from='mgr.14100 172.21.15.6:0/2556401518' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T04:40:04.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:04 smithi006 ceph-mon[29835]: Manager daemon x is now available 2024-05-09T04:40:04.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:04 smithi006 ceph-mon[29835]: from='mgr.14100 172.21.15.6:0/2556401518' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T04:40:04.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:04 smithi006 ceph-mon[29835]: from='mgr.14100 172.21.15.6:0/2556401518' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T04:40:05.881 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:05 smithi006 ceph-mon[29835]: from='mgr.14100 172.21.15.6:0/2556401518' entity='mgr.x' 2024-05-09T04:40:05.881 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:05 smithi006 ceph-mon[29835]: from='mgr.14100 172.21.15.6:0/2556401518' entity='mgr.x' 2024-05-09T04:40:05.881 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:05 smithi006 ceph-mon[29835]: from='mgr.14100 172.21.15.6:0/2556401518' entity='mgr.x' 2024-05-09T04:40:05.881 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:05 smithi006 ceph-mon[29835]: mgrmap e3: x(active, since 1.01265s) 2024-05-09T04:40:06.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:06 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/655073083' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T04:40:07.312 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-05-09T04:40:07.312 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-05-09T04:40:07.313 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsid": "f74d813c-0dbd-11ef-bc97-c7b262605968", 2024-05-09T04:40:07.313 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T04:40:07.314 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T04:40:07.314 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T04:40:07.314 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T04:40:07.314 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:40:07.314 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T04:40:07.314 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T04:40:07.314 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 0 2024-05-09T04:40:07.315 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-05-09T04:40:07.315 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T04:40:07.315 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T04:40:07.315 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-05-09T04:40:07.316 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-05-09T04:40:07.316 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T04:40:07.316 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T04:40:07.316 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T04:40:07.316 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T04:40:07.316 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:40:07.316 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T04:40:07.316 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T04:40:07.317 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T04:40:07.317 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T04:40:07.317 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T04:40:07.320 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T04:40:07.320 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T04:40:07.321 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T04:40:07.321 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:40:07.321 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T04:40:07.321 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T04:40:07.321 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T04:40:07.321 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T04:40:07.321 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T04:40:07.321 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T04:40:07.321 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T04:40:07.321 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T04:40:07.321 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T04:40:07.321 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:40:07.321 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T04:40:07.321 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T04:40:07.321 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T04:40:07.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T04:40:07.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:40:07.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T04:40:07.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-09T04:40:07.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T04:40:07.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T04:40:07.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T04:40:07.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T04:40:07.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T04:40:07.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-05-09T04:40:07.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T04:40:07.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:40:07.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T04:40:07.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T04:40:07.323 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T04:39:35.288282+0000", 2024-05-09T04:40:07.323 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T04:40:07.323 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:40:07.323 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T04:40:07.323 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-05-09T04:40:07.327 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.257+0000 7ffbaf200700 1 Processor -- start 2024-05-09T04:40:07.328 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.257+0000 7ffbaf200700 1 -- start start 2024-05-09T04:40:07.328 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.257+0000 7ffbaf200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbb0110730 0x7ffbb0112b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:07.328 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.257+0000 7ffbaf200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb0113120 con 0x7ffbb0110730 2024-05-09T04:40:07.328 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.258+0000 7ffbade00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbb0110730 0x7ffbb0112b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:07.328 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.258+0000 7ffbade00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbb0110730 0x7ffbb0112b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35408/0 (socket says 172.21.15.6:35408) 2024-05-09T04:40:07.328 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.258+0000 7ffbade00700 1 -- 172.21.15.6:0/828597341 learned_addr learned my addr 172.21.15.6:0/828597341 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:07.328 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.258+0000 7ffbade00700 1 -- 172.21.15.6:0/828597341 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb01132a0 con 0x7ffbb0110730 2024-05-09T04:40:07.328 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.258+0000 7ffbade00700 1 --2- 172.21.15.6:0/828597341 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbb0110730 0x7ffbb0112b50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ffba00098a0 tx=0x7ffba0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a38267f8ff024f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:07.328 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.259+0000 7ffbad400700 1 -- 172.21.15.6:0/828597341 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffba0004160 con 0x7ffbb0110730 2024-05-09T04:40:07.328 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.259+0000 7ffbad400700 1 -- 172.21.15.6:0/828597341 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ffba0004300 con 0x7ffbb0110730 2024-05-09T04:40:07.328 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.259+0000 7ffbad400700 1 -- 172.21.15.6:0/828597341 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffba0011af0 con 0x7ffbb0110730 2024-05-09T04:40:07.328 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.259+0000 7ffbaf200700 1 -- 172.21.15.6:0/828597341 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbb0110730 msgr2=0x7ffbb0112b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:07.328 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.259+0000 7ffbaf200700 1 --2- 172.21.15.6:0/828597341 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbb0110730 0x7ffbb0112b50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ffba00098a0 tx=0x7ffba0009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:07.329 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.260+0000 7ffbaf200700 1 -- 172.21.15.6:0/828597341 shutdown_connections 2024-05-09T04:40:07.329 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.260+0000 7ffbaf200700 1 --2- 172.21.15.6:0/828597341 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbb0110730 0x7ffbb0112b50 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:07.329 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.260+0000 7ffbaf200700 1 -- 172.21.15.6:0/828597341 >> 172.21.15.6:0/828597341 conn(0x7ffbb007f020 msgr2=0x7ffbb0081440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:07.329 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.260+0000 7ffbaf200700 1 -- 172.21.15.6:0/828597341 shutdown_connections 2024-05-09T04:40:07.329 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.260+0000 7ffbaf200700 1 -- 172.21.15.6:0/828597341 wait complete. 2024-05-09T04:40:07.329 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.261+0000 7ffbaf200700 1 Processor -- start 2024-05-09T04:40:07.329 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.261+0000 7ffbaf200700 1 -- start start 2024-05-09T04:40:07.329 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.261+0000 7ffbaf200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbb0110730 0x7ffbb01a4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:07.329 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.261+0000 7ffbaf200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb0114320 con 0x7ffbb0110730 2024-05-09T04:40:07.329 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.262+0000 7ffbade00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbb0110730 0x7ffbb01a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:07.329 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.262+0000 7ffbade00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbb0110730 0x7ffbb01a4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35420/0 (socket says 172.21.15.6:35420) 2024-05-09T04:40:07.329 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.262+0000 7ffbade00700 1 -- 172.21.15.6:0/655073083 learned_addr learned my addr 172.21.15.6:0/655073083 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:07.330 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.262+0000 7ffbade00700 1 -- 172.21.15.6:0/655073083 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb01a48c0 con 0x7ffbb0110730 2024-05-09T04:40:07.330 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.262+0000 7ffbade00700 1 --2- 172.21.15.6:0/655073083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbb0110730 0x7ffbb01a4380 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ffba00117b0 tx=0x7ffba0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:07.330 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.263+0000 7ffba6a00700 1 -- 172.21.15.6:0/655073083 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffba00039c0 con 0x7ffbb0110730 2024-05-09T04:40:07.330 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.263+0000 7ffba6a00700 1 -- 172.21.15.6:0/655073083 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ffba0003b60 con 0x7ffbb0110730 2024-05-09T04:40:07.330 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.263+0000 7ffbaf200700 1 -- 172.21.15.6:0/655073083 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb01a4b50 con 0x7ffbb0110730 2024-05-09T04:40:07.330 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.263+0000 7ffba6a00700 1 -- 172.21.15.6:0/655073083 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffba00196a0 con 0x7ffbb0110730 2024-05-09T04:40:07.330 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.263+0000 7ffbaf200700 1 -- 172.21.15.6:0/655073083 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb01a7760 con 0x7ffbb0110730 2024-05-09T04:40:07.330 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.264+0000 7ffbaf200700 1 -- 172.21.15.6:0/655073083 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbb0064e20 con 0x7ffbb0110730 2024-05-09T04:40:07.330 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.264+0000 7ffba6a00700 1 -- 172.21.15.6:0/655073083 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 3) ==== 50105+0+0 (secure 0 0 0) 0x7ffba001a430 con 0x7ffbb0110730 2024-05-09T04:40:07.330 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.265+0000 7ffba6a00700 1 --2- 172.21.15.6:0/655073083 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7ffb9403e350 0x7ffb94040860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:07.330 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.265+0000 7ffba6a00700 1 -- 172.21.15.6:0/655073083 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ffba0053860 con 0x7ffbb0110730 2024-05-09T04:40:07.331 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.265+0000 7ffba7e00700 1 --2- 172.21.15.6:0/655073083 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7ffb9403e350 0x7ffb94040860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:07.331 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.266+0000 7ffba7e00700 1 --2- 172.21.15.6:0/655073083 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7ffb9403e350 0x7ffb94040860 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ffb98006fd0 tx=0x7ffb98006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:07.331 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.271+0000 7ffba6a00700 1 -- 172.21.15.6:0/655073083 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffba000b3e0 con 0x7ffbb0110730 2024-05-09T04:40:07.331 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.597+0000 7ffbaf200700 1 -- 172.21.15.6:0/655073083 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ffbb01a7de0 con 0x7ffbb0110730 2024-05-09T04:40:07.331 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.598+0000 7ffba6a00700 1 -- 172.21.15.6:0/655073083 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7ffba00172b0 con 0x7ffbb0110730 2024-05-09T04:40:07.331 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.601+0000 7ffbaf200700 1 -- 172.21.15.6:0/655073083 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7ffb9403e350 msgr2=0x7ffb94040860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:07.331 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.601+0000 7ffbaf200700 1 --2- 172.21.15.6:0/655073083 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7ffb9403e350 0x7ffb94040860 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ffb98006fd0 tx=0x7ffb98006e40 comp rx=0 tx=0).stop 2024-05-09T04:40:07.331 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.601+0000 7ffbaf200700 1 -- 172.21.15.6:0/655073083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbb0110730 msgr2=0x7ffbb01a4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:07.331 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.601+0000 7ffbaf200700 1 --2- 172.21.15.6:0/655073083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbb0110730 0x7ffbb01a4380 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ffba00117b0 tx=0x7ffba0011b70 comp rx=0 tx=0).stop 2024-05-09T04:40:07.331 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.601+0000 7ffbaf200700 1 -- 172.21.15.6:0/655073083 shutdown_connections 2024-05-09T04:40:07.331 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.601+0000 7ffbaf200700 1 --2- 172.21.15.6:0/655073083 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7ffb9403e350 0x7ffb94040860 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:07.331 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.601+0000 7ffbaf200700 1 --2- 172.21.15.6:0/655073083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffbb0110730 0x7ffbb01a4380 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:07.331 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.601+0000 7ffbaf200700 1 -- 172.21.15.6:0/655073083 >> 172.21.15.6:0/655073083 conn(0x7ffbb007f020 msgr2=0x7ffbb007f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:07.331 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.601+0000 7ffbaf200700 1 -- 172.21.15.6:0/655073083 shutdown_connections 2024-05-09T04:40:07.331 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.602+0000 7ffbaf200700 1 -- 172.21.15.6:0/655073083 wait complete. 2024-05-09T04:40:07.331 INFO:teuthology.orchestra.run.smithi006.stdout:mgr is available 2024-05-09T04:40:07.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:07 smithi006 ceph-mon[29835]: mgrmap e4: x(active, since 2s) 2024-05-09T04:40:08.885 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:08 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/685395004' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-09T04:40:08.897 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-05-09T04:40:08.897 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [global] 2024-05-09T04:40:08.897 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout fsid = f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:40:08.897 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-09T04:40:08.898 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-09T04:40:08.898 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-09T04:40:08.898 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-09T04:40:08.898 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-09T04:40:08.905 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-05-09T04:40:08.906 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-09T04:40:08.906 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-09T04:40:08.906 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-05-09T04:40:08.906 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [osd] 2024-05-09T04:40:08.906 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-09T04:40:08.906 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-09T04:40:08.906 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-09T04:40:08.906 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-09T04:40:08.906 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-09T04:40:08.906 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.015+0000 7fa6a8c00700 1 Processor -- start 2024-05-09T04:40:08.906 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.015+0000 7fa6a8c00700 1 -- start start 2024-05-09T04:40:08.906 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.016+0000 7fa6a8c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6a407edc0 0x7fa6a407d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:08.906 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.016+0000 7fa6a8c00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a407d780 con 0x7fa6a407edc0 2024-05-09T04:40:08.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.016+0000 7fa6a1e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6a407edc0 0x7fa6a407d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:08.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.017+0000 7fa6a1e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6a407edc0 0x7fa6a407d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58250/0 (socket says 172.21.15.6:58250) 2024-05-09T04:40:08.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.017+0000 7fa6a1e00700 1 -- 172.21.15.6:0/3102832862 learned_addr learned my addr 172.21.15.6:0/3102832862 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:08.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.017+0000 7fa6a1e00700 1 -- 172.21.15.6:0/3102832862 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a407d900 con 0x7fa6a407edc0 2024-05-09T04:40:08.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.018+0000 7fa6a1e00700 1 --2- 172.21.15.6:0/3102832862 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6a407edc0 0x7fa6a407d240 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa6940098a0 tx=0x7fa694009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c36d81a7871ced1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:08.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.018+0000 7fa6a0a00700 1 -- 172.21.15.6:0/3102832862 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa694004160 con 0x7fa6a407edc0 2024-05-09T04:40:08.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.018+0000 7fa6a0a00700 1 -- 172.21.15.6:0/3102832862 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa694004300 con 0x7fa6a407edc0 2024-05-09T04:40:08.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.018+0000 7fa6a0a00700 1 -- 172.21.15.6:0/3102832862 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa694011af0 con 0x7fa6a407edc0 2024-05-09T04:40:08.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.019+0000 7fa6a8c00700 1 -- 172.21.15.6:0/3102832862 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6a407edc0 msgr2=0x7fa6a407d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:08.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.019+0000 7fa6a8c00700 1 --2- 172.21.15.6:0/3102832862 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6a407edc0 0x7fa6a407d240 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa6940098a0 tx=0x7fa694009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:08.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.019+0000 7fa6a8c00700 1 -- 172.21.15.6:0/3102832862 shutdown_connections 2024-05-09T04:40:08.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.019+0000 7fa6a8c00700 1 --2- 172.21.15.6:0/3102832862 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6a407edc0 0x7fa6a407d240 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:08.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.019+0000 7fa6a8c00700 1 -- 172.21.15.6:0/3102832862 >> 172.21.15.6:0/3102832862 conn(0x7fa6a4105120 msgr2=0x7fa6a41075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:08.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.019+0000 7fa6a8c00700 1 -- 172.21.15.6:0/3102832862 shutdown_connections 2024-05-09T04:40:08.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.019+0000 7fa6a8c00700 1 -- 172.21.15.6:0/3102832862 wait complete. 2024-05-09T04:40:08.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.020+0000 7fa6a8c00700 1 Processor -- start 2024-05-09T04:40:08.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.020+0000 7fa6a8c00700 1 -- start start 2024-05-09T04:40:08.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.021+0000 7fa6a8c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6a407edc0 0x7fa6a41a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:08.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.021+0000 7fa6a8c00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a41100e0 con 0x7fa6a407edc0 2024-05-09T04:40:08.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.021+0000 7fa6a1e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6a407edc0 0x7fa6a41a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:08.909 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.021+0000 7fa6a1e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6a407edc0 0x7fa6a41a00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58266/0 (socket says 172.21.15.6:58266) 2024-05-09T04:40:08.909 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.021+0000 7fa6a1e00700 1 -- 172.21.15.6:0/685395004 learned_addr learned my addr 172.21.15.6:0/685395004 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:08.909 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.021+0000 7fa6a1e00700 1 -- 172.21.15.6:0/685395004 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a41a0620 con 0x7fa6a407edc0 2024-05-09T04:40:08.909 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.022+0000 7fa6a1e00700 1 --2- 172.21.15.6:0/685395004 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6a407edc0 0x7fa6a41a00e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa6940117b0 tx=0x7fa694011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:08.909 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.022+0000 7fa69aa00700 1 -- 172.21.15.6:0/685395004 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa694003af0 con 0x7fa6a407edc0 2024-05-09T04:40:08.909 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.022+0000 7fa69aa00700 1 -- 172.21.15.6:0/685395004 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa694020070 con 0x7fa6a407edc0 2024-05-09T04:40:08.909 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.022+0000 7fa6a8c00700 1 -- 172.21.15.6:0/685395004 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a41a08b0 con 0x7fa6a407edc0 2024-05-09T04:40:08.909 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.022+0000 7fa69aa00700 1 -- 172.21.15.6:0/685395004 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa694019630 con 0x7fa6a407edc0 2024-05-09T04:40:08.909 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.022+0000 7fa6a8c00700 1 -- 172.21.15.6:0/685395004 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a41a34c0 con 0x7fa6a407edc0 2024-05-09T04:40:08.910 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.024+0000 7fa6a8c00700 1 -- 172.21.15.6:0/685395004 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6a4064e20 con 0x7fa6a407edc0 2024-05-09T04:40:08.910 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.024+0000 7fa69aa00700 1 -- 172.21.15.6:0/685395004 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fa6940198a0 con 0x7fa6a407edc0 2024-05-09T04:40:08.910 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.024+0000 7fa69aa00700 1 --2- 172.21.15.6:0/685395004 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7fa68803e470 0x7fa688040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:08.910 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.024+0000 7fa69aa00700 1 -- 172.21.15.6:0/685395004 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa694053bc0 con 0x7fa6a407edc0 2024-05-09T04:40:08.910 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.025+0000 7fa6a1400700 1 --2- 172.21.15.6:0/685395004 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7fa68803e470 0x7fa688040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:08.910 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.026+0000 7fa6a1400700 1 --2- 172.21.15.6:0/685395004 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7fa68803e470 0x7fa688040980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa690006fd0 tx=0x7fa690006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:08.910 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.031+0000 7fa69aa00700 1 -- 172.21.15.6:0/685395004 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa6940535e0 con 0x7fa6a407edc0 2024-05-09T04:40:08.910 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.267+0000 7fa6a8c00700 1 -- 172.21.15.6:0/685395004 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fa6a41a3690 con 0x7fa6a407edc0 2024-05-09T04:40:08.911 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.271+0000 7fa69aa00700 1 -- 172.21.15.6:0/685395004 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7fa6940042a0 con 0x7fa6a407edc0 2024-05-09T04:40:08.911 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.274+0000 7fa6a8c00700 1 -- 172.21.15.6:0/685395004 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7fa68803e470 msgr2=0x7fa688040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:08.911 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.274+0000 7fa6a8c00700 1 --2- 172.21.15.6:0/685395004 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7fa68803e470 0x7fa688040980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa690006fd0 tx=0x7fa690006e40 comp rx=0 tx=0).stop 2024-05-09T04:40:08.911 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.274+0000 7fa6a8c00700 1 -- 172.21.15.6:0/685395004 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6a407edc0 msgr2=0x7fa6a41a00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:08.911 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.274+0000 7fa6a8c00700 1 --2- 172.21.15.6:0/685395004 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6a407edc0 0x7fa6a41a00e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa6940117b0 tx=0x7fa694011b70 comp rx=0 tx=0).stop 2024-05-09T04:40:08.911 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.274+0000 7fa6a8c00700 1 -- 172.21.15.6:0/685395004 shutdown_connections 2024-05-09T04:40:08.911 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.274+0000 7fa6a8c00700 1 --2- 172.21.15.6:0/685395004 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7fa68803e470 0x7fa688040980 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:08.912 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.274+0000 7fa6a8c00700 1 --2- 172.21.15.6:0/685395004 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6a407edc0 0x7fa6a41a00e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:08.912 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.275+0000 7fa6a8c00700 1 -- 172.21.15.6:0/685395004 >> 172.21.15.6:0/685395004 conn(0x7fa6a4105120 msgr2=0x7fa6a4105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:08.912 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.275+0000 7fa6a8c00700 1 -- 172.21.15.6:0/685395004 shutdown_connections 2024-05-09T04:40:08.912 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:08.275+0000 7fa6a8c00700 1 -- 172.21.15.6:0/685395004 wait complete. 2024-05-09T04:40:08.913 INFO:teuthology.orchestra.run.smithi006.stdout:Enabling cephadm module... 2024-05-09T04:40:10.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:09 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/148072736' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-09T04:40:10.196 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:10 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: ignoring --setuser ceph since I am not root 2024-05-09T04:40:10.197 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:10 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: ignoring --setgroup ceph since I am not root 2024-05-09T04:40:10.625 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:10 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:10.339+0000 7f48aec84200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T04:40:10.626 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:10 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:10.586+0000 7f48aec84200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T04:40:10.640 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.602+0000 7f87d3c00700 1 Processor -- start 2024-05-09T04:40:10.640 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.602+0000 7f87d3c00700 1 -- start start 2024-05-09T04:40:10.641 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.603+0000 7f87d3c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87cc07ee00 0x7f87cc07d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:10.641 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.603+0000 7f87d3c00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc07d7c0 con 0x7f87cc07ee00 2024-05-09T04:40:10.641 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.603+0000 7f87d1200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87cc07ee00 0x7f87cc07d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:10.641 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.604+0000 7f87d1200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87cc07ee00 0x7f87cc07d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58280/0 (socket says 172.21.15.6:58280) 2024-05-09T04:40:10.641 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.604+0000 7f87d1200700 1 -- 172.21.15.6:0/1719638861 learned_addr learned my addr 172.21.15.6:0/1719638861 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:10.641 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.604+0000 7f87d1200700 1 -- 172.21.15.6:0/1719638861 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87cc07d940 con 0x7f87cc07ee00 2024-05-09T04:40:10.641 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.604+0000 7f87d1200700 1 --2- 172.21.15.6:0/1719638861 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87cc07ee00 0x7f87cc07d280 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f87c00098a0 tx=0x7f87c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3e3cd3c11f2784 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:10.641 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.605+0000 7f87cb400700 1 -- 172.21.15.6:0/1719638861 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87c0004160 con 0x7f87cc07ee00 2024-05-09T04:40:10.641 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.605+0000 7f87cb400700 1 -- 172.21.15.6:0/1719638861 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f87c0004300 con 0x7f87cc07ee00 2024-05-09T04:40:10.641 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.605+0000 7f87cb400700 1 -- 172.21.15.6:0/1719638861 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87c0011af0 con 0x7f87cc07ee00 2024-05-09T04:40:10.641 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.606+0000 7f87d3c00700 1 -- 172.21.15.6:0/1719638861 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87cc07ee00 msgr2=0x7f87cc07d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:10.641 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.606+0000 7f87d3c00700 1 --2- 172.21.15.6:0/1719638861 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87cc07ee00 0x7f87cc07d280 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f87c00098a0 tx=0x7f87c0009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:10.641 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.606+0000 7f87d3c00700 1 -- 172.21.15.6:0/1719638861 shutdown_connections 2024-05-09T04:40:10.642 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.606+0000 7f87d3c00700 1 --2- 172.21.15.6:0/1719638861 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87cc07ee00 0x7f87cc07d280 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:10.642 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.606+0000 7f87d3c00700 1 -- 172.21.15.6:0/1719638861 >> 172.21.15.6:0/1719638861 conn(0x7f87cc105120 msgr2=0x7f87cc1075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:10.642 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.606+0000 7f87d3c00700 1 -- 172.21.15.6:0/1719638861 shutdown_connections 2024-05-09T04:40:10.642 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.606+0000 7f87d3c00700 1 -- 172.21.15.6:0/1719638861 wait complete. 2024-05-09T04:40:10.642 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.607+0000 7f87d3c00700 1 Processor -- start 2024-05-09T04:40:10.642 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.607+0000 7f87d3c00700 1 -- start start 2024-05-09T04:40:10.642 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.608+0000 7f87d3c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87cc07ee00 0x7f87cc1a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:10.642 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.608+0000 7f87d3c00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc1100a0 con 0x7f87cc07ee00 2024-05-09T04:40:10.642 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.608+0000 7f87d1200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87cc07ee00 0x7f87cc1a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:10.642 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.608+0000 7f87d1200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87cc07ee00 0x7f87cc1a0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58294/0 (socket says 172.21.15.6:58294) 2024-05-09T04:40:10.642 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.608+0000 7f87d1200700 1 -- 172.21.15.6:0/148072736 learned_addr learned my addr 172.21.15.6:0/148072736 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:10.642 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.608+0000 7f87d1200700 1 -- 172.21.15.6:0/148072736 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87cc1a0680 con 0x7f87cc07ee00 2024-05-09T04:40:10.642 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.609+0000 7f87d1200700 1 --2- 172.21.15.6:0/148072736 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87cc07ee00 0x7f87cc1a0140 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f87c00117b0 tx=0x7f87c0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:10.643 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.609+0000 7f87c9600700 1 -- 172.21.15.6:0/148072736 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87c0003af0 con 0x7f87cc07ee00 2024-05-09T04:40:10.643 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.609+0000 7f87c9600700 1 -- 172.21.15.6:0/148072736 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f87c0020070 con 0x7f87cc07ee00 2024-05-09T04:40:10.643 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.609+0000 7f87d3c00700 1 -- 172.21.15.6:0/148072736 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87cc1a0910 con 0x7f87cc07ee00 2024-05-09T04:40:10.643 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.609+0000 7f87c9600700 1 -- 172.21.15.6:0/148072736 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87c0019630 con 0x7f87cc07ee00 2024-05-09T04:40:10.643 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.609+0000 7f87d3c00700 1 -- 172.21.15.6:0/148072736 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87cc1a3520 con 0x7f87cc07ee00 2024-05-09T04:40:10.643 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.610+0000 7f87c9600700 1 -- 172.21.15.6:0/148072736 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f87c001a430 con 0x7f87cc07ee00 2024-05-09T04:40:10.643 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.611+0000 7f87bea00700 1 -- 172.21.15.6:0/148072736 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87a8005330 con 0x7f87cc07ee00 2024-05-09T04:40:10.643 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.611+0000 7f87c9600700 1 --2- 172.21.15.6:0/148072736 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7f87b403e470 0x7f87b4040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:10.643 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.611+0000 7f87c9600700 1 -- 172.21.15.6:0/148072736 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f87c0053a60 con 0x7f87cc07ee00 2024-05-09T04:40:10.643 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.611+0000 7f87cbe00700 1 --2- 172.21.15.6:0/148072736 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7f87b403e470 0x7f87b4040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:10.643 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.612+0000 7f87cbe00700 1 --2- 172.21.15.6:0/148072736 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7f87b403e470 0x7f87b4040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f87b8006fd0 tx=0x7f87b8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:10.643 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.617+0000 7f87c9600700 1 -- 172.21.15.6:0/148072736 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f87c0021de0 con 0x7f87cc07ee00 2024-05-09T04:40:10.643 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.918+0000 7f87bea00700 1 -- 172.21.15.6:0/148072736 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f87a8005620 con 0x7f87cc07ee00 2024-05-09T04:40:10.644 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.972+0000 7f87c9600700 1 -- 172.21.15.6:0/148072736 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f87c000be00 con 0x7f87cc07ee00 2024-05-09T04:40:10.644 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.976+0000 7f87c9600700 1 -- 172.21.15.6:0/148072736 <== mon.0 v2:172.21.15.6:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f87c001a9c0 con 0x7f87cc07ee00 2024-05-09T04:40:10.644 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.977+0000 7f87bea00700 1 -- 172.21.15.6:0/148072736 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7f87b403e470 msgr2=0x7f87b4040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:10.644 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.977+0000 7f87bea00700 1 --2- 172.21.15.6:0/148072736 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7f87b403e470 0x7f87b4040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f87b8006fd0 tx=0x7f87b8006e40 comp rx=0 tx=0).stop 2024-05-09T04:40:10.644 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.977+0000 7f87bea00700 1 -- 172.21.15.6:0/148072736 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87cc07ee00 msgr2=0x7f87cc1a0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:10.644 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.977+0000 7f87bea00700 1 --2- 172.21.15.6:0/148072736 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87cc07ee00 0x7f87cc1a0140 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f87c00117b0 tx=0x7f87c0011b70 comp rx=0 tx=0).stop 2024-05-09T04:40:10.644 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.978+0000 7f87bea00700 1 -- 172.21.15.6:0/148072736 shutdown_connections 2024-05-09T04:40:10.644 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.978+0000 7f87bea00700 1 --2- 172.21.15.6:0/148072736 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7f87b403e470 0x7f87b4040980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:10.644 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.978+0000 7f87bea00700 1 --2- 172.21.15.6:0/148072736 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87cc07ee00 0x7f87cc1a0140 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:10.644 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.978+0000 7f87bea00700 1 -- 172.21.15.6:0/148072736 >> 172.21.15.6:0/148072736 conn(0x7f87cc105120 msgr2=0x7f87cc105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:10.644 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.978+0000 7f87bea00700 1 -- 172.21.15.6:0/148072736 shutdown_connections 2024-05-09T04:40:10.644 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:09.978+0000 7f87bea00700 1 -- 172.21.15.6:0/148072736 wait complete. 2024-05-09T04:40:10.931 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:10 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:10.725+0000 7f48aec84200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T04:40:11.196 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:11 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:11.142+0000 7f48aec84200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T04:40:11.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:10 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/148072736' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-09T04:40:11.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:10 smithi006 ceph-mon[29835]: mgrmap e5: x(active, since 5s) 2024-05-09T04:40:11.563 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:11 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:11.264+0000 7f48aec84200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T04:40:12.373 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:11 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1243567483' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-09T04:40:12.389 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-05-09T04:40:12.389 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-09T04:40:12.389 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-09T04:40:12.389 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-09T04:40:12.389 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-09T04:40:12.389 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-05-09T04:40:12.389 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.354+0000 7fc8f7c00700 1 Processor -- start 2024-05-09T04:40:12.389 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.354+0000 7fc8f7c00700 1 -- start start 2024-05-09T04:40:12.389 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.355+0000 7fc8f7c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8f007edc0 0x7fc8f007d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:12.389 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.355+0000 7fc8f7c00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f007d780 con 0x7fc8f007edc0 2024-05-09T04:40:12.389 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.355+0000 7fc8f5200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8f007edc0 0x7fc8f007d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:12.389 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.355+0000 7fc8f5200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8f007edc0 0x7fc8f007d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58316/0 (socket says 172.21.15.6:58316) 2024-05-09T04:40:12.390 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.355+0000 7fc8f5200700 1 -- 172.21.15.6:0/3586852008 learned_addr learned my addr 172.21.15.6:0/3586852008 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:12.390 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.356+0000 7fc8f5200700 1 -- 172.21.15.6:0/3586852008 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f007d900 con 0x7fc8f007edc0 2024-05-09T04:40:12.390 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.356+0000 7fc8f5200700 1 --2- 172.21.15.6:0/3586852008 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8f007edc0 0x7fc8f007d240 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc8e40098a0 tx=0x7fc8e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74351e7ac5f58000 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:12.390 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.356+0000 7fc8ef400700 1 -- 172.21.15.6:0/3586852008 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8e4004160 con 0x7fc8f007edc0 2024-05-09T04:40:12.390 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.356+0000 7fc8ef400700 1 -- 172.21.15.6:0/3586852008 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc8e4004300 con 0x7fc8f007edc0 2024-05-09T04:40:12.390 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.356+0000 7fc8ef400700 1 -- 172.21.15.6:0/3586852008 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8e4011af0 con 0x7fc8f007edc0 2024-05-09T04:40:12.390 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.357+0000 7fc8f7c00700 1 -- 172.21.15.6:0/3586852008 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8f007edc0 msgr2=0x7fc8f007d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:12.390 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.357+0000 7fc8f7c00700 1 --2- 172.21.15.6:0/3586852008 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8f007edc0 0x7fc8f007d240 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc8e40098a0 tx=0x7fc8e4009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:12.390 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.357+0000 7fc8f7c00700 1 -- 172.21.15.6:0/3586852008 shutdown_connections 2024-05-09T04:40:12.390 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.357+0000 7fc8f7c00700 1 --2- 172.21.15.6:0/3586852008 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8f007edc0 0x7fc8f007d240 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:12.390 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.357+0000 7fc8f7c00700 1 -- 172.21.15.6:0/3586852008 >> 172.21.15.6:0/3586852008 conn(0x7fc8f0105120 msgr2=0x7fc8f01075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:12.390 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.358+0000 7fc8f7c00700 1 -- 172.21.15.6:0/3586852008 shutdown_connections 2024-05-09T04:40:12.390 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.358+0000 7fc8f7c00700 1 -- 172.21.15.6:0/3586852008 wait complete. 2024-05-09T04:40:12.391 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.359+0000 7fc8f7c00700 1 Processor -- start 2024-05-09T04:40:12.391 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.359+0000 7fc8f7c00700 1 -- start start 2024-05-09T04:40:12.391 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.359+0000 7fc8f7c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8f007edc0 0x7fc8f01a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:12.391 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.359+0000 7fc8f7c00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f01100e0 con 0x7fc8f007edc0 2024-05-09T04:40:12.391 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.360+0000 7fc8f5200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8f007edc0 0x7fc8f01a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:12.391 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.360+0000 7fc8f5200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8f007edc0 0x7fc8f01a00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58332/0 (socket says 172.21.15.6:58332) 2024-05-09T04:40:12.391 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.360+0000 7fc8f5200700 1 -- 172.21.15.6:0/1243567483 learned_addr learned my addr 172.21.15.6:0/1243567483 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:12.391 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.360+0000 7fc8f5200700 1 -- 172.21.15.6:0/1243567483 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f01a0620 con 0x7fc8f007edc0 2024-05-09T04:40:12.391 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.360+0000 7fc8f5200700 1 --2- 172.21.15.6:0/1243567483 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8f007edc0 0x7fc8f01a00e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4009fb0 tx=0x7fc8e4011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:12.391 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.360+0000 7fc8ed600700 1 -- 172.21.15.6:0/1243567483 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8e40036a0 con 0x7fc8f007edc0 2024-05-09T04:40:12.391 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.361+0000 7fc8f7c00700 1 -- 172.21.15.6:0/1243567483 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8f01a08b0 con 0x7fc8f007edc0 2024-05-09T04:40:12.391 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.361+0000 7fc8ed600700 1 -- 172.21.15.6:0/1243567483 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc8e401a430 con 0x7fc8f007edc0 2024-05-09T04:40:12.391 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.361+0000 7fc8ed600700 1 -- 172.21.15.6:0/1243567483 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8e4019550 con 0x7fc8f007edc0 2024-05-09T04:40:12.391 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.361+0000 7fc8f7c00700 1 -- 172.21.15.6:0/1243567483 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8f01a34c0 con 0x7fc8f007edc0 2024-05-09T04:40:12.391 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.362+0000 7fc8ed600700 1 -- 172.21.15.6:0/1243567483 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fc8e4019850 con 0x7fc8f007edc0 2024-05-09T04:40:12.392 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.362+0000 7fc8f7c00700 1 -- 172.21.15.6:0/1243567483 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8f0064e20 con 0x7fc8f007edc0 2024-05-09T04:40:12.392 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.363+0000 7fc8ed600700 1 --2- 172.21.15.6:0/1243567483 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7fc8d803e420 0x7fc8d8040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:12.392 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.363+0000 7fc8ed600700 1 -- 172.21.15.6:0/1243567483 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc8e4053890 con 0x7fc8f007edc0 2024-05-09T04:40:12.392 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.363+0000 7fc8efe00700 1 -- 172.21.15.6:0/1243567483 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7fc8d803e420 msgr2=0x7fc8d8040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3021503778 2024-05-09T04:40:12.392 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.363+0000 7fc8efe00700 1 --2- 172.21.15.6:0/1243567483 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7fc8d803e420 0x7fc8d8040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T04:40:12.392 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.368+0000 7fc8ed600700 1 -- 172.21.15.6:0/1243567483 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc8e4017e90 con 0x7fc8f007edc0 2024-05-09T04:40:12.392 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.564+0000 7fc8efe00700 1 -- 172.21.15.6:0/1243567483 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7fc8d803e420 msgr2=0x7fc8d8040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3021503778 2024-05-09T04:40:12.392 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.564+0000 7fc8efe00700 1 --2- 172.21.15.6:0/1243567483 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7fc8d803e420 0x7fc8d8040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T04:40:12.392 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.662+0000 7fc8f7c00700 1 -- 172.21.15.6:0/1243567483 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fc8f01a3b40 con 0x7fc8f007edc0 2024-05-09T04:40:12.392 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.663+0000 7fc8ed600700 1 -- 172.21.15.6:0/1243567483 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fc8e40532b0 con 0x7fc8f007edc0 2024-05-09T04:40:12.392 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.666+0000 7fc8f7c00700 1 -- 172.21.15.6:0/1243567483 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7fc8d803e420 msgr2=0x7fc8d8040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T04:40:12.392 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.666+0000 7fc8f7c00700 1 --2- 172.21.15.6:0/1243567483 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7fc8d803e420 0x7fc8d8040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:12.392 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.666+0000 7fc8f7c00700 1 -- 172.21.15.6:0/1243567483 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8f007edc0 msgr2=0x7fc8f01a00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:12.392 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.666+0000 7fc8f7c00700 1 --2- 172.21.15.6:0/1243567483 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8f007edc0 0x7fc8f01a00e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4009fb0 tx=0x7fc8e4011940 comp rx=0 tx=0).stop 2024-05-09T04:40:12.393 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.666+0000 7fc8f7c00700 1 -- 172.21.15.6:0/1243567483 shutdown_connections 2024-05-09T04:40:12.393 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.666+0000 7fc8f7c00700 1 --2- 172.21.15.6:0/1243567483 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7fc8d803e420 0x7fc8d8040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:12.393 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.666+0000 7fc8f7c00700 1 --2- 172.21.15.6:0/1243567483 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8f007edc0 0x7fc8f01a00e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:12.393 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.666+0000 7fc8f7c00700 1 -- 172.21.15.6:0/1243567483 >> 172.21.15.6:0/1243567483 conn(0x7fc8f0105120 msgr2=0x7fc8f0105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:12.393 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.666+0000 7fc8f7c00700 1 -- 172.21.15.6:0/1243567483 shutdown_connections 2024-05-09T04:40:12.393 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.666+0000 7fc8f7c00700 1 -- 172.21.15.6:0/1243567483 wait complete. 2024-05-09T04:40:12.393 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for the mgr to restart... 2024-05-09T04:40:12.393 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mgr epoch 5... 2024-05-09T04:40:12.946 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:12 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:12.880+0000 7f48aec84200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T04:40:13.340 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:13 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:13.121+0000 7f48aec84200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T04:40:13.341 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:13 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:13.341+0000 7f48aec84200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T04:40:13.866 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:13 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:13.738+0000 7f48aec84200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T04:40:13.866 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:13 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:13.866+0000 7f48aec84200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T04:40:14.387 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:14 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:14.124+0000 7f48aec84200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T04:40:14.696 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:14 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:14.388+0000 7f48aec84200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T04:40:15.526 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:15 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:15.225+0000 7f48aec84200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T04:40:15.526 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:15 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:15.526+0000 7f48aec84200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T04:40:16.446 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:16 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:16.100+0000 7f48aec84200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T04:40:16.446 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:16 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:16.232+0000 7f48aec84200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T04:40:17.196 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:16 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:16.718+0000 7f48aec84200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T04:40:17.663 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:17 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:17.198+0000 7f48aec84200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T04:40:17.664 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:17 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:17.329+0000 7f48aec84200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T04:40:17.946 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:17 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:17.663+0000 7f48aec84200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T04:40:17.946 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:17 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:17.851+0000 7f48aec84200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T04:40:18.397 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:17 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:17.983+0000 7f48aec84200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T04:40:18.696 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:18 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:18.396+0000 7f48aec84200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T04:40:18.696 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:18 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:18.527+0000 7f48aec84200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T04:40:19.634 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:19 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:19.336+0000 7f48aec84200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T04:40:19.946 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:19 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:19.634+0000 7f48aec84200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T04:40:19.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:19 smithi006 ceph-mon[29835]: Active manager daemon x restarted 2024-05-09T04:40:19.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:19 smithi006 ceph-mon[29835]: Activating manager daemon x 2024-05-09T04:40:19.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:19 smithi006 ceph-mon[29835]: osdmap e2: 0 total, 0 up, 0 in 2024-05-09T04:40:19.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:19 smithi006 ceph-mon[29835]: mgrmap e6: x(active, starting, since 0.0118181s) 2024-05-09T04:40:19.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:19 smithi006 ceph-mon[29835]: from='mgr.14120 172.21.15.6:0/4038153362' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:40:19.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:19 smithi006 ceph-mon[29835]: from='mgr.14120 172.21.15.6:0/4038153362' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T04:40:19.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:19 smithi006 ceph-mon[29835]: from='mgr.14120 172.21.15.6:0/4038153362' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T04:40:19.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:19 smithi006 ceph-mon[29835]: from='mgr.14120 172.21.15.6:0/4038153362' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T04:40:19.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:19 smithi006 ceph-mon[29835]: from='mgr.14120 172.21.15.6:0/4038153362' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T04:40:19.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:19 smithi006 ceph-mon[29835]: Manager daemon x is now available 2024-05-09T04:40:20.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:20 smithi006 ceph-mon[29835]: Found migration_current of "None". Setting to last migration. 2024-05-09T04:40:20.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:20 smithi006 ceph-mon[29835]: from='mgr.14120 172.21.15.6:0/4038153362' entity='mgr.x' 2024-05-09T04:40:20.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:20 smithi006 ceph-mon[29835]: from='mgr.14120 172.21.15.6:0/4038153362' entity='mgr.x' 2024-05-09T04:40:20.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:20 smithi006 ceph-mon[29835]: from='mgr.14120 172.21.15.6:0/4038153362' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:40:20.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:20 smithi006 ceph-mon[29835]: from='mgr.14120 172.21.15.6:0/4038153362' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:40:20.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:20 smithi006 ceph-mon[29835]: from='mgr.14120 172.21.15.6:0/4038153362' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T04:40:20.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:20 smithi006 ceph-mon[29835]: from='mgr.14120 172.21.15.6:0/4038153362' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T04:40:20.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:20 smithi006 ceph-mon[29835]: mgrmap e7: x(active, since 1.01887s) 2024-05-09T04:40:21.363 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-05-09T04:40:21.368 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-09T04:40:21.368 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-09T04:40:21.368 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-05-09T04:40:21.368 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.002+0000 7f4a64400700 1 Processor -- start 2024-05-09T04:40:21.368 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.002+0000 7f4a64400700 1 -- start start 2024-05-09T04:40:21.368 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.003+0000 7f4a64400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a5c07ede0 0x7f4a5c07d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:21.369 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.003+0000 7f4a64400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c07d7a0 con 0x7f4a5c07ede0 2024-05-09T04:40:21.369 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.003+0000 7f4a61a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a5c07ede0 0x7f4a5c07d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:21.369 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.003+0000 7f4a61a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a5c07ede0 0x7f4a5c07d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58340/0 (socket says 172.21.15.6:58340) 2024-05-09T04:40:21.369 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.003+0000 7f4a61a00700 1 -- 172.21.15.6:0/1260859691 learned_addr learned my addr 172.21.15.6:0/1260859691 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:21.369 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.004+0000 7f4a61a00700 1 -- 172.21.15.6:0/1260859691 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a5c07d920 con 0x7f4a5c07ede0 2024-05-09T04:40:21.369 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.004+0000 7f4a61a00700 1 --2- 172.21.15.6:0/1260859691 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a5c07ede0 0x7f4a5c07d260 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009e80 tx=0x7f4a5000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97cb30bd5aaf7686 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:21.369 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.004+0000 7f4a5be00700 1 -- 172.21.15.6:0/1260859691 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a50017070 con 0x7f4a5c07ede0 2024-05-09T04:40:21.369 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.004+0000 7f4a5be00700 1 -- 172.21.15.6:0/1260859691 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4a5000b8c0 con 0x7f4a5c07ede0 2024-05-09T04:40:21.369 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.004+0000 7f4a5be00700 1 -- 172.21.15.6:0/1260859691 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a50004490 con 0x7f4a5c07ede0 2024-05-09T04:40:21.369 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.005+0000 7f4a64400700 1 -- 172.21.15.6:0/1260859691 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a5c07ede0 msgr2=0x7f4a5c07d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:21.369 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.005+0000 7f4a64400700 1 --2- 172.21.15.6:0/1260859691 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a5c07ede0 0x7f4a5c07d260 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009e80 tx=0x7f4a5000b330 comp rx=0 tx=0).stop 2024-05-09T04:40:21.369 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.006+0000 7f4a64400700 1 -- 172.21.15.6:0/1260859691 shutdown_connections 2024-05-09T04:40:21.369 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.006+0000 7f4a64400700 1 --2- 172.21.15.6:0/1260859691 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a5c07ede0 0x7f4a5c07d260 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:21.369 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.006+0000 7f4a64400700 1 -- 172.21.15.6:0/1260859691 >> 172.21.15.6:0/1260859691 conn(0x7f4a5c105120 msgr2=0x7f4a5c1075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:21.370 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.006+0000 7f4a64400700 1 -- 172.21.15.6:0/1260859691 shutdown_connections 2024-05-09T04:40:21.370 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.006+0000 7f4a64400700 1 -- 172.21.15.6:0/1260859691 wait complete. 2024-05-09T04:40:21.370 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.007+0000 7f4a64400700 1 Processor -- start 2024-05-09T04:40:21.370 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.007+0000 7f4a64400700 1 -- start start 2024-05-09T04:40:21.370 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.007+0000 7f4a64400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a5c07ede0 0x7f4a5c1a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:21.370 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.007+0000 7f4a64400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c1100c0 con 0x7f4a5c07ede0 2024-05-09T04:40:21.370 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.008+0000 7f4a61a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a5c07ede0 0x7f4a5c1a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:21.370 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.008+0000 7f4a61a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a5c07ede0 0x7f4a5c1a00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58354/0 (socket says 172.21.15.6:58354) 2024-05-09T04:40:21.370 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.008+0000 7f4a61a00700 1 -- 172.21.15.6:0/3231386976 learned_addr learned my addr 172.21.15.6:0/3231386976 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:21.370 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.008+0000 7f4a61a00700 1 -- 172.21.15.6:0/3231386976 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a5c1a0620 con 0x7f4a5c07ede0 2024-05-09T04:40:21.370 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.008+0000 7f4a61a00700 1 --2- 172.21.15.6:0/3231386976 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a5c07ede0 0x7f4a5c1a00e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4a50004220 tx=0x7f4a5000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:21.370 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.009+0000 7f4a5a000700 1 -- 172.21.15.6:0/3231386976 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a50020020 con 0x7f4a5c07ede0 2024-05-09T04:40:21.370 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.009+0000 7f4a5a000700 1 -- 172.21.15.6:0/3231386976 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4a50012430 con 0x7f4a5c07ede0 2024-05-09T04:40:21.371 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.009+0000 7f4a64400700 1 -- 172.21.15.6:0/3231386976 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a5c1a08b0 con 0x7f4a5c07ede0 2024-05-09T04:40:21.371 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.009+0000 7f4a5a000700 1 -- 172.21.15.6:0/3231386976 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a50017040 con 0x7f4a5c07ede0 2024-05-09T04:40:21.371 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.009+0000 7f4a64400700 1 -- 172.21.15.6:0/3231386976 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a5c1a34c0 con 0x7f4a5c07ede0 2024-05-09T04:40:21.371 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.010+0000 7f4a5a000700 1 -- 172.21.15.6:0/3231386976 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f4a50023020 con 0x7f4a5c07ede0 2024-05-09T04:40:21.371 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.011+0000 7f4a5a000700 1 --2- 172.21.15.6:0/3231386976 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7f4a4403e4d0 0x7f4a440409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:21.371 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.011+0000 7f4a5a000700 1 -- 172.21.15.6:0/3231386976 --> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4a440410f0 con 0x7f4a4403e4d0 2024-05-09T04:40:21.371 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.011+0000 7f4a5a000700 1 -- 172.21.15.6:0/3231386976 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4a5000f050 con 0x7f4a5c07ede0 2024-05-09T04:40:21.371 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.011+0000 7f4a61000700 1 -- 172.21.15.6:0/3231386976 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7f4a4403e4d0 msgr2=0x7f4a440409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3021503778 2024-05-09T04:40:21.371 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.011+0000 7f4a61000700 1 --2- 172.21.15.6:0/3231386976 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7f4a4403e4d0 0x7f4a440409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T04:40:21.374 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.212+0000 7f4a61000700 1 -- 172.21.15.6:0/3231386976 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7f4a4403e4d0 msgr2=0x7f4a440409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3021503778 2024-05-09T04:40:21.374 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.212+0000 7f4a61000700 1 --2- 172.21.15.6:0/3231386976 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7f4a4403e4d0 0x7f4a440409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T04:40:21.374 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.613+0000 7f4a61000700 1 -- 172.21.15.6:0/3231386976 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7f4a4403e4d0 msgr2=0x7f4a440409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3021503778 2024-05-09T04:40:21.374 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:13.613+0000 7f4a61000700 1 --2- 172.21.15.6:0/3231386976 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7f4a4403e4d0 0x7f4a440409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T04:40:21.374 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.414+0000 7f4a61000700 1 -- 172.21.15.6:0/3231386976 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7f4a4403e4d0 msgr2=0x7f4a440409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3021503778 2024-05-09T04:40:21.375 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.414+0000 7f4a61000700 1 --2- 172.21.15.6:0/3231386976 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7f4a4403e4d0 0x7f4a440409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T04:40:21.375 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:16.016+0000 7f4a61000700 1 -- 172.21.15.6:0/3231386976 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7f4a4403e4d0 msgr2=0x7f4a440409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3021503778 2024-05-09T04:40:21.375 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:16.016+0000 7f4a61000700 1 --2- 172.21.15.6:0/3231386976 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7f4a4403e4d0 0x7f4a440409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-09T04:40:21.375 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.218+0000 7f4a61000700 1 -- 172.21.15.6:0/3231386976 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7f4a4403e4d0 msgr2=0x7f4a440409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3021503778 2024-05-09T04:40:21.375 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.218+0000 7f4a61000700 1 --2- 172.21.15.6:0/3231386976 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7f4a4403e4d0 0x7f4a440409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-05-09T04:40:21.375 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.646+0000 7f4a5a000700 1 -- 172.21.15.6:0/3231386976 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7f4a50012670 con 0x7f4a5c07ede0 2024-05-09T04:40:21.375 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.646+0000 7f4a5a000700 1 -- 172.21.15.6:0/3231386976 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7f4a4403e4d0 msgr2=0x7f4a440409e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T04:40:21.375 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.646+0000 7f4a5a000700 1 --2- 172.21.15.6:0/3231386976 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7f4a4403e4d0 0x7f4a440409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:21.375 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.650+0000 7f4a5a000700 1 -- 172.21.15.6:0/3231386976 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f4a50054d30 con 0x7f4a5c07ede0 2024-05-09T04:40:21.375 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.650+0000 7f4a5a000700 1 --2- 172.21.15.6:0/3231386976 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f4a44041e10 0x7f4a44044230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:21.375 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.650+0000 7f4a5a000700 1 -- 172.21.15.6:0/3231386976 --> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4a50051c50 con 0x7f4a44041e10 2024-05-09T04:40:21.375 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.652+0000 7f4a61000700 1 --2- 172.21.15.6:0/3231386976 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f4a44041e10 0x7f4a44044230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:21.375 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.655+0000 7f4a61000700 1 --2- 172.21.15.6:0/3231386976 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f4a44041e10 0x7f4a44044230 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4a48003a10 tx=0x7f4a480092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:21.375 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.658+0000 7f4a5a000700 1 -- 172.21.15.6:0/3231386976 <== mgr.14120 v2:172.21.15.6:6800/937713898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f4a50051c50 con 0x7f4a44041e10 2024-05-09T04:40:21.376 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.667+0000 7f4a64400700 1 -- 172.21.15.6:0/3231386976 --> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f4a5c064e20 con 0x7f4a44041e10 2024-05-09T04:40:21.376 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.668+0000 7f4a5a000700 1 -- 172.21.15.6:0/3231386976 <== mgr.14120 v2:172.21.15.6:6800/937713898 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f4a5c064e20 con 0x7f4a44041e10 2024-05-09T04:40:21.376 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.668+0000 7f4a64400700 1 -- 172.21.15.6:0/3231386976 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f4a44041e10 msgr2=0x7f4a44044230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:21.376 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.668+0000 7f4a64400700 1 --2- 172.21.15.6:0/3231386976 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f4a44041e10 0x7f4a44044230 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4a48003a10 tx=0x7f4a480092b0 comp rx=0 tx=0).stop 2024-05-09T04:40:21.376 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.668+0000 7f4a64400700 1 -- 172.21.15.6:0/3231386976 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a5c07ede0 msgr2=0x7f4a5c1a00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:21.376 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.668+0000 7f4a64400700 1 --2- 172.21.15.6:0/3231386976 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a5c07ede0 0x7f4a5c1a00e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4a50004220 tx=0x7f4a5000bd50 comp rx=0 tx=0).stop 2024-05-09T04:40:21.376 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.669+0000 7f4a64400700 1 -- 172.21.15.6:0/3231386976 shutdown_connections 2024-05-09T04:40:21.376 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.669+0000 7f4a64400700 1 --2- 172.21.15.6:0/3231386976 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f4a44041e10 0x7f4a44044230 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:21.376 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.669+0000 7f4a64400700 1 --2- 172.21.15.6:0/3231386976 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a5c07ede0 0x7f4a5c1a00e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:21.376 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.669+0000 7f4a64400700 1 --2- 172.21.15.6:0/3231386976 >> [v2:172.21.15.6:6800/3021503778,v1:172.21.15.6:6801/3021503778] conn(0x7f4a4403e4d0 0x7f4a440409e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:21.376 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.669+0000 7f4a64400700 1 -- 172.21.15.6:0/3231386976 >> 172.21.15.6:0/3231386976 conn(0x7f4a5c105120 msgr2=0x7f4a5c105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:21.376 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.669+0000 7f4a64400700 1 -- 172.21.15.6:0/3231386976 shutdown_connections 2024-05-09T04:40:21.376 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.669+0000 7f4a64400700 1 -- 172.21.15.6:0/3231386976 wait complete. 2024-05-09T04:40:21.376 INFO:teuthology.orchestra.run.smithi006.stdout:mgr epoch 5 is available 2024-05-09T04:40:21.377 INFO:teuthology.orchestra.run.smithi006.stdout:Setting orchestrator backend to cephadm... 2024-05-09T04:40:21.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:21 smithi006 ceph-mon[29835]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-09T04:40:21.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:21 smithi006 ceph-mon[29835]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-09T04:40:21.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:21 smithi006 ceph-mon[29835]: from='mgr.14120 172.21.15.6:0/4038153362' entity='mgr.x' 2024-05-09T04:40:21.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:21 smithi006 ceph-mon[29835]: from='mgr.14120 172.21.15.6:0/4038153362' entity='mgr.x' 2024-05-09T04:40:23.006 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.036+0000 7fc057c00700 1 Processor -- start 2024-05-09T04:40:23.006 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.036+0000 7fc057c00700 1 -- start start 2024-05-09T04:40:23.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.037+0000 7fc057c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc05010a550 0x7fc05010c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:23.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.037+0000 7fc057c00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc05007ce80 con 0x7fc05010a550 2024-05-09T04:40:23.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.037+0000 7fc055200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc05010a550 0x7fc05010c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:23.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.037+0000 7fc055200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc05010a550 0x7fc05010c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34926/0 (socket says 172.21.15.6:34926) 2024-05-09T04:40:23.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.037+0000 7fc055200700 1 -- 172.21.15.6:0/2408798861 learned_addr learned my addr 172.21.15.6:0/2408798861 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:23.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.038+0000 7fc055200700 1 -- 172.21.15.6:0/2408798861 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc05010ceb0 con 0x7fc05010a550 2024-05-09T04:40:23.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.038+0000 7fc055200700 1 --2- 172.21.15.6:0/2408798861 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc05010a550 0x7fc05010c970 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc0440098a0 tx=0x7fc044009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89ef22793dff6aa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:23.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.038+0000 7fc04f400700 1 -- 172.21.15.6:0/2408798861 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc044004160 con 0x7fc05010a550 2024-05-09T04:40:23.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.038+0000 7fc04f400700 1 -- 172.21.15.6:0/2408798861 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc044004300 con 0x7fc05010a550 2024-05-09T04:40:23.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.038+0000 7fc04f400700 1 -- 172.21.15.6:0/2408798861 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc044011af0 con 0x7fc05010a550 2024-05-09T04:40:23.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.039+0000 7fc057c00700 1 -- 172.21.15.6:0/2408798861 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc05010a550 msgr2=0x7fc05010c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:23.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.039+0000 7fc057c00700 1 --2- 172.21.15.6:0/2408798861 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc05010a550 0x7fc05010c970 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc0440098a0 tx=0x7fc044009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:23.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.039+0000 7fc057c00700 1 -- 172.21.15.6:0/2408798861 shutdown_connections 2024-05-09T04:40:23.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.039+0000 7fc057c00700 1 --2- 172.21.15.6:0/2408798861 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc05010a550 0x7fc05010c970 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:23.008 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.039+0000 7fc057c00700 1 -- 172.21.15.6:0/2408798861 >> 172.21.15.6:0/2408798861 conn(0x7fc050104070 msgr2=0x7fc0501064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:23.008 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.040+0000 7fc057c00700 1 -- 172.21.15.6:0/2408798861 shutdown_connections 2024-05-09T04:40:23.008 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.040+0000 7fc057c00700 1 -- 172.21.15.6:0/2408798861 wait complete. 2024-05-09T04:40:23.008 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.041+0000 7fc057c00700 1 Processor -- start 2024-05-09T04:40:23.008 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.041+0000 7fc057c00700 1 -- start start 2024-05-09T04:40:23.008 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.041+0000 7fc057c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc05010a550 0x7fc0501a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:23.008 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.041+0000 7fc057c00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc05010de70 con 0x7fc05010a550 2024-05-09T04:40:23.008 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.041+0000 7fc055200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc05010a550 0x7fc0501a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:23.008 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.041+0000 7fc055200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc05010a550 0x7fc0501a00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34940/0 (socket says 172.21.15.6:34940) 2024-05-09T04:40:23.008 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.041+0000 7fc055200700 1 -- 172.21.15.6:0/2056695379 learned_addr learned my addr 172.21.15.6:0/2056695379 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:23.008 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.042+0000 7fc055200700 1 -- 172.21.15.6:0/2056695379 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0501a05f0 con 0x7fc05010a550 2024-05-09T04:40:23.008 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.042+0000 7fc055200700 1 --2- 172.21.15.6:0/2056695379 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc05010a550 0x7fc0501a00b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc0440117b0 tx=0x7fc044011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:23.008 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.042+0000 7fc04d600700 1 -- 172.21.15.6:0/2056695379 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc044003af0 con 0x7fc05010a550 2024-05-09T04:40:23.008 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.042+0000 7fc04d600700 1 -- 172.21.15.6:0/2056695379 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc044020070 con 0x7fc05010a550 2024-05-09T04:40:23.009 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.042+0000 7fc057c00700 1 -- 172.21.15.6:0/2056695379 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0501a0880 con 0x7fc05010a550 2024-05-09T04:40:23.009 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.042+0000 7fc04d600700 1 -- 172.21.15.6:0/2056695379 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc044019630 con 0x7fc05010a550 2024-05-09T04:40:23.009 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.042+0000 7fc057c00700 1 -- 172.21.15.6:0/2056695379 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0501a3490 con 0x7fc05010a550 2024-05-09T04:40:23.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.044+0000 7fc057c00700 1 -- 172.21.15.6:0/2056695379 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc050064e20 con 0x7fc05010a550 2024-05-09T04:40:23.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.044+0000 7fc04d600700 1 -- 172.21.15.6:0/2056695379 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7fc0440198a0 con 0x7fc05010a550 2024-05-09T04:40:23.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.045+0000 7fc04d600700 1 --2- 172.21.15.6:0/2056695379 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7fc03803e3a0 0x7fc0380408b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:23.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.045+0000 7fc04d600700 1 -- 172.21.15.6:0/2056695379 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc044053ad0 con 0x7fc05010a550 2024-05-09T04:40:23.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.045+0000 7fc04fe00700 1 --2- 172.21.15.6:0/2056695379 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7fc03803e3a0 0x7fc0380408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:23.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.046+0000 7fc04fe00700 1 --2- 172.21.15.6:0/2056695379 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7fc03803e3a0 0x7fc0380408b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc03c006fd0 tx=0x7fc03c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:23.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.051+0000 7fc04d600700 1 -- 172.21.15.6:0/2056695379 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc0440042a0 con 0x7fc05010a550 2024-05-09T04:40:23.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.271+0000 7fc04d600700 1 -- 172.21.15.6:0/2056695379 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fc04401a9c0 con 0x7fc05010a550 2024-05-09T04:40:23.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.304+0000 7fc057c00700 1 -- 172.21.15.6:0/2056695379 --> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fc050106120 con 0x7fc03803e3a0 2024-05-09T04:40:23.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.315+0000 7fc04d600700 1 -- 172.21.15.6:0/2056695379 <== mgr.14120 v2:172.21.15.6:6800/937713898 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc050106120 con 0x7fc03803e3a0 2024-05-09T04:40:23.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.318+0000 7fc057c00700 1 -- 172.21.15.6:0/2056695379 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7fc03803e3a0 msgr2=0x7fc0380408b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:23.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.318+0000 7fc057c00700 1 --2- 172.21.15.6:0/2056695379 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7fc03803e3a0 0x7fc0380408b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc03c006fd0 tx=0x7fc03c006e40 comp rx=0 tx=0).stop 2024-05-09T04:40:23.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.318+0000 7fc057c00700 1 -- 172.21.15.6:0/2056695379 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc05010a550 msgr2=0x7fc0501a00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:23.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.318+0000 7fc057c00700 1 --2- 172.21.15.6:0/2056695379 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc05010a550 0x7fc0501a00b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc0440117b0 tx=0x7fc044011b70 comp rx=0 tx=0).stop 2024-05-09T04:40:23.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.319+0000 7fc057c00700 1 -- 172.21.15.6:0/2056695379 shutdown_connections 2024-05-09T04:40:23.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.319+0000 7fc057c00700 1 --2- 172.21.15.6:0/2056695379 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7fc03803e3a0 0x7fc0380408b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:23.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.319+0000 7fc057c00700 1 --2- 172.21.15.6:0/2056695379 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc05010a550 0x7fc0501a00b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:23.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.319+0000 7fc057c00700 1 -- 172.21.15.6:0/2056695379 >> 172.21.15.6:0/2056695379 conn(0x7fc050104070 msgr2=0x7fc050105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:23.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.319+0000 7fc057c00700 1 -- 172.21.15.6:0/2056695379 shutdown_connections 2024-05-09T04:40:23.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.319+0000 7fc057c00700 1 -- 172.21.15.6:0/2056695379 wait complete. 2024-05-09T04:40:23.555 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:23 smithi006 ceph-mon[29835]: mgrmap e8: x(active, since 2s) 2024-05-09T04:40:23.556 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:23 smithi006 ceph-mon[29835]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:40:23.556 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:23 smithi006 ceph-mon[29835]: from='mgr.14120 172.21.15.6:0/4038153362' entity='mgr.x' 2024-05-09T04:40:23.556 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:23 smithi006 ceph-mon[29835]: from='mgr.14120 172.21.15.6:0/4038153362' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:40:23.556 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:23 smithi006 ceph-mon[29835]: [09/May/2024:04:40:22] ENGINE Bus STARTING 2024-05-09T04:40:23.556 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:23 smithi006 ceph-mon[29835]: [09/May/2024:04:40:22] ENGINE Serving on https://172.21.15.6:7150 2024-05-09T04:40:23.556 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:23 smithi006 ceph-mon[29835]: [09/May/2024:04:40:22] ENGINE Serving on http://172.21.15.6:8765 2024-05-09T04:40:23.556 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:23 smithi006 ceph-mon[29835]: [09/May/2024:04:40:22] ENGINE Bus STARTED 2024-05-09T04:40:23.556 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:23 smithi006 ceph-mon[29835]: from='mgr.14120 172.21.15.6:0/4038153362' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:40:24.592 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:24 smithi006 ceph-mon[29835]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:40:24.609 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-09T04:40:24.610 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.712+0000 7f2286200700 1 Processor -- start 2024-05-09T04:40:24.610 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.712+0000 7f2286200700 1 -- start start 2024-05-09T04:40:24.611 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.713+0000 7f2286200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f228010bcc0 0x7f228010c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:24.611 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.713+0000 7f2286200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f228010c6c0 con 0x7f228010bcc0 2024-05-09T04:40:24.611 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.713+0000 7f227f400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f228010bcc0 0x7f228010c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:24.611 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.713+0000 7f227f400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f228010bcc0 0x7f228010c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34944/0 (socket says 172.21.15.6:34944) 2024-05-09T04:40:24.611 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.713+0000 7f227f400700 1 -- 172.21.15.6:0/2427431202 learned_addr learned my addr 172.21.15.6:0/2427431202 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:24.612 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.714+0000 7f227f400700 1 -- 172.21.15.6:0/2427431202 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228010ce90 con 0x7f228010bcc0 2024-05-09T04:40:24.612 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.714+0000 7f227f400700 1 --2- 172.21.15.6:0/2427431202 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f228010bcc0 0x7f228010c0d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f22700098a0 tx=0x7f2270009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3cac132997c6be4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:24.612 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.714+0000 7f227e000700 1 -- 172.21.15.6:0/2427431202 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2270004160 con 0x7f228010bcc0 2024-05-09T04:40:24.612 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.714+0000 7f227e000700 1 -- 172.21.15.6:0/2427431202 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2270004300 con 0x7f228010bcc0 2024-05-09T04:40:24.612 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.714+0000 7f227e000700 1 -- 172.21.15.6:0/2427431202 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2270011af0 con 0x7f228010bcc0 2024-05-09T04:40:24.615 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.715+0000 7f2286200700 1 -- 172.21.15.6:0/2427431202 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f228010bcc0 msgr2=0x7f228010c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:24.615 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.715+0000 7f2286200700 1 --2- 172.21.15.6:0/2427431202 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f228010bcc0 0x7f228010c0d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f22700098a0 tx=0x7f2270009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:24.615 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.716+0000 7f2286200700 1 -- 172.21.15.6:0/2427431202 shutdown_connections 2024-05-09T04:40:24.615 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.716+0000 7f2286200700 1 --2- 172.21.15.6:0/2427431202 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f228010bcc0 0x7f228010c0d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:24.615 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.716+0000 7f2286200700 1 -- 172.21.15.6:0/2427431202 >> 172.21.15.6:0/2427431202 conn(0x7f228007f1f0 msgr2=0x7f2280109d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:24.615 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.716+0000 7f2286200700 1 -- 172.21.15.6:0/2427431202 shutdown_connections 2024-05-09T04:40:24.615 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.716+0000 7f2286200700 1 -- 172.21.15.6:0/2427431202 wait complete. 2024-05-09T04:40:24.615 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.717+0000 7f2286200700 1 Processor -- start 2024-05-09T04:40:24.615 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.717+0000 7f2286200700 1 -- start start 2024-05-09T04:40:24.615 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.717+0000 7f2286200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f228010bcc0 0x7f22801a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:24.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.717+0000 7f2286200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2280110240 con 0x7f228010bcc0 2024-05-09T04:40:24.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.718+0000 7f227f400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f228010bcc0 0x7f22801a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:24.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.718+0000 7f227f400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f228010bcc0 0x7f22801a0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34946/0 (socket says 172.21.15.6:34946) 2024-05-09T04:40:24.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.718+0000 7f227f400700 1 -- 172.21.15.6:0/4160606906 learned_addr learned my addr 172.21.15.6:0/4160606906 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:24.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.718+0000 7f227f400700 1 -- 172.21.15.6:0/4160606906 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22801a0660 con 0x7f228010bcc0 2024-05-09T04:40:24.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.719+0000 7f227f400700 1 --2- 172.21.15.6:0/4160606906 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f228010bcc0 0x7f22801a0120 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2270011890 tx=0x7f2270011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:24.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.719+0000 7f2277e00700 1 -- 172.21.15.6:0/4160606906 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22700039e0 con 0x7f228010bcc0 2024-05-09T04:40:24.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.719+0000 7f2277e00700 1 -- 172.21.15.6:0/4160606906 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2270003b80 con 0x7f228010bcc0 2024-05-09T04:40:24.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.719+0000 7f2277e00700 1 -- 172.21.15.6:0/4160606906 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22700246e0 con 0x7f228010bcc0 2024-05-09T04:40:24.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.719+0000 7f2286200700 1 -- 172.21.15.6:0/4160606906 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22801a08f0 con 0x7f228010bcc0 2024-05-09T04:40:24.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.719+0000 7f2286200700 1 -- 172.21.15.6:0/4160606906 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22801a3500 con 0x7f228010bcc0 2024-05-09T04:40:24.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.720+0000 7f2286200700 1 -- 172.21.15.6:0/4160606906 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2280064e20 con 0x7f228010bcc0 2024-05-09T04:40:24.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.721+0000 7f2277e00700 1 -- 172.21.15.6:0/4160606906 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f2270022020 con 0x7f228010bcc0 2024-05-09T04:40:24.616 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.721+0000 7f2277e00700 1 --2- 172.21.15.6:0/4160606906 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f226403e470 0x7f2264040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:24.617 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.721+0000 7f2277e00700 1 -- 172.21.15.6:0/4160606906 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f22700537a0 con 0x7f228010bcc0 2024-05-09T04:40:24.617 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.721+0000 7f227ea00700 1 --2- 172.21.15.6:0/4160606906 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f226403e470 0x7f2264040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:24.618 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.722+0000 7f227ea00700 1 --2- 172.21.15.6:0/4160606906 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f226403e470 0x7f2264040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f226c006fd0 tx=0x7f226c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:24.618 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.727+0000 7f2277e00700 1 -- 172.21.15.6:0/4160606906 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f22700212c0 con 0x7f228010bcc0 2024-05-09T04:40:24.619 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.968+0000 7f2286200700 1 -- 172.21.15.6:0/4160606906 --> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f2280109930 con 0x7f226403e470 2024-05-09T04:40:24.619 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.969+0000 7f2277e00700 1 -- 172.21.15.6:0/4160606906 <== mgr.14120 v2:172.21.15.6:6800/937713898 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f2280109930 con 0x7f226403e470 2024-05-09T04:40:24.619 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.972+0000 7f2286200700 1 -- 172.21.15.6:0/4160606906 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f226403e470 msgr2=0x7f2264040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:24.619 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.972+0000 7f2286200700 1 --2- 172.21.15.6:0/4160606906 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f226403e470 0x7f2264040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f226c006fd0 tx=0x7f226c006e40 comp rx=0 tx=0).stop 2024-05-09T04:40:24.619 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.972+0000 7f2286200700 1 -- 172.21.15.6:0/4160606906 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f228010bcc0 msgr2=0x7f22801a0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:24.619 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.972+0000 7f2286200700 1 --2- 172.21.15.6:0/4160606906 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f228010bcc0 0x7f22801a0120 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2270011890 tx=0x7f2270011970 comp rx=0 tx=0).stop 2024-05-09T04:40:24.619 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.973+0000 7f2286200700 1 -- 172.21.15.6:0/4160606906 shutdown_connections 2024-05-09T04:40:24.619 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.973+0000 7f2286200700 1 --2- 172.21.15.6:0/4160606906 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f226403e470 0x7f2264040980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:24.619 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.973+0000 7f2286200700 1 --2- 172.21.15.6:0/4160606906 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f228010bcc0 0x7f22801a0120 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:24.619 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.973+0000 7f2286200700 1 -- 172.21.15.6:0/4160606906 >> 172.21.15.6:0/4160606906 conn(0x7f228007f1f0 msgr2=0x7f2280109220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:24.619 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.973+0000 7f2286200700 1 -- 172.21.15.6:0/4160606906 shutdown_connections 2024-05-09T04:40:24.619 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.973+0000 7f2286200700 1 -- 172.21.15.6:0/4160606906 wait complete. 2024-05-09T04:40:24.619 INFO:teuthology.orchestra.run.smithi006.stdout:Generating ssh key... 2024-05-09T04:40:26.197 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:25 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: Generating public/private rsa key pair. 2024-05-09T04:40:26.197 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:25 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: Your identification has been saved in /tmp/tmp3ua_qwa8/key. 2024-05-09T04:40:26.197 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:25 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: Your public key has been saved in /tmp/tmp3ua_qwa8/key.pub. 2024-05-09T04:40:26.197 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:25 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: The key fingerprint is: 2024-05-09T04:40:26.198 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:25 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: SHA256:hLYc0z3dTeXgsPg00hZ83JKpqLQ1RCqzqOPPBBBiKWs ceph-f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:40:26.198 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:25 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: The key's randomart image is: 2024-05-09T04:40:26.198 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:25 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: +---[RSA 3072]----+ 2024-05-09T04:40:26.198 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:25 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: |o.. ...o..++| 2024-05-09T04:40:26.198 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:25 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: |+o o o.+.**=.| 2024-05-09T04:40:26.198 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:25 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: |o. B +.=.Boo.o| 2024-05-09T04:40:26.198 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:25 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: |.E + O. +*.. | 2024-05-09T04:40:26.198 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:25 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: |. . . +.S+ .. | 2024-05-09T04:40:26.198 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:25 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: | o o | 2024-05-09T04:40:26.198 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:25 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: | o . | 2024-05-09T04:40:26.198 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:25 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: | . + | 2024-05-09T04:40:26.198 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:25 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: | ..o | 2024-05-09T04:40:26.198 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:25 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: +----[SHA256]-----+ 2024-05-09T04:40:26.465 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.421+0000 7fe2aca00700 1 Processor -- start 2024-05-09T04:40:26.466 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.421+0000 7fe2aca00700 1 -- start start 2024-05-09T04:40:26.466 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.422+0000 7fe2aca00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2a810a400 0x7fe2a810c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:26.466 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.422+0000 7fe2aca00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a807ce80 con 0x7fe2a810a400 2024-05-09T04:40:26.467 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.422+0000 7fe2a6800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2a810a400 0x7fe2a810c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:26.467 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.422+0000 7fe2a6800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2a810a400 0x7fe2a810c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34962/0 (socket says 172.21.15.6:34962) 2024-05-09T04:40:26.467 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.422+0000 7fe2a6800700 1 -- 172.21.15.6:0/2434986427 learned_addr learned my addr 172.21.15.6:0/2434986427 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:26.467 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.423+0000 7fe2a6800700 1 -- 172.21.15.6:0/2434986427 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a810cd60 con 0x7fe2a810a400 2024-05-09T04:40:26.468 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.423+0000 7fe2a6800700 1 --2- 172.21.15.6:0/2434986427 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2a810a400 0x7fe2a810c820 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe2980098a0 tx=0x7fe298009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139f138c92fce1e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:26.468 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.424+0000 7fe2a5e00700 1 -- 172.21.15.6:0/2434986427 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe298004160 con 0x7fe2a810a400 2024-05-09T04:40:26.468 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.424+0000 7fe2a5e00700 1 -- 172.21.15.6:0/2434986427 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe298004300 con 0x7fe2a810a400 2024-05-09T04:40:26.469 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.424+0000 7fe2a5e00700 1 -- 172.21.15.6:0/2434986427 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe298011af0 con 0x7fe2a810a400 2024-05-09T04:40:26.469 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.425+0000 7fe2aca00700 1 -- 172.21.15.6:0/2434986427 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2a810a400 msgr2=0x7fe2a810c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:26.469 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.425+0000 7fe2aca00700 1 --2- 172.21.15.6:0/2434986427 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2a810a400 0x7fe2a810c820 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe2980098a0 tx=0x7fe298009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:26.470 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.425+0000 7fe2aca00700 1 -- 172.21.15.6:0/2434986427 shutdown_connections 2024-05-09T04:40:26.470 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.425+0000 7fe2aca00700 1 --2- 172.21.15.6:0/2434986427 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2a810a400 0x7fe2a810c820 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:26.470 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.425+0000 7fe2aca00700 1 -- 172.21.15.6:0/2434986427 >> 172.21.15.6:0/2434986427 conn(0x7fe2a8103ea0 msgr2=0x7fe2a8106310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:26.471 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.425+0000 7fe2aca00700 1 -- 172.21.15.6:0/2434986427 shutdown_connections 2024-05-09T04:40:26.471 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.425+0000 7fe2aca00700 1 -- 172.21.15.6:0/2434986427 wait complete. 2024-05-09T04:40:26.471 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.427+0000 7fe2aca00700 1 Processor -- start 2024-05-09T04:40:26.471 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.427+0000 7fe2aca00700 1 -- start start 2024-05-09T04:40:26.471 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.428+0000 7fe2aca00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2a810a400 0x7fe2a819fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:26.471 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.428+0000 7fe2aca00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a810dd20 con 0x7fe2a810a400 2024-05-09T04:40:26.471 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.428+0000 7fe2a6800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2a810a400 0x7fe2a819fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:26.474 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.428+0000 7fe2a6800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2a810a400 0x7fe2a819fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34968/0 (socket says 172.21.15.6:34968) 2024-05-09T04:40:26.474 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.428+0000 7fe2a6800700 1 -- 172.21.15.6:0/2582686488 learned_addr learned my addr 172.21.15.6:0/2582686488 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:26.474 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.429+0000 7fe2a6800700 1 -- 172.21.15.6:0/2582686488 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a81a0530 con 0x7fe2a810a400 2024-05-09T04:40:26.474 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.429+0000 7fe2a6800700 1 --2- 172.21.15.6:0/2582686488 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2a810a400 0x7fe2a819fff0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe298009fb0 tx=0x7fe298011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:26.474 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.429+0000 7fe29f400700 1 -- 172.21.15.6:0/2582686488 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe298011bc0 con 0x7fe2a810a400 2024-05-09T04:40:26.474 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.429+0000 7fe29f400700 1 -- 172.21.15.6:0/2582686488 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe298024440 con 0x7fe2a810a400 2024-05-09T04:40:26.474 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.429+0000 7fe29f400700 1 -- 172.21.15.6:0/2582686488 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe29801a5c0 con 0x7fe2a810a400 2024-05-09T04:40:26.474 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.430+0000 7fe2aca00700 1 -- 172.21.15.6:0/2582686488 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2a81a07c0 con 0x7fe2a810a400 2024-05-09T04:40:26.474 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.430+0000 7fe2aca00700 1 -- 172.21.15.6:0/2582686488 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2a81a33d0 con 0x7fe2a810a400 2024-05-09T04:40:26.474 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.431+0000 7fe2aca00700 1 -- 172.21.15.6:0/2582686488 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2a8064e20 con 0x7fe2a810a400 2024-05-09T04:40:26.474 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.431+0000 7fe29f400700 1 -- 172.21.15.6:0/2582686488 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fe2980038b0 con 0x7fe2a810a400 2024-05-09T04:40:26.475 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.432+0000 7fe29f400700 1 --2- 172.21.15.6:0/2582686488 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7fe28c03e470 0x7fe28c040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:26.475 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.432+0000 7fe29f400700 1 -- 172.21.15.6:0/2582686488 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe298053770 con 0x7fe2a810a400 2024-05-09T04:40:26.475 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.432+0000 7fe29fe00700 1 --2- 172.21.15.6:0/2582686488 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7fe28c03e470 0x7fe28c040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:26.475 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.433+0000 7fe29fe00700 1 --2- 172.21.15.6:0/2582686488 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7fe28c03e470 0x7fe28c040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe290006fd0 tx=0x7fe290006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:26.475 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.438+0000 7fe29f400700 1 -- 172.21.15.6:0/2582686488 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe29800f2e0 con 0x7fe2a810a400 2024-05-09T04:40:26.475 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.675+0000 7fe2aca00700 1 -- 172.21.15.6:0/2582686488 --> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fe2a802e9a0 con 0x7fe28c03e470 2024-05-09T04:40:26.475 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.863+0000 7fe29f400700 1 -- 172.21.15.6:0/2582686488 <== mgr.14120 v2:172.21.15.6:6800/937713898 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe2a802e9a0 con 0x7fe28c03e470 2024-05-09T04:40:26.475 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.866+0000 7fe2aca00700 1 -- 172.21.15.6:0/2582686488 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7fe28c03e470 msgr2=0x7fe28c040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:26.475 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.866+0000 7fe2aca00700 1 --2- 172.21.15.6:0/2582686488 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7fe28c03e470 0x7fe28c040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe290006fd0 tx=0x7fe290006e40 comp rx=0 tx=0).stop 2024-05-09T04:40:26.476 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.866+0000 7fe2aca00700 1 -- 172.21.15.6:0/2582686488 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2a810a400 msgr2=0x7fe2a819fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:26.476 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.866+0000 7fe2aca00700 1 --2- 172.21.15.6:0/2582686488 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2a810a400 0x7fe2a819fff0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe298009fb0 tx=0x7fe298011890 comp rx=0 tx=0).stop 2024-05-09T04:40:26.476 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.867+0000 7fe2aca00700 1 -- 172.21.15.6:0/2582686488 shutdown_connections 2024-05-09T04:40:26.476 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.867+0000 7fe2aca00700 1 --2- 172.21.15.6:0/2582686488 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7fe28c03e470 0x7fe28c040980 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:26.476 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.867+0000 7fe2aca00700 1 --2- 172.21.15.6:0/2582686488 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2a810a400 0x7fe2a819fff0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:26.476 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.867+0000 7fe2aca00700 1 -- 172.21.15.6:0/2582686488 >> 172.21.15.6:0/2582686488 conn(0x7fe2a8103ea0 msgr2=0x7fe2a8108380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:26.476 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.867+0000 7fe2aca00700 1 -- 172.21.15.6:0/2582686488 shutdown_connections 2024-05-09T04:40:26.477 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.867+0000 7fe2aca00700 1 -- 172.21.15.6:0/2582686488 wait complete. 2024-05-09T04:40:27.018 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:26 smithi006 ceph-mon[29835]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:40:27.018 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:26 smithi006 ceph-mon[29835]: Generating ssh key... 2024-05-09T04:40:27.018 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:26 smithi006 ceph-mon[29835]: from='mgr.14120 172.21.15.6:0/4038153362' entity='mgr.x' 2024-05-09T04:40:27.018 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:26 smithi006 ceph-mon[29835]: from='mgr.14120 172.21.15.6:0/4038153362' entity='mgr.x' 2024-05-09T04:40:28.037 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDB+JkWLcjz04FbFR+6XOfUfNnZH+rJAq5tjOeYbHh3TcznCg/0nM6iyOUco1vjTVoqdzlKXJY7qsE4+22GnosJ88Fis5m/VxM1TWSZ09rnmW0q4F7wQc1pWjYWf03VbSdLHFIjPDorkdw4Lc6T8zfrQNA18mBWqkfNuAB+5Zo1618k06uAYiFlomc0eDs0/0BFdAx5nXV+2L5VKxBS4nlJjz8HtU9z0ghAQccIpVdJ+DdSZBwxun8r/WsXEJ6MutWgMLuOvso6R7QJ0SN3uKSAQunpnW55vXMS8ZAC7BLzhSOZc+zAc54RQi361VZErU1O8oARjX3jv0a3f0LkeDDJklm/N8aAUvadAHth10A6YUVGqA0a4XeSFRcSx2vllSN3UWaeCoL69+EOiJbWryie3YR4/cbqZODvHPA2xkxlWUVIwdFEZOxxQhr9p0GqKh1y4QjLVWnduPhiM0S4zKE34rCCe4q+QT/uJAko3UWGq4OHzRg70TVrGJGgdjAXK8s= ceph-f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:40:28.037 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.164+0000 7fda47e00700 1 Processor -- start 2024-05-09T04:40:28.038 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.165+0000 7fda47e00700 1 -- start start 2024-05-09T04:40:28.038 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.165+0000 7fda47e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda40110880 0x7fda40110cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:28.038 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.165+0000 7fda47e00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fda40111300 con 0x7fda40110880 2024-05-09T04:40:28.038 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.165+0000 7fda45400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda40110880 0x7fda40110cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:28.038 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.166+0000 7fda45400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda40110880 0x7fda40110cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54080/0 (socket says 172.21.15.6:54080) 2024-05-09T04:40:28.038 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.166+0000 7fda45400700 1 -- 172.21.15.6:0/3570521334 learned_addr learned my addr 172.21.15.6:0/3570521334 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:28.038 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.166+0000 7fda45400700 1 -- 172.21.15.6:0/3570521334 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda40113610 con 0x7fda40110880 2024-05-09T04:40:28.043 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.167+0000 7fda45400700 1 --2- 172.21.15.6:0/3570521334 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda40110880 0x7fda40110cf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fda340098a0 tx=0x7fda34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc48b89da3bc847 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:28.043 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.167+0000 7fda3fe00700 1 -- 172.21.15.6:0/3570521334 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda34004160 con 0x7fda40110880 2024-05-09T04:40:28.043 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.167+0000 7fda3fe00700 1 -- 172.21.15.6:0/3570521334 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fda34004300 con 0x7fda40110880 2024-05-09T04:40:28.043 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.167+0000 7fda3fe00700 1 -- 172.21.15.6:0/3570521334 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda34011af0 con 0x7fda40110880 2024-05-09T04:40:28.044 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.168+0000 7fda47e00700 1 -- 172.21.15.6:0/3570521334 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda40110880 msgr2=0x7fda40110cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:28.044 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.168+0000 7fda47e00700 1 --2- 172.21.15.6:0/3570521334 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda40110880 0x7fda40110cf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fda340098a0 tx=0x7fda34009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:28.044 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.168+0000 7fda47e00700 1 -- 172.21.15.6:0/3570521334 shutdown_connections 2024-05-09T04:40:28.044 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.168+0000 7fda47e00700 1 --2- 172.21.15.6:0/3570521334 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda40110880 0x7fda40110cf0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:28.044 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.168+0000 7fda47e00700 1 -- 172.21.15.6:0/3570521334 >> 172.21.15.6:0/3570521334 conn(0x7fda4007f1f0 msgr2=0x7fda40081640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:28.044 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.168+0000 7fda47e00700 1 -- 172.21.15.6:0/3570521334 shutdown_connections 2024-05-09T04:40:28.044 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.168+0000 7fda47e00700 1 -- 172.21.15.6:0/3570521334 wait complete. 2024-05-09T04:40:28.044 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.169+0000 7fda47e00700 1 Processor -- start 2024-05-09T04:40:28.044 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.170+0000 7fda47e00700 1 -- start start 2024-05-09T04:40:28.044 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.170+0000 7fda47e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda40110880 0x7fda401a4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:28.044 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.170+0000 7fda47e00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fda40114570 con 0x7fda40110880 2024-05-09T04:40:28.044 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.170+0000 7fda45400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda40110880 0x7fda401a4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:28.044 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.170+0000 7fda45400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda40110880 0x7fda401a4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54094/0 (socket says 172.21.15.6:54094) 2024-05-09T04:40:28.044 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.170+0000 7fda45400700 1 -- 172.21.15.6:0/3585548876 learned_addr learned my addr 172.21.15.6:0/3585548876 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:28.045 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.171+0000 7fda45400700 1 -- 172.21.15.6:0/3585548876 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda401a4aa0 con 0x7fda40110880 2024-05-09T04:40:28.045 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.171+0000 7fda45400700 1 --2- 172.21.15.6:0/3585548876 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda40110880 0x7fda401a4560 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fda340117b0 tx=0x7fda34011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:28.045 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.171+0000 7fda3e000700 1 -- 172.21.15.6:0/3585548876 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda34003af0 con 0x7fda40110880 2024-05-09T04:40:28.045 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.171+0000 7fda3e000700 1 -- 172.21.15.6:0/3585548876 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fda34020070 con 0x7fda40110880 2024-05-09T04:40:28.045 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.171+0000 7fda3e000700 1 -- 172.21.15.6:0/3585548876 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda34019630 con 0x7fda40110880 2024-05-09T04:40:28.045 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.171+0000 7fda47e00700 1 -- 172.21.15.6:0/3585548876 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda401a4d30 con 0x7fda40110880 2024-05-09T04:40:28.045 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.172+0000 7fda47e00700 1 -- 172.21.15.6:0/3585548876 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda401a7940 con 0x7fda40110880 2024-05-09T04:40:28.045 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.173+0000 7fda47e00700 1 -- 172.21.15.6:0/3585548876 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda40064e20 con 0x7fda40110880 2024-05-09T04:40:28.045 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.173+0000 7fda3e000700 1 -- 172.21.15.6:0/3585548876 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fda3401a9c0 con 0x7fda40110880 2024-05-09T04:40:28.045 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.173+0000 7fda3e000700 1 --2- 172.21.15.6:0/3585548876 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7fda2803e4c0 0x7fda280409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:28.045 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.174+0000 7fda3e000700 1 -- 172.21.15.6:0/3585548876 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fda34053b50 con 0x7fda40110880 2024-05-09T04:40:28.045 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.174+0000 7fda44a00700 1 --2- 172.21.15.6:0/3585548876 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7fda2803e4c0 0x7fda280409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:28.045 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.175+0000 7fda44a00700 1 --2- 172.21.15.6:0/3585548876 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7fda2803e4c0 0x7fda280409d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fda2c006fd0 tx=0x7fda2c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:28.045 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.180+0000 7fda3e000700 1 -- 172.21.15.6:0/3585548876 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fda340172b0 con 0x7fda40110880 2024-05-09T04:40:28.046 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.419+0000 7fda47e00700 1 -- 172.21.15.6:0/3585548876 --> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fda40081360 con 0x7fda2803e4c0 2024-05-09T04:40:28.046 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.421+0000 7fda3e000700 1 -- 172.21.15.6:0/3585548876 <== mgr.14120 v2:172.21.15.6:6800/937713898 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fda40081360 con 0x7fda2803e4c0 2024-05-09T04:40:28.046 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.424+0000 7fda47e00700 1 -- 172.21.15.6:0/3585548876 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7fda2803e4c0 msgr2=0x7fda280409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:28.046 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.424+0000 7fda47e00700 1 --2- 172.21.15.6:0/3585548876 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7fda2803e4c0 0x7fda280409d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fda2c006fd0 tx=0x7fda2c006e40 comp rx=0 tx=0).stop 2024-05-09T04:40:28.046 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.424+0000 7fda47e00700 1 -- 172.21.15.6:0/3585548876 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda40110880 msgr2=0x7fda401a4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:28.046 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.424+0000 7fda47e00700 1 --2- 172.21.15.6:0/3585548876 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda40110880 0x7fda401a4560 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fda340117b0 tx=0x7fda34011b70 comp rx=0 tx=0).stop 2024-05-09T04:40:28.046 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.424+0000 7fda47e00700 1 -- 172.21.15.6:0/3585548876 shutdown_connections 2024-05-09T04:40:28.047 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.424+0000 7fda47e00700 1 --2- 172.21.15.6:0/3585548876 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7fda2803e4c0 0x7fda280409d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:28.047 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.424+0000 7fda47e00700 1 --2- 172.21.15.6:0/3585548876 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda40110880 0x7fda401a4560 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:28.047 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.424+0000 7fda47e00700 1 -- 172.21.15.6:0/3585548876 >> 172.21.15.6:0/3585548876 conn(0x7fda4007f1f0 msgr2=0x7fda40080c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:28.047 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.424+0000 7fda47e00700 1 -- 172.21.15.6:0/3585548876 shutdown_connections 2024-05-09T04:40:28.047 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.425+0000 7fda47e00700 1 -- 172.21.15.6:0/3585548876 wait complete. 2024-05-09T04:40:28.047 INFO:teuthology.orchestra.run.smithi006.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-09T04:40:28.047 INFO:teuthology.orchestra.run.smithi006.stdout:Adding key to root@localhost authorized_keys... 2024-05-09T04:40:28.048 INFO:teuthology.orchestra.run.smithi006.stdout:Adding host smithi006... 2024-05-09T04:40:28.365 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:28 smithi006 ceph-mon[29835]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:40:29.246 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:29 smithi006 ceph-mon[29835]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi006", "addr": "172.21.15.6", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:40:31.327 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:31 smithi006 ceph-mon[29835]: Deploying cephadm binary to smithi006 2024-05-09T04:40:36.128 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout Added host 'smithi006' with addr '172.21.15.6' 2024-05-09T04:40:36.128 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.732+0000 7f7f05000700 1 Processor -- start 2024-05-09T04:40:36.128 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.733+0000 7f7f05000700 1 -- start start 2024-05-09T04:40:36.128 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.733+0000 7f7f05000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f0007f6a0 0x7f7f0007dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:36.128 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.733+0000 7f7f05000700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0007e010 con 0x7f7f0007f6a0 2024-05-09T04:40:36.128 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.734+0000 7f7efde00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f0007f6a0 0x7f7f0007dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:36.128 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.734+0000 7f7efde00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f0007f6a0 0x7f7f0007dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54104/0 (socket says 172.21.15.6:54104) 2024-05-09T04:40:36.128 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.734+0000 7f7efde00700 1 -- 172.21.15.6:0/1376380052 learned_addr learned my addr 172.21.15.6:0/1376380052 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:36.128 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.734+0000 7f7efde00700 1 -- 172.21.15.6:0/1376380052 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f0007e190 con 0x7f7f0007f6a0 2024-05-09T04:40:36.128 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.735+0000 7f7efde00700 1 --2- 172.21.15.6:0/1376380052 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f0007f6a0 0x7f7f0007dad0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7ef00098a0 tx=0x7f7ef0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=773ac9e90c525ae5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:36.128 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.735+0000 7f7efca00700 1 -- 172.21.15.6:0/1376380052 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ef0004160 con 0x7f7f0007f6a0 2024-05-09T04:40:36.129 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.735+0000 7f7efca00700 1 -- 172.21.15.6:0/1376380052 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7ef0004300 con 0x7f7f0007f6a0 2024-05-09T04:40:36.129 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.735+0000 7f7efca00700 1 -- 172.21.15.6:0/1376380052 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ef0011af0 con 0x7f7f0007f6a0 2024-05-09T04:40:36.129 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.736+0000 7f7f05000700 1 -- 172.21.15.6:0/1376380052 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f0007f6a0 msgr2=0x7f7f0007dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:36.129 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.736+0000 7f7f05000700 1 --2- 172.21.15.6:0/1376380052 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f0007f6a0 0x7f7f0007dad0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7ef00098a0 tx=0x7f7ef0009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:36.129 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.736+0000 7f7f05000700 1 -- 172.21.15.6:0/1376380052 shutdown_connections 2024-05-09T04:40:36.129 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.736+0000 7f7f05000700 1 --2- 172.21.15.6:0/1376380052 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f0007f6a0 0x7f7f0007dad0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:36.129 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.736+0000 7f7f05000700 1 -- 172.21.15.6:0/1376380052 >> 172.21.15.6:0/1376380052 conn(0x7f7f001051c0 msgr2=0x7f7f00107610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:36.129 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.736+0000 7f7f05000700 1 -- 172.21.15.6:0/1376380052 shutdown_connections 2024-05-09T04:40:36.129 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.736+0000 7f7f05000700 1 -- 172.21.15.6:0/1376380052 wait complete. 2024-05-09T04:40:36.129 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.738+0000 7f7f05000700 1 Processor -- start 2024-05-09T04:40:36.129 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.738+0000 7f7f05000700 1 -- start start 2024-05-09T04:40:36.129 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.738+0000 7f7f05000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f0007f6a0 0x7f7f001a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:36.129 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.739+0000 7f7f05000700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7f001100a0 con 0x7f7f0007f6a0 2024-05-09T04:40:36.130 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.739+0000 7f7efde00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f0007f6a0 0x7f7f001a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:36.130 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.739+0000 7f7efde00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f0007f6a0 0x7f7f001a00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54116/0 (socket says 172.21.15.6:54116) 2024-05-09T04:40:36.130 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.739+0000 7f7efde00700 1 -- 172.21.15.6:0/2406566076 learned_addr learned my addr 172.21.15.6:0/2406566076 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:36.130 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.739+0000 7f7efde00700 1 -- 172.21.15.6:0/2406566076 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f001a05f0 con 0x7f7f0007f6a0 2024-05-09T04:40:36.130 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.740+0000 7f7efde00700 1 --2- 172.21.15.6:0/2406566076 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f0007f6a0 0x7f7f001a00b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7ef00117b0 tx=0x7f7ef0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:36.130 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.740+0000 7f7ef6a00700 1 -- 172.21.15.6:0/2406566076 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ef0003af0 con 0x7f7f0007f6a0 2024-05-09T04:40:36.130 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.740+0000 7f7ef6a00700 1 -- 172.21.15.6:0/2406566076 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7ef0020070 con 0x7f7f0007f6a0 2024-05-09T04:40:36.130 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.740+0000 7f7ef6a00700 1 -- 172.21.15.6:0/2406566076 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ef0019630 con 0x7f7f0007f6a0 2024-05-09T04:40:36.130 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.740+0000 7f7f05000700 1 -- 172.21.15.6:0/2406566076 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f001a0880 con 0x7f7f0007f6a0 2024-05-09T04:40:36.130 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.740+0000 7f7f05000700 1 -- 172.21.15.6:0/2406566076 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f001a3490 con 0x7f7f0007f6a0 2024-05-09T04:40:36.130 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.741+0000 7f7f05000700 1 -- 172.21.15.6:0/2406566076 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f00064e20 con 0x7f7f0007f6a0 2024-05-09T04:40:36.130 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.742+0000 7f7ef6a00700 1 -- 172.21.15.6:0/2406566076 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f7ef001a9c0 con 0x7f7f0007f6a0 2024-05-09T04:40:36.130 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.742+0000 7f7ef6a00700 1 --2- 172.21.15.6:0/2406566076 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f7ee403e4c0 0x7f7ee40409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:36.130 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.742+0000 7f7ef6a00700 1 -- 172.21.15.6:0/2406566076 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7ef0053ac0 con 0x7f7f0007f6a0 2024-05-09T04:40:36.131 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.743+0000 7f7efd400700 1 --2- 172.21.15.6:0/2406566076 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f7ee403e4c0 0x7f7ee40409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:36.131 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.743+0000 7f7efd400700 1 --2- 172.21.15.6:0/2406566076 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f7ee403e4c0 0x7f7ee40409d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7eec006fd0 tx=0x7f7eec006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:36.131 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.748+0000 7f7ef6a00700 1 -- 172.21.15.6:0/2406566076 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7ef00172b0 con 0x7f7f0007f6a0 2024-05-09T04:40:36.131 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.991+0000 7f7f05000700 1 -- 172.21.15.6:0/2406566076 --> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi006", "addr": "172.21.15.6", "target": ["mon-mgr", ""]}) -- 0x7f7f00107310 con 0x7f7ee403e4c0 2024-05-09T04:40:36.131 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.366+0000 7f7ef6a00700 1 -- 172.21.15.6:0/2406566076 <== mgr.14120 v2:172.21.15.6:6800/937713898 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7f7f00107310 con 0x7f7ee403e4c0 2024-05-09T04:40:36.131 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.370+0000 7f7f05000700 1 -- 172.21.15.6:0/2406566076 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f7ee403e4c0 msgr2=0x7f7ee40409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:36.131 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.370+0000 7f7f05000700 1 --2- 172.21.15.6:0/2406566076 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f7ee403e4c0 0x7f7ee40409d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7eec006fd0 tx=0x7f7eec006e40 comp rx=0 tx=0).stop 2024-05-09T04:40:36.131 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.371+0000 7f7f05000700 1 -- 172.21.15.6:0/2406566076 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f0007f6a0 msgr2=0x7f7f001a00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:36.131 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.371+0000 7f7f05000700 1 --2- 172.21.15.6:0/2406566076 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f0007f6a0 0x7f7f001a00b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7ef00117b0 tx=0x7f7ef0011b70 comp rx=0 tx=0).stop 2024-05-09T04:40:36.131 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.371+0000 7f7f05000700 1 -- 172.21.15.6:0/2406566076 shutdown_connections 2024-05-09T04:40:36.131 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.371+0000 7f7f05000700 1 --2- 172.21.15.6:0/2406566076 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f7ee403e4c0 0x7f7ee40409d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:36.131 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.371+0000 7f7f05000700 1 --2- 172.21.15.6:0/2406566076 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f0007f6a0 0x7f7f001a00b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:36.132 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.371+0000 7f7f05000700 1 -- 172.21.15.6:0/2406566076 >> 172.21.15.6:0/2406566076 conn(0x7f7f001051c0 msgr2=0x7f7f00106c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:36.132 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.371+0000 7f7f05000700 1 -- 172.21.15.6:0/2406566076 shutdown_connections 2024-05-09T04:40:36.132 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.371+0000 7f7f05000700 1 -- 172.21.15.6:0/2406566076 wait complete. 2024-05-09T04:40:36.132 INFO:teuthology.orchestra.run.smithi006.stdout:Deploying unmanaged mon service... 2024-05-09T04:40:36.668 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:36 smithi006 ceph-mon[29835]: from='mgr.14120 172.21.15.6:0/4038153362' entity='mgr.x' 2024-05-09T04:40:36.668 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:36 smithi006 ceph-mon[29835]: Added host smithi006 2024-05-09T04:40:36.668 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:36 smithi006 ceph-mon[29835]: from='mgr.14120 172.21.15.6:0/4038153362' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:40:37.842 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-09T04:40:37.842 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.832+0000 7f0d3ac00700 1 Processor -- start 2024-05-09T04:40:37.842 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.832+0000 7f0d3ac00700 1 -- start start 2024-05-09T04:40:37.842 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.833+0000 7f0d3ac00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d3410bb70 0x7f0d3410bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:37.842 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.833+0000 7f0d3ac00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3410c570 con 0x7f0d3410bb70 2024-05-09T04:40:37.844 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.833+0000 7f0d39800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d3410bb70 0x7f0d3410bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:37.844 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.834+0000 7f0d39800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d3410bb70 0x7f0d3410bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54118/0 (socket says 172.21.15.6:54118) 2024-05-09T04:40:37.844 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.834+0000 7f0d39800700 1 -- 172.21.15.6:0/2557888073 learned_addr learned my addr 172.21.15.6:0/2557888073 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:37.844 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.834+0000 7f0d39800700 1 -- 172.21.15.6:0/2557888073 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d3410ccf0 con 0x7f0d3410bb70 2024-05-09T04:40:37.844 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.835+0000 7f0d39800700 1 --2- 172.21.15.6:0/2557888073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d3410bb70 0x7f0d3410bf80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0d280098a0 tx=0x7f0d28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7825e0949ce0abe9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:37.844 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.835+0000 7f0d33e00700 1 -- 172.21.15.6:0/2557888073 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d28004160 con 0x7f0d3410bb70 2024-05-09T04:40:37.844 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.835+0000 7f0d33e00700 1 -- 172.21.15.6:0/2557888073 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0d28004300 con 0x7f0d3410bb70 2024-05-09T04:40:37.844 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.835+0000 7f0d33e00700 1 -- 172.21.15.6:0/2557888073 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d28011af0 con 0x7f0d3410bb70 2024-05-09T04:40:37.845 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.836+0000 7f0d3ac00700 1 -- 172.21.15.6:0/2557888073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d3410bb70 msgr2=0x7f0d3410bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:37.845 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.836+0000 7f0d3ac00700 1 --2- 172.21.15.6:0/2557888073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d3410bb70 0x7f0d3410bf80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0d280098a0 tx=0x7f0d28009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:37.845 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.836+0000 7f0d3ac00700 1 -- 172.21.15.6:0/2557888073 shutdown_connections 2024-05-09T04:40:37.845 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.836+0000 7f0d3ac00700 1 --2- 172.21.15.6:0/2557888073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d3410bb70 0x7f0d3410bf80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:37.845 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.836+0000 7f0d3ac00700 1 -- 172.21.15.6:0/2557888073 >> 172.21.15.6:0/2557888073 conn(0x7f0d3407f000 msgr2=0x7f0d34109d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:37.849 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.836+0000 7f0d3ac00700 1 -- 172.21.15.6:0/2557888073 shutdown_connections 2024-05-09T04:40:37.849 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.836+0000 7f0d3ac00700 1 -- 172.21.15.6:0/2557888073 wait complete. 2024-05-09T04:40:37.849 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.838+0000 7f0d3ac00700 1 Processor -- start 2024-05-09T04:40:37.849 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.838+0000 7f0d3ac00700 1 -- start start 2024-05-09T04:40:37.849 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.838+0000 7f0d3ac00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d341a0060 0x7f0d341a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:37.849 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.838+0000 7f0d3ac00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0d341100a0 con 0x7f0d341a0060 2024-05-09T04:40:37.849 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.839+0000 7f0d39800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d341a0060 0x7f0d341a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:37.849 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.839+0000 7f0d39800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d341a0060 0x7f0d341a0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49924/0 (socket says 172.21.15.6:49924) 2024-05-09T04:40:37.850 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.839+0000 7f0d39800700 1 -- 172.21.15.6:0/2585393881 learned_addr learned my addr 172.21.15.6:0/2585393881 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:37.850 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.839+0000 7f0d39800700 1 -- 172.21.15.6:0/2585393881 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d341a0a10 con 0x7f0d341a0060 2024-05-09T04:40:37.850 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.840+0000 7f0d39800700 1 --2- 172.21.15.6:0/2585393881 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d341a0060 0x7f0d341a0490 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0d28000c00 tx=0x7f0d28004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:37.850 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.840+0000 7f0d32000700 1 -- 172.21.15.6:0/2585393881 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d28011bb0 con 0x7f0d341a0060 2024-05-09T04:40:37.850 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.840+0000 7f0d32000700 1 -- 172.21.15.6:0/2585393881 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0d28003b30 con 0x7f0d341a0060 2024-05-09T04:40:37.850 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.840+0000 7f0d3ac00700 1 -- 172.21.15.6:0/2585393881 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d341a0ca0 con 0x7f0d341a0060 2024-05-09T04:40:37.850 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.840+0000 7f0d32000700 1 -- 172.21.15.6:0/2585393881 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d2801acc0 con 0x7f0d341a0060 2024-05-09T04:40:37.850 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.840+0000 7f0d3ac00700 1 -- 172.21.15.6:0/2585393881 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d341a37d0 con 0x7f0d341a0060 2024-05-09T04:40:37.850 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.841+0000 7f0d3ac00700 1 -- 172.21.15.6:0/2585393881 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d34064e20 con 0x7f0d341a0060 2024-05-09T04:40:37.850 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.841+0000 7f0d32000700 1 -- 172.21.15.6:0/2585393881 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f0d2802d440 con 0x7f0d341a0060 2024-05-09T04:40:37.850 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.842+0000 7f0d32000700 1 --2- 172.21.15.6:0/2585393881 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f0d1c03e4c0 0x7f0d1c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:37.850 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.842+0000 7f0d32000700 1 -- 172.21.15.6:0/2585393881 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0d28053690 con 0x7f0d341a0060 2024-05-09T04:40:37.850 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.842+0000 7f0d38e00700 1 --2- 172.21.15.6:0/2585393881 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f0d1c03e4c0 0x7f0d1c0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:37.851 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.843+0000 7f0d38e00700 1 --2- 172.21.15.6:0/2585393881 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f0d1c03e4c0 0x7f0d1c0409d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0d20006fd0 tx=0x7f0d20006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:37.851 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.848+0000 7f0d32000700 1 -- 172.21.15.6:0/2585393881 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0d28029260 con 0x7f0d341a0060 2024-05-09T04:40:37.851 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:37.107+0000 7f0d3ac00700 1 -- 172.21.15.6:0/2585393881 --> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f0d34063c00 con 0x7f0d1c03e4c0 2024-05-09T04:40:37.851 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:37.117+0000 7f0d32000700 1 -- 172.21.15.6:0/2585393881 <== mgr.14120 v2:172.21.15.6:6800/937713898 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0d34063c00 con 0x7f0d1c03e4c0 2024-05-09T04:40:37.851 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:37.121+0000 7f0d3ac00700 1 -- 172.21.15.6:0/2585393881 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f0d1c03e4c0 msgr2=0x7f0d1c0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:37.851 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:37.121+0000 7f0d3ac00700 1 --2- 172.21.15.6:0/2585393881 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f0d1c03e4c0 0x7f0d1c0409d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0d20006fd0 tx=0x7f0d20006e40 comp rx=0 tx=0).stop 2024-05-09T04:40:37.851 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:37.121+0000 7f0d3ac00700 1 -- 172.21.15.6:0/2585393881 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d341a0060 msgr2=0x7f0d341a0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:37.851 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:37.121+0000 7f0d3ac00700 1 --2- 172.21.15.6:0/2585393881 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d341a0060 0x7f0d341a0490 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0d28000c00 tx=0x7f0d28004260 comp rx=0 tx=0).stop 2024-05-09T04:40:37.851 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:37.121+0000 7f0d3ac00700 1 -- 172.21.15.6:0/2585393881 shutdown_connections 2024-05-09T04:40:37.851 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:37.121+0000 7f0d3ac00700 1 --2- 172.21.15.6:0/2585393881 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f0d1c03e4c0 0x7f0d1c0409d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:37.851 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:37.121+0000 7f0d3ac00700 1 --2- 172.21.15.6:0/2585393881 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d341a0060 0x7f0d341a0490 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:37.851 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:37.121+0000 7f0d3ac00700 1 -- 172.21.15.6:0/2585393881 >> 172.21.15.6:0/2585393881 conn(0x7f0d3407f000 msgr2=0x7f0d341094e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:37.851 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:37.121+0000 7f0d3ac00700 1 -- 172.21.15.6:0/2585393881 shutdown_connections 2024-05-09T04:40:37.851 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:37.122+0000 7f0d3ac00700 1 -- 172.21.15.6:0/2585393881 wait complete. 2024-05-09T04:40:37.852 INFO:teuthology.orchestra.run.smithi006.stdout:Deploying unmanaged mgr service... 2024-05-09T04:40:38.368 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:38 smithi006 ceph-mon[29835]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:40:38.368 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:38 smithi006 ceph-mon[29835]: Saving service mon spec with placement count:5 2024-05-09T04:40:38.368 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:38 smithi006 ceph-mon[29835]: from='mgr.14120 172.21.15.6:0/4038153362' entity='mgr.x' 2024-05-09T04:40:40.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-09T04:40:40.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.107+0000 7f052d400700 1 Processor -- start 2024-05-09T04:40:40.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.107+0000 7f052d400700 1 -- start start 2024-05-09T04:40:40.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.108+0000 7f052d400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0528109b10 0x7f0528109f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:40.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.108+0000 7f052d400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f052810a510 con 0x7f0528109b10 2024-05-09T04:40:40.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.108+0000 7f0526800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0528109b10 0x7f0528109f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:40.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.108+0000 7f0526800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0528109b10 0x7f0528109f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49938/0 (socket says 172.21.15.6:49938) 2024-05-09T04:40:40.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.108+0000 7f0526800700 1 -- 172.21.15.6:0/1731148573 learned_addr learned my addr 172.21.15.6:0/1731148573 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:40.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.109+0000 7f0526800700 1 -- 172.21.15.6:0/1731148573 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f052810ac90 con 0x7f0528109b10 2024-05-09T04:40:40.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.109+0000 7f0526800700 1 --2- 172.21.15.6:0/1731148573 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0528109b10 0x7f0528109f20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f05180098a0 tx=0x7f0518009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5162fe8b2ee00f42 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:40.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.110+0000 7f0525400700 1 -- 172.21.15.6:0/1731148573 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0518004160 con 0x7f0528109b10 2024-05-09T04:40:40.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.110+0000 7f0525400700 1 -- 172.21.15.6:0/1731148573 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0518004300 con 0x7f0528109b10 2024-05-09T04:40:40.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.110+0000 7f0525400700 1 -- 172.21.15.6:0/1731148573 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0518011af0 con 0x7f0528109b10 2024-05-09T04:40:40.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.111+0000 7f052d400700 1 -- 172.21.15.6:0/1731148573 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0528109b10 msgr2=0x7f0528109f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:40.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.111+0000 7f052d400700 1 --2- 172.21.15.6:0/1731148573 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0528109b10 0x7f0528109f20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f05180098a0 tx=0x7f0518009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:40.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.111+0000 7f052d400700 1 -- 172.21.15.6:0/1731148573 shutdown_connections 2024-05-09T04:40:40.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.111+0000 7f052d400700 1 --2- 172.21.15.6:0/1731148573 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0528109b10 0x7f0528109f20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:40.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.111+0000 7f052d400700 1 -- 172.21.15.6:0/1731148573 >> 172.21.15.6:0/1731148573 conn(0x7f0528105230 msgr2=0x7f0528107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:40.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.111+0000 7f052d400700 1 -- 172.21.15.6:0/1731148573 shutdown_connections 2024-05-09T04:40:40.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.111+0000 7f052d400700 1 -- 172.21.15.6:0/1731148573 wait complete. 2024-05-09T04:40:40.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.112+0000 7f052d400700 1 Processor -- start 2024-05-09T04:40:40.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.112+0000 7f052d400700 1 -- start start 2024-05-09T04:40:40.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.113+0000 7f052d400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0528109b10 0x7f052819df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:40.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.113+0000 7f052d400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f052810e040 con 0x7f0528109b10 2024-05-09T04:40:40.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.113+0000 7f0526800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0528109b10 0x7f052819df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:40.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.113+0000 7f0526800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0528109b10 0x7f052819df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49946/0 (socket says 172.21.15.6:49946) 2024-05-09T04:40:40.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.113+0000 7f0526800700 1 -- 172.21.15.6:0/2056169338 learned_addr learned my addr 172.21.15.6:0/2056169338 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:40.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.113+0000 7f0526800700 1 -- 172.21.15.6:0/2056169338 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f052819e460 con 0x7f0528109b10 2024-05-09T04:40:40.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.114+0000 7f0526800700 1 --2- 172.21.15.6:0/2056169338 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0528109b10 0x7f052819df20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0518009fb0 tx=0x7f0518011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:40.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.114+0000 7f051f400700 1 -- 172.21.15.6:0/2056169338 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05180036a0 con 0x7f0528109b10 2024-05-09T04:40:40.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.114+0000 7f051f400700 1 -- 172.21.15.6:0/2056169338 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0518003cd0 con 0x7f0528109b10 2024-05-09T04:40:40.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.114+0000 7f052d400700 1 -- 172.21.15.6:0/2056169338 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f052819e6f0 con 0x7f0528109b10 2024-05-09T04:40:40.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.114+0000 7f051f400700 1 -- 172.21.15.6:0/2056169338 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f051801a430 con 0x7f0528109b10 2024-05-09T04:40:40.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.114+0000 7f052d400700 1 -- 172.21.15.6:0/2056169338 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05281a1300 con 0x7f0528109b10 2024-05-09T04:40:40.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.115+0000 7f052d400700 1 -- 172.21.15.6:0/2056169338 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0528064e20 con 0x7f0528109b10 2024-05-09T04:40:40.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.116+0000 7f051f400700 1 -- 172.21.15.6:0/2056169338 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f0518024440 con 0x7f0528109b10 2024-05-09T04:40:40.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.116+0000 7f051f400700 1 --2- 172.21.15.6:0/2056169338 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f050c03e470 0x7f050c040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:40.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.116+0000 7f051f400700 1 -- 172.21.15.6:0/2056169338 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0518053620 con 0x7f0528109b10 2024-05-09T04:40:40.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.117+0000 7f0525e00700 1 --2- 172.21.15.6:0/2056169338 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f050c03e470 0x7f050c040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:40.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.118+0000 7f0525e00700 1 --2- 172.21.15.6:0/2056169338 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f050c03e470 0x7f050c040980 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0514006fd0 tx=0x7f0514006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:40.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.122+0000 7f051f400700 1 -- 172.21.15.6:0/2056169338 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0518024810 con 0x7f0528109b10 2024-05-09T04:40:40.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.374+0000 7f052d400700 1 -- 172.21.15.6:0/2056169338 --> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f0528107260 con 0x7f050c03e470 2024-05-09T04:40:40.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.388+0000 7f051f400700 1 -- 172.21.15.6:0/2056169338 <== mgr.14120 v2:172.21.15.6:6800/937713898 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0528107260 con 0x7f050c03e470 2024-05-09T04:40:40.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.391+0000 7f052d400700 1 -- 172.21.15.6:0/2056169338 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f050c03e470 msgr2=0x7f050c040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:40.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.391+0000 7f052d400700 1 --2- 172.21.15.6:0/2056169338 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f050c03e470 0x7f050c040980 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0514006fd0 tx=0x7f0514006e40 comp rx=0 tx=0).stop 2024-05-09T04:40:40.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.391+0000 7f052d400700 1 -- 172.21.15.6:0/2056169338 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0528109b10 msgr2=0x7f052819df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:40.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.391+0000 7f052d400700 1 --2- 172.21.15.6:0/2056169338 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0528109b10 0x7f052819df20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0518009fb0 tx=0x7f0518011890 comp rx=0 tx=0).stop 2024-05-09T04:40:40.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.392+0000 7f052d400700 1 -- 172.21.15.6:0/2056169338 shutdown_connections 2024-05-09T04:40:40.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.392+0000 7f052d400700 1 --2- 172.21.15.6:0/2056169338 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f050c03e470 0x7f050c040980 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:40.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.392+0000 7f052d400700 1 --2- 172.21.15.6:0/2056169338 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0528109b10 0x7f052819df20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:40.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.392+0000 7f052d400700 1 -- 172.21.15.6:0/2056169338 >> 172.21.15.6:0/2056169338 conn(0x7f0528105230 msgr2=0x7f0528106b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:40.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.392+0000 7f052d400700 1 -- 172.21.15.6:0/2056169338 shutdown_connections 2024-05-09T04:40:40.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:39.392+0000 7f052d400700 1 -- 172.21.15.6:0/2056169338 wait complete. 2024-05-09T04:40:40.696 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:40 smithi006 ceph-mon[29835]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:40:40.696 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:40 smithi006 ceph-mon[29835]: Saving service mgr spec with placement count:2 2024-05-09T04:40:40.696 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:40 smithi006 ceph-mon[29835]: from='mgr.14120 172.21.15.6:0/4038153362' entity='mgr.x' 2024-05-09T04:40:40.696 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:40 smithi006 ceph-mon[29835]: from='mgr.14120 172.21.15.6:0/4038153362' entity='mgr.x' 2024-05-09T04:40:40.696 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:40 smithi006 ceph-mon[29835]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:40:41.696 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:41 smithi006 ceph-mon[29835]: from='mgr.14120 172.21.15.6:0/4038153362' entity='mgr.x' 2024-05-09T04:40:41.696 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:41 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1152956944' entity='client.admin' 2024-05-09T04:40:41.750 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.782+0000 7fd28e400700 1 Processor -- start 2024-05-09T04:40:41.751 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.783+0000 7fd28e400700 1 -- start start 2024-05-09T04:40:41.751 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.783+0000 7fd28e400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd28807f6b0 0x7fd28807db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:41.751 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.783+0000 7fd28e400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd28807e100 con 0x7fd28807f6b0 2024-05-09T04:40:41.751 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.783+0000 7fd287400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd28807f6b0 0x7fd28807db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:41.751 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.784+0000 7fd287400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd28807f6b0 0x7fd28807db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49948/0 (socket says 172.21.15.6:49948) 2024-05-09T04:40:41.751 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.784+0000 7fd287400700 1 -- 172.21.15.6:0/3670486210 learned_addr learned my addr 172.21.15.6:0/3670486210 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:41.751 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.784+0000 7fd287400700 1 -- 172.21.15.6:0/3670486210 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd28807e280 con 0x7fd28807f6b0 2024-05-09T04:40:41.751 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.784+0000 7fd287400700 1 --2- 172.21.15.6:0/3670486210 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd28807f6b0 0x7fd28807db30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd2780098a0 tx=0x7fd278009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad3ac8b127d6b89 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:41.751 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.785+0000 7fd286a00700 1 -- 172.21.15.6:0/3670486210 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd278004160 con 0x7fd28807f6b0 2024-05-09T04:40:41.751 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.785+0000 7fd286a00700 1 -- 172.21.15.6:0/3670486210 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd278004300 con 0x7fd28807f6b0 2024-05-09T04:40:41.751 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.785+0000 7fd286a00700 1 -- 172.21.15.6:0/3670486210 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd278011a20 con 0x7fd28807f6b0 2024-05-09T04:40:41.752 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.786+0000 7fd28e400700 1 -- 172.21.15.6:0/3670486210 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd28807f6b0 msgr2=0x7fd28807db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:41.752 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.786+0000 7fd28e400700 1 --2- 172.21.15.6:0/3670486210 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd28807f6b0 0x7fd28807db30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd2780098a0 tx=0x7fd278009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:41.752 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.786+0000 7fd28e400700 1 -- 172.21.15.6:0/3670486210 shutdown_connections 2024-05-09T04:40:41.752 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.786+0000 7fd28e400700 1 --2- 172.21.15.6:0/3670486210 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd28807f6b0 0x7fd28807db30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:41.752 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.786+0000 7fd28e400700 1 -- 172.21.15.6:0/3670486210 >> 172.21.15.6:0/3670486210 conn(0x7fd288105230 msgr2=0x7fd288107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:41.752 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.786+0000 7fd28e400700 1 -- 172.21.15.6:0/3670486210 shutdown_connections 2024-05-09T04:40:41.752 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.787+0000 7fd28e400700 1 -- 172.21.15.6:0/3670486210 wait complete. 2024-05-09T04:40:41.752 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.788+0000 7fd28e400700 1 Processor -- start 2024-05-09T04:40:41.752 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.788+0000 7fd28e400700 1 -- start start 2024-05-09T04:40:41.752 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.788+0000 7fd28e400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd28807f6b0 0x7fd2881a0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:41.752 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.788+0000 7fd28e400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd288110130 con 0x7fd28807f6b0 2024-05-09T04:40:41.752 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.788+0000 7fd287400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd28807f6b0 0x7fd2881a0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:41.752 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.789+0000 7fd287400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd28807f6b0 0x7fd2881a0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49956/0 (socket says 172.21.15.6:49956) 2024-05-09T04:40:41.752 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.789+0000 7fd287400700 1 -- 172.21.15.6:0/1152956944 learned_addr learned my addr 172.21.15.6:0/1152956944 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:41.752 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.789+0000 7fd287400700 1 -- 172.21.15.6:0/1152956944 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2881a05c0 con 0x7fd28807f6b0 2024-05-09T04:40:41.753 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.789+0000 7fd287400700 1 --2- 172.21.15.6:0/1152956944 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd28807f6b0 0x7fd2881a0080 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd278000c00 tx=0x7fd278011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:41.753 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.790+0000 7fd284c00700 1 -- 172.21.15.6:0/1152956944 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd278003aa0 con 0x7fd28807f6b0 2024-05-09T04:40:41.753 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.790+0000 7fd284c00700 1 -- 172.21.15.6:0/1152956944 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd27801b8f0 con 0x7fd28807f6b0 2024-05-09T04:40:41.753 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.790+0000 7fd28e400700 1 -- 172.21.15.6:0/1152956944 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2881a0850 con 0x7fd28807f6b0 2024-05-09T04:40:41.753 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.790+0000 7fd28e400700 1 -- 172.21.15.6:0/1152956944 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2881a3460 con 0x7fd28807f6b0 2024-05-09T04:40:41.753 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.790+0000 7fd284c00700 1 -- 172.21.15.6:0/1152956944 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd278023890 con 0x7fd28807f6b0 2024-05-09T04:40:41.753 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.791+0000 7fd28e400700 1 -- 172.21.15.6:0/1152956944 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd288064e20 con 0x7fd28807f6b0 2024-05-09T04:40:41.753 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.791+0000 7fd284c00700 1 -- 172.21.15.6:0/1152956944 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fd27802c3f0 con 0x7fd28807f6b0 2024-05-09T04:40:41.753 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.792+0000 7fd284c00700 1 --2- 172.21.15.6:0/1152956944 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7fd26c03e4c0 0x7fd26c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:41.753 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.792+0000 7fd284c00700 1 -- 172.21.15.6:0/1152956944 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd278053d80 con 0x7fd28807f6b0 2024-05-09T04:40:41.753 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.792+0000 7fd27fe00700 1 --2- 172.21.15.6:0/1152956944 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7fd26c03e4c0 0x7fd26c0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:41.753 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.793+0000 7fd27fe00700 1 --2- 172.21.15.6:0/1152956944 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7fd26c03e4c0 0x7fd26c0409d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd270006fd0 tx=0x7fd270006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:41.753 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:40.798+0000 7fd284c00700 1 -- 172.21.15.6:0/1152956944 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd27801f030 con 0x7fd28807f6b0 2024-05-09T04:40:41.754 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:41.034+0000 7fd28e400700 1 -- 172.21.15.6:0/1152956944 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fd2881a3a90 con 0x7fd28807f6b0 2024-05-09T04:40:41.754 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:41.041+0000 7fd284c00700 1 -- 172.21.15.6:0/1152956944 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fd27800f2e0 con 0x7fd28807f6b0 2024-05-09T04:40:41.754 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:41.045+0000 7fd28e400700 1 -- 172.21.15.6:0/1152956944 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7fd26c03e4c0 msgr2=0x7fd26c0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:41.754 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:41.045+0000 7fd28e400700 1 --2- 172.21.15.6:0/1152956944 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7fd26c03e4c0 0x7fd26c0409d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd270006fd0 tx=0x7fd270006e40 comp rx=0 tx=0).stop 2024-05-09T04:40:41.754 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:41.045+0000 7fd28e400700 1 -- 172.21.15.6:0/1152956944 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd28807f6b0 msgr2=0x7fd2881a0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:41.754 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:41.045+0000 7fd28e400700 1 --2- 172.21.15.6:0/1152956944 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd28807f6b0 0x7fd2881a0080 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd278000c00 tx=0x7fd278011aa0 comp rx=0 tx=0).stop 2024-05-09T04:40:41.754 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:41.046+0000 7fd28e400700 1 -- 172.21.15.6:0/1152956944 shutdown_connections 2024-05-09T04:40:41.754 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:41.046+0000 7fd28e400700 1 --2- 172.21.15.6:0/1152956944 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7fd26c03e4c0 0x7fd26c0409d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:41.754 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:41.046+0000 7fd28e400700 1 --2- 172.21.15.6:0/1152956944 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd28807f6b0 0x7fd2881a0080 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:41.754 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:41.046+0000 7fd28e400700 1 -- 172.21.15.6:0/1152956944 >> 172.21.15.6:0/1152956944 conn(0x7fd288105230 msgr2=0x7fd288106b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:41.754 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:41.046+0000 7fd28e400700 1 -- 172.21.15.6:0/1152956944 shutdown_connections 2024-05-09T04:40:41.754 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:41.046+0000 7fd28e400700 1 -- 172.21.15.6:0/1152956944 wait complete. 2024-05-09T04:40:42.715 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:42 smithi006 ceph-mon[29835]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:40:43.699 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.464+0000 7f5afa400700 1 Processor -- start 2024-05-09T04:40:43.700 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.464+0000 7f5afa400700 1 -- start start 2024-05-09T04:40:43.700 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.465+0000 7f5afa400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5af4108820 0x7f5af4108c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:43.700 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.465+0000 7f5afa400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5af4109220 con 0x7f5af4108820 2024-05-09T04:40:43.700 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.465+0000 7f5af9000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5af4108820 0x7f5af4108c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:43.700 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.465+0000 7f5af9000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5af4108820 0x7f5af4108c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49962/0 (socket says 172.21.15.6:49962) 2024-05-09T04:40:43.700 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.465+0000 7f5af9000700 1 -- 172.21.15.6:0/3785662193 learned_addr learned my addr 172.21.15.6:0/3785662193 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:43.700 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.466+0000 7f5af9000700 1 -- 172.21.15.6:0/3785662193 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af41099f0 con 0x7f5af4108820 2024-05-09T04:40:43.700 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.466+0000 7f5af9000700 1 --2- 172.21.15.6:0/3785662193 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5af4108820 0x7f5af4108c30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009e80 tx=0x7f5aec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b461968e1442f543 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:43.700 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.466+0000 7f5af3400700 1 -- 172.21.15.6:0/3785662193 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5aec017070 con 0x7f5af4108820 2024-05-09T04:40:43.700 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.466+0000 7f5af3400700 1 -- 172.21.15.6:0/3785662193 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5aec00b8c0 con 0x7f5af4108820 2024-05-09T04:40:43.700 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.466+0000 7f5af3400700 1 -- 172.21.15.6:0/3785662193 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5aec004490 con 0x7f5af4108820 2024-05-09T04:40:43.700 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.467+0000 7f5afa400700 1 -- 172.21.15.6:0/3785662193 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5af4108820 msgr2=0x7f5af4108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:43.700 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.467+0000 7f5afa400700 1 --2- 172.21.15.6:0/3785662193 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5af4108820 0x7f5af4108c30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009e80 tx=0x7f5aec00b330 comp rx=0 tx=0).stop 2024-05-09T04:40:43.701 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.467+0000 7f5afa400700 1 -- 172.21.15.6:0/3785662193 shutdown_connections 2024-05-09T04:40:43.701 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.467+0000 7f5afa400700 1 --2- 172.21.15.6:0/3785662193 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5af4108820 0x7f5af4108c30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:43.701 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.467+0000 7f5afa400700 1 -- 172.21.15.6:0/3785662193 >> 172.21.15.6:0/3785662193 conn(0x7f5af4103ea0 msgr2=0x7f5af4106310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:43.701 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.470+0000 7f5afa400700 1 -- 172.21.15.6:0/3785662193 shutdown_connections 2024-05-09T04:40:43.701 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.470+0000 7f5afa400700 1 -- 172.21.15.6:0/3785662193 wait complete. 2024-05-09T04:40:43.701 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.471+0000 7f5afa400700 1 Processor -- start 2024-05-09T04:40:43.701 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.471+0000 7f5afa400700 1 -- start start 2024-05-09T04:40:43.701 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.472+0000 7f5afa400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5af4108820 0x7f5af41a0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:43.701 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.472+0000 7f5afa400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5af410cda0 con 0x7f5af4108820 2024-05-09T04:40:43.701 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.472+0000 7f5af9000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5af4108820 0x7f5af41a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:43.701 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.472+0000 7f5af9000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5af4108820 0x7f5af41a0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49974/0 (socket says 172.21.15.6:49974) 2024-05-09T04:40:43.701 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.472+0000 7f5af9000700 1 -- 172.21.15.6:0/4040625459 learned_addr learned my addr 172.21.15.6:0/4040625459 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:43.701 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.472+0000 7f5af9000700 1 -- 172.21.15.6:0/4040625459 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af41a0550 con 0x7f5af4108820 2024-05-09T04:40:43.701 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.473+0000 7f5af9000700 1 --2- 172.21.15.6:0/4040625459 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5af4108820 0x7f5af41a0010 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009fd0 tx=0x7f5aec00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:43.701 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.473+0000 7f5af1600700 1 -- 172.21.15.6:0/4040625459 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5aec020020 con 0x7f5af4108820 2024-05-09T04:40:43.702 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.473+0000 7f5af1600700 1 -- 172.21.15.6:0/4040625459 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5aec012430 con 0x7f5af4108820 2024-05-09T04:40:43.702 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.473+0000 7f5af1600700 1 -- 172.21.15.6:0/4040625459 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5aec017040 con 0x7f5af4108820 2024-05-09T04:40:43.702 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.473+0000 7f5afa400700 1 -- 172.21.15.6:0/4040625459 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af41a07e0 con 0x7f5af4108820 2024-05-09T04:40:43.702 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.473+0000 7f5afa400700 1 -- 172.21.15.6:0/4040625459 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af41a33f0 con 0x7f5af4108820 2024-05-09T04:40:43.702 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.475+0000 7f5afa400700 1 -- 172.21.15.6:0/4040625459 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5af4064e20 con 0x7f5af4108820 2024-05-09T04:40:43.702 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.475+0000 7f5af1600700 1 -- 172.21.15.6:0/4040625459 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f5aec023020 con 0x7f5af4108820 2024-05-09T04:40:43.702 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.475+0000 7f5af1600700 1 --2- 172.21.15.6:0/4040625459 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f5ad803e4c0 0x7f5ad80409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:43.702 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.476+0000 7f5af1600700 1 -- 172.21.15.6:0/4040625459 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5aec00f050 con 0x7f5af4108820 2024-05-09T04:40:43.702 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.476+0000 7f5af3e00700 1 --2- 172.21.15.6:0/4040625459 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f5ad803e4c0 0x7f5ad80409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:43.702 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.477+0000 7f5af3e00700 1 --2- 172.21.15.6:0/4040625459 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f5ad803e4c0 0x7f5ad80409d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5adc006fd0 tx=0x7f5adc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:43.702 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.483+0000 7f5af1600700 1 -- 172.21.15.6:0/4040625459 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5aec02a020 con 0x7f5af4108820 2024-05-09T04:40:43.702 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.724+0000 7f5afa400700 1 -- 172.21.15.6:0/4040625459 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f5af402e9a0 con 0x7f5af4108820 2024-05-09T04:40:43.702 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.728+0000 7f5af1600700 1 -- 172.21.15.6:0/4040625459 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f5aec02ade0 con 0x7f5af4108820 2024-05-09T04:40:43.703 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.732+0000 7f5afa400700 1 -- 172.21.15.6:0/4040625459 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f5ad803e4c0 msgr2=0x7f5ad80409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:43.703 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.732+0000 7f5afa400700 1 --2- 172.21.15.6:0/4040625459 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f5ad803e4c0 0x7f5ad80409d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5adc006fd0 tx=0x7f5adc006e40 comp rx=0 tx=0).stop 2024-05-09T04:40:43.703 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.732+0000 7f5afa400700 1 -- 172.21.15.6:0/4040625459 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5af4108820 msgr2=0x7f5af41a0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:43.703 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.732+0000 7f5afa400700 1 --2- 172.21.15.6:0/4040625459 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5af4108820 0x7f5af41a0010 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009fd0 tx=0x7f5aec00bd50 comp rx=0 tx=0).stop 2024-05-09T04:40:43.703 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.733+0000 7f5afa400700 1 -- 172.21.15.6:0/4040625459 shutdown_connections 2024-05-09T04:40:43.703 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.733+0000 7f5afa400700 1 --2- 172.21.15.6:0/4040625459 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f5ad803e4c0 0x7f5ad80409d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:43.703 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.733+0000 7f5afa400700 1 --2- 172.21.15.6:0/4040625459 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5af4108820 0x7f5af41a0010 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:43.703 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.733+0000 7f5afa400700 1 -- 172.21.15.6:0/4040625459 >> 172.21.15.6:0/4040625459 conn(0x7f5af4103ea0 msgr2=0x7f5af4104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:43.703 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.733+0000 7f5afa400700 1 -- 172.21.15.6:0/4040625459 shutdown_connections 2024-05-09T04:40:43.703 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:42.733+0000 7f5afa400700 1 -- 172.21.15.6:0/4040625459 wait complete. 2024-05-09T04:40:43.703 INFO:teuthology.orchestra.run.smithi006.stdout:Enabling the dashboard module... 2024-05-09T04:40:43.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:43 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/4040625459' entity='client.admin' 2024-05-09T04:40:43.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:43 smithi006 ceph-mon[29835]: from='mgr.14120 172.21.15.6:0/4038153362' entity='mgr.x' 2024-05-09T04:40:45.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:44 smithi006 ceph-mon[29835]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:40:45.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:44 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/3753983404' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-09T04:40:45.947 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:45 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: ignoring --setuser ceph since I am not root 2024-05-09T04:40:45.947 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:45 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: ignoring --setgroup ceph since I am not root 2024-05-09T04:40:46.246 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:45 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:45.992+0000 7f9442fa4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T04:40:46.349 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.389+0000 7f676bc00700 1 Processor -- start 2024-05-09T04:40:46.349 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.390+0000 7f676bc00700 1 -- start start 2024-05-09T04:40:46.350 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.390+0000 7f676bc00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f676407ede0 0x7f676407d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:46.350 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.390+0000 7f676bc00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f676407d7a0 con 0x7f676407ede0 2024-05-09T04:40:46.350 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.391+0000 7f6769200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f676407ede0 0x7f676407d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:46.350 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.391+0000 7f6769200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f676407ede0 0x7f676407d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49980/0 (socket says 172.21.15.6:49980) 2024-05-09T04:40:46.350 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.391+0000 7f6769200700 1 -- 172.21.15.6:0/3229546612 learned_addr learned my addr 172.21.15.6:0/3229546612 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:46.350 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.391+0000 7f6769200700 1 -- 172.21.15.6:0/3229546612 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f676407d920 con 0x7f676407ede0 2024-05-09T04:40:46.350 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.391+0000 7f6769200700 1 --2- 172.21.15.6:0/3229546612 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f676407ede0 0x7f676407d260 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f67540098a0 tx=0x7f6754009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb3051bd160940d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:46.350 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.392+0000 7f6763e00700 1 -- 172.21.15.6:0/3229546612 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6754004160 con 0x7f676407ede0 2024-05-09T04:40:46.350 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.392+0000 7f6763e00700 1 -- 172.21.15.6:0/3229546612 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6754004300 con 0x7f676407ede0 2024-05-09T04:40:46.350 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.392+0000 7f6763e00700 1 -- 172.21.15.6:0/3229546612 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6754011a20 con 0x7f676407ede0 2024-05-09T04:40:46.350 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.393+0000 7f676bc00700 1 -- 172.21.15.6:0/3229546612 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f676407ede0 msgr2=0x7f676407d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:46.350 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.393+0000 7f676bc00700 1 --2- 172.21.15.6:0/3229546612 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f676407ede0 0x7f676407d260 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f67540098a0 tx=0x7f6754009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:46.351 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.393+0000 7f676bc00700 1 -- 172.21.15.6:0/3229546612 shutdown_connections 2024-05-09T04:40:46.351 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.393+0000 7f676bc00700 1 --2- 172.21.15.6:0/3229546612 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f676407ede0 0x7f676407d260 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:46.351 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.393+0000 7f676bc00700 1 -- 172.21.15.6:0/3229546612 >> 172.21.15.6:0/3229546612 conn(0x7f6764105120 msgr2=0x7f67641075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:46.351 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.393+0000 7f676bc00700 1 -- 172.21.15.6:0/3229546612 shutdown_connections 2024-05-09T04:40:46.351 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.393+0000 7f676bc00700 1 -- 172.21.15.6:0/3229546612 wait complete. 2024-05-09T04:40:46.351 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.394+0000 7f676bc00700 1 Processor -- start 2024-05-09T04:40:46.351 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.394+0000 7f676bc00700 1 -- start start 2024-05-09T04:40:46.351 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.395+0000 7f676bc00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f676407ede0 0x7f67641a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:46.351 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.395+0000 7f676bc00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6764110100 con 0x7f676407ede0 2024-05-09T04:40:46.351 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.395+0000 7f6769200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f676407ede0 0x7f67641a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:46.351 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.395+0000 7f6769200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f676407ede0 0x7f67641a0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49986/0 (socket says 172.21.15.6:49986) 2024-05-09T04:40:46.351 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.395+0000 7f6769200700 1 -- 172.21.15.6:0/3753983404 learned_addr learned my addr 172.21.15.6:0/3753983404 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:46.351 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.395+0000 7f6769200700 1 -- 172.21.15.6:0/3753983404 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67641a05d0 con 0x7f676407ede0 2024-05-09T04:40:46.352 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.396+0000 7f6769200700 1 --2- 172.21.15.6:0/3753983404 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f676407ede0 0x7f67641a0090 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6754000c00 tx=0x7f6754011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:46.352 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.396+0000 7f6762000700 1 -- 172.21.15.6:0/3753983404 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6754003aa0 con 0x7f676407ede0 2024-05-09T04:40:46.352 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.396+0000 7f6762000700 1 -- 172.21.15.6:0/3753983404 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f675401b8f0 con 0x7f676407ede0 2024-05-09T04:40:46.352 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.396+0000 7f6762000700 1 -- 172.21.15.6:0/3753983404 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6754023890 con 0x7f676407ede0 2024-05-09T04:40:46.352 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.396+0000 7f676bc00700 1 -- 172.21.15.6:0/3753983404 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67641a0860 con 0x7f676407ede0 2024-05-09T04:40:46.352 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.396+0000 7f676bc00700 1 -- 172.21.15.6:0/3753983404 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67641a3470 con 0x7f676407ede0 2024-05-09T04:40:46.352 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.398+0000 7f676bc00700 1 -- 172.21.15.6:0/3753983404 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6764064e20 con 0x7f676407ede0 2024-05-09T04:40:46.352 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.398+0000 7f6762000700 1 -- 172.21.15.6:0/3753983404 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f675401b440 con 0x7f676407ede0 2024-05-09T04:40:46.352 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.399+0000 7f6762000700 1 --2- 172.21.15.6:0/3753983404 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f674803e470 0x7f6748040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:46.352 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.399+0000 7f6762000700 1 -- 172.21.15.6:0/3753983404 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6754053e80 con 0x7f676407ede0 2024-05-09T04:40:46.352 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.399+0000 7f675be00700 1 --2- 172.21.15.6:0/3753983404 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f674803e470 0x7f6748040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:46.352 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.400+0000 7f675be00700 1 --2- 172.21.15.6:0/3753983404 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f674803e470 0x7f6748040980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f674c006fd0 tx=0x7f674c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:46.352 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.404+0000 7f6762000700 1 -- 172.21.15.6:0/3753983404 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f675401f030 con 0x7f676407ede0 2024-05-09T04:40:46.353 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:44.709+0000 7f676bc00700 1 -- 172.21.15.6:0/3753983404 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f67641a3a60 con 0x7f676407ede0 2024-05-09T04:40:46.353 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:45.571+0000 7f6762000700 1 -- 172.21.15.6:0/3753983404 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f675401bda0 con 0x7f676407ede0 2024-05-09T04:40:46.353 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:45.575+0000 7f676bc00700 1 -- 172.21.15.6:0/3753983404 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f674803e470 msgr2=0x7f6748040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:46.353 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:45.575+0000 7f676bc00700 1 --2- 172.21.15.6:0/3753983404 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f674803e470 0x7f6748040980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f674c006fd0 tx=0x7f674c006e40 comp rx=0 tx=0).stop 2024-05-09T04:40:46.353 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:45.575+0000 7f676bc00700 1 -- 172.21.15.6:0/3753983404 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f676407ede0 msgr2=0x7f67641a0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:46.353 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:45.575+0000 7f676bc00700 1 --2- 172.21.15.6:0/3753983404 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f676407ede0 0x7f67641a0090 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6754000c00 tx=0x7f6754011aa0 comp rx=0 tx=0).stop 2024-05-09T04:40:46.353 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:45.576+0000 7f676bc00700 1 -- 172.21.15.6:0/3753983404 shutdown_connections 2024-05-09T04:40:46.353 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:45.576+0000 7f676bc00700 1 --2- 172.21.15.6:0/3753983404 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f674803e470 0x7f6748040980 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:46.353 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:45.576+0000 7f676bc00700 1 --2- 172.21.15.6:0/3753983404 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f676407ede0 0x7f67641a0090 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:46.353 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:45.576+0000 7f676bc00700 1 -- 172.21.15.6:0/3753983404 >> 172.21.15.6:0/3753983404 conn(0x7f6764105120 msgr2=0x7f6764105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:46.353 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:45.576+0000 7f676bc00700 1 -- 172.21.15.6:0/3753983404 shutdown_connections 2024-05-09T04:40:46.353 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:45.576+0000 7f676bc00700 1 -- 172.21.15.6:0/3753983404 wait complete. 2024-05-09T04:40:46.572 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:46 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:46.245+0000 7f9442fa4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T04:40:46.572 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:46 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:46.386+0000 7f9442fa4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T04:40:46.914 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:46 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/3753983404' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-09T04:40:46.914 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:46 smithi006 ceph-mon[29835]: mgrmap e9: x(active, since 25s) 2024-05-09T04:40:46.915 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:46 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:46.807+0000 7f9442fa4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T04:40:47.196 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:46 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:46.930+0000 7f9442fa4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T04:40:47.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:47 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/3193398719' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-09T04:40:48.100 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-05-09T04:40:48.101 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-09T04:40:48.102 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-09T04:40:48.102 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-09T04:40:48.102 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-09T04:40:48.102 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-05-09T04:40:48.104 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.035+0000 7f6b57000700 1 Processor -- start 2024-05-09T04:40:48.104 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.035+0000 7f6b57000700 1 -- start start 2024-05-09T04:40:48.104 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.036+0000 7f6b57000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b50109b20 0x7f6b50109f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:48.104 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.036+0000 7f6b57000700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5010a520 con 0x7f6b50109b20 2024-05-09T04:40:48.104 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.036+0000 7f6b4fe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b50109b20 0x7f6b50109f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:48.104 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.037+0000 7f6b4fe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b50109b20 0x7f6b50109f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43438/0 (socket says 172.21.15.6:43438) 2024-05-09T04:40:48.104 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.037+0000 7f6b4fe00700 1 -- 172.21.15.6:0/4178081285 learned_addr learned my addr 172.21.15.6:0/4178081285 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:48.104 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.037+0000 7f6b4fe00700 1 -- 172.21.15.6:0/4178081285 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b5010acf0 con 0x7f6b50109b20 2024-05-09T04:40:48.104 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.037+0000 7f6b4fe00700 1 --2- 172.21.15.6:0/4178081285 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b50109b20 0x7f6b50109f30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6b400098a0 tx=0x7f6b40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4076249416df9780 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:48.104 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.038+0000 7f6b4ea00700 1 -- 172.21.15.6:0/4178081285 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b40004160 con 0x7f6b50109b20 2024-05-09T04:40:48.105 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.038+0000 7f6b4ea00700 1 -- 172.21.15.6:0/4178081285 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b40004300 con 0x7f6b50109b20 2024-05-09T04:40:48.105 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.038+0000 7f6b4ea00700 1 -- 172.21.15.6:0/4178081285 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b40011a20 con 0x7f6b50109b20 2024-05-09T04:40:48.105 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.039+0000 7f6b57000700 1 -- 172.21.15.6:0/4178081285 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b50109b20 msgr2=0x7f6b50109f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:48.105 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.039+0000 7f6b57000700 1 --2- 172.21.15.6:0/4178081285 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b50109b20 0x7f6b50109f30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6b400098a0 tx=0x7f6b40009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:48.105 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.039+0000 7f6b57000700 1 -- 172.21.15.6:0/4178081285 shutdown_connections 2024-05-09T04:40:48.105 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.039+0000 7f6b57000700 1 --2- 172.21.15.6:0/4178081285 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b50109b20 0x7f6b50109f30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:48.108 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.039+0000 7f6b57000700 1 -- 172.21.15.6:0/4178081285 >> 172.21.15.6:0/4178081285 conn(0x7f6b501051e0 msgr2=0x7f6b50107630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:48.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.039+0000 7f6b57000700 1 -- 172.21.15.6:0/4178081285 shutdown_connections 2024-05-09T04:40:48.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.039+0000 7f6b57000700 1 -- 172.21.15.6:0/4178081285 wait complete. 2024-05-09T04:40:48.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.040+0000 7f6b57000700 1 Processor -- start 2024-05-09T04:40:48.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.041+0000 7f6b57000700 1 -- start start 2024-05-09T04:40:48.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.041+0000 7f6b57000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b50109b20 0x7f6b501a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:48.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.041+0000 7f6b57000700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5010e0a0 con 0x7f6b50109b20 2024-05-09T04:40:48.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.041+0000 7f6b4fe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b50109b20 0x7f6b501a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:48.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.041+0000 7f6b4fe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b50109b20 0x7f6b501a00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43444/0 (socket says 172.21.15.6:43444) 2024-05-09T04:40:48.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.041+0000 7f6b4fe00700 1 -- 172.21.15.6:0/3193398719 learned_addr learned my addr 172.21.15.6:0/3193398719 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:48.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.042+0000 7f6b4fe00700 1 -- 172.21.15.6:0/3193398719 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b501a0630 con 0x7f6b50109b20 2024-05-09T04:40:48.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.042+0000 7f6b4fe00700 1 --2- 172.21.15.6:0/3193398719 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b50109b20 0x7f6b501a00f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6b40003e70 tx=0x7f6b40011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:48.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.042+0000 7f6b4cc00700 1 -- 172.21.15.6:0/3193398719 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b40003aa0 con 0x7f6b50109b20 2024-05-09T04:40:48.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.042+0000 7f6b4cc00700 1 -- 172.21.15.6:0/3193398719 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b4001a8e0 con 0x7f6b50109b20 2024-05-09T04:40:48.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.042+0000 7f6b4cc00700 1 -- 172.21.15.6:0/3193398719 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b40024870 con 0x7f6b50109b20 2024-05-09T04:40:48.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.042+0000 7f6b57000700 1 -- 172.21.15.6:0/3193398719 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b501a08c0 con 0x7f6b50109b20 2024-05-09T04:40:48.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.042+0000 7f6b57000700 1 -- 172.21.15.6:0/3193398719 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b501a34d0 con 0x7f6b50109b20 2024-05-09T04:40:48.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.044+0000 7f6b57000700 1 -- 172.21.15.6:0/3193398719 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b50064e20 con 0x7f6b50109b20 2024-05-09T04:40:48.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.044+0000 7f6b4cc00700 1 -- 172.21.15.6:0/3193398719 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f6b40022020 con 0x7f6b50109b20 2024-05-09T04:40:48.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.045+0000 7f6b4cc00700 1 --2- 172.21.15.6:0/3193398719 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f6b3403e520 0x7f6b34040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:48.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.045+0000 7f6b4cc00700 1 -- 172.21.15.6:0/3193398719 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6b40053940 con 0x7f6b50109b20 2024-05-09T04:40:48.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.045+0000 7f6b4f400700 1 -- 172.21.15.6:0/3193398719 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f6b3403e520 msgr2=0x7f6b34040a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/937713898 2024-05-09T04:40:48.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.045+0000 7f6b4f400700 1 --2- 172.21.15.6:0/3193398719 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f6b3403e520 0x7f6b34040a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T04:40:48.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.051+0000 7f6b4cc00700 1 -- 172.21.15.6:0/3193398719 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6b400532d0 con 0x7f6b50109b20 2024-05-09T04:40:48.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.246+0000 7f6b4f400700 1 -- 172.21.15.6:0/3193398719 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f6b3403e520 msgr2=0x7f6b34040a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/937713898 2024-05-09T04:40:48.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.246+0000 7f6b4f400700 1 --2- 172.21.15.6:0/3193398719 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f6b3403e520 0x7f6b34040a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T04:40:48.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.346+0000 7f6b57000700 1 -- 172.21.15.6:0/3193398719 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f6b501a36a0 con 0x7f6b50109b20 2024-05-09T04:40:48.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.346+0000 7f6b4cc00700 1 -- 172.21.15.6:0/3193398719 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f6b40016df0 con 0x7f6b50109b20 2024-05-09T04:40:48.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.349+0000 7f6b57000700 1 -- 172.21.15.6:0/3193398719 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f6b3403e520 msgr2=0x7f6b34040a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T04:40:48.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.349+0000 7f6b57000700 1 --2- 172.21.15.6:0/3193398719 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f6b3403e520 0x7f6b34040a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:48.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.349+0000 7f6b57000700 1 -- 172.21.15.6:0/3193398719 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b50109b20 msgr2=0x7f6b501a00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:48.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.349+0000 7f6b57000700 1 --2- 172.21.15.6:0/3193398719 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b50109b20 0x7f6b501a00f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6b40003e70 tx=0x7f6b40011aa0 comp rx=0 tx=0).stop 2024-05-09T04:40:48.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.349+0000 7f6b57000700 1 -- 172.21.15.6:0/3193398719 shutdown_connections 2024-05-09T04:40:48.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.349+0000 7f6b57000700 1 --2- 172.21.15.6:0/3193398719 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f6b3403e520 0x7f6b34040a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:48.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.349+0000 7f6b57000700 1 --2- 172.21.15.6:0/3193398719 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b50109b20 0x7f6b501a00f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:48.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.349+0000 7f6b57000700 1 -- 172.21.15.6:0/3193398719 >> 172.21.15.6:0/3193398719 conn(0x7f6b501051e0 msgr2=0x7f6b5007d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:48.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.349+0000 7f6b57000700 1 -- 172.21.15.6:0/3193398719 shutdown_connections 2024-05-09T04:40:48.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:47.350+0000 7f6b57000700 1 -- 172.21.15.6:0/3193398719 wait complete. 2024-05-09T04:40:48.112 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for the mgr to restart... 2024-05-09T04:40:48.112 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mgr epoch 9... 2024-05-09T04:40:48.687 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:48 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:48.591+0000 7f9442fa4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T04:40:48.946 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:48 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:48.842+0000 7f9442fa4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T04:40:49.423 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:49 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:49.067+0000 7f9442fa4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T04:40:49.696 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:49 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:49.465+0000 7f9442fa4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T04:40:49.696 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:49 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:49.593+0000 7f9442fa4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T04:40:50.121 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:49 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:49.852+0000 7f9442fa4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T04:40:50.121 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:50 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:50.120+0000 7f9442fa4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T04:40:51.251 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:50 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:50.947+0000 7f9442fa4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T04:40:51.251 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:51 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:51.250+0000 7f9442fa4200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T04:40:52.196 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:51 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:51.833+0000 7f9442fa4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T04:40:52.196 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:51 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:51.968+0000 7f9442fa4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T04:40:52.922 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:52 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:52.447+0000 7f9442fa4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T04:40:52.922 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:52 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:52.922+0000 7f9442fa4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T04:40:53.196 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:53 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:53.053+0000 7f9442fa4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T04:40:53.696 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:53 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:53.393+0000 7f9442fa4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T04:40:53.696 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:53 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:53.581+0000 7f9442fa4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T04:40:54.126 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:53 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:53.717+0000 7f9442fa4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T04:40:54.446 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:54 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:54.125+0000 7f9442fa4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T04:40:54.446 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:54 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:54.255+0000 7f9442fa4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T04:40:55.353 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:55 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:55.053+0000 7f9442fa4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T04:40:55.696 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:40:55 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:40:55.352+0000 7f9442fa4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T04:40:55.696 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:55 smithi006 ceph-mon[29835]: Active manager daemon x restarted 2024-05-09T04:40:55.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:55 smithi006 ceph-mon[29835]: Activating manager daemon x 2024-05-09T04:40:55.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:55 smithi006 ceph-mon[29835]: osdmap e3: 0 total, 0 up, 0 in 2024-05-09T04:40:55.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:55 smithi006 ceph-mon[29835]: mgrmap e10: x(active, starting, since 0.0123338s) 2024-05-09T04:40:55.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:55 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:40:55.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:55 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T04:40:55.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:55 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T04:40:55.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:55 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T04:40:55.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:55 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T04:40:55.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:55 smithi006 ceph-mon[29835]: Manager daemon x is now available 2024-05-09T04:40:56.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:40:56.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T04:40:56.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T04:40:56.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:56 smithi006 ceph-mon[29835]: mgrmap e11: x(active, since 1.0139s) 2024-05-09T04:40:57.077 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-05-09T04:40:57.078 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-09T04:40:57.078 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-09T04:40:57.078 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-05-09T04:40:57.078 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.810+0000 7f52e7400700 1 Processor -- start 2024-05-09T04:40:57.078 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.811+0000 7f52e7400700 1 -- start start 2024-05-09T04:40:57.078 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.811+0000 7f52e7400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52e010a550 0x7f52e010c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:57.078 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.811+0000 7f52e7400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f52e007ce80 con 0x7f52e010a550 2024-05-09T04:40:57.078 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.811+0000 7f52e4a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52e010a550 0x7f52e010c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:57.085 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.812+0000 7f52e4a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52e010a550 0x7f52e010c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43446/0 (socket says 172.21.15.6:43446) 2024-05-09T04:40:57.086 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.812+0000 7f52e4a00700 1 -- 172.21.15.6:0/278591258 learned_addr learned my addr 172.21.15.6:0/278591258 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:57.086 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.812+0000 7f52e4a00700 1 -- 172.21.15.6:0/278591258 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e010ceb0 con 0x7f52e010a550 2024-05-09T04:40:57.086 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.812+0000 7f52e4a00700 1 --2- 172.21.15.6:0/278591258 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52e010a550 0x7f52e010c970 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f52d40098a0 tx=0x7f52d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1cf72498d756953 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:57.086 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.813+0000 7f52df400700 1 -- 172.21.15.6:0/278591258 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52d4004160 con 0x7f52e010a550 2024-05-09T04:40:57.086 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.813+0000 7f52df400700 1 -- 172.21.15.6:0/278591258 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52d4004300 con 0x7f52e010a550 2024-05-09T04:40:57.086 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.813+0000 7f52df400700 1 -- 172.21.15.6:0/278591258 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52d4011a20 con 0x7f52e010a550 2024-05-09T04:40:57.086 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.814+0000 7f52e7400700 1 -- 172.21.15.6:0/278591258 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52e010a550 msgr2=0x7f52e010c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:57.086 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.814+0000 7f52e7400700 1 --2- 172.21.15.6:0/278591258 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52e010a550 0x7f52e010c970 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f52d40098a0 tx=0x7f52d4009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:57.086 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.814+0000 7f52e7400700 1 -- 172.21.15.6:0/278591258 shutdown_connections 2024-05-09T04:40:57.086 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.814+0000 7f52e7400700 1 --2- 172.21.15.6:0/278591258 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52e010a550 0x7f52e010c970 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:57.086 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.814+0000 7f52e7400700 1 -- 172.21.15.6:0/278591258 >> 172.21.15.6:0/278591258 conn(0x7f52e0104070 msgr2=0x7f52e01064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:57.086 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.814+0000 7f52e7400700 1 -- 172.21.15.6:0/278591258 shutdown_connections 2024-05-09T04:40:57.086 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.814+0000 7f52e7400700 1 -- 172.21.15.6:0/278591258 wait complete. 2024-05-09T04:40:57.087 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.816+0000 7f52e7400700 1 Processor -- start 2024-05-09T04:40:57.087 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.816+0000 7f52e7400700 1 -- start start 2024-05-09T04:40:57.087 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.817+0000 7f52e7400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52e010a550 0x7f52e01a0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:57.087 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.817+0000 7f52e7400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f52e010deb0 con 0x7f52e010a550 2024-05-09T04:40:57.087 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.817+0000 7f52e4a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52e010a550 0x7f52e01a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:57.087 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.817+0000 7f52e4a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52e010a550 0x7f52e01a0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43448/0 (socket says 172.21.15.6:43448) 2024-05-09T04:40:57.087 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.817+0000 7f52e4a00700 1 -- 172.21.15.6:0/1563749706 learned_addr learned my addr 172.21.15.6:0/1563749706 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:57.087 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.817+0000 7f52e4a00700 1 -- 172.21.15.6:0/1563749706 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e01a0560 con 0x7f52e010a550 2024-05-09T04:40:57.087 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.818+0000 7f52e4a00700 1 --2- 172.21.15.6:0/1563749706 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52e010a550 0x7f52e01a0020 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f52d4000c00 tx=0x7f52d4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:57.087 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.818+0000 7f52dd600700 1 -- 172.21.15.6:0/1563749706 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52d4003aa0 con 0x7f52e010a550 2024-05-09T04:40:57.087 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.818+0000 7f52dd600700 1 -- 172.21.15.6:0/1563749706 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52d401b8f0 con 0x7f52e010a550 2024-05-09T04:40:57.087 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.818+0000 7f52dd600700 1 -- 172.21.15.6:0/1563749706 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52d4023890 con 0x7f52e010a550 2024-05-09T04:40:57.087 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.818+0000 7f52e7400700 1 -- 172.21.15.6:0/1563749706 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52e01a07f0 con 0x7f52e010a550 2024-05-09T04:40:57.088 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.818+0000 7f52e7400700 1 -- 172.21.15.6:0/1563749706 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52e01a3400 con 0x7f52e010a550 2024-05-09T04:40:57.088 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.820+0000 7f52dd600700 1 -- 172.21.15.6:0/1563749706 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f52d401b440 con 0x7f52e010a550 2024-05-09T04:40:57.088 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.821+0000 7f52dd600700 1 --2- 172.21.15.6:0/1563749706 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f52c803e520 0x7f52c8040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:57.088 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.821+0000 7f52dd600700 1 -- 172.21.15.6:0/1563749706 --> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f52c8041140 con 0x7f52c803e520 2024-05-09T04:40:57.088 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.821+0000 7f52dd600700 1 -- 172.21.15.6:0/1563749706 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f52d4053f60 con 0x7f52e010a550 2024-05-09T04:40:57.088 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.821+0000 7f52dfe00700 1 -- 172.21.15.6:0/1563749706 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f52c803e520 msgr2=0x7f52c8040a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/937713898 2024-05-09T04:40:57.088 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:48.821+0000 7f52dfe00700 1 --2- 172.21.15.6:0/1563749706 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f52c803e520 0x7f52c8040a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T04:40:57.088 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:49.022+0000 7f52dfe00700 1 -- 172.21.15.6:0/1563749706 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f52c803e520 msgr2=0x7f52c8040a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/937713898 2024-05-09T04:40:57.088 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:49.022+0000 7f52dfe00700 1 --2- 172.21.15.6:0/1563749706 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f52c803e520 0x7f52c8040a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T04:40:57.088 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:49.423+0000 7f52dfe00700 1 -- 172.21.15.6:0/1563749706 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f52c803e520 msgr2=0x7f52c8040a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/937713898 2024-05-09T04:40:57.088 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:49.423+0000 7f52dfe00700 1 --2- 172.21.15.6:0/1563749706 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f52c803e520 0x7f52c8040a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T04:40:57.088 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:50.224+0000 7f52dfe00700 1 -- 172.21.15.6:0/1563749706 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f52c803e520 msgr2=0x7f52c8040a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/937713898 2024-05-09T04:40:57.089 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:50.224+0000 7f52dfe00700 1 --2- 172.21.15.6:0/1563749706 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f52c803e520 0x7f52c8040a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T04:40:57.089 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:51.826+0000 7f52dfe00700 1 -- 172.21.15.6:0/1563749706 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f52c803e520 msgr2=0x7f52c8040a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/937713898 2024-05-09T04:40:57.089 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:51.826+0000 7f52dfe00700 1 --2- 172.21.15.6:0/1563749706 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f52c803e520 0x7f52c8040a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-09T04:40:57.089 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:55.029+0000 7f52dfe00700 1 -- 172.21.15.6:0/1563749706 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f52c803e520 msgr2=0x7f52c8040a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/937713898 2024-05-09T04:40:57.089 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:55.030+0000 7f52dfe00700 1 --2- 172.21.15.6:0/1563749706 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f52c803e520 0x7f52c8040a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-05-09T04:40:57.089 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:55.363+0000 7f52dd600700 1 -- 172.21.15.6:0/1563749706 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7f52d400b3e0 con 0x7f52e010a550 2024-05-09T04:40:57.089 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:55.364+0000 7f52dd600700 1 -- 172.21.15.6:0/1563749706 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f52c803e520 msgr2=0x7f52c8040a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T04:40:57.089 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:55.364+0000 7f52dd600700 1 --2- 172.21.15.6:0/1563749706 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f52c803e520 0x7f52c8040a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:57.089 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:56.366+0000 7f52dd600700 1 -- 172.21.15.6:0/1563749706 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f52d402c940 con 0x7f52e010a550 2024-05-09T04:40:57.089 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:56.366+0000 7f52dd600700 1 --2- 172.21.15.6:0/1563749706 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f52d4053af0 0x7f52c8044000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:57.089 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:56.367+0000 7f52dd600700 1 -- 172.21.15.6:0/1563749706 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f52c8041140 con 0x7f52d4053af0 2024-05-09T04:40:57.089 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:56.370+0000 7f52dfe00700 1 --2- 172.21.15.6:0/1563749706 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f52d4053af0 0x7f52c8044000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:57.089 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:56.373+0000 7f52dfe00700 1 --2- 172.21.15.6:0/1563749706 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f52d4053af0 0x7f52c8044000 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f52cc003a10 tx=0x7f52cc009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:57.089 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:56.376+0000 7f52dd600700 1 -- 172.21.15.6:0/1563749706 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f52c8041140 con 0x7f52d4053af0 2024-05-09T04:40:57.090 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:56.386+0000 7f52e7400700 1 -- 172.21.15.6:0/1563749706 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f52e0064e20 con 0x7f52d4053af0 2024-05-09T04:40:57.090 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:56.386+0000 7f52dd600700 1 -- 172.21.15.6:0/1563749706 <== mgr.14152 v2:172.21.15.6:6800/2997423326 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f52e0064e20 con 0x7f52d4053af0 2024-05-09T04:40:57.090 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:56.386+0000 7f52e7400700 1 -- 172.21.15.6:0/1563749706 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f52d4053af0 msgr2=0x7f52c8044000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:57.090 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:56.386+0000 7f52e7400700 1 --2- 172.21.15.6:0/1563749706 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f52d4053af0 0x7f52c8044000 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f52cc003a10 tx=0x7f52cc009270 comp rx=0 tx=0).stop 2024-05-09T04:40:57.090 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:56.387+0000 7f52e7400700 1 -- 172.21.15.6:0/1563749706 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52e010a550 msgr2=0x7f52e01a0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:57.090 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:56.387+0000 7f52e7400700 1 --2- 172.21.15.6:0/1563749706 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52e010a550 0x7f52e01a0020 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f52d4000c00 tx=0x7f52d4011aa0 comp rx=0 tx=0).stop 2024-05-09T04:40:57.090 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:56.387+0000 7f52e7400700 1 -- 172.21.15.6:0/1563749706 shutdown_connections 2024-05-09T04:40:57.090 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:56.387+0000 7f52e7400700 1 --2- 172.21.15.6:0/1563749706 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f52d4053af0 0x7f52c8044000 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:57.090 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:56.387+0000 7f52e7400700 1 --2- 172.21.15.6:0/1563749706 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52e010a550 0x7f52e01a0020 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:57.090 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:56.387+0000 7f52e7400700 1 --2- 172.21.15.6:0/1563749706 >> [v2:172.21.15.6:6800/937713898,v1:172.21.15.6:6801/937713898] conn(0x7f52c803e520 0x7f52c8040a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:57.090 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:56.387+0000 7f52e7400700 1 -- 172.21.15.6:0/1563749706 >> 172.21.15.6:0/1563749706 conn(0x7f52e0104070 msgr2=0x7f52e0105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:57.090 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:56.387+0000 7f52e7400700 1 -- 172.21.15.6:0/1563749706 shutdown_connections 2024-05-09T04:40:57.090 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:56.387+0000 7f52e7400700 1 -- 172.21.15.6:0/1563749706 wait complete. 2024-05-09T04:40:57.091 INFO:teuthology.orchestra.run.smithi006.stdout:mgr epoch 9 is available 2024-05-09T04:40:57.091 INFO:teuthology.orchestra.run.smithi006.stdout:Generating a dashboard self-signed certificate... 2024-05-09T04:40:57.696 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:57 smithi006 ceph-mon[29835]: [09/May/2024:04:40:56] ENGINE Bus STARTING 2024-05-09T04:40:57.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:57 smithi006 ceph-mon[29835]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-09T04:40:57.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:57 smithi006 ceph-mon[29835]: [09/May/2024:04:40:56] ENGINE Serving on http://172.21.15.6:8765 2024-05-09T04:40:57.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:57 smithi006 ceph-mon[29835]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-09T04:40:57.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:57 smithi006 ceph-mon[29835]: [09/May/2024:04:40:56] ENGINE Serving on https://172.21.15.6:7150 2024-05-09T04:40:57.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:57 smithi006 ceph-mon[29835]: [09/May/2024:04:40:56] ENGINE Bus STARTED 2024-05-09T04:40:58.696 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:58 smithi006 ceph-mon[29835]: mgrmap e12: x(active, since 2s) 2024-05-09T04:40:59.378 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-09T04:40:59.378 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.171+0000 7f418b800700 1 Processor -- start 2024-05-09T04:40:59.378 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.171+0000 7f418b800700 1 -- start start 2024-05-09T04:40:59.378 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.172+0000 7f418b800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4184109b20 0x7f4184109f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:59.378 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.172+0000 7f418b800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f418410a520 con 0x7f4184109b20 2024-05-09T04:40:59.378 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.172+0000 7f4188e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4184109b20 0x7f4184109f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:59.380 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.172+0000 7f4188e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4184109b20 0x7f4184109f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55896/0 (socket says 172.21.15.6:55896) 2024-05-09T04:40:59.380 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.172+0000 7f4188e00700 1 -- 172.21.15.6:0/1350979373 learned_addr learned my addr 172.21.15.6:0/1350979373 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:59.380 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.173+0000 7f4188e00700 1 -- 172.21.15.6:0/1350979373 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f418410acf0 con 0x7f4184109b20 2024-05-09T04:40:59.380 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.173+0000 7f4188e00700 1 --2- 172.21.15.6:0/1350979373 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4184109b20 0x7f4184109f30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f41780098a0 tx=0x7f4178009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c723803ebe7ec7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:59.380 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.173+0000 7f4183400700 1 -- 172.21.15.6:0/1350979373 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4178004160 con 0x7f4184109b20 2024-05-09T04:40:59.380 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.173+0000 7f4183400700 1 -- 172.21.15.6:0/1350979373 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4178004300 con 0x7f4184109b20 2024-05-09T04:40:59.380 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.173+0000 7f4183400700 1 -- 172.21.15.6:0/1350979373 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4178011a20 con 0x7f4184109b20 2024-05-09T04:40:59.380 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.174+0000 7f418b800700 1 -- 172.21.15.6:0/1350979373 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4184109b20 msgr2=0x7f4184109f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:59.380 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.174+0000 7f418b800700 1 --2- 172.21.15.6:0/1350979373 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4184109b20 0x7f4184109f30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f41780098a0 tx=0x7f4178009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:59.380 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.174+0000 7f418b800700 1 -- 172.21.15.6:0/1350979373 shutdown_connections 2024-05-09T04:40:59.380 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.174+0000 7f418b800700 1 --2- 172.21.15.6:0/1350979373 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4184109b20 0x7f4184109f30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:59.381 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.174+0000 7f418b800700 1 -- 172.21.15.6:0/1350979373 >> 172.21.15.6:0/1350979373 conn(0x7f41841051e0 msgr2=0x7f4184107630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:59.382 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.174+0000 7f418b800700 1 -- 172.21.15.6:0/1350979373 shutdown_connections 2024-05-09T04:40:59.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.174+0000 7f418b800700 1 -- 172.21.15.6:0/1350979373 wait complete. 2024-05-09T04:40:59.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.176+0000 7f418b800700 1 Processor -- start 2024-05-09T04:40:59.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.176+0000 7f418b800700 1 -- start start 2024-05-09T04:40:59.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.176+0000 7f418b800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4184109b20 0x7f41841a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:59.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.176+0000 7f418b800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f418410e0a0 con 0x7f4184109b20 2024-05-09T04:40:59.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.176+0000 7f4188e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4184109b20 0x7f41841a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:59.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.176+0000 7f4188e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4184109b20 0x7f41841a00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55910/0 (socket says 172.21.15.6:55910) 2024-05-09T04:40:59.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.176+0000 7f4188e00700 1 -- 172.21.15.6:0/1939818399 learned_addr learned my addr 172.21.15.6:0/1939818399 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:40:59.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.177+0000 7f4188e00700 1 -- 172.21.15.6:0/1939818399 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41841a0630 con 0x7f4184109b20 2024-05-09T04:40:59.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.177+0000 7f4188e00700 1 --2- 172.21.15.6:0/1939818399 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4184109b20 0x7f41841a00f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4178003e70 tx=0x7f4178011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:59.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.177+0000 7f4181600700 1 -- 172.21.15.6:0/1939818399 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4178003aa0 con 0x7f4184109b20 2024-05-09T04:40:59.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.177+0000 7f4181600700 1 -- 172.21.15.6:0/1939818399 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f417801a8e0 con 0x7f4184109b20 2024-05-09T04:40:59.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.177+0000 7f418b800700 1 -- 172.21.15.6:0/1939818399 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41841a08c0 con 0x7f4184109b20 2024-05-09T04:40:59.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.177+0000 7f4181600700 1 -- 172.21.15.6:0/1939818399 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4178024870 con 0x7f4184109b20 2024-05-09T04:40:59.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.177+0000 7f418b800700 1 -- 172.21.15.6:0/1939818399 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41841a34d0 con 0x7f4184109b20 2024-05-09T04:40:59.386 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.179+0000 7f418b800700 1 -- 172.21.15.6:0/1939818399 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4184064e20 con 0x7f4184109b20 2024-05-09T04:40:59.386 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.179+0000 7f4181600700 1 -- 172.21.15.6:0/1939818399 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f4178022020 con 0x7f4184109b20 2024-05-09T04:40:59.386 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.180+0000 7f4181600700 1 --2- 172.21.15.6:0/1939818399 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f416c03e4c0 0x7f416c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:59.386 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.180+0000 7f4181600700 1 -- 172.21.15.6:0/1939818399 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4178053ba0 con 0x7f4184109b20 2024-05-09T04:40:59.386 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.180+0000 7f4183e00700 1 --2- 172.21.15.6:0/1939818399 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f416c03e4c0 0x7f416c0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:59.386 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.181+0000 7f4183e00700 1 --2- 172.21.15.6:0/1939818399 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f416c03e4c0 0x7f416c0409d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4170006fd0 tx=0x7f4170006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:59.387 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.186+0000 7f4181600700 1 -- 172.21.15.6:0/1939818399 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f417800b3e0 con 0x7f4184109b20 2024-05-09T04:40:59.387 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.432+0000 7f418b800700 1 -- 172.21.15.6:0/1939818399 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f418407d8f0 con 0x7f416c03e4c0 2024-05-09T04:40:59.387 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.627+0000 7f4181600700 1 -- 172.21.15.6:0/1939818399 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f418407d8f0 con 0x7f416c03e4c0 2024-05-09T04:40:59.387 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.630+0000 7f418b800700 1 -- 172.21.15.6:0/1939818399 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f416c03e4c0 msgr2=0x7f416c0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:59.387 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.630+0000 7f418b800700 1 --2- 172.21.15.6:0/1939818399 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f416c03e4c0 0x7f416c0409d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4170006fd0 tx=0x7f4170006e40 comp rx=0 tx=0).stop 2024-05-09T04:40:59.387 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.630+0000 7f418b800700 1 -- 172.21.15.6:0/1939818399 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4184109b20 msgr2=0x7f41841a00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:59.387 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.630+0000 7f418b800700 1 --2- 172.21.15.6:0/1939818399 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4184109b20 0x7f41841a00f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4178003e70 tx=0x7f4178011aa0 comp rx=0 tx=0).stop 2024-05-09T04:40:59.387 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.630+0000 7f418b800700 1 -- 172.21.15.6:0/1939818399 shutdown_connections 2024-05-09T04:40:59.387 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.630+0000 7f418b800700 1 --2- 172.21.15.6:0/1939818399 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f416c03e4c0 0x7f416c0409d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:59.387 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.630+0000 7f418b800700 1 --2- 172.21.15.6:0/1939818399 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4184109b20 0x7f41841a00f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:59.387 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.630+0000 7f418b800700 1 -- 172.21.15.6:0/1939818399 >> 172.21.15.6:0/1939818399 conn(0x7f41841051e0 msgr2=0x7f418407d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:59.387 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.631+0000 7f418b800700 1 -- 172.21.15.6:0/1939818399 shutdown_connections 2024-05-09T04:40:59.387 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:58.631+0000 7f418b800700 1 -- 172.21.15.6:0/1939818399 wait complete. 2024-05-09T04:40:59.388 INFO:teuthology.orchestra.run.smithi006.stdout:Creating initial admin user... 2024-05-09T04:40:59.676 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:59 smithi006 ceph-mon[29835]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:40:59.676 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:59 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:40:59.677 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:59 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:40:59.677 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:40:59 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:01.308 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$pNvBLFB4s2m9CEqwPVGOYuab1C2Ewd6sA5JAz.2ePiaT34k56jZ/6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715229660, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-09T04:41:01.308 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.042+0000 7fe96a200700 1 Processor -- start 2024-05-09T04:41:01.308 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.042+0000 7fe96a200700 1 -- start start 2024-05-09T04:41:01.308 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.042+0000 7fe96a200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe96407eda0 0x7fe96407d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:01.308 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.042+0000 7fe96a200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe96407d760 con 0x7fe96407eda0 2024-05-09T04:41:01.308 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.043+0000 7fe963400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe96407eda0 0x7fe96407d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:01.308 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.043+0000 7fe963400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe96407eda0 0x7fe96407d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55924/0 (socket says 172.21.15.6:55924) 2024-05-09T04:41:01.308 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.043+0000 7fe963400700 1 -- 172.21.15.6:0/2957947178 learned_addr learned my addr 172.21.15.6:0/2957947178 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:41:01.308 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.043+0000 7fe963400700 1 -- 172.21.15.6:0/2957947178 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe96407d8e0 con 0x7fe96407eda0 2024-05-09T04:41:01.309 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.044+0000 7fe963400700 1 --2- 172.21.15.6:0/2957947178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe96407eda0 0x7fe96407d220 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe9540098a0 tx=0x7fe954009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6780b954ca3f423 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:01.309 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.044+0000 7fe962000700 1 -- 172.21.15.6:0/2957947178 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe954004160 con 0x7fe96407eda0 2024-05-09T04:41:01.309 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.044+0000 7fe962000700 1 -- 172.21.15.6:0/2957947178 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe954004300 con 0x7fe96407eda0 2024-05-09T04:41:01.309 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.044+0000 7fe962000700 1 -- 172.21.15.6:0/2957947178 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe954011a20 con 0x7fe96407eda0 2024-05-09T04:41:01.309 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.045+0000 7fe96a200700 1 -- 172.21.15.6:0/2957947178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe96407eda0 msgr2=0x7fe96407d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:01.309 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.045+0000 7fe96a200700 1 --2- 172.21.15.6:0/2957947178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe96407eda0 0x7fe96407d220 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe9540098a0 tx=0x7fe954009bb0 comp rx=0 tx=0).stop 2024-05-09T04:41:01.309 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.045+0000 7fe96a200700 1 -- 172.21.15.6:0/2957947178 shutdown_connections 2024-05-09T04:41:01.309 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.045+0000 7fe96a200700 1 --2- 172.21.15.6:0/2957947178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe96407eda0 0x7fe96407d220 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:01.309 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.045+0000 7fe96a200700 1 -- 172.21.15.6:0/2957947178 >> 172.21.15.6:0/2957947178 conn(0x7fe964105120 msgr2=0x7fe9641075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:01.309 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.046+0000 7fe96a200700 1 -- 172.21.15.6:0/2957947178 shutdown_connections 2024-05-09T04:41:01.309 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.046+0000 7fe96a200700 1 -- 172.21.15.6:0/2957947178 wait complete. 2024-05-09T04:41:01.309 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.047+0000 7fe96a200700 1 Processor -- start 2024-05-09T04:41:01.310 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.047+0000 7fe96a200700 1 -- start start 2024-05-09T04:41:01.310 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.048+0000 7fe96a200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe96407eda0 0x7fe9641a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:01.310 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.048+0000 7fe96a200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe9641100e0 con 0x7fe96407eda0 2024-05-09T04:41:01.310 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.048+0000 7fe963400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe96407eda0 0x7fe9641a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:01.310 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.048+0000 7fe963400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe96407eda0 0x7fe9641a00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55930/0 (socket says 172.21.15.6:55930) 2024-05-09T04:41:01.310 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.048+0000 7fe963400700 1 -- 172.21.15.6:0/4203417186 learned_addr learned my addr 172.21.15.6:0/4203417186 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:41:01.310 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.048+0000 7fe963400700 1 -- 172.21.15.6:0/4203417186 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9641a0600 con 0x7fe96407eda0 2024-05-09T04:41:01.310 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.049+0000 7fe963400700 1 --2- 172.21.15.6:0/4203417186 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe96407eda0 0x7fe9641a00c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe954000c00 tx=0x7fe954011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:01.310 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.049+0000 7fe95be00700 1 -- 172.21.15.6:0/4203417186 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe954003aa0 con 0x7fe96407eda0 2024-05-09T04:41:01.310 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.049+0000 7fe95be00700 1 -- 172.21.15.6:0/4203417186 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe95401b8f0 con 0x7fe96407eda0 2024-05-09T04:41:01.310 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.049+0000 7fe96a200700 1 -- 172.21.15.6:0/4203417186 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9641a0890 con 0x7fe96407eda0 2024-05-09T04:41:01.310 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.049+0000 7fe95be00700 1 -- 172.21.15.6:0/4203417186 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe954023890 con 0x7fe96407eda0 2024-05-09T04:41:01.310 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.049+0000 7fe96a200700 1 -- 172.21.15.6:0/4203417186 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9641a34a0 con 0x7fe96407eda0 2024-05-09T04:41:01.311 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.051+0000 7fe96a200700 1 -- 172.21.15.6:0/4203417186 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe964064e20 con 0x7fe96407eda0 2024-05-09T04:41:01.311 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.051+0000 7fe95be00700 1 -- 172.21.15.6:0/4203417186 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fe95401b440 con 0x7fe96407eda0 2024-05-09T04:41:01.311 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.052+0000 7fe95be00700 1 --2- 172.21.15.6:0/4203417186 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe94803e520 0x7fe948040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:01.311 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.052+0000 7fe95be00700 1 -- 172.21.15.6:0/4203417186 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe954053f20 con 0x7fe96407eda0 2024-05-09T04:41:01.311 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.052+0000 7fe962a00700 1 --2- 172.21.15.6:0/4203417186 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe94803e520 0x7fe948040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:01.311 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.053+0000 7fe962a00700 1 --2- 172.21.15.6:0/4203417186 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe94803e520 0x7fe948040a30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe950006fd0 tx=0x7fe950006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:01.311 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.058+0000 7fe95be00700 1 -- 172.21.15.6:0/4203417186 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe95401f030 con 0x7fe96407eda0 2024-05-09T04:41:01.311 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.316+0000 7fe96a200700 1 -- 172.21.15.6:0/4203417186 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fe9641a3730 con 0x7fe94803e520 2024-05-09T04:41:01.311 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.576+0000 7fe95be00700 1 -- 172.21.15.6:0/4203417186 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fe9641a3730 con 0x7fe94803e520 2024-05-09T04:41:01.311 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.580+0000 7fe96a200700 1 -- 172.21.15.6:0/4203417186 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe94803e520 msgr2=0x7fe948040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:01.311 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.580+0000 7fe96a200700 1 --2- 172.21.15.6:0/4203417186 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe94803e520 0x7fe948040a30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe950006fd0 tx=0x7fe950006e40 comp rx=0 tx=0).stop 2024-05-09T04:41:01.311 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.580+0000 7fe96a200700 1 -- 172.21.15.6:0/4203417186 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe96407eda0 msgr2=0x7fe9641a00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:01.312 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.580+0000 7fe96a200700 1 --2- 172.21.15.6:0/4203417186 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe96407eda0 0x7fe9641a00c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe954000c00 tx=0x7fe954011aa0 comp rx=0 tx=0).stop 2024-05-09T04:41:01.312 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.580+0000 7fe96a200700 1 -- 172.21.15.6:0/4203417186 shutdown_connections 2024-05-09T04:41:01.312 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.580+0000 7fe96a200700 1 --2- 172.21.15.6:0/4203417186 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe94803e520 0x7fe948040a30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:01.312 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.580+0000 7fe96a200700 1 --2- 172.21.15.6:0/4203417186 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe96407eda0 0x7fe9641a00c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:01.312 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.580+0000 7fe96a200700 1 -- 172.21.15.6:0/4203417186 >> 172.21.15.6:0/4203417186 conn(0x7fe964105120 msgr2=0x7fe964105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:01.312 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.580+0000 7fe96a200700 1 -- 172.21.15.6:0/4203417186 shutdown_connections 2024-05-09T04:41:01.312 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:00.580+0000 7fe96a200700 1 -- 172.21.15.6:0/4203417186 wait complete. 2024-05-09T04:41:01.312 INFO:teuthology.orchestra.run.smithi006.stdout:Fetching dashboard port number... 2024-05-09T04:41:01.832 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:01 smithi006 ceph-mon[29835]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:41:01.832 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:01 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:01.832 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:01 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:02.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:02 smithi006 ceph-mon[29835]: mgrmap e13: x(active, since 6s) 2024-05-09T04:41:02.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:02 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1113427719' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-09T04:41:03.807 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 8443 2024-05-09T04:41:03.808 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.968+0000 7f9d94600700 1 Processor -- start 2024-05-09T04:41:03.808 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.968+0000 7f9d94600700 1 -- start start 2024-05-09T04:41:03.808 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.968+0000 7f9d94600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d8c108970 0x7f9d8c108d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:03.808 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.968+0000 7f9d94600700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c109370 con 0x7f9d8c108970 2024-05-09T04:41:03.808 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.969+0000 7f9d91c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d8c108970 0x7f9d8c108d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:03.809 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.969+0000 7f9d91c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d8c108970 0x7f9d8c108d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55946/0 (socket says 172.21.15.6:55946) 2024-05-09T04:41:03.809 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.969+0000 7f9d91c00700 1 -- 172.21.15.6:0/2802544226 learned_addr learned my addr 172.21.15.6:0/2802544226 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:41:03.809 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.970+0000 7f9d91c00700 1 -- 172.21.15.6:0/2802544226 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d8c109b40 con 0x7f9d8c108970 2024-05-09T04:41:03.809 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.970+0000 7f9d91c00700 1 --2- 172.21.15.6:0/2802544226 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d8c108970 0x7f9d8c108d80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9d800098a0 tx=0x7f9d80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d21faf44f3dc60f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:03.809 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.971+0000 7f9d8be00700 1 -- 172.21.15.6:0/2802544226 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d80004160 con 0x7f9d8c108970 2024-05-09T04:41:03.809 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.971+0000 7f9d8be00700 1 -- 172.21.15.6:0/2802544226 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d80004300 con 0x7f9d8c108970 2024-05-09T04:41:03.809 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.971+0000 7f9d8be00700 1 -- 172.21.15.6:0/2802544226 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d80011a20 con 0x7f9d8c108970 2024-05-09T04:41:03.809 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.971+0000 7f9d94600700 1 -- 172.21.15.6:0/2802544226 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d8c108970 msgr2=0x7f9d8c108d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:03.809 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.971+0000 7f9d94600700 1 --2- 172.21.15.6:0/2802544226 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d8c108970 0x7f9d8c108d80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9d800098a0 tx=0x7f9d80009bb0 comp rx=0 tx=0).stop 2024-05-09T04:41:03.809 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.972+0000 7f9d94600700 1 -- 172.21.15.6:0/2802544226 shutdown_connections 2024-05-09T04:41:03.809 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.972+0000 7f9d94600700 1 --2- 172.21.15.6:0/2802544226 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d8c108970 0x7f9d8c108d80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:03.809 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.972+0000 7f9d94600700 1 -- 172.21.15.6:0/2802544226 >> 172.21.15.6:0/2802544226 conn(0x7f9d8c104070 msgr2=0x7f9d8c1064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:03.809 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.972+0000 7f9d94600700 1 -- 172.21.15.6:0/2802544226 shutdown_connections 2024-05-09T04:41:03.809 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.972+0000 7f9d94600700 1 -- 172.21.15.6:0/2802544226 wait complete. 2024-05-09T04:41:03.810 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.973+0000 7f9d94600700 1 Processor -- start 2024-05-09T04:41:03.810 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.973+0000 7f9d94600700 1 -- start start 2024-05-09T04:41:03.810 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.973+0000 7f9d94600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d8c108970 0x7f9d8c1a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:03.810 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.973+0000 7f9d94600700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c10cef0 con 0x7f9d8c108970 2024-05-09T04:41:03.810 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.974+0000 7f9d91c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d8c108970 0x7f9d8c1a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:03.810 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.974+0000 7f9d91c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d8c108970 0x7f9d8c1a00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55956/0 (socket says 172.21.15.6:55956) 2024-05-09T04:41:03.813 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.974+0000 7f9d91c00700 1 -- 172.21.15.6:0/1113427719 learned_addr learned my addr 172.21.15.6:0/1113427719 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:41:03.813 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.974+0000 7f9d91c00700 1 -- 172.21.15.6:0/1113427719 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d8c1a0600 con 0x7f9d8c108970 2024-05-09T04:41:03.813 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.975+0000 7f9d91c00700 1 --2- 172.21.15.6:0/1113427719 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d8c108970 0x7f9d8c1a00c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9d80003e70 tx=0x7f9d80011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:03.813 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.975+0000 7f9d8a000700 1 -- 172.21.15.6:0/1113427719 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d80003af0 con 0x7f9d8c108970 2024-05-09T04:41:03.814 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.975+0000 7f9d8a000700 1 -- 172.21.15.6:0/1113427719 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d8001a8e0 con 0x7f9d8c108970 2024-05-09T04:41:03.814 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.975+0000 7f9d94600700 1 -- 172.21.15.6:0/1113427719 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d8c1a0890 con 0x7f9d8c108970 2024-05-09T04:41:03.814 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.975+0000 7f9d8a000700 1 -- 172.21.15.6:0/1113427719 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d80024870 con 0x7f9d8c108970 2024-05-09T04:41:03.814 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.975+0000 7f9d94600700 1 -- 172.21.15.6:0/1113427719 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d8c1a34a0 con 0x7f9d8c108970 2024-05-09T04:41:03.814 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.976+0000 7f9d8a000700 1 -- 172.21.15.6:0/1113427719 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f9d80022020 con 0x7f9d8c108970 2024-05-09T04:41:03.814 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.977+0000 7f9d94600700 1 -- 172.21.15.6:0/1113427719 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d8c064e20 con 0x7f9d8c108970 2024-05-09T04:41:03.814 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.977+0000 7f9d8a000700 1 --2- 172.21.15.6:0/1113427719 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9d7403e510 0x7f9d74040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:03.814 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.977+0000 7f9d8a000700 1 -- 172.21.15.6:0/1113427719 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9d80053c20 con 0x7f9d8c108970 2024-05-09T04:41:03.814 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.978+0000 7f9d91200700 1 --2- 172.21.15.6:0/1113427719 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9d7403e510 0x7f9d74040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:03.814 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.979+0000 7f9d91200700 1 --2- 172.21.15.6:0/1113427719 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9d7403e510 0x7f9d74040a20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9d78006fd0 tx=0x7f9d78006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:03.814 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:01.984+0000 7f9d8a000700 1 -- 172.21.15.6:0/1113427719 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9d8000e260 con 0x7f9d8c108970 2024-05-09T04:41:03.814 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:02.237+0000 7f9d94600700 1 -- 172.21.15.6:0/1113427719 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f9d8c1a3670 con 0x7f9d8c108970 2024-05-09T04:41:03.814 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:02.238+0000 7f9d8a000700 1 -- 172.21.15.6:0/1113427719 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f9d8001ad90 con 0x7f9d8c108970 2024-05-09T04:41:03.815 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:02.241+0000 7f9d94600700 1 -- 172.21.15.6:0/1113427719 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9d7403e510 msgr2=0x7f9d74040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:03.815 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:02.241+0000 7f9d94600700 1 --2- 172.21.15.6:0/1113427719 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9d7403e510 0x7f9d74040a20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9d78006fd0 tx=0x7f9d78006e40 comp rx=0 tx=0).stop 2024-05-09T04:41:03.815 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:02.241+0000 7f9d94600700 1 -- 172.21.15.6:0/1113427719 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d8c108970 msgr2=0x7f9d8c1a00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:03.815 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:02.241+0000 7f9d94600700 1 --2- 172.21.15.6:0/1113427719 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d8c108970 0x7f9d8c1a00c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9d80003e70 tx=0x7f9d80011aa0 comp rx=0 tx=0).stop 2024-05-09T04:41:03.815 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:02.241+0000 7f9d94600700 1 -- 172.21.15.6:0/1113427719 shutdown_connections 2024-05-09T04:41:03.815 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:02.241+0000 7f9d94600700 1 --2- 172.21.15.6:0/1113427719 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9d7403e510 0x7f9d74040a20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:03.815 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:02.241+0000 7f9d94600700 1 --2- 172.21.15.6:0/1113427719 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9d8c108970 0x7f9d8c1a00c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:03.815 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:02.242+0000 7f9d94600700 1 -- 172.21.15.6:0/1113427719 >> 172.21.15.6:0/1113427719 conn(0x7f9d8c104070 msgr2=0x7f9d8c105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:03.815 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:02.242+0000 7f9d94600700 1 -- 172.21.15.6:0/1113427719 shutdown_connections 2024-05-09T04:41:03.815 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:02.242+0000 7f9d94600700 1 -- 172.21.15.6:0/1113427719 wait complete. 2024-05-09T04:41:03.845 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T04:41:03.845 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout disabled 2024-05-09T04:41:03.871 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T04:41:03.871 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2024-05-09T04:41:03.871 INFO:teuthology.orchestra.run.smithi006.stdout:firewalld.service is not enabled 2024-05-09T04:41:03.871 INFO:teuthology.orchestra.run.smithi006.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-09T04:41:03.875 INFO:teuthology.orchestra.run.smithi006.stdout:Ceph Dashboard is now available at: 2024-05-09T04:41:03.875 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:41:03.875 INFO:teuthology.orchestra.run.smithi006.stdout: URL: https://smithi006.front.sepia.ceph.com:8443/ 2024-05-09T04:41:03.875 INFO:teuthology.orchestra.run.smithi006.stdout: User: admin 2024-05-09T04:41:03.876 INFO:teuthology.orchestra.run.smithi006.stdout: Password: gbe6gupu0u 2024-05-09T04:41:03.876 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:41:03.876 INFO:teuthology.orchestra.run.smithi006.stdout:Saving cluster configuration to /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/config directory 2024-05-09T04:41:03.877 INFO:teuthology.orchestra.run.smithi006.stdout:Enabling autotune for osd_memory_target 2024-05-09T04:41:05.491 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.572+0000 7f0697e00700 1 Processor -- start 2024-05-09T04:41:05.491 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.572+0000 7f0697e00700 1 -- start start 2024-05-09T04:41:05.491 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.573+0000 7f0697e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f069810bb10 0x7f069810bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:05.491 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.573+0000 7f0697e00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f069810c510 con 0x7f069810bb10 2024-05-09T04:41:05.492 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.573+0000 7f0696a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f069810bb10 0x7f069810bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:05.492 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.573+0000 7f0696a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f069810bb10 0x7f069810bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55966/0 (socket says 172.21.15.6:55966) 2024-05-09T04:41:05.492 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.573+0000 7f0696a00700 1 -- 172.21.15.6:0/1262609528 learned_addr learned my addr 172.21.15.6:0/1262609528 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:41:05.492 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.574+0000 7f0696a00700 1 -- 172.21.15.6:0/1262609528 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f069810cce0 con 0x7f069810bb10 2024-05-09T04:41:05.492 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.574+0000 7f0696a00700 1 --2- 172.21.15.6:0/1262609528 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f069810bb10 0x7f069810bf20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f06880098a0 tx=0x7f0688009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765bbec72926fdd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:05.492 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.574+0000 7f0695600700 1 -- 172.21.15.6:0/1262609528 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0688004160 con 0x7f069810bb10 2024-05-09T04:41:05.496 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.574+0000 7f0695600700 1 -- 172.21.15.6:0/1262609528 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0688004300 con 0x7f069810bb10 2024-05-09T04:41:05.496 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.574+0000 7f0695600700 1 -- 172.21.15.6:0/1262609528 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0688011a20 con 0x7f069810bb10 2024-05-09T04:41:05.496 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.575+0000 7f0697e00700 1 -- 172.21.15.6:0/1262609528 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f069810bb10 msgr2=0x7f069810bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:05.497 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.575+0000 7f0697e00700 1 --2- 172.21.15.6:0/1262609528 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f069810bb10 0x7f069810bf20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f06880098a0 tx=0x7f0688009bb0 comp rx=0 tx=0).stop 2024-05-09T04:41:05.497 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.575+0000 7f0697e00700 1 -- 172.21.15.6:0/1262609528 shutdown_connections 2024-05-09T04:41:05.497 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.575+0000 7f0697e00700 1 --2- 172.21.15.6:0/1262609528 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f069810bb10 0x7f069810bf20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:05.497 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.575+0000 7f0697e00700 1 -- 172.21.15.6:0/1262609528 >> 172.21.15.6:0/1262609528 conn(0x7f06981074c0 msgr2=0x7f06981098e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:05.497 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.576+0000 7f0697e00700 1 -- 172.21.15.6:0/1262609528 shutdown_connections 2024-05-09T04:41:05.497 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.576+0000 7f0697e00700 1 -- 172.21.15.6:0/1262609528 wait complete. 2024-05-09T04:41:05.497 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.577+0000 7f0697e00700 1 Processor -- start 2024-05-09T04:41:05.497 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.577+0000 7f0697e00700 1 -- start start 2024-05-09T04:41:05.497 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.577+0000 7f0697e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f069810bb10 0x7f069819ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:05.497 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.577+0000 7f0697e00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0698110090 con 0x7f069810bb10 2024-05-09T04:41:05.497 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.578+0000 7f0696a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f069810bb10 0x7f069819ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:05.497 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.578+0000 7f0696a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f069810bb10 0x7f069819ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55976/0 (socket says 172.21.15.6:55976) 2024-05-09T04:41:05.497 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.578+0000 7f0696a00700 1 -- 172.21.15.6:0/1454548438 learned_addr learned my addr 172.21.15.6:0/1454548438 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:41:05.498 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.578+0000 7f0696a00700 1 -- 172.21.15.6:0/1454548438 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06981a04f0 con 0x7f069810bb10 2024-05-09T04:41:05.498 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.578+0000 7f0696a00700 1 --2- 172.21.15.6:0/1454548438 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f069810bb10 0x7f069819ffb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0688009130 tx=0x7f0688011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:05.498 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.579+0000 7f068f400700 1 -- 172.21.15.6:0/1454548438 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0688003aa0 con 0x7f069810bb10 2024-05-09T04:41:05.498 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.579+0000 7f068f400700 1 -- 172.21.15.6:0/1454548438 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f068801a8e0 con 0x7f069810bb10 2024-05-09T04:41:05.498 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.579+0000 7f068f400700 1 -- 172.21.15.6:0/1454548438 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06880248f0 con 0x7f069810bb10 2024-05-09T04:41:05.498 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.579+0000 7f0697e00700 1 -- 172.21.15.6:0/1454548438 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06981a0780 con 0x7f069810bb10 2024-05-09T04:41:05.498 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.579+0000 7f0697e00700 1 -- 172.21.15.6:0/1454548438 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06981a3390 con 0x7f069810bb10 2024-05-09T04:41:05.498 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.580+0000 7f068cc00700 1 -- 172.21.15.6:0/1454548438 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0670005330 con 0x7f069810bb10 2024-05-09T04:41:05.498 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.580+0000 7f068f400700 1 -- 172.21.15.6:0/1454548438 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f0688022020 con 0x7f069810bb10 2024-05-09T04:41:05.498 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.581+0000 7f068f400700 1 --2- 172.21.15.6:0/1454548438 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f067c03e5c0 0x7f067c040ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:05.498 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.581+0000 7f068f400700 1 -- 172.21.15.6:0/1454548438 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0688053b20 con 0x7f069810bb10 2024-05-09T04:41:05.498 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.581+0000 7f0696000700 1 --2- 172.21.15.6:0/1454548438 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f067c03e5c0 0x7f067c040ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:05.498 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.582+0000 7f0696000700 1 --2- 172.21.15.6:0/1454548438 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f067c03e5c0 0x7f067c040ad0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0684006fd0 tx=0x7f0684006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:05.499 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.586+0000 7f068f400700 1 -- 172.21.15.6:0/1454548438 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f068801ad40 con 0x7f069810bb10 2024-05-09T04:41:05.499 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.825+0000 7f068cc00700 1 -- 172.21.15.6:0/1454548438 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f0670005620 con 0x7f069810bb10 2024-05-09T04:41:05.499 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.826+0000 7f068f400700 1 -- 172.21.15.6:0/1454548438 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f068801a430 con 0x7f069810bb10 2024-05-09T04:41:05.499 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.829+0000 7f068cc00700 1 -- 172.21.15.6:0/1454548438 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f067c03e5c0 msgr2=0x7f067c040ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:05.499 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.829+0000 7f068cc00700 1 --2- 172.21.15.6:0/1454548438 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f067c03e5c0 0x7f067c040ad0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0684006fd0 tx=0x7f0684006e40 comp rx=0 tx=0).stop 2024-05-09T04:41:05.499 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.829+0000 7f068cc00700 1 -- 172.21.15.6:0/1454548438 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f069810bb10 msgr2=0x7f069819ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:05.499 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.829+0000 7f068cc00700 1 --2- 172.21.15.6:0/1454548438 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f069810bb10 0x7f069819ffb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0688009130 tx=0x7f0688011aa0 comp rx=0 tx=0).stop 2024-05-09T04:41:05.499 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.830+0000 7f068cc00700 1 -- 172.21.15.6:0/1454548438 shutdown_connections 2024-05-09T04:41:05.499 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.830+0000 7f068cc00700 1 --2- 172.21.15.6:0/1454548438 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f067c03e5c0 0x7f067c040ad0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:05.499 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.830+0000 7f068cc00700 1 --2- 172.21.15.6:0/1454548438 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f069810bb10 0x7f069819ffb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:05.499 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.830+0000 7f068cc00700 1 -- 172.21.15.6:0/1454548438 >> 172.21.15.6:0/1454548438 conn(0x7f06981074c0 msgr2=0x7f0698108d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:05.500 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.830+0000 7f068cc00700 1 -- 172.21.15.6:0/1454548438 shutdown_connections 2024-05-09T04:41:05.500 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:04.830+0000 7f068cc00700 1 -- 172.21.15.6:0/1454548438 wait complete. 2024-05-09T04:41:07.201 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.187+0000 7f1aa4c00700 1 Processor -- start 2024-05-09T04:41:07.201 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.188+0000 7f1aa4c00700 1 -- start start 2024-05-09T04:41:07.201 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.188+0000 7f1aa4c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1aa0109b00 0x7f1aa0109f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:07.201 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.188+0000 7f1aa4c00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa010a500 con 0x7f1aa0109b00 2024-05-09T04:41:07.201 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.189+0000 7f1a9de00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1aa0109b00 0x7f1aa0109f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:07.201 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.189+0000 7f1a9de00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1aa0109b00 0x7f1aa0109f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55986/0 (socket says 172.21.15.6:55986) 2024-05-09T04:41:07.201 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.189+0000 7f1a9de00700 1 -- 172.21.15.6:0/3025734623 learned_addr learned my addr 172.21.15.6:0/3025734623 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:41:07.201 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.189+0000 7f1a9de00700 1 -- 172.21.15.6:0/3025734623 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa010ac80 con 0x7f1aa0109b00 2024-05-09T04:41:07.201 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.190+0000 7f1a9de00700 1 --2- 172.21.15.6:0/3025734623 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1aa0109b00 0x7f1aa0109f10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1a90009e80 tx=0x7f1a9000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7a01e43e3de10f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:07.201 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.190+0000 7f1a9ca00700 1 -- 172.21.15.6:0/3025734623 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a90017070 con 0x7f1aa0109b00 2024-05-09T04:41:07.202 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.190+0000 7f1a9ca00700 1 -- 172.21.15.6:0/3025734623 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a9000b8c0 con 0x7f1aa0109b00 2024-05-09T04:41:07.202 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.190+0000 7f1a9ca00700 1 -- 172.21.15.6:0/3025734623 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a90004490 con 0x7f1aa0109b00 2024-05-09T04:41:07.202 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.191+0000 7f1aa4c00700 1 -- 172.21.15.6:0/3025734623 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1aa0109b00 msgr2=0x7f1aa0109f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:07.202 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.191+0000 7f1aa4c00700 1 --2- 172.21.15.6:0/3025734623 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1aa0109b00 0x7f1aa0109f10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1a90009e80 tx=0x7f1a9000b330 comp rx=0 tx=0).stop 2024-05-09T04:41:07.202 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.191+0000 7f1aa4c00700 1 -- 172.21.15.6:0/3025734623 shutdown_connections 2024-05-09T04:41:07.202 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.191+0000 7f1aa4c00700 1 --2- 172.21.15.6:0/3025734623 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1aa0109b00 0x7f1aa0109f10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:07.202 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.191+0000 7f1aa4c00700 1 -- 172.21.15.6:0/3025734623 >> 172.21.15.6:0/3025734623 conn(0x7f1aa0105120 msgr2=0x7f1aa01075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:07.202 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.191+0000 7f1aa4c00700 1 -- 172.21.15.6:0/3025734623 shutdown_connections 2024-05-09T04:41:07.202 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.191+0000 7f1aa4c00700 1 -- 172.21.15.6:0/3025734623 wait complete. 2024-05-09T04:41:07.202 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.192+0000 7f1aa4c00700 1 Processor -- start 2024-05-09T04:41:07.202 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.193+0000 7f1aa4c00700 1 -- start start 2024-05-09T04:41:07.202 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.193+0000 7f1aa4c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1aa0109b00 0x7f1aa007ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:07.202 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.193+0000 7f1aa4c00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa010e030 con 0x7f1aa0109b00 2024-05-09T04:41:07.203 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.193+0000 7f1a9de00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1aa0109b00 0x7f1aa007ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:07.203 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.194+0000 7f1a9de00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1aa0109b00 0x7f1aa007ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55998/0 (socket says 172.21.15.6:55998) 2024-05-09T04:41:07.203 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.194+0000 7f1a9de00700 1 -- 172.21.15.6:0/3053220163 learned_addr learned my addr 172.21.15.6:0/3053220163 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:41:07.203 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.194+0000 7f1a9de00700 1 -- 172.21.15.6:0/3053220163 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa007efb0 con 0x7f1aa0109b00 2024-05-09T04:41:07.203 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.194+0000 7f1a9de00700 1 --2- 172.21.15.6:0/3053220163 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1aa0109b00 0x7f1aa007ea70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1a9000bc80 tx=0x7f1a9000bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:07.203 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.194+0000 7f1a96a00700 1 -- 172.21.15.6:0/3053220163 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a90020020 con 0x7f1aa0109b00 2024-05-09T04:41:07.203 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.195+0000 7f1aa4c00700 1 -- 172.21.15.6:0/3053220163 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aa007d200 con 0x7f1aa0109b00 2024-05-09T04:41:07.203 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.195+0000 7f1a96a00700 1 -- 172.21.15.6:0/3053220163 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a90003b50 con 0x7f1aa0109b00 2024-05-09T04:41:07.203 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.195+0000 7f1a96a00700 1 -- 172.21.15.6:0/3053220163 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a90017040 con 0x7f1aa0109b00 2024-05-09T04:41:07.203 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.195+0000 7f1aa4c00700 1 -- 172.21.15.6:0/3053220163 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aa007d740 con 0x7f1aa0109b00 2024-05-09T04:41:07.203 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.196+0000 7f1a96a00700 1 -- 172.21.15.6:0/3053220163 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f1a90023020 con 0x7f1aa0109b00 2024-05-09T04:41:07.203 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.196+0000 7f1aa4c00700 1 -- 172.21.15.6:0/3053220163 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aa0064e20 con 0x7f1aa0109b00 2024-05-09T04:41:07.203 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.197+0000 7f1a96a00700 1 --2- 172.21.15.6:0/3053220163 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f1a8403e5c0 0x7f1a84040ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:07.203 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.197+0000 7f1a96a00700 1 -- 172.21.15.6:0/3053220163 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1a9000f050 con 0x7f1aa0109b00 2024-05-09T04:41:07.203 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.197+0000 7f1a9d400700 1 --2- 172.21.15.6:0/3053220163 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f1a8403e5c0 0x7f1a84040ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:07.203 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.198+0000 7f1a9d400700 1 --2- 172.21.15.6:0/3053220163 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f1a8403e5c0 0x7f1a84040ad0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c006fd0 tx=0x7f1a8c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:07.204 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.202+0000 7f1a96a00700 1 -- 172.21.15.6:0/3053220163 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1a9002a020 con 0x7f1aa0109b00 2024-05-09T04:41:07.204 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.566+0000 7f1aa4c00700 1 -- 172.21.15.6:0/3053220163 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f1aa007de60 con 0x7f1aa0109b00 2024-05-09T04:41:07.204 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.573+0000 7f1a96a00700 1 -- 172.21.15.6:0/3053220163 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f1a9002a200 con 0x7f1aa0109b00 2024-05-09T04:41:07.204 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.576+0000 7f1aa4c00700 1 -- 172.21.15.6:0/3053220163 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f1a8403e5c0 msgr2=0x7f1a84040ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:07.204 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.576+0000 7f1aa4c00700 1 --2- 172.21.15.6:0/3053220163 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f1a8403e5c0 0x7f1a84040ad0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c006fd0 tx=0x7f1a8c006e40 comp rx=0 tx=0).stop 2024-05-09T04:41:07.204 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.576+0000 7f1aa4c00700 1 -- 172.21.15.6:0/3053220163 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1aa0109b00 msgr2=0x7f1aa007ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:07.204 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.576+0000 7f1aa4c00700 1 --2- 172.21.15.6:0/3053220163 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1aa0109b00 0x7f1aa007ea70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1a9000bc80 tx=0x7f1a9000bd60 comp rx=0 tx=0).stop 2024-05-09T04:41:07.204 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.576+0000 7f1aa4c00700 1 -- 172.21.15.6:0/3053220163 shutdown_connections 2024-05-09T04:41:07.204 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.576+0000 7f1aa4c00700 1 --2- 172.21.15.6:0/3053220163 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f1a8403e5c0 0x7f1a84040ad0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:07.204 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.576+0000 7f1aa4c00700 1 --2- 172.21.15.6:0/3053220163 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1aa0109b00 0x7f1aa007ea70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:07.204 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.576+0000 7f1aa4c00700 1 -- 172.21.15.6:0/3053220163 >> 172.21.15.6:0/3053220163 conn(0x7f1aa0105120 msgr2=0x7f1aa0105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:07.204 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.576+0000 7f1aa4c00700 1 -- 172.21.15.6:0/3053220163 shutdown_connections 2024-05-09T04:41:07.204 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-05-09T04:41:06.576+0000 7f1aa4c00700 1 -- 172.21.15.6:0/3053220163 wait complete. 2024-05-09T04:41:07.204 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-09T04:41:07.204 INFO:teuthology.orchestra.run.smithi006.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-09T04:41:07.205 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:41:07.205 INFO:teuthology.orchestra.run.smithi006.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-09T04:41:07.205 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:41:07.205 INFO:teuthology.orchestra.run.smithi006.stdout:Or, if you are only running a single cluster on this host: 2024-05-09T04:41:07.205 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:41:07.205 INFO:teuthology.orchestra.run.smithi006.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-09T04:41:07.205 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:41:07.205 INFO:teuthology.orchestra.run.smithi006.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-09T04:41:07.205 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:41:07.205 INFO:teuthology.orchestra.run.smithi006.stdout: ceph telemetry on 2024-05-09T04:41:07.205 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:41:07.205 INFO:teuthology.orchestra.run.smithi006.stdout:For more information see: 2024-05-09T04:41:07.205 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:41:07.205 INFO:teuthology.orchestra.run.smithi006.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-09T04:41:07.205 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:41:07.205 INFO:teuthology.orchestra.run.smithi006.stdout:Bootstrap complete. 2024-05-09T04:41:07.299 INFO:tasks.cephadm:Fetching config... 2024-05-09T04:41:07.300 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T04:41:07.300 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-09T04:41:07.353 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-09T04:41:07.353 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T04:41:07.353 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-09T04:41:07.436 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-09T04:41:07.436 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T04:41:07.436 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-09T04:41:07.567 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-09T04:41:07.567 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T04:41:07.567 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-09T04:41:07.618 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-09T04:41:07.619 DEBUG:teuthology.orchestra.run.smithi006:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDB+JkWLcjz04FbFR+6XOfUfNnZH+rJAq5tjOeYbHh3TcznCg/0nM6iyOUco1vjTVoqdzlKXJY7qsE4+22GnosJ88Fis5m/VxM1TWSZ09rnmW0q4F7wQc1pWjYWf03VbSdLHFIjPDorkdw4Lc6T8zfrQNA18mBWqkfNuAB+5Zo1618k06uAYiFlomc0eDs0/0BFdAx5nXV+2L5VKxBS4nlJjz8HtU9z0ghAQccIpVdJ+DdSZBwxun8r/WsXEJ6MutWgMLuOvso6R7QJ0SN3uKSAQunpnW55vXMS8ZAC7BLzhSOZc+zAc54RQi361VZErU1O8oARjX3jv0a3f0LkeDDJklm/N8aAUvadAHth10A6YUVGqA0a4XeSFRcSx2vllSN3UWaeCoL69+EOiJbWryie3YR4/cbqZODvHPA2xkxlWUVIwdFEZOxxQhr9p0GqKh1y4QjLVWnduPhiM0S4zKE34rCCe4q+QT/uJAko3UWGq4OHzRg70TVrGJGgdjAXK8s= ceph-f74d813c-0dbd-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-09T04:41:07.735 INFO:teuthology.orchestra.run.smithi006.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDB+JkWLcjz04FbFR+6XOfUfNnZH+rJAq5tjOeYbHh3TcznCg/0nM6iyOUco1vjTVoqdzlKXJY7qsE4+22GnosJ88Fis5m/VxM1TWSZ09rnmW0q4F7wQc1pWjYWf03VbSdLHFIjPDorkdw4Lc6T8zfrQNA18mBWqkfNuAB+5Zo1618k06uAYiFlomc0eDs0/0BFdAx5nXV+2L5VKxBS4nlJjz8HtU9z0ghAQccIpVdJ+DdSZBwxun8r/WsXEJ6MutWgMLuOvso6R7QJ0SN3uKSAQunpnW55vXMS8ZAC7BLzhSOZc+zAc54RQi361VZErU1O8oARjX3jv0a3f0LkeDDJklm/N8aAUvadAHth10A6YUVGqA0a4XeSFRcSx2vllSN3UWaeCoL69+EOiJbWryie3YR4/cbqZODvHPA2xkxlWUVIwdFEZOxxQhr9p0GqKh1y4QjLVWnduPhiM0S4zKE34rCCe4q+QT/uJAko3UWGq4OHzRg70TVrGJGgdjAXK8s= ceph-f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:41:07.769 DEBUG:teuthology.orchestra.run.smithi169:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDB+JkWLcjz04FbFR+6XOfUfNnZH+rJAq5tjOeYbHh3TcznCg/0nM6iyOUco1vjTVoqdzlKXJY7qsE4+22GnosJ88Fis5m/VxM1TWSZ09rnmW0q4F7wQc1pWjYWf03VbSdLHFIjPDorkdw4Lc6T8zfrQNA18mBWqkfNuAB+5Zo1618k06uAYiFlomc0eDs0/0BFdAx5nXV+2L5VKxBS4nlJjz8HtU9z0ghAQccIpVdJ+DdSZBwxun8r/WsXEJ6MutWgMLuOvso6R7QJ0SN3uKSAQunpnW55vXMS8ZAC7BLzhSOZc+zAc54RQi361VZErU1O8oARjX3jv0a3f0LkeDDJklm/N8aAUvadAHth10A6YUVGqA0a4XeSFRcSx2vllSN3UWaeCoL69+EOiJbWryie3YR4/cbqZODvHPA2xkxlWUVIwdFEZOxxQhr9p0GqKh1y4QjLVWnduPhiM0S4zKE34rCCe4q+QT/uJAko3UWGq4OHzRg70TVrGJGgdjAXK8s= ceph-f74d813c-0dbd-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-09T04:41:07.810 INFO:teuthology.orchestra.run.smithi169.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDB+JkWLcjz04FbFR+6XOfUfNnZH+rJAq5tjOeYbHh3TcznCg/0nM6iyOUco1vjTVoqdzlKXJY7qsE4+22GnosJ88Fis5m/VxM1TWSZ09rnmW0q4F7wQc1pWjYWf03VbSdLHFIjPDorkdw4Lc6T8zfrQNA18mBWqkfNuAB+5Zo1618k06uAYiFlomc0eDs0/0BFdAx5nXV+2L5VKxBS4nlJjz8HtU9z0ghAQccIpVdJ+DdSZBwxun8r/WsXEJ6MutWgMLuOvso6R7QJ0SN3uKSAQunpnW55vXMS8ZAC7BLzhSOZc+zAc54RQi361VZErU1O8oARjX3jv0a3f0LkeDDJklm/N8aAUvadAHth10A6YUVGqA0a4XeSFRcSx2vllSN3UWaeCoL69+EOiJbWryie3YR4/cbqZODvHPA2xkxlWUVIwdFEZOxxQhr9p0GqKh1y4QjLVWnduPhiM0S4zKE34rCCe4q+QT/uJAko3UWGq4OHzRg70TVrGJGgdjAXK8s= ceph-f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:41:07.822 DEBUG:teuthology.orchestra.run.smithi186:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDB+JkWLcjz04FbFR+6XOfUfNnZH+rJAq5tjOeYbHh3TcznCg/0nM6iyOUco1vjTVoqdzlKXJY7qsE4+22GnosJ88Fis5m/VxM1TWSZ09rnmW0q4F7wQc1pWjYWf03VbSdLHFIjPDorkdw4Lc6T8zfrQNA18mBWqkfNuAB+5Zo1618k06uAYiFlomc0eDs0/0BFdAx5nXV+2L5VKxBS4nlJjz8HtU9z0ghAQccIpVdJ+DdSZBwxun8r/WsXEJ6MutWgMLuOvso6R7QJ0SN3uKSAQunpnW55vXMS8ZAC7BLzhSOZc+zAc54RQi361VZErU1O8oARjX3jv0a3f0LkeDDJklm/N8aAUvadAHth10A6YUVGqA0a4XeSFRcSx2vllSN3UWaeCoL69+EOiJbWryie3YR4/cbqZODvHPA2xkxlWUVIwdFEZOxxQhr9p0GqKh1y4QjLVWnduPhiM0S4zKE34rCCe4q+QT/uJAko3UWGq4OHzRg70TVrGJGgdjAXK8s= ceph-f74d813c-0dbd-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-09T04:41:07.864 INFO:teuthology.orchestra.run.smithi186.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDB+JkWLcjz04FbFR+6XOfUfNnZH+rJAq5tjOeYbHh3TcznCg/0nM6iyOUco1vjTVoqdzlKXJY7qsE4+22GnosJ88Fis5m/VxM1TWSZ09rnmW0q4F7wQc1pWjYWf03VbSdLHFIjPDorkdw4Lc6T8zfrQNA18mBWqkfNuAB+5Zo1618k06uAYiFlomc0eDs0/0BFdAx5nXV+2L5VKxBS4nlJjz8HtU9z0ghAQccIpVdJ+DdSZBwxun8r/WsXEJ6MutWgMLuOvso6R7QJ0SN3uKSAQunpnW55vXMS8ZAC7BLzhSOZc+zAc54RQi361VZErU1O8oARjX3jv0a3f0LkeDDJklm/N8aAUvadAHth10A6YUVGqA0a4XeSFRcSx2vllSN3UWaeCoL69+EOiJbWryie3YR4/cbqZODvHPA2xkxlWUVIwdFEZOxxQhr9p0GqKh1y4QjLVWnduPhiM0S4zKE34rCCe4q+QT/uJAko3UWGq4OHzRg70TVrGJGgdjAXK8s= ceph-f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:41:07.872 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:07 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/3053220163' entity='client.admin' 2024-05-09T04:41:07.876 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-09T04:41:08.319 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:41:11.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.012+0000 7f2f68e00700 1 -- 172.21.15.6:0/3735002286 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f64105de0 msgr2=0x7f2f641061f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:11.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.012+0000 7f2f68e00700 1 --2- 172.21.15.6:0/3735002286 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f64105de0 0x7f2f641061f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2f54009910 tx=0x7f2f54009c20 comp rx=0 tx=0).stop 2024-05-09T04:41:11.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.013+0000 7f2f68e00700 1 -- 172.21.15.6:0/3735002286 shutdown_connections 2024-05-09T04:41:11.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.013+0000 7f2f68e00700 1 --2- 172.21.15.6:0/3735002286 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f64105de0 0x7f2f641061f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:11.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.013+0000 7f2f68e00700 1 -- 172.21.15.6:0/3735002286 >> 172.21.15.6:0/3735002286 conn(0x7f2f64101420 msgr2=0x7f2f641038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:11.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.013+0000 7f2f68e00700 1 -- 172.21.15.6:0/3735002286 shutdown_connections 2024-05-09T04:41:11.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.013+0000 7f2f68e00700 1 -- 172.21.15.6:0/3735002286 wait complete. 2024-05-09T04:41:11.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.014+0000 7f2f68e00700 1 Processor -- start 2024-05-09T04:41:11.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.014+0000 7f2f68e00700 1 -- start start 2024-05-09T04:41:11.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.015+0000 7f2f68e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f64105de0 0x7f2f6419b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:11.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.015+0000 7f2f68e00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6410bda0 con 0x7f2f64105de0 2024-05-09T04:41:11.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.015+0000 7f2f61e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f64105de0 0x7f2f6419b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:11.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.015+0000 7f2f61e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f64105de0 0x7f2f6419b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56510/0 (socket says 172.21.15.6:56510) 2024-05-09T04:41:11.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.015+0000 7f2f61e00700 1 -- 172.21.15.6:0/812040467 learned_addr learned my addr 172.21.15.6:0/812040467 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:41:11.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.016+0000 7f2f61e00700 1 -- 172.21.15.6:0/812040467 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6419be30 con 0x7f2f64105de0 2024-05-09T04:41:11.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.016+0000 7f2f61e00700 1 --2- 172.21.15.6:0/812040467 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f64105de0 0x7f2f6419b8f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2f540057d0 tx=0x7f2f54005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:11.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.016+0000 7f2f5aa00700 1 -- 172.21.15.6:0/812040467 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f54027070 con 0x7f2f64105de0 2024-05-09T04:41:11.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.017+0000 7f2f5aa00700 1 -- 172.21.15.6:0/812040467 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f5400be10 con 0x7f2f64105de0 2024-05-09T04:41:11.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.017+0000 7f2f5aa00700 1 -- 172.21.15.6:0/812040467 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f54022dd0 con 0x7f2f64105de0 2024-05-09T04:41:11.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.017+0000 7f2f68e00700 1 -- 172.21.15.6:0/812040467 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f6419c0c0 con 0x7f2f64105de0 2024-05-09T04:41:11.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.017+0000 7f2f68e00700 1 -- 172.21.15.6:0/812040467 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f6419ecd0 con 0x7f2f64105de0 2024-05-09T04:41:11.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.018+0000 7f2f68e00700 1 -- 172.21.15.6:0/812040467 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f6410c520 con 0x7f2f64105de0 2024-05-09T04:41:11.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.018+0000 7f2f5aa00700 1 -- 172.21.15.6:0/812040467 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f2f5402a070 con 0x7f2f64105de0 2024-05-09T04:41:11.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.019+0000 7f2f5aa00700 1 --2- 172.21.15.6:0/812040467 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f2f4803e510 0x7f2f48040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:11.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.019+0000 7f2f5aa00700 1 -- 172.21.15.6:0/812040467 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2f5401e030 con 0x7f2f64105de0 2024-05-09T04:41:11.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.019+0000 7f2f61400700 1 --2- 172.21.15.6:0/812040467 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f2f4803e510 0x7f2f48040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:11.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.020+0000 7f2f61400700 1 --2- 172.21.15.6:0/812040467 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f2f4803e510 0x7f2f48040a20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2f50006fd0 tx=0x7f2f50006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:11.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.025+0000 7f2f5aa00700 1 -- 172.21.15.6:0/812040467 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2f54013070 con 0x7f2f64105de0 2024-05-09T04:41:11.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.260+0000 7f2f68e00700 1 -- 172.21.15.6:0/812040467 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f2f6402e700 con 0x7f2f64105de0 2024-05-09T04:41:11.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.267+0000 7f2f5aa00700 1 -- 172.21.15.6:0/812040467 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f2f540292c0 con 0x7f2f64105de0 2024-05-09T04:41:11.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.270+0000 7f2f68e00700 1 -- 172.21.15.6:0/812040467 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f2f4803e510 msgr2=0x7f2f48040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:11.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.270+0000 7f2f68e00700 1 --2- 172.21.15.6:0/812040467 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f2f4803e510 0x7f2f48040a20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2f50006fd0 tx=0x7f2f50006e40 comp rx=0 tx=0).stop 2024-05-09T04:41:11.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.271+0000 7f2f68e00700 1 -- 172.21.15.6:0/812040467 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f64105de0 msgr2=0x7f2f6419b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:11.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.271+0000 7f2f68e00700 1 --2- 172.21.15.6:0/812040467 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f64105de0 0x7f2f6419b8f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2f540057d0 tx=0x7f2f54005800 comp rx=0 tx=0).stop 2024-05-09T04:41:11.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.271+0000 7f2f68e00700 1 -- 172.21.15.6:0/812040467 shutdown_connections 2024-05-09T04:41:11.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.271+0000 7f2f68e00700 1 --2- 172.21.15.6:0/812040467 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f2f4803e510 0x7f2f48040a20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:11.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.271+0000 7f2f68e00700 1 --2- 172.21.15.6:0/812040467 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f64105de0 0x7f2f6419b8f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:11.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.271+0000 7f2f68e00700 1 -- 172.21.15.6:0/812040467 >> 172.21.15.6:0/812040467 conn(0x7f2f64101420 msgr2=0x7f2f641020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:11.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.271+0000 7f2f68e00700 1 -- 172.21.15.6:0/812040467 shutdown_connections 2024-05-09T04:41:11.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:11.271+0000 7f2f68e00700 1 -- 172.21.15.6:0/812040467 wait complete. 2024-05-09T04:41:11.649 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:11 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:11.649 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:11 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:11.649 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:11 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi006", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:41:11.649 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:11 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:11.649 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:11 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:41:11.649 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:11 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:11.649 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:11 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/812040467' entity='client.admin' 2024-05-09T04:41:11.649 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:11 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:41:11.650 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:11 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:11.978 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-09T04:41:11.979 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-09T04:41:12.425 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:41:14.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.505+0000 7fe572a00700 1 -- 172.21.15.6:0/594753872 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe56c1050f0 msgr2=0x7fe56c105500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:14.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.505+0000 7fe572a00700 1 --2- 172.21.15.6:0/594753872 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe56c1050f0 0x7fe56c105500 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe56000b0a0 tx=0x7fe56000b3b0 comp rx=0 tx=0).stop 2024-05-09T04:41:14.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.506+0000 7fe572a00700 1 -- 172.21.15.6:0/594753872 shutdown_connections 2024-05-09T04:41:14.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.506+0000 7fe572a00700 1 --2- 172.21.15.6:0/594753872 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe56c1050f0 0x7fe56c105500 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:14.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.506+0000 7fe572a00700 1 -- 172.21.15.6:0/594753872 >> 172.21.15.6:0/594753872 conn(0x7fe56c0ff050 msgr2=0x7fe56c101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:14.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.506+0000 7fe572a00700 1 -- 172.21.15.6:0/594753872 shutdown_connections 2024-05-09T04:41:14.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.506+0000 7fe572a00700 1 -- 172.21.15.6:0/594753872 wait complete. 2024-05-09T04:41:14.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.507+0000 7fe572a00700 1 Processor -- start 2024-05-09T04:41:14.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.507+0000 7fe572a00700 1 -- start start 2024-05-09T04:41:14.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.507+0000 7fe572a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe56c1050f0 0x7fe56c077380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:14.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.507+0000 7fe572a00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c109ad0 con 0x7fe56c1050f0 2024-05-09T04:41:14.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.508+0000 7fe571600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe56c1050f0 0x7fe56c077380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:14.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.508+0000 7fe571600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe56c1050f0 0x7fe56c077380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56536/0 (socket says 172.21.15.6:56536) 2024-05-09T04:41:14.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.508+0000 7fe571600700 1 -- 172.21.15.6:0/2746321609 learned_addr learned my addr 172.21.15.6:0/2746321609 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:41:14.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.508+0000 7fe571600700 1 -- 172.21.15.6:0/2746321609 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe56c0778c0 con 0x7fe56c1050f0 2024-05-09T04:41:14.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.508+0000 7fe571600700 1 --2- 172.21.15.6:0/2746321609 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe56c1050f0 0x7fe56c077380 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe560007670 tx=0x7fe560003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:14.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.509+0000 7fe56a000700 1 -- 172.21.15.6:0/2746321609 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe560021020 con 0x7fe56c1050f0 2024-05-09T04:41:14.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.509+0000 7fe56a000700 1 -- 172.21.15.6:0/2746321609 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5600044b0 con 0x7fe56c1050f0 2024-05-09T04:41:14.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.509+0000 7fe56a000700 1 -- 172.21.15.6:0/2746321609 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe560017040 con 0x7fe56c1050f0 2024-05-09T04:41:14.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.509+0000 7fe572a00700 1 -- 172.21.15.6:0/2746321609 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe56c075b10 con 0x7fe56c1050f0 2024-05-09T04:41:14.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.509+0000 7fe572a00700 1 -- 172.21.15.6:0/2746321609 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe56c075fd0 con 0x7fe56c1050f0 2024-05-09T04:41:14.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.510+0000 7fe572a00700 1 -- 172.21.15.6:0/2746321609 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe56c10a250 con 0x7fe56c1050f0 2024-05-09T04:41:14.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.511+0000 7fe56a000700 1 -- 172.21.15.6:0/2746321609 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7fe560024020 con 0x7fe56c1050f0 2024-05-09T04:41:14.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.511+0000 7fe56a000700 1 --2- 172.21.15.6:0/2746321609 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe55403e5b0 0x7fe554040ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:14.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.511+0000 7fe56a000700 1 -- 172.21.15.6:0/2746321609 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe56000f050 con 0x7fe56c1050f0 2024-05-09T04:41:14.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.511+0000 7fe570c00700 1 --2- 172.21.15.6:0/2746321609 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe55403e5b0 0x7fe554040ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:14.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.512+0000 7fe570c00700 1 --2- 172.21.15.6:0/2746321609 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe55403e5b0 0x7fe554040ac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe558006fd0 tx=0x7fe558006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:14.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.517+0000 7fe56a000700 1 -- 172.21.15.6:0/2746321609 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe56002b020 con 0x7fe56c1050f0 2024-05-09T04:41:14.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.756+0000 7fe572a00700 1 -- 172.21.15.6:0/2746321609 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fe56c0761a0 con 0x7fe55403e5b0 2024-05-09T04:41:14.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.765+0000 7fe56a000700 1 -- 172.21.15.6:0/2746321609 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe56c0761a0 con 0x7fe55403e5b0 2024-05-09T04:41:14.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.769+0000 7fe572a00700 1 -- 172.21.15.6:0/2746321609 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe55403e5b0 msgr2=0x7fe554040ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:14.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.769+0000 7fe572a00700 1 --2- 172.21.15.6:0/2746321609 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe55403e5b0 0x7fe554040ac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe558006fd0 tx=0x7fe558006e40 comp rx=0 tx=0).stop 2024-05-09T04:41:14.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.769+0000 7fe572a00700 1 -- 172.21.15.6:0/2746321609 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe56c1050f0 msgr2=0x7fe56c077380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:14.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.769+0000 7fe572a00700 1 --2- 172.21.15.6:0/2746321609 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe56c1050f0 0x7fe56c077380 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe560007670 tx=0x7fe560003ce0 comp rx=0 tx=0).stop 2024-05-09T04:41:14.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.769+0000 7fe572a00700 1 -- 172.21.15.6:0/2746321609 shutdown_connections 2024-05-09T04:41:14.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.769+0000 7fe572a00700 1 --2- 172.21.15.6:0/2746321609 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe55403e5b0 0x7fe554040ac0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:14.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.769+0000 7fe572a00700 1 --2- 172.21.15.6:0/2746321609 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe56c1050f0 0x7fe56c077380 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:14.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.769+0000 7fe572a00700 1 -- 172.21.15.6:0/2746321609 >> 172.21.15.6:0/2746321609 conn(0x7fe56c0ff050 msgr2=0x7fe56c0ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:14.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.769+0000 7fe572a00700 1 -- 172.21.15.6:0/2746321609 shutdown_connections 2024-05-09T04:41:14.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:14.769+0000 7fe572a00700 1 -- 172.21.15.6:0/2746321609 wait complete. 2024-05-09T04:41:15.503 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi169 2024-05-09T04:41:15.504 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-09T04:41:15.504 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/etc/ceph/ceph.conf 2024-05-09T04:41:15.522 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-09T04:41:15.522 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T04:41:15.580 INFO:tasks.cephadm:Adding host smithi169 to orchestrator... 2024-05-09T04:41:15.580 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph orch host add smithi169 2024-05-09T04:41:16.005 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:41:16.155 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:15 smithi006 ceph-mon[29835]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:41:16.155 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:15 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:16.155 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:15 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:41:16.155 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:15 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:16.155 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:15 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:41:16.155 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:15 smithi006 ceph-mon[29835]: Updating smithi006:/etc/ceph/ceph.conf 2024-05-09T04:41:17.179 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:16 smithi006 ceph-mon[29835]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:17.179 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:16 smithi006 ceph-mon[29835]: Updating smithi006:/var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:41:17.924 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:17 smithi006 ceph-mon[29835]: Updating smithi006:/etc/ceph/ceph.client.admin.keyring 2024-05-09T04:41:17.924 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:17 smithi006 ceph-mon[29835]: Updating smithi006:/var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T04:41:17.924 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:17 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:17.924 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:17 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:17.924 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:17 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:18.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:18.175+0000 7ffacfc00700 1 -- 172.21.15.6:0/494475 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffac8103dd0 msgr2=0x7ffac81041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:18.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:18.175+0000 7ffacfc00700 1 --2- 172.21.15.6:0/494475 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffac8103dd0 0x7ffac81041e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffab800b0a0 tx=0x7ffab800b3b0 comp rx=0 tx=0).stop 2024-05-09T04:41:18.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:18.175+0000 7ffacfc00700 1 -- 172.21.15.6:0/494475 shutdown_connections 2024-05-09T04:41:18.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:18.175+0000 7ffacfc00700 1 --2- 172.21.15.6:0/494475 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffac8103dd0 0x7ffac81041e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:18.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:18.175+0000 7ffacfc00700 1 -- 172.21.15.6:0/494475 >> 172.21.15.6:0/494475 conn(0x7ffac80ff660 msgr2=0x7ffac8101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:18.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:18.176+0000 7ffacfc00700 1 -- 172.21.15.6:0/494475 shutdown_connections 2024-05-09T04:41:18.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:18.176+0000 7ffacfc00700 1 -- 172.21.15.6:0/494475 wait complete. 2024-05-09T04:41:18.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:18.177+0000 7ffacfc00700 1 Processor -- start 2024-05-09T04:41:18.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:18.177+0000 7ffacfc00700 1 -- start start 2024-05-09T04:41:18.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:18.177+0000 7ffacfc00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffac8103dd0 0x7ffac8075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:18.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:18.178+0000 7ffacfc00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ffac810e0c0 con 0x7ffac8103dd0 2024-05-09T04:41:18.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:18.178+0000 7ffacd200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffac8103dd0 0x7ffac8075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:18.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:18.178+0000 7ffacd200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffac8103dd0 0x7ffac8075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60376/0 (socket says 172.21.15.6:60376) 2024-05-09T04:41:18.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:18.178+0000 7ffacd200700 1 -- 172.21.15.6:0/2927488104 learned_addr learned my addr 172.21.15.6:0/2927488104 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:41:18.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:18.178+0000 7ffacd200700 1 -- 172.21.15.6:0/2927488104 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac8077910 con 0x7ffac8103dd0 2024-05-09T04:41:18.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:18.179+0000 7ffacd200700 1 --2- 172.21.15.6:0/2927488104 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffac8103dd0 0x7ffac8075ac0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ffab800d010 tx=0x7ffab8003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:18.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:18.179+0000 7ffac6000700 1 -- 172.21.15.6:0/2927488104 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffab8021020 con 0x7ffac8103dd0 2024-05-09T04:41:18.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:18.179+0000 7ffac6000700 1 -- 172.21.15.6:0/2927488104 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffab80044b0 con 0x7ffac8103dd0 2024-05-09T04:41:18.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:18.179+0000 7ffacfc00700 1 -- 172.21.15.6:0/2927488104 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffac8076000 con 0x7ffac8103dd0 2024-05-09T04:41:18.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:18.179+0000 7ffac6000700 1 -- 172.21.15.6:0/2927488104 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffab8017040 con 0x7ffac8103dd0 2024-05-09T04:41:18.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:18.179+0000 7ffacfc00700 1 -- 172.21.15.6:0/2927488104 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffac8076460 con 0x7ffac8103dd0 2024-05-09T04:41:18.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:18.180+0000 7ffacfc00700 1 -- 172.21.15.6:0/2927488104 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffac8105280 con 0x7ffac8103dd0 2024-05-09T04:41:18.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:18.181+0000 7ffac6000700 1 -- 172.21.15.6:0/2927488104 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7ffab8024020 con 0x7ffac8103dd0 2024-05-09T04:41:18.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:18.181+0000 7ffac6000700 1 --2- 172.21.15.6:0/2927488104 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ffaac03e560 0x7ffaac040a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:18.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:18.181+0000 7ffac6000700 1 -- 172.21.15.6:0/2927488104 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ffab800f050 con 0x7ffac8103dd0 2024-05-09T04:41:18.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:18.181+0000 7ffabfe00700 1 --2- 172.21.15.6:0/2927488104 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ffaac03e560 0x7ffaac040a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:18.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:18.182+0000 7ffabfe00700 1 --2- 172.21.15.6:0/2927488104 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ffaac03e560 0x7ffaac040a70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ffab0006fd0 tx=0x7ffab0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:18.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:18.187+0000 7ffac6000700 1 -- 172.21.15.6:0/2927488104 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffab802b020 con 0x7ffac8103dd0 2024-05-09T04:41:18.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:18.429+0000 7ffacfc00700 1 -- 172.21.15.6:0/2927488104 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi169", "target": ["mon-mgr", ""]}) -- 0x7ffac8076ff0 con 0x7ffaac03e560 2024-05-09T04:41:19.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:18 smithi006 ceph-mon[29835]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:19.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:19 smithi006 ceph-mon[29835]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi169", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:41:21.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:20 smithi006 ceph-mon[29835]: Deploying cephadm binary to smithi169 2024-05-09T04:41:21.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:20 smithi006 ceph-mon[29835]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:23.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:22 smithi006 ceph-mon[29835]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:23.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:23.325+0000 7ffac6000700 1 -- 172.21.15.6:0/2927488104 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7ffac8076ff0 con 0x7ffaac03e560 2024-05-09T04:41:23.327 INFO:teuthology.orchestra.run.smithi006.stdout:Added host 'smithi169' with addr '172.21.15.169' 2024-05-09T04:41:23.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:23.330+0000 7ffacfc00700 1 -- 172.21.15.6:0/2927488104 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ffaac03e560 msgr2=0x7ffaac040a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:23.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:23.330+0000 7ffacfc00700 1 --2- 172.21.15.6:0/2927488104 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ffaac03e560 0x7ffaac040a70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ffab0006fd0 tx=0x7ffab0006e40 comp rx=0 tx=0).stop 2024-05-09T04:41:23.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:23.330+0000 7ffacfc00700 1 -- 172.21.15.6:0/2927488104 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffac8103dd0 msgr2=0x7ffac8075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:23.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:23.330+0000 7ffacfc00700 1 --2- 172.21.15.6:0/2927488104 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffac8103dd0 0x7ffac8075ac0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ffab800d010 tx=0x7ffab8003ce0 comp rx=0 tx=0).stop 2024-05-09T04:41:23.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:23.330+0000 7ffacfc00700 1 -- 172.21.15.6:0/2927488104 shutdown_connections 2024-05-09T04:41:23.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:23.330+0000 7ffacfc00700 1 --2- 172.21.15.6:0/2927488104 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ffaac03e560 0x7ffaac040a70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:23.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:23.330+0000 7ffacfc00700 1 --2- 172.21.15.6:0/2927488104 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffac8103dd0 0x7ffac8075ac0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:23.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:23.330+0000 7ffacfc00700 1 -- 172.21.15.6:0/2927488104 >> 172.21.15.6:0/2927488104 conn(0x7ffac80ff660 msgr2=0x7ffac8109900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:23.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:23.330+0000 7ffacfc00700 1 -- 172.21.15.6:0/2927488104 shutdown_connections 2024-05-09T04:41:23.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:23.331+0000 7ffacfc00700 1 -- 172.21.15.6:0/2927488104 wait complete. 2024-05-09T04:41:24.055 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-09T04:41:24.515 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:41:24.662 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:24 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:24.662 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:24 smithi006 ceph-mon[29835]: Added host smithi169 2024-05-09T04:41:24.662 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:24 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:41:24.662 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:24 smithi006 ceph-mon[29835]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:24.662 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:24 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:25.691 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:25 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:26.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.622+0000 7f654be00700 1 -- 172.21.15.6:0/2985472453 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6544103df0 msgr2=0x7f6544104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:26.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.622+0000 7f654be00700 1 --2- 172.21.15.6:0/2985472453 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6544103df0 0x7f6544104200 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6538009910 tx=0x7f6538009c20 comp rx=0 tx=0).stop 2024-05-09T04:41:26.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.622+0000 7f654be00700 1 -- 172.21.15.6:0/2985472453 shutdown_connections 2024-05-09T04:41:26.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.622+0000 7f654be00700 1 --2- 172.21.15.6:0/2985472453 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6544103df0 0x7f6544104200 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:26.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.622+0000 7f654be00700 1 -- 172.21.15.6:0/2985472453 >> 172.21.15.6:0/2985472453 conn(0x7f65440ff6a0 msgr2=0x7f6544101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:26.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.622+0000 7f654be00700 1 -- 172.21.15.6:0/2985472453 shutdown_connections 2024-05-09T04:41:26.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.622+0000 7f654be00700 1 -- 172.21.15.6:0/2985472453 wait complete. 2024-05-09T04:41:26.623 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:26 smithi006 ceph-mon[29835]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:26.623 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:26 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:26.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.623+0000 7f654be00700 1 Processor -- start 2024-05-09T04:41:26.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.624+0000 7f654be00700 1 -- start start 2024-05-09T04:41:26.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.624+0000 7f654be00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6544103df0 0x7f65441a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:26.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.624+0000 7f654be00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f654410e0e0 con 0x7f6544103df0 2024-05-09T04:41:26.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.624+0000 7f6549400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6544103df0 0x7f65441a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:26.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.624+0000 7f6549400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6544103df0 0x7f65441a1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60388/0 (socket says 172.21.15.6:60388) 2024-05-09T04:41:26.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.624+0000 7f6549400700 1 -- 172.21.15.6:0/948763977 learned_addr learned my addr 172.21.15.6:0/948763977 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:41:26.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.625+0000 7f6549400700 1 -- 172.21.15.6:0/948763977 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65441a1cc0 con 0x7f6544103df0 2024-05-09T04:41:26.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.625+0000 7f6549400700 1 --2- 172.21.15.6:0/948763977 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6544103df0 0x7f65441a1780 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6538006010 tx=0x7f6538005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:26.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.625+0000 7f6542000700 1 -- 172.21.15.6:0/948763977 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6538026070 con 0x7f6544103df0 2024-05-09T04:41:26.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.625+0000 7f6542000700 1 -- 172.21.15.6:0/948763977 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f653800bba0 con 0x7f6544103df0 2024-05-09T04:41:26.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.625+0000 7f6542000700 1 -- 172.21.15.6:0/948763977 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6538003910 con 0x7f6544103df0 2024-05-09T04:41:26.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.626+0000 7f654be00700 1 -- 172.21.15.6:0/948763977 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65441a1fb0 con 0x7f6544103df0 2024-05-09T04:41:26.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.626+0000 7f654be00700 1 -- 172.21.15.6:0/948763977 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f654419b9a0 con 0x7f6544103df0 2024-05-09T04:41:26.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.627+0000 7f654be00700 1 -- 172.21.15.6:0/948763977 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65441052a0 con 0x7f6544103df0 2024-05-09T04:41:26.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.627+0000 7f6542000700 1 -- 172.21.15.6:0/948763977 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f6538029070 con 0x7f6544103df0 2024-05-09T04:41:26.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.628+0000 7f6542000700 1 --2- 172.21.15.6:0/948763977 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f652c03e510 0x7f652c040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:26.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.628+0000 7f6542000700 1 -- 172.21.15.6:0/948763977 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f653801e070 con 0x7f6544103df0 2024-05-09T04:41:26.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.628+0000 7f6548a00700 1 --2- 172.21.15.6:0/948763977 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f652c03e510 0x7f652c040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:26.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.629+0000 7f6548a00700 1 --2- 172.21.15.6:0/948763977 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f652c03e510 0x7f652c040a20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6530006fd0 tx=0x7f6530006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:26.634 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.634+0000 7f6542000700 1 -- 172.21.15.6:0/948763977 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6538013070 con 0x7f6544103df0 2024-05-09T04:41:26.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.879+0000 7f654be00700 1 -- 172.21.15.6:0/948763977 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f654419ced0 con 0x7f652c03e510 2024-05-09T04:41:26.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.883+0000 7f6542000700 1 -- 172.21.15.6:0/948763977 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f654419ced0 con 0x7f652c03e510 2024-05-09T04:41:26.883 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:41:26.883 INFO:teuthology.orchestra.run.smithi006.stdout:[{"addr": "172.21.15.6", "hostname": "smithi006", "labels": [], "status": ""}, {"addr": "172.21.15.169", "hostname": "smithi169", "labels": [], "status": ""}] 2024-05-09T04:41:26.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.886+0000 7f654be00700 1 -- 172.21.15.6:0/948763977 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f652c03e510 msgr2=0x7f652c040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:26.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.886+0000 7f654be00700 1 --2- 172.21.15.6:0/948763977 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f652c03e510 0x7f652c040a20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6530006fd0 tx=0x7f6530006e40 comp rx=0 tx=0).stop 2024-05-09T04:41:26.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.886+0000 7f654be00700 1 -- 172.21.15.6:0/948763977 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6544103df0 msgr2=0x7f65441a1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:26.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.886+0000 7f654be00700 1 --2- 172.21.15.6:0/948763977 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6544103df0 0x7f65441a1780 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6538006010 tx=0x7f6538005780 comp rx=0 tx=0).stop 2024-05-09T04:41:26.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.886+0000 7f654be00700 1 -- 172.21.15.6:0/948763977 shutdown_connections 2024-05-09T04:41:26.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.886+0000 7f654be00700 1 --2- 172.21.15.6:0/948763977 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f652c03e510 0x7f652c040a20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:26.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.886+0000 7f654be00700 1 --2- 172.21.15.6:0/948763977 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6544103df0 0x7f65441a1780 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:26.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.886+0000 7f654be00700 1 -- 172.21.15.6:0/948763977 >> 172.21.15.6:0/948763977 conn(0x7f65440ff6a0 msgr2=0x7f6544075b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:26.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.887+0000 7f654be00700 1 -- 172.21.15.6:0/948763977 shutdown_connections 2024-05-09T04:41:26.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:26.887+0000 7f654be00700 1 -- 172.21.15.6:0/948763977 wait complete. 2024-05-09T04:41:27.573 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi186 2024-05-09T04:41:27.573 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-05-09T04:41:27.574 DEBUG:teuthology.orchestra.run.smithi186:> dd of=/etc/ceph/ceph.conf 2024-05-09T04:41:27.591 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-05-09T04:41:27.592 DEBUG:teuthology.orchestra.run.smithi186:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T04:41:27.650 INFO:tasks.cephadm:Adding host smithi186 to orchestrator... 2024-05-09T04:41:27.650 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph orch host add smithi186 2024-05-09T04:41:28.115 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:41:28.577 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:28 smithi006 ceph-mon[29835]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:41:28.577 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:28 smithi006 ceph-mon[29835]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:30.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:30.205+0000 7f5c03e00700 1 -- 172.21.15.6:0/2821342009 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5bfc10a3d0 msgr2=0x7f5bfc10a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:30.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:30.205+0000 7f5c03e00700 1 --2- 172.21.15.6:0/2821342009 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5bfc10a3d0 0x7f5bfc10a7e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0009910 tx=0x7f5bf0009c20 comp rx=0 tx=0).stop 2024-05-09T04:41:30.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:30.205+0000 7f5c03e00700 1 -- 172.21.15.6:0/2821342009 shutdown_connections 2024-05-09T04:41:30.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:30.206+0000 7f5c03e00700 1 --2- 172.21.15.6:0/2821342009 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5bfc10a3d0 0x7f5bfc10a7e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:30.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:30.206+0000 7f5c03e00700 1 -- 172.21.15.6:0/2821342009 >> 172.21.15.6:0/2821342009 conn(0x7f5bfc0799f0 msgr2=0x7f5bfc1085e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:30.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:30.206+0000 7f5c03e00700 1 -- 172.21.15.6:0/2821342009 shutdown_connections 2024-05-09T04:41:30.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:30.206+0000 7f5c03e00700 1 -- 172.21.15.6:0/2821342009 wait complete. 2024-05-09T04:41:30.206 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:30 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:30.206 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:30 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:30.206 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:30 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:30.206 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:30 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:30.206 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:30 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi169", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:41:30.206 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:30 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:30.206 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:30 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:41:30.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:30 smithi006 ceph-mon[29835]: Updating smithi169:/etc/ceph/ceph.conf 2024-05-09T04:41:30.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:30 smithi006 ceph-mon[29835]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:30.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:30 smithi006 ceph-mon[29835]: Updating smithi169:/var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:41:30.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:30.207+0000 7f5c03e00700 1 Processor -- start 2024-05-09T04:41:30.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:30.207+0000 7f5c03e00700 1 -- start start 2024-05-09T04:41:30.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:30.208+0000 7f5c03e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5bfc10a3d0 0x7f5bfc19ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:30.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:30.208+0000 7f5c03e00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5bfc1146c0 con 0x7f5bfc10a3d0 2024-05-09T04:41:30.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:30.208+0000 7f5c01400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5bfc10a3d0 0x7f5bfc19ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:30.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:30.208+0000 7f5c01400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5bfc10a3d0 0x7f5bfc19ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44962/0 (socket says 172.21.15.6:44962) 2024-05-09T04:41:30.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:30.208+0000 7f5c01400700 1 -- 172.21.15.6:0/3148235447 learned_addr learned my addr 172.21.15.6:0/3148235447 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:41:30.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:30.208+0000 7f5c01400700 1 -- 172.21.15.6:0/3148235447 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bfc1a04e0 con 0x7f5bfc10a3d0 2024-05-09T04:41:30.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:30.209+0000 7f5c01400700 1 --2- 172.21.15.6:0/3148235447 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5bfc10a3d0 0x7f5bfc19ffa0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0006010 tx=0x7f5bf0005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:30.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:30.209+0000 7f5bfa000700 1 -- 172.21.15.6:0/3148235447 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bf0026070 con 0x7f5bfc10a3d0 2024-05-09T04:41:30.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:30.209+0000 7f5bfa000700 1 -- 172.21.15.6:0/3148235447 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bf000bba0 con 0x7f5bfc10a3d0 2024-05-09T04:41:30.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:30.209+0000 7f5c03e00700 1 -- 172.21.15.6:0/3148235447 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bfc1a07d0 con 0x7f5bfc10a3d0 2024-05-09T04:41:30.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:30.209+0000 7f5bfa000700 1 -- 172.21.15.6:0/3148235447 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bf0003910 con 0x7f5bfc10a3d0 2024-05-09T04:41:30.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:30.209+0000 7f5c03e00700 1 -- 172.21.15.6:0/3148235447 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bfc1a4780 con 0x7f5bfc10a3d0 2024-05-09T04:41:30.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:30.211+0000 7f5c03e00700 1 -- 172.21.15.6:0/3148235447 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bfc10b880 con 0x7f5bfc10a3d0 2024-05-09T04:41:30.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:30.211+0000 7f5bfa000700 1 -- 172.21.15.6:0/3148235447 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f5bf0029070 con 0x7f5bfc10a3d0 2024-05-09T04:41:30.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:30.212+0000 7f5bfa000700 1 --2- 172.21.15.6:0/3148235447 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f5be403e570 0x7f5be4040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:30.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:30.212+0000 7f5bfa000700 1 -- 172.21.15.6:0/3148235447 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5bf001e070 con 0x7f5bfc10a3d0 2024-05-09T04:41:30.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:30.212+0000 7f5c00a00700 1 --2- 172.21.15.6:0/3148235447 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f5be403e570 0x7f5be4040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:30.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:30.213+0000 7f5c00a00700 1 --2- 172.21.15.6:0/3148235447 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f5be403e570 0x7f5be4040a80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5be8006fd0 tx=0x7f5be8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:30.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:30.220+0000 7f5bfa000700 1 -- 172.21.15.6:0/3148235447 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5bf0013070 con 0x7f5bfc10a3d0 2024-05-09T04:41:30.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:30.468+0000 7f5c03e00700 1 -- 172.21.15.6:0/3148235447 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi186", "target": ["mon-mgr", ""]}) -- 0x7f5bfc1a12b0 con 0x7f5be403e570 2024-05-09T04:41:31.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:31 smithi006 ceph-mon[29835]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-05-09T04:41:31.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:31 smithi006 ceph-mon[29835]: Updating smithi169:/var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T04:41:31.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:31 smithi006 ceph-mon[29835]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi186", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:41:31.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:31 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:31.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:31 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:31.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:31 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:32.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:32 smithi006 ceph-mon[29835]: Deploying cephadm binary to smithi186 2024-05-09T04:41:32.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:32 smithi006 ceph-mon[29835]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:34.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:34 smithi006 ceph-mon[29835]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:35.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:35.390+0000 7f5bfa000700 1 -- 172.21.15.6:0/3148235447 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f5bfc1a12b0 con 0x7f5be403e570 2024-05-09T04:41:35.389 INFO:teuthology.orchestra.run.smithi006.stdout:Added host 'smithi186' with addr '172.21.15.186' 2024-05-09T04:41:35.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:35.394+0000 7f5c03e00700 1 -- 172.21.15.6:0/3148235447 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f5be403e570 msgr2=0x7f5be4040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:35.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:35.394+0000 7f5c03e00700 1 --2- 172.21.15.6:0/3148235447 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f5be403e570 0x7f5be4040a80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5be8006fd0 tx=0x7f5be8006e40 comp rx=0 tx=0).stop 2024-05-09T04:41:35.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:35.394+0000 7f5c03e00700 1 -- 172.21.15.6:0/3148235447 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5bfc10a3d0 msgr2=0x7f5bfc19ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:35.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:35.394+0000 7f5c03e00700 1 --2- 172.21.15.6:0/3148235447 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5bfc10a3d0 0x7f5bfc19ffa0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0006010 tx=0x7f5bf0005780 comp rx=0 tx=0).stop 2024-05-09T04:41:35.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:35.394+0000 7f5c03e00700 1 -- 172.21.15.6:0/3148235447 shutdown_connections 2024-05-09T04:41:35.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:35.394+0000 7f5c03e00700 1 --2- 172.21.15.6:0/3148235447 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f5be403e570 0x7f5be4040a80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:35.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:35.394+0000 7f5c03e00700 1 --2- 172.21.15.6:0/3148235447 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5bfc10a3d0 0x7f5bfc19ffa0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:35.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:35.394+0000 7f5c03e00700 1 -- 172.21.15.6:0/3148235447 >> 172.21.15.6:0/3148235447 conn(0x7f5bfc0799f0 msgr2=0x7f5bfc10bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:35.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:35.394+0000 7f5c03e00700 1 -- 172.21.15.6:0/3148235447 shutdown_connections 2024-05-09T04:41:35.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:35.395+0000 7f5c03e00700 1 -- 172.21.15.6:0/3148235447 wait complete. 2024-05-09T04:41:36.225 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-09T04:41:36.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:36 smithi006 ceph-mon[29835]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:36.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:36 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:36.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:36 smithi006 ceph-mon[29835]: Added host smithi186 2024-05-09T04:41:36.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:36 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:41:36.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:36 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:36.688 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:41:37.851 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:37 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:38.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.707+0000 7f73c8c00700 1 -- 172.21.15.6:0/2695482424 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73c4075d30 msgr2=0x7f73c4076140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:38.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.707+0000 7f73c8c00700 1 --2- 172.21.15.6:0/2695482424 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73c4075d30 0x7f73c4076140 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f73b4009910 tx=0x7f73b4009c20 comp rx=0 tx=0).stop 2024-05-09T04:41:38.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.708+0000 7f73c8c00700 1 -- 172.21.15.6:0/2695482424 shutdown_connections 2024-05-09T04:41:38.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.708+0000 7f73c8c00700 1 --2- 172.21.15.6:0/2695482424 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73c4075d30 0x7f73c4076140 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:38.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.708+0000 7f73c8c00700 1 -- 172.21.15.6:0/2695482424 >> 172.21.15.6:0/2695482424 conn(0x7f73c40ff6a0 msgr2=0x7f73c4101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:38.707 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:38 smithi006 ceph-mon[29835]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:38.708 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:38 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:38.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.708+0000 7f73c8c00700 1 -- 172.21.15.6:0/2695482424 shutdown_connections 2024-05-09T04:41:38.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.708+0000 7f73c8c00700 1 -- 172.21.15.6:0/2695482424 wait complete. 2024-05-09T04:41:38.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.709+0000 7f73c8c00700 1 Processor -- start 2024-05-09T04:41:38.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.709+0000 7f73c8c00700 1 -- start start 2024-05-09T04:41:38.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.710+0000 7f73c8c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73c4075d30 0x7f73c41a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:38.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.710+0000 7f73c8c00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f73c4110340 con 0x7f73c4075d30 2024-05-09T04:41:38.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.710+0000 7f73c1e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73c4075d30 0x7f73c41a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:38.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.710+0000 7f73c1e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73c4075d30 0x7f73c41a1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45096/0 (socket says 172.21.15.6:45096) 2024-05-09T04:41:38.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.710+0000 7f73c1e00700 1 -- 172.21.15.6:0/2032289448 learned_addr learned my addr 172.21.15.6:0/2032289448 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:41:38.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.711+0000 7f73c1e00700 1 -- 172.21.15.6:0/2032289448 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73c41a1cc0 con 0x7f73c4075d30 2024-05-09T04:41:38.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.711+0000 7f73c1e00700 1 --2- 172.21.15.6:0/2032289448 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73c4075d30 0x7f73c41a1780 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f73b4006010 tx=0x7f73b4005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:38.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.711+0000 7f73bb400700 1 -- 172.21.15.6:0/2032289448 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73b4026070 con 0x7f73c4075d30 2024-05-09T04:41:38.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.711+0000 7f73bb400700 1 -- 172.21.15.6:0/2032289448 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73b400bc00 con 0x7f73c4075d30 2024-05-09T04:41:38.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.711+0000 7f73c8c00700 1 -- 172.21.15.6:0/2032289448 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73c41a1f50 con 0x7f73c4075d30 2024-05-09T04:41:38.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.711+0000 7f73bb400700 1 -- 172.21.15.6:0/2032289448 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73b4003910 con 0x7f73c4075d30 2024-05-09T04:41:38.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.711+0000 7f73c8c00700 1 -- 172.21.15.6:0/2032289448 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73c419b9a0 con 0x7f73c4075d30 2024-05-09T04:41:38.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.713+0000 7f73c8c00700 1 -- 172.21.15.6:0/2032289448 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73c40772a0 con 0x7f73c4075d30 2024-05-09T04:41:38.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.713+0000 7f73bb400700 1 -- 172.21.15.6:0/2032289448 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f73b4029070 con 0x7f73c4075d30 2024-05-09T04:41:38.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.713+0000 7f73bb400700 1 --2- 172.21.15.6:0/2032289448 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f73a803e570 0x7f73a8040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:38.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.713+0000 7f73bb400700 1 -- 172.21.15.6:0/2032289448 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f73b401e070 con 0x7f73c4075d30 2024-05-09T04:41:38.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.714+0000 7f73b9400700 1 --2- 172.21.15.6:0/2032289448 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f73a803e570 0x7f73a8040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:38.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.715+0000 7f73b9400700 1 --2- 172.21.15.6:0/2032289448 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f73a803e570 0x7f73a8040a80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f73ac00ad30 tx=0x7f73ac0093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:38.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.720+0000 7f73bb400700 1 -- 172.21.15.6:0/2032289448 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f73b4013070 con 0x7f73c4075d30 2024-05-09T04:41:38.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.965+0000 7f73c8c00700 1 -- 172.21.15.6:0/2032289448 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f73c419ced0 con 0x7f73a803e570 2024-05-09T04:41:38.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.968+0000 7f73bb400700 1 -- 172.21.15.6:0/2032289448 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f73c419ced0 con 0x7f73a803e570 2024-05-09T04:41:38.967 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:41:38.967 INFO:teuthology.orchestra.run.smithi006.stdout:[{"addr": "172.21.15.6", "hostname": "smithi006", "labels": [], "status": ""}, {"addr": "172.21.15.169", "hostname": "smithi169", "labels": [], "status": ""}, {"addr": "172.21.15.186", "hostname": "smithi186", "labels": [], "status": ""}] 2024-05-09T04:41:38.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.971+0000 7f73c8c00700 1 -- 172.21.15.6:0/2032289448 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f73a803e570 msgr2=0x7f73a8040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:38.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.971+0000 7f73c8c00700 1 --2- 172.21.15.6:0/2032289448 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f73a803e570 0x7f73a8040a80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f73ac00ad30 tx=0x7f73ac0093f0 comp rx=0 tx=0).stop 2024-05-09T04:41:38.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.971+0000 7f73c8c00700 1 -- 172.21.15.6:0/2032289448 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73c4075d30 msgr2=0x7f73c41a1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:38.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.971+0000 7f73c8c00700 1 --2- 172.21.15.6:0/2032289448 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73c4075d30 0x7f73c41a1780 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f73b4006010 tx=0x7f73b4005780 comp rx=0 tx=0).stop 2024-05-09T04:41:38.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.971+0000 7f73c8c00700 1 -- 172.21.15.6:0/2032289448 shutdown_connections 2024-05-09T04:41:38.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.971+0000 7f73c8c00700 1 --2- 172.21.15.6:0/2032289448 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f73a803e570 0x7f73a8040a80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:38.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.971+0000 7f73c8c00700 1 --2- 172.21.15.6:0/2032289448 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73c4075d30 0x7f73c41a1780 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:38.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.971+0000 7f73c8c00700 1 -- 172.21.15.6:0/2032289448 >> 172.21.15.6:0/2032289448 conn(0x7f73c40ff6a0 msgr2=0x7f73c4107e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:38.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.971+0000 7f73c8c00700 1 -- 172.21.15.6:0/2032289448 shutdown_connections 2024-05-09T04:41:38.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:38.971+0000 7f73c8c00700 1 -- 172.21.15.6:0/2032289448 wait complete. 2024-05-09T04:41:39.689 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-09T04:41:39.690 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd crush tunables default 2024-05-09T04:41:40.148 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:41:40.578 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:40 smithi006 ceph-mon[29835]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:41:40.579 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:40 smithi006 ceph-mon[29835]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:42.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:42.225+0000 7f89ffe00700 1 -- 172.21.15.6:0/2517686875 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a001088a0 msgr2=0x7f8a00108cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:42.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:42.225+0000 7f89ffe00700 1 --2- 172.21.15.6:0/2517686875 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a001088a0 0x7f8a00108cb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f89f0009910 tx=0x7f89f0009c20 comp rx=0 tx=0).stop 2024-05-09T04:41:42.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:42.225+0000 7f89ffe00700 1 -- 172.21.15.6:0/2517686875 shutdown_connections 2024-05-09T04:41:42.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:42.225+0000 7f89ffe00700 1 --2- 172.21.15.6:0/2517686875 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a001088a0 0x7f8a00108cb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:42.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:42.225+0000 7f89ffe00700 1 -- 172.21.15.6:0/2517686875 >> 172.21.15.6:0/2517686875 conn(0x7f8a00078450 msgr2=0x7f8a00078890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:42.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:42.225+0000 7f89ffe00700 1 -- 172.21.15.6:0/2517686875 shutdown_connections 2024-05-09T04:41:42.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:42.226+0000 7f89ffe00700 1 -- 172.21.15.6:0/2517686875 wait complete. 2024-05-09T04:41:42.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:42.227+0000 7f89ffe00700 1 Processor -- start 2024-05-09T04:41:42.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:42.227+0000 7f89ffe00700 1 -- start start 2024-05-09T04:41:42.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:42.227+0000 7f89ffe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a001088a0 0x7f8a001a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:42.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:42.227+0000 7f89ffe00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0010cdf0 con 0x7f8a001088a0 2024-05-09T04:41:42.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:42.228+0000 7f89fea00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a001088a0 0x7f8a001a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:42.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:42.228+0000 7f89fea00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a001088a0 0x7f8a001a1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45110/0 (socket says 172.21.15.6:45110) 2024-05-09T04:41:42.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:42.228+0000 7f89fea00700 1 -- 172.21.15.6:0/2370318114 learned_addr learned my addr 172.21.15.6:0/2370318114 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:41:42.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:42.228+0000 7f89fea00700 1 -- 172.21.15.6:0/2370318114 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a001a1ba0 con 0x7f8a001088a0 2024-05-09T04:41:42.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:42.229+0000 7f89fea00700 1 --2- 172.21.15.6:0/2370318114 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a001088a0 0x7f8a001a1660 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f89f00092e0 tx=0x7f89f0005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:42.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:42.229+0000 7f89f7400700 1 -- 172.21.15.6:0/2370318114 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89f0026070 con 0x7f8a001088a0 2024-05-09T04:41:42.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:42.229+0000 7f89f7400700 1 -- 172.21.15.6:0/2370318114 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89f000b6f0 con 0x7f8a001088a0 2024-05-09T04:41:42.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:42.229+0000 7f89ffe00700 1 -- 172.21.15.6:0/2370318114 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a001a1e30 con 0x7f8a001088a0 2024-05-09T04:41:42.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:42.229+0000 7f89f7400700 1 -- 172.21.15.6:0/2370318114 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89f00037f0 con 0x7f8a001088a0 2024-05-09T04:41:42.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:42.229+0000 7f89ffe00700 1 -- 172.21.15.6:0/2370318114 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a0019b840 con 0x7f8a001088a0 2024-05-09T04:41:42.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:42.230+0000 7f89ffe00700 1 -- 172.21.15.6:0/2370318114 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a00103fb0 con 0x7f8a001088a0 2024-05-09T04:41:42.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:42.231+0000 7f89f7400700 1 -- 172.21.15.6:0/2370318114 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f89f0029070 con 0x7f8a001088a0 2024-05-09T04:41:42.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:42.231+0000 7f89f7400700 1 --2- 172.21.15.6:0/2370318114 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f89e403e570 0x7f89e4040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:42.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:42.232+0000 7f89f7400700 1 -- 172.21.15.6:0/2370318114 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f89f001e070 con 0x7f8a001088a0 2024-05-09T04:41:42.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:42.232+0000 7f89fe000700 1 --2- 172.21.15.6:0/2370318114 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f89e403e570 0x7f89e4040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:42.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:42.233+0000 7f89fe000700 1 --2- 172.21.15.6:0/2370318114 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f89e403e570 0x7f89e4040a80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f89ec006fd0 tx=0x7f89ec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:42.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:42.237+0000 7f89f7400700 1 -- 172.21.15.6:0/2370318114 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f89f0013070 con 0x7f8a001088a0 2024-05-09T04:41:42.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:42.487+0000 7f89ffe00700 1 -- 172.21.15.6:0/2370318114 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f8a0019cd50 con 0x7f8a001088a0 2024-05-09T04:41:42.486 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:42 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:42.486 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:42 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:42.486 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:42 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:42.487 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:42 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:42.487 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:42 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi186", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:41:42.487 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:42 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:42.487 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:42 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:41:42.487 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:42 smithi006 ceph-mon[29835]: Updating smithi186:/etc/ceph/ceph.conf 2024-05-09T04:41:42.487 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:42 smithi006 ceph-mon[29835]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:42.487 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:42 smithi006 ceph-mon[29835]: Updating smithi186:/var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:41:43.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:43.340+0000 7f89f7400700 1 -- 172.21.15.6:0/2370318114 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f89f000d340 con 0x7f8a001088a0 2024-05-09T04:41:43.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:43.344+0000 7f89ffe00700 1 -- 172.21.15.6:0/2370318114 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f89e403e570 msgr2=0x7f89e4040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:43.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:43.344+0000 7f89ffe00700 1 --2- 172.21.15.6:0/2370318114 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f89e403e570 0x7f89e4040a80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f89ec006fd0 tx=0x7f89ec006e40 comp rx=0 tx=0).stop 2024-05-09T04:41:43.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:43.344+0000 7f89ffe00700 1 -- 172.21.15.6:0/2370318114 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a001088a0 msgr2=0x7f8a001a1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:43.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:43.344+0000 7f89ffe00700 1 --2- 172.21.15.6:0/2370318114 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a001088a0 0x7f8a001a1660 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f89f00092e0 tx=0x7f89f0005780 comp rx=0 tx=0).stop 2024-05-09T04:41:43.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:43.345+0000 7f89ffe00700 1 -- 172.21.15.6:0/2370318114 shutdown_connections 2024-05-09T04:41:43.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:43.345+0000 7f89ffe00700 1 --2- 172.21.15.6:0/2370318114 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f89e403e570 0x7f89e4040a80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:43.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:43.345+0000 7f89ffe00700 1 --2- 172.21.15.6:0/2370318114 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a001088a0 0x7f8a001a1660 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:43.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:43.345+0000 7f89ffe00700 1 -- 172.21.15.6:0/2370318114 >> 172.21.15.6:0/2370318114 conn(0x7f8a00078450 msgr2=0x7f8a0006ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:43.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:43.345+0000 7f89ffe00700 1 -- 172.21.15.6:0/2370318114 shutdown_connections 2024-05-09T04:41:43.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:41:43.345+0000 7f89ffe00700 1 -- 172.21.15.6:0/2370318114 wait complete. 2024-05-09T04:41:43.345 INFO:teuthology.orchestra.run.smithi006.stderr:adjusted tunables profile to default 2024-05-09T04:41:43.696 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:43 smithi006 ceph-mon[29835]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-05-09T04:41:43.696 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:43 smithi006 ceph-mon[29835]: Updating smithi186:/var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T04:41:43.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:43 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/2370318114' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-09T04:41:43.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:43 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:43.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:43 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:43.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:43 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:44.068 INFO:tasks.cephadm:Adding mon.a on smithi006 2024-05-09T04:41:44.069 INFO:tasks.cephadm:Adding mon.b on smithi169 2024-05-09T04:41:44.069 INFO:tasks.cephadm:Adding mon.c on smithi186 2024-05-09T04:41:44.069 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph orch apply mon '3;smithi006:172.21.15.6=a;smithi169:172.21.15.169=b;smithi186:172.21.15.186=c' 2024-05-09T04:41:44.424 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:41:44.696 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:44 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/2370318114' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-09T04:41:44.696 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:44 smithi006 ceph-mon[29835]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T04:41:44.696 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:44 smithi006 ceph-mon[29835]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:45.825 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:45.825+0000 7fd2d2261700 1 -- 172.21.15.186:0/3835953698 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2cc108a90 msgr2=0x7fd2cc108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:45.825 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:45.825+0000 7fd2d2261700 1 --2- 172.21.15.186:0/3835953698 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2cc108a90 0x7fd2cc108ea0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc009910 tx=0x7fd2bc009c20 comp rx=0 tx=0).stop 2024-05-09T04:41:45.825 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:45.825+0000 7fd2d2261700 1 -- 172.21.15.186:0/3835953698 shutdown_connections 2024-05-09T04:41:45.825 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:45.825+0000 7fd2d2261700 1 --2- 172.21.15.186:0/3835953698 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2cc108a90 0x7fd2cc108ea0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:45.825 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:45.825+0000 7fd2d2261700 1 -- 172.21.15.186:0/3835953698 >> 172.21.15.186:0/3835953698 conn(0x7fd2cc0785b0 msgr2=0x7fd2cc078a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:45.826 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:45.825+0000 7fd2d2261700 1 -- 172.21.15.186:0/3835953698 shutdown_connections 2024-05-09T04:41:45.826 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:45.825+0000 7fd2d2261700 1 -- 172.21.15.186:0/3835953698 wait complete. 2024-05-09T04:41:45.827 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:45.826+0000 7fd2d2261700 1 Processor -- start 2024-05-09T04:41:45.827 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:45.826+0000 7fd2d2261700 1 -- start start 2024-05-09T04:41:45.827 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:45.827+0000 7fd2d2261700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2cc108a90 0x7fd2cc19e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:45.827 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:45.827+0000 7fd2d2261700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc10cfe0 con 0x7fd2cc108a90 2024-05-09T04:41:45.827 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:45.827+0000 7fd2cb7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2cc108a90 0x7fd2cc19e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:45.827 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:45.827+0000 7fd2cb7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2cc108a90 0x7fd2cc19e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.186:40126/0 (socket says 172.21.15.186:40126) 2024-05-09T04:41:45.827 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:45.827+0000 7fd2cb7fe700 1 -- 172.21.15.186:0/281852130 learned_addr learned my addr 172.21.15.186:0/281852130 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-05-09T04:41:45.828 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:45.828+0000 7fd2cb7fe700 1 -- 172.21.15.186:0/281852130 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2cc19ea30 con 0x7fd2cc108a90 2024-05-09T04:41:45.828 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:45.828+0000 7fd2cb7fe700 1 --2- 172.21.15.186:0/281852130 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2cc108a90 0x7fd2cc19e4f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc0056d0 tx=0x7fd2bc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:45.828 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:45.828+0000 7fd2c97fa700 1 -- 172.21.15.186:0/281852130 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2bc027070 con 0x7fd2cc108a90 2024-05-09T04:41:45.828 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:45.829+0000 7fd2d2261700 1 -- 172.21.15.186:0/281852130 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2cc19ed20 con 0x7fd2cc108a90 2024-05-09T04:41:45.828 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:45.829+0000 7fd2c97fa700 1 -- 172.21.15.186:0/281852130 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2bc00be10 con 0x7fd2cc108a90 2024-05-09T04:41:45.829 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:45.829+0000 7fd2c97fa700 1 -- 172.21.15.186:0/281852130 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2bc022d20 con 0x7fd2cc108a90 2024-05-09T04:41:45.829 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:45.829+0000 7fd2d2261700 1 -- 172.21.15.186:0/281852130 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2cc198850 con 0x7fd2cc108a90 2024-05-09T04:41:45.829 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:45.830+0000 7fd2d2261700 1 -- 172.21.15.186:0/281852130 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2cc1041a0 con 0x7fd2cc108a90 2024-05-09T04:41:45.829 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:45.830+0000 7fd2c97fa700 1 -- 172.21.15.186:0/281852130 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7fd2bc02a070 con 0x7fd2cc108a90 2024-05-09T04:41:45.830 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:45.831+0000 7fd2c97fa700 1 --2- 172.21.15.186:0/281852130 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd2b803e510 0x7fd2b8040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:45.830 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:45.831+0000 7fd2c97fa700 1 -- 172.21.15.186:0/281852130 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd2bc01e030 con 0x7fd2cc108a90 2024-05-09T04:41:45.830 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:45.831+0000 7fd2c3fff700 1 --2- 172.21.15.186:0/281852130 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd2b803e510 0x7fd2b8040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:45.831 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:45.832+0000 7fd2c3fff700 1 --2- 172.21.15.186:0/281852130 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd2b803e510 0x7fd2b8040a20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4006fd0 tx=0x7fd2b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:45.846 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:45.847+0000 7fd2c97fa700 1 -- 172.21.15.186:0/281852130 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd2bc031020 con 0x7fd2cc108a90 2024-05-09T04:41:46.057 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:46.058+0000 7fd2d2261700 1 -- 172.21.15.186:0/281852130 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi006:172.21.15.6=a;smithi169:172.21.15.169=b;smithi186:172.21.15.186=c", "target": ["mon-mgr", ""]}) -- 0x7fd2cc19ef50 con 0x7fd2b803e510 2024-05-09T04:41:46.074 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:46.075+0000 7fd2c97fa700 1 -- 172.21.15.186:0/281852130 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd2cc19ef50 con 0x7fd2b803e510 2024-05-09T04:41:46.074 INFO:teuthology.orchestra.run.smithi186.stdout:Scheduled mon update... 2024-05-09T04:41:46.077 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:46.077+0000 7fd2d2261700 1 -- 172.21.15.186:0/281852130 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd2b803e510 msgr2=0x7fd2b8040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:46.077 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:46.077+0000 7fd2d2261700 1 --2- 172.21.15.186:0/281852130 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd2b803e510 0x7fd2b8040a20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4006fd0 tx=0x7fd2b4006e40 comp rx=0 tx=0).stop 2024-05-09T04:41:46.077 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:46.077+0000 7fd2d2261700 1 -- 172.21.15.186:0/281852130 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2cc108a90 msgr2=0x7fd2cc19e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:46.077 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:46.077+0000 7fd2d2261700 1 --2- 172.21.15.186:0/281852130 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2cc108a90 0x7fd2cc19e4f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc0056d0 tx=0x7fd2bc0057b0 comp rx=0 tx=0).stop 2024-05-09T04:41:46.077 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:46.077+0000 7fd2d2261700 1 -- 172.21.15.186:0/281852130 shutdown_connections 2024-05-09T04:41:46.077 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:46.077+0000 7fd2d2261700 1 --2- 172.21.15.186:0/281852130 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd2b803e510 0x7fd2b8040a20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:46.077 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:46.077+0000 7fd2d2261700 1 --2- 172.21.15.186:0/281852130 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2cc108a90 0x7fd2cc19e4f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:46.077 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:46.077+0000 7fd2d2261700 1 -- 172.21.15.186:0/281852130 >> 172.21.15.186:0/281852130 conn(0x7fd2cc0785b0 msgr2=0x7fd2cc0ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:46.077 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:46.077+0000 7fd2d2261700 1 -- 172.21.15.186:0/281852130 shutdown_connections 2024-05-09T04:41:46.077 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:46.077+0000 7fd2d2261700 1 -- 172.21.15.186:0/281852130 wait complete. 2024-05-09T04:41:46.567 DEBUG:teuthology.orchestra.run.smithi169:mon.b> sudo journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mon.b.service 2024-05-09T04:41:46.570 DEBUG:teuthology.orchestra.run.smithi186:mon.c> sudo journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mon.c.service 2024-05-09T04:41:46.572 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-09T04:41:46.572 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T04:41:46.696 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:46 smithi006 ceph-mon[29835]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:46.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:46 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:46.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:46 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:41:46.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:46 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:46.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:46 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:41:46.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:46 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:46.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:46 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T04:41:46.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:46 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:47.288 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:41:47.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:47 smithi006 ceph-mon[29835]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi006:172.21.15.6=a;smithi169:172.21.15.169=b;smithi186:172.21.15.186=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:41:47.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:47 smithi006 ceph-mon[29835]: Saving service mon spec with placement smithi006:172.21.15.6=a;smithi169:172.21.15.169=b;smithi186:172.21.15.186=c;count:3 2024-05-09T04:41:47.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:47 smithi006 ceph-mon[29835]: Deploying daemon mon.c on smithi186 2024-05-09T04:41:48.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:48 smithi006 ceph-mon[29835]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:50.256 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.256+0000 7fec78e0a700 1 -- 172.21.15.186:0/3273327542 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec74108a90 msgr2=0x7fec74108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:50.257 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.256+0000 7fec78e0a700 1 --2- 172.21.15.186:0/3273327542 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec74108a90 0x7fec74108ea0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fec64009910 tx=0x7fec64009c20 comp rx=0 tx=0).stop 2024-05-09T04:41:50.257 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.256+0000 7fec78e0a700 1 -- 172.21.15.186:0/3273327542 shutdown_connections 2024-05-09T04:41:50.257 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.256+0000 7fec78e0a700 1 --2- 172.21.15.186:0/3273327542 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec74108a90 0x7fec74108ea0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:50.257 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.256+0000 7fec78e0a700 1 -- 172.21.15.186:0/3273327542 >> 172.21.15.186:0/3273327542 conn(0x7fec740785b0 msgr2=0x7fec74078a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:50.257 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.256+0000 7fec78e0a700 1 -- 172.21.15.186:0/3273327542 shutdown_connections 2024-05-09T04:41:50.257 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.256+0000 7fec78e0a700 1 -- 172.21.15.186:0/3273327542 wait complete. 2024-05-09T04:41:50.258 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.258+0000 7fec78e0a700 1 Processor -- start 2024-05-09T04:41:50.258 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.258+0000 7fec78e0a700 1 -- start start 2024-05-09T04:41:50.258 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.258+0000 7fec78e0a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec74108a90 0x7fec741a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:50.258 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.258+0000 7fec78e0a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fec7410cfe0 con 0x7fec74108a90 2024-05-09T04:41:50.258 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.259+0000 7fec7259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec74108a90 0x7fec741a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:50.258 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.259+0000 7fec7259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec74108a90 0x7fec741a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.186:55066/0 (socket says 172.21.15.186:55066) 2024-05-09T04:41:50.258 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.259+0000 7fec7259c700 1 -- 172.21.15.186:0/1575284274 learned_addr learned my addr 172.21.15.186:0/1575284274 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-05-09T04:41:50.259 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.259+0000 7fec7259c700 1 -- 172.21.15.186:0/1575284274 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec741a1ca0 con 0x7fec74108a90 2024-05-09T04:41:50.259 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.260+0000 7fec7259c700 1 --2- 172.21.15.186:0/1575284274 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec74108a90 0x7fec741a1760 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fec640056d0 tx=0x7fec640057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:50.260 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.260+0000 7fec6b7fe700 1 -- 172.21.15.186:0/1575284274 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec64027070 con 0x7fec74108a90 2024-05-09T04:41:50.260 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.260+0000 7fec78e0a700 1 -- 172.21.15.186:0/1575284274 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec741a1f90 con 0x7fec74108a90 2024-05-09T04:41:50.260 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.260+0000 7fec6b7fe700 1 -- 172.21.15.186:0/1575284274 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec6400be10 con 0x7fec74108a90 2024-05-09T04:41:50.260 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.260+0000 7fec6b7fe700 1 -- 172.21.15.186:0/1575284274 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec64022d20 con 0x7fec74108a90 2024-05-09T04:41:50.260 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.260+0000 7fec78e0a700 1 -- 172.21.15.186:0/1575284274 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec7419b980 con 0x7fec74108a90 2024-05-09T04:41:50.261 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.261+0000 7fec78e0a700 1 -- 172.21.15.186:0/1575284274 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec741041a0 con 0x7fec74108a90 2024-05-09T04:41:50.261 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.261+0000 7fec6b7fe700 1 -- 172.21.15.186:0/1575284274 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7fec6402a070 con 0x7fec74108a90 2024-05-09T04:41:50.261 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.262+0000 7fec6b7fe700 1 --2- 172.21.15.186:0/1575284274 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fec6003e510 0x7fec60040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:50.261 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.262+0000 7fec6b7fe700 1 -- 172.21.15.186:0/1575284274 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fec6401e030 con 0x7fec74108a90 2024-05-09T04:41:50.262 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.262+0000 7fec71d9b700 1 --2- 172.21.15.186:0/1575284274 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fec6003e510 0x7fec60040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:50.262 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.263+0000 7fec71d9b700 1 --2- 172.21.15.186:0/1575284274 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fec6003e510 0x7fec60040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fec5c006fd0 tx=0x7fec5c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:50.267 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.268+0000 7fec6b7fe700 1 -- 172.21.15.186:0/1575284274 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fec64013070 con 0x7fec74108a90 2024-05-09T04:41:50.564 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.564+0000 7fec78e0a700 1 -- 172.21.15.186:0/1575284274 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fec7419ceb0 con 0x7fec74108a90 2024-05-09T04:41:50.565 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.565+0000 7fec6b7fe700 1 -- 172.21.15.186:0/1575284274 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+741 (secure 0 0 0) 0x7fec64031090 con 0x7fec74108a90 2024-05-09T04:41:50.565 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-05-09T04:41:50.565 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":1,"fsid":"f74d813c-0dbd-11ef-bc97-c7b262605968","modified":"2024-05-09T04:39:30.984541Z","created":"2024-05-09T04:39:30.984541Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:3300","nonce":0},{"type":"v1","addr":"172.21.15.6:6789","nonce":0}]},"addr":"172.21.15.6:6789/0","public_addr":"172.21.15.6:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-09T04:41:50.567 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.568+0000 7fec78e0a700 1 -- 172.21.15.186:0/1575284274 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fec6003e510 msgr2=0x7fec60040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:50.567 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.568+0000 7fec78e0a700 1 --2- 172.21.15.186:0/1575284274 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fec6003e510 0x7fec60040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fec5c006fd0 tx=0x7fec5c006e40 comp rx=0 tx=0).stop 2024-05-09T04:41:50.567 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.568+0000 7fec78e0a700 1 -- 172.21.15.186:0/1575284274 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec74108a90 msgr2=0x7fec741a1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:50.567 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.568+0000 7fec78e0a700 1 --2- 172.21.15.186:0/1575284274 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec74108a90 0x7fec741a1760 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fec640056d0 tx=0x7fec640057b0 comp rx=0 tx=0).stop 2024-05-09T04:41:50.568 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.568+0000 7fec78e0a700 1 -- 172.21.15.186:0/1575284274 shutdown_connections 2024-05-09T04:41:50.568 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.568+0000 7fec78e0a700 1 --2- 172.21.15.186:0/1575284274 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fec6003e510 0x7fec60040a20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:50.568 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.568+0000 7fec78e0a700 1 --2- 172.21.15.186:0/1575284274 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec74108a90 0x7fec741a1760 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:50.568 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.568+0000 7fec78e0a700 1 -- 172.21.15.186:0/1575284274 >> 172.21.15.186:0/1575284274 conn(0x7fec740785b0 msgr2=0x7fec74104560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:50.568 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.568+0000 7fec78e0a700 1 -- 172.21.15.186:0/1575284274 shutdown_connections 2024-05-09T04:41:50.568 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:50.568+0000 7fec78e0a700 1 -- 172.21.15.186:0/1575284274 wait complete. 2024-05-09T04:41:50.568 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 1 2024-05-09T04:41:50.696 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:50 smithi006 ceph-mon[29835]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:51.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:51 smithi186 ceph-mon[34668]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-09T04:41:51.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:51 smithi006 ceph-mon[29835]: from='client.? 172.21.15.186:0/1575284274' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T04:41:52.270 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-09T04:41:52.270 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T04:41:52.506 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T04:41:56.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:56 smithi186 ceph-mon[34668]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:56.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:41:56.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:56 smithi186 ceph-mon[34668]: mon.a calling monitor election 2024-05-09T04:41:56.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:41:56.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:41:56.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:56 smithi186 ceph-mon[34668]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:56.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:41:56.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:56 smithi186 ceph-mon[34668]: mon.c calling monitor election 2024-05-09T04:41:56.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:41:56.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:56 smithi186 ceph-mon[34668]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:56.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:41:56.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:41:56.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:56 smithi186 ceph-mon[34668]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T04:41:56.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:56 smithi186 ceph-mon[34668]: monmap epoch 2 2024-05-09T04:41:56.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:56 smithi186 ceph-mon[34668]: fsid f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:41:56.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:56 smithi186 ceph-mon[34668]: last_changed 2024-05-09T04:41:51.498362+0000 2024-05-09T04:41:56.851 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:56 smithi186 ceph-mon[34668]: created 2024-05-09T04:39:30.984541+0000 2024-05-09T04:41:56.851 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:56 smithi186 ceph-mon[34668]: min_mon_release 19 (squid) 2024-05-09T04:41:56.851 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:56 smithi186 ceph-mon[34668]: election_strategy: 1 2024-05-09T04:41:56.851 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:56 smithi186 ceph-mon[34668]: 0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-09T04:41:56.851 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:56 smithi186 ceph-mon[34668]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-05-09T04:41:56.851 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:56 smithi186 ceph-mon[34668]: fsmap 2024-05-09T04:41:56.851 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:56 smithi186 ceph-mon[34668]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T04:41:56.851 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:56 smithi186 ceph-mon[34668]: mgrmap e13: x(active, since 61s) 2024-05-09T04:41:56.851 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:56 smithi186 ceph-mon[34668]: overall HEALTH_OK 2024-05-09T04:41:56.851 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:56.851 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:56.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:56 smithi006 ceph-mon[29835]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:56.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:41:56.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:56 smithi006 ceph-mon[29835]: mon.a calling monitor election 2024-05-09T04:41:56.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:41:56.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:41:56.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:56 smithi006 ceph-mon[29835]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:56.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:41:56.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:56 smithi006 ceph-mon[29835]: mon.c calling monitor election 2024-05-09T04:41:56.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:41:56.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:56 smithi006 ceph-mon[29835]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:56.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:41:56.949 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:41:56.949 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:56 smithi006 ceph-mon[29835]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T04:41:56.949 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:56 smithi006 ceph-mon[29835]: monmap epoch 2 2024-05-09T04:41:56.949 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:56 smithi006 ceph-mon[29835]: fsid f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:41:56.949 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:56 smithi006 ceph-mon[29835]: last_changed 2024-05-09T04:41:51.498362+0000 2024-05-09T04:41:56.949 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:56 smithi006 ceph-mon[29835]: created 2024-05-09T04:39:30.984541+0000 2024-05-09T04:41:56.949 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:56 smithi006 ceph-mon[29835]: min_mon_release 19 (squid) 2024-05-09T04:41:56.949 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:56 smithi006 ceph-mon[29835]: election_strategy: 1 2024-05-09T04:41:56.949 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:56 smithi006 ceph-mon[29835]: 0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-09T04:41:56.949 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:56 smithi006 ceph-mon[29835]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-05-09T04:41:56.949 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:56 smithi006 ceph-mon[29835]: fsmap 2024-05-09T04:41:56.949 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:56 smithi006 ceph-mon[29835]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T04:41:56.949 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:56 smithi006 ceph-mon[29835]: mgrmap e13: x(active, since 61s) 2024-05-09T04:41:56.949 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:56 smithi006 ceph-mon[29835]: overall HEALTH_OK 2024-05-09T04:41:56.949 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:56.950 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:56.974 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.973+0000 7fecbff32700 1 -- 172.21.15.186:0/13086473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feca00035c0 msgr2=0x7feca0005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:56.975 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.973+0000 7fecbff32700 1 --2- 172.21.15.186:0/13086473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feca00035c0 0x7feca0005aa0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fecb8103ba0 tx=0x7fecb000feb0 comp rx=0 tx=0).stop 2024-05-09T04:41:56.975 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.973+0000 7fecbff32700 1 -- 172.21.15.186:0/13086473 shutdown_connections 2024-05-09T04:41:56.975 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.973+0000 7fecbff32700 1 --2- 172.21.15.186:0/13086473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feca00035c0 0x7feca0005aa0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:56.975 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.973+0000 7fecbff32700 1 -- 172.21.15.186:0/13086473 >> 172.21.15.186:0/13086473 conn(0x7fecb80ff790 msgr2=0x7fecb8101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:56.975 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.973+0000 7fecbff32700 1 -- 172.21.15.186:0/13086473 shutdown_connections 2024-05-09T04:41:56.975 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.973+0000 7fecbff32700 1 -- 172.21.15.186:0/13086473 wait complete. 2024-05-09T04:41:56.976 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.975+0000 7fecbff32700 1 Processor -- start 2024-05-09T04:41:56.976 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.975+0000 7fecbff32700 1 -- start start 2024-05-09T04:41:56.976 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.975+0000 7fecbff32700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feca00035c0 0x7fecb819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:56.976 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.975+0000 7fecbff32700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecb819fbe0 0x7fecb81998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:56.977 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.975+0000 7fecbff32700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fecb810bc20 con 0x7feca00035c0 2024-05-09T04:41:56.977 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.975+0000 7fecbff32700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fecb810baa0 con 0x7fecb819fbe0 2024-05-09T04:41:56.977 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.975+0000 7fecbd4cd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecb819fbe0 0x7fecb81998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:56.977 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.975+0000 7fecbd4cd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecb819fbe0 0x7fecb81998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:37186/0 (socket says 172.21.15.186:37186) 2024-05-09T04:41:56.977 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.975+0000 7fecbd4cd700 1 -- 172.21.15.186:0/4018104089 learned_addr learned my addr 172.21.15.186:0/4018104089 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-05-09T04:41:56.977 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.975+0000 7fecbd4cd700 1 -- 172.21.15.186:0/4018104089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecb819fbe0 msgr2=0x7fecb81998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-09T04:41:56.977 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.975+0000 7fecbd4cd700 1 -- 172.21.15.186:0/4018104089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecb819fbe0 msgr2=0x7fecb81998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T04:41:56.977 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.975+0000 7fecbd4cd700 1 --2- 172.21.15.186:0/4018104089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecb819fbe0 0x7fecb81998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T04:41:56.977 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.976+0000 7fecbd4cd700 1 --2- 172.21.15.186:0/4018104089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecb819fbe0 0x7fecb81998c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T04:41:56.977 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.976+0000 7fecbdcce700 1 --2- 172.21.15.186:0/4018104089 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feca00035c0 0x7fecb819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:56.978 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.976+0000 7fecbdcce700 1 -- 172.21.15.186:0/4018104089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecb819fbe0 msgr2=0x7fecb81998c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T04:41:56.978 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.976+0000 7fecbdcce700 1 --2- 172.21.15.186:0/4018104089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecb819fbe0 0x7fecb81998c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:56.978 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.976+0000 7fecbdcce700 1 -- 172.21.15.186:0/4018104089 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb81a01b0 con 0x7feca00035c0 2024-05-09T04:41:56.978 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.977+0000 7fecbdcce700 1 --2- 172.21.15.186:0/4018104089 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feca00035c0 0x7fecb819f6a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fecb0000c00 tx=0x7fecb00050d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:56.978 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.977+0000 7fecb6ffd700 1 -- 172.21.15.186:0/4018104089 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fecb0004aa0 con 0x7feca00035c0 2024-05-09T04:41:56.979 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.977+0000 7fecbff32700 1 -- 172.21.15.186:0/4018104089 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecb8199e60 con 0x7feca00035c0 2024-05-09T04:41:56.979 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.977+0000 7fecb6ffd700 1 -- 172.21.15.186:0/4018104089 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecb0004030 con 0x7feca00035c0 2024-05-09T04:41:56.979 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.977+0000 7fecb6ffd700 1 -- 172.21.15.186:0/4018104089 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fecb0011bb0 con 0x7feca00035c0 2024-05-09T04:41:56.979 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.978+0000 7fecbff32700 1 -- 172.21.15.186:0/4018104089 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecb819a3f0 con 0x7feca00035c0 2024-05-09T04:41:56.979 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.979+0000 7fecbff32700 1 -- 172.21.15.186:0/4018104089 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecb81053b0 con 0x7feca00035c0 2024-05-09T04:41:56.980 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.980+0000 7fecb6ffd700 1 -- 172.21.15.186:0/4018104089 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7fecb0023070 con 0x7feca00035c0 2024-05-09T04:41:56.980 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.980+0000 7fecb6ffd700 1 --2- 172.21.15.186:0/4018104089 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7feca403e300 0x7feca4040810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:56.980 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.980+0000 7fecb6ffd700 1 -- 172.21.15.186:0/4018104089 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fecb0020050 con 0x7feca00035c0 2024-05-09T04:41:56.981 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.980+0000 7fecbd4cd700 1 --2- 172.21.15.186:0/4018104089 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7feca403e300 0x7feca4040810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:56.982 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.981+0000 7fecbd4cd700 1 --2- 172.21.15.186:0/4018104089 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7feca403e300 0x7feca4040810 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fecac009400 tx=0x7fecac007040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:56.997 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:56.997+0000 7fecb6ffd700 1 -- 172.21.15.186:0/4018104089 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fecb0028020 con 0x7feca00035c0 2024-05-09T04:41:57.289 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:57.288+0000 7fecbff32700 1 -- 172.21.15.186:0/4018104089 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fecb819adf0 con 0x7feca00035c0 2024-05-09T04:41:57.290 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:57.289+0000 7fecb6ffd700 1 -- 172.21.15.186:0/4018104089 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7fecb0033300 con 0x7feca00035c0 2024-05-09T04:41:57.290 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-05-09T04:41:57.290 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":2,"fsid":"f74d813c-0dbd-11ef-bc97-c7b262605968","modified":"2024-05-09T04:41:51.498362Z","created":"2024-05-09T04:39:30.984541Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:3300","nonce":0},{"type":"v1","addr":"172.21.15.6:6789","nonce":0}]},"addr":"172.21.15.6:6789/0","public_addr":"172.21.15.6:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:3300","nonce":0},{"type":"v1","addr":"172.21.15.186:6789","nonce":0}]},"addr":"172.21.15.186:6789/0","public_addr":"172.21.15.186:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-09T04:41:57.293 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:57.292+0000 7fecbff32700 1 -- 172.21.15.186:0/4018104089 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7feca403e300 msgr2=0x7feca4040810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:57.293 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:57.292+0000 7fecbff32700 1 --2- 172.21.15.186:0/4018104089 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7feca403e300 0x7feca4040810 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fecac009400 tx=0x7fecac007040 comp rx=0 tx=0).stop 2024-05-09T04:41:57.293 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:57.292+0000 7fecbff32700 1 -- 172.21.15.186:0/4018104089 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feca00035c0 msgr2=0x7fecb819f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:57.293 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:57.292+0000 7fecbff32700 1 --2- 172.21.15.186:0/4018104089 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feca00035c0 0x7fecb819f6a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fecb0000c00 tx=0x7fecb00050d0 comp rx=0 tx=0).stop 2024-05-09T04:41:57.293 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:57.292+0000 7fecbff32700 1 -- 172.21.15.186:0/4018104089 shutdown_connections 2024-05-09T04:41:57.293 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:57.292+0000 7fecbff32700 1 --2- 172.21.15.186:0/4018104089 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7feca403e300 0x7feca4040810 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:57.293 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:57.292+0000 7fecbff32700 1 --2- 172.21.15.186:0/4018104089 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feca00035c0 0x7fecb819f6a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:57.293 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:57.292+0000 7fecbff32700 1 --2- 172.21.15.186:0/4018104089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecb819fbe0 0x7fecb81998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:57.293 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:57.292+0000 7fecbff32700 1 -- 172.21.15.186:0/4018104089 >> 172.21.15.186:0/4018104089 conn(0x7fecb80ff790 msgr2=0x7fecb8107440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:57.293 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:57.292+0000 7fecbff32700 1 -- 172.21.15.186:0/4018104089 shutdown_connections 2024-05-09T04:41:57.293 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:41:57.292+0000 7fecbff32700 1 -- 172.21.15.186:0/4018104089 wait complete. 2024-05-09T04:41:57.294 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 2 2024-05-09T04:41:57.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:57 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:57.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:57 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T04:41:57.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:57 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:57.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:57 smithi186 ceph-mon[34668]: Deploying daemon mon.b on smithi169 2024-05-09T04:41:57.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:57 smithi186 ceph-mon[34668]: from='client.? 172.21.15.186:0/4018104089' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T04:41:57.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:57 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:41:57.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:57 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:41:57.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:57 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T04:41:57.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:57 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:57.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:57 smithi006 ceph-mon[29835]: Deploying daemon mon.b on smithi169 2024-05-09T04:41:57.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:57 smithi006 ceph-mon[29835]: from='client.? 172.21.15.186:0/4018104089' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T04:41:57.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:57 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:41:58.808 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-09T04:41:58.809 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T04:41:58.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:41:58 smithi006 ceph-mon[29835]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:59.053 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T04:41:59.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:41:58 smithi186 ceph-mon[34668]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:00.415 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: mon.b@-1(???) e0 preinit fsid f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:42:00.415 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: mon.b@-1(synchronizing).mds e1 new map 2024-05-09T04:42:00.415 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-09T04:42:00.416 INFO:journalctl@ceph.mon.b.smithi169.stdout: e1 2024-05-09T04:42:00.416 INFO:journalctl@ceph.mon.b.smithi169.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T04:42:00.416 INFO:journalctl@ceph.mon.b.smithi169.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T04:42:00.416 INFO:journalctl@ceph.mon.b.smithi169.stdout: legacy client fscid: -1 2024-05-09T04:42:00.416 INFO:journalctl@ceph.mon.b.smithi169.stdout: 2024-05-09T04:42:00.416 INFO:journalctl@ceph.mon.b.smithi169.stdout: No filesystems configured 2024-05-09T04:42:00.416 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-09T04:42:00.416 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-09T04:42:00.416 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-09T04:42:00.416 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-09T04:42:00.416 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-09T04:42:00.416 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-09T04:42:00.670 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-09T04:42:00.670 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T04:42:00.670 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T04:42:00.670 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T04:42:00.670 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:00.671 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:00.671 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: Added host smithi169 2024-05-09T04:42:00.671 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:00.671 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:00.671 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:00.671 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:00.671 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:00.671 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:00.671 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:42:00.671 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:00.671 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:00.671 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:00.671 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:00.672 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:00.672 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi169", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:42:00.672 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:00.672 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:00.672 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: Updating smithi169:/etc/ceph/ceph.conf 2024-05-09T04:42:00.672 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:00.672 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: Updating smithi169:/var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:42:00.672 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-05-09T04:42:00.672 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: Updating smithi169:/var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T04:42:00.672 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi186", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:42:00.672 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:00.672 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:00.672 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:00.672 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: Deploying cephadm binary to smithi186 2024-05-09T04:42:00.672 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:00.672 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:00.673 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:00.673 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:00.673 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: Added host smithi186 2024-05-09T04:42:00.673 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:00.673 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:00.674 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:00.674 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:00.674 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:00.674 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:42:00.674 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:00.674 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:00.674 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:00.674 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:00.674 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:00.674 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi186", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:42:00.674 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:00.674 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:00.674 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: Updating smithi186:/etc/ceph/ceph.conf 2024-05-09T04:42:00.674 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:00.675 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: Updating smithi186:/var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:42:00.675 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-05-09T04:42:00.675 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: Updating smithi186:/var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T04:42:00.675 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/2370318114' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-09T04:42:00.675 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:00.675 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:00.675 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:00.675 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/2370318114' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-09T04:42:00.675 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T04:42:00.675 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:00.675 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:00.675 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:00.675 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:00.676 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:00.676 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:00.676 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:00.676 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T04:42:00.676 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:00.676 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi006:172.21.15.6=a;smithi169:172.21.15.169=b;smithi186:172.21.15.186=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:42:00.676 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: Saving service mon spec with placement smithi006:172.21.15.6=a;smithi169:172.21.15.169=b;smithi186:172.21.15.186=c;count:3 2024-05-09T04:42:00.676 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: Deploying daemon mon.c on smithi186 2024-05-09T04:42:00.676 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:00.676 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:00.676 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='client.? 172.21.15.186:0/1575284274' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T04:42:00.676 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:00.676 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:42:00.676 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: mon.a calling monitor election 2024-05-09T04:42:00.676 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:42:00.676 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:42:00.677 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:00.677 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:42:00.677 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: mon.c calling monitor election 2024-05-09T04:42:00.677 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:42:00.677 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:00.677 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:42:00.677 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:42:00.677 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T04:42:00.677 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: monmap epoch 2 2024-05-09T04:42:00.677 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: fsid f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:42:00.677 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: last_changed 2024-05-09T04:41:51.498362+0000 2024-05-09T04:42:00.677 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: created 2024-05-09T04:39:30.984541+0000 2024-05-09T04:42:00.677 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: min_mon_release 19 (squid) 2024-05-09T04:42:00.677 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: election_strategy: 1 2024-05-09T04:42:00.677 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: 0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-09T04:42:00.678 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-05-09T04:42:00.678 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: fsmap 2024-05-09T04:42:00.678 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T04:42:00.678 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: mgrmap e13: x(active, since 61s) 2024-05-09T04:42:00.678 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: overall HEALTH_OK 2024-05-09T04:42:00.678 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:00.678 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:00.678 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:00.678 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T04:42:00.678 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:00.678 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: Deploying daemon mon.b on smithi169 2024-05-09T04:42:00.678 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='client.? 172.21.15.186:0/4018104089' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T04:42:00.678 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:42:00.678 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:00.679 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 ceph-mon[34075]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-09T04:42:00.679 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 bash[34054]: 1ca93128349d94aaf4848614aad9fac3d2fdfaa260dd2809c70d0eb4e380f5c2 2024-05-09T04:42:00.679 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:00 smithi169 systemd[1]: Started Ceph mon.b for f74d813c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:42:05.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:05 smithi186 ceph-mon[34668]: mon.c calling monitor election 2024-05-09T04:42:05.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:05 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:42:05.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:05 smithi186 ceph-mon[34668]: mon.a calling monitor election 2024-05-09T04:42:05.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:05 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:42:05.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:05 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:42:05.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:05 smithi186 ceph-mon[34668]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:05.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:05 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:42:05.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:05 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:42:05.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:05 smithi186 ceph-mon[34668]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:05.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:05 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:42:05.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:05 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:42:05.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:05 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:42:05.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:05 smithi186 ceph-mon[34668]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T04:42:05.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:05 smithi186 ceph-mon[34668]: monmap epoch 3 2024-05-09T04:42:05.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:05 smithi186 ceph-mon[34668]: fsid f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:42:05.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:05 smithi186 ceph-mon[34668]: last_changed 2024-05-09T04:42:00.423052+0000 2024-05-09T04:42:05.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:05 smithi186 ceph-mon[34668]: created 2024-05-09T04:39:30.984541+0000 2024-05-09T04:42:05.851 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:05 smithi186 ceph-mon[34668]: min_mon_release 19 (squid) 2024-05-09T04:42:05.851 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:05 smithi186 ceph-mon[34668]: election_strategy: 1 2024-05-09T04:42:05.851 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:05 smithi186 ceph-mon[34668]: 0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-09T04:42:05.851 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:05 smithi186 ceph-mon[34668]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-05-09T04:42:05.851 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:05 smithi186 ceph-mon[34668]: 2: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.b 2024-05-09T04:42:05.851 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:05 smithi186 ceph-mon[34668]: fsmap 2024-05-09T04:42:05.851 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:05 smithi186 ceph-mon[34668]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T04:42:05.851 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:05 smithi186 ceph-mon[34668]: mgrmap e13: x(active, since 70s) 2024-05-09T04:42:05.851 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:05 smithi186 ceph-mon[34668]: overall HEALTH_OK 2024-05-09T04:42:05.851 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:05 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:05.851 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:05 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:05.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:05 smithi006 ceph-mon[29835]: mon.c calling monitor election 2024-05-09T04:42:05.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:05 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:42:05.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:05 smithi006 ceph-mon[29835]: mon.a calling monitor election 2024-05-09T04:42:05.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:05 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:42:05.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:05 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:42:05.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:05 smithi006 ceph-mon[29835]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:05.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:05 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:42:05.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:05 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:42:05.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:05 smithi006 ceph-mon[29835]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:05.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:05 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:42:05.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:05 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:42:05.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:05 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:42:05.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:05 smithi006 ceph-mon[29835]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T04:42:05.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:05 smithi006 ceph-mon[29835]: monmap epoch 3 2024-05-09T04:42:05.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:05 smithi006 ceph-mon[29835]: fsid f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:42:05.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:05 smithi006 ceph-mon[29835]: last_changed 2024-05-09T04:42:00.423052+0000 2024-05-09T04:42:05.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:05 smithi006 ceph-mon[29835]: created 2024-05-09T04:39:30.984541+0000 2024-05-09T04:42:05.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:05 smithi006 ceph-mon[29835]: min_mon_release 19 (squid) 2024-05-09T04:42:05.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:05 smithi006 ceph-mon[29835]: election_strategy: 1 2024-05-09T04:42:05.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:05 smithi006 ceph-mon[29835]: 0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-09T04:42:05.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:05 smithi006 ceph-mon[29835]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-05-09T04:42:05.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:05 smithi006 ceph-mon[29835]: 2: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.b 2024-05-09T04:42:05.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:05 smithi006 ceph-mon[29835]: fsmap 2024-05-09T04:42:05.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:05 smithi006 ceph-mon[29835]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T04:42:05.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:05 smithi006 ceph-mon[29835]: mgrmap e13: x(active, since 70s) 2024-05-09T04:42:05.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:05 smithi006 ceph-mon[29835]: overall HEALTH_OK 2024-05-09T04:42:05.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:05 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:05.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:05 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:06.594 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.594+0000 7f4034f19700 1 -- 172.21.15.186:0/573892747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40180062f0 msgr2=0x7f40180087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:06.595 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.594+0000 7f4034f19700 1 --2- 172.21.15.186:0/573892747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40180062f0 0x7f40180087b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4030104d00 tx=0x7f402000fe90 comp rx=0 tx=0).stop 2024-05-09T04:42:06.595 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.594+0000 7f4034f19700 1 -- 172.21.15.186:0/573892747 shutdown_connections 2024-05-09T04:42:06.595 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.594+0000 7f4034f19700 1 --2- 172.21.15.186:0/573892747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40180062f0 0x7f40180087b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:06.596 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.594+0000 7f4034f19700 1 -- 172.21.15.186:0/573892747 >> 172.21.15.186:0/573892747 conn(0x7f40300ff7a0 msgr2=0x7f4030101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:42:06.596 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.594+0000 7f4034f19700 1 -- 172.21.15.186:0/573892747 shutdown_connections 2024-05-09T04:42:06.596 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.594+0000 7f4034f19700 1 -- 172.21.15.186:0/573892747 wait complete. 2024-05-09T04:42:06.601 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.595+0000 7f4034f19700 1 Processor -- start 2024-05-09T04:42:06.601 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.595+0000 7f4034f19700 1 -- start start 2024-05-09T04:42:06.601 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.596+0000 7f4034f19700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40180062f0 0x7f403019f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:42:06.601 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.596+0000 7f4034f19700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f403019fef0 0x7f4030199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:42:06.601 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.596+0000 7f4034f19700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f403019a080 0x7f403019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:42:06.602 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.596+0000 7f4034f19700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f403010bba0 con 0x7f40180062f0 2024-05-09T04:42:06.602 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.596+0000 7f4034f19700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f403010ba20 con 0x7f403019fef0 2024-05-09T04:42:06.602 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.596+0000 7f4034f19700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f403010bd20 con 0x7f403019a080 2024-05-09T04:42:06.602 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.596+0000 7f402ed9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f403019a080 0x7f403019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:42:06.602 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.596+0000 7f402ed9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f403019a080 0x7f403019a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:59550/0 (socket says 172.21.15.186:59550) 2024-05-09T04:42:06.602 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.596+0000 7f402ed9d700 1 -- 172.21.15.186:0/3057701427 learned_addr learned my addr 172.21.15.186:0/3057701427 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-05-09T04:42:06.603 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.596+0000 7f402dd9b700 1 --2- 172.21.15.186:0/3057701427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f403019fef0 0x7f4030199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:42:06.603 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.596+0000 7f402e59c700 1 --2- 172.21.15.186:0/3057701427 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40180062f0 0x7f403019f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:42:06.603 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.596+0000 7f402dd9b700 1 -- 172.21.15.186:0/3057701427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f403019fef0 msgr2=0x7f4030199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-09T04:42:06.603 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.596+0000 7f402dd9b700 1 -- 172.21.15.186:0/3057701427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f403019fef0 msgr2=0x7f4030199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T04:42:06.603 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.596+0000 7f402dd9b700 1 --2- 172.21.15.186:0/3057701427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f403019fef0 0x7f4030199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T04:42:06.603 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.596+0000 7f402dd9b700 1 --2- 172.21.15.186:0/3057701427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f403019fef0 0x7f4030199b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T04:42:06.603 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.596+0000 7f402ed9d700 1 -- 172.21.15.186:0/3057701427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f403019fef0 msgr2=0x7f4030199b40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T04:42:06.603 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.596+0000 7f402ed9d700 1 --2- 172.21.15.186:0/3057701427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f403019fef0 0x7f4030199b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:06.603 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.596+0000 7f402ed9d700 1 -- 172.21.15.186:0/3057701427 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40180062f0 msgr2=0x7f403019f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:06.603 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.596+0000 7f402ed9d700 1 --2- 172.21.15.186:0/3057701427 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40180062f0 0x7f403019f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:06.603 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.596+0000 7f402ed9d700 1 -- 172.21.15.186:0/3057701427 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40301a6760 con 0x7f403019a080 2024-05-09T04:42:06.604 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.597+0000 7f402ed9d700 1 --2- 172.21.15.186:0/3057701427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f403019a080 0x7f403019a490 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f402800ba50 tx=0x7f402800be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:42:06.604 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.597+0000 7f401f7fe700 1 -- 172.21.15.186:0/3057701427 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4028019070 con 0x7f403019a080 2024-05-09T04:42:06.604 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.597+0000 7f401f7fe700 1 -- 172.21.15.186:0/3057701427 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4028014440 con 0x7f403019a080 2024-05-09T04:42:06.604 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.597+0000 7f4034f19700 1 -- 172.21.15.186:0/3057701427 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40301a6a50 con 0x7f403019a080 2024-05-09T04:42:06.604 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.597+0000 7f401f7fe700 1 -- 172.21.15.186:0/3057701427 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4028007590 con 0x7f403019a080 2024-05-09T04:42:06.604 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.597+0000 7f4034f19700 1 -- 172.21.15.186:0/3057701427 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40301a6f60 con 0x7f403019a080 2024-05-09T04:42:06.604 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.598+0000 7f4034f19700 1 -- 172.21.15.186:0/3057701427 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4030105380 con 0x7f403019a080 2024-05-09T04:42:06.604 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.598+0000 7f401f7fe700 1 -- 172.21.15.186:0/3057701427 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f402801c070 con 0x7f403019a080 2024-05-09T04:42:06.604 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.598+0000 7f401f7fe700 1 --2- 172.21.15.186:0/3057701427 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f401003e9a0 0x7f4010040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:42:06.604 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.598+0000 7f401f7fe700 1 -- 172.21.15.186:0/3057701427 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4028051e10 con 0x7f403019a080 2024-05-09T04:42:06.604 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.599+0000 7f402e59c700 1 --2- 172.21.15.186:0/3057701427 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f401003e9a0 0x7f4010040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:42:06.604 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.600+0000 7f402e59c700 1 --2- 172.21.15.186:0/3057701427 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f401003e9a0 0x7f4010040eb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f402000fa40 tx=0x7f4020005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:42:06.605 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.604+0000 7f401f7fe700 1 -- 172.21.15.186:0/3057701427 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f402800f160 con 0x7f403019a080 2024-05-09T04:42:06.813 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:06 smithi186 ceph-mon[34668]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:06.813 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:06 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:06.813 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:06 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:06.813 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:06 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:06.813 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:06 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:42:06.902 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.902+0000 7f4034f19700 1 -- 172.21.15.186:0/3057701427 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f40301a71f0 con 0x7f403019a080 2024-05-09T04:42:06.902 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.902+0000 7f401f7fe700 1 -- 172.21.15.186:0/3057701427 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f4028012c40 con 0x7f403019a080 2024-05-09T04:42:06.902 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-05-09T04:42:06.903 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":3,"fsid":"f74d813c-0dbd-11ef-bc97-c7b262605968","modified":"2024-05-09T04:42:00.423052Z","created":"2024-05-09T04:39:30.984541Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:3300","nonce":0},{"type":"v1","addr":"172.21.15.6:6789","nonce":0}]},"addr":"172.21.15.6:6789/0","public_addr":"172.21.15.6:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:3300","nonce":0},{"type":"v1","addr":"172.21.15.186:6789","nonce":0}]},"addr":"172.21.15.186:6789/0","public_addr":"172.21.15.186:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:3300","nonce":0},{"type":"v1","addr":"172.21.15.169:6789","nonce":0}]},"addr":"172.21.15.169:6789/0","public_addr":"172.21.15.169:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-09T04:42:06.905 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.905+0000 7f4034f19700 1 -- 172.21.15.186:0/3057701427 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f401003e9a0 msgr2=0x7f4010040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:06.905 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.905+0000 7f4034f19700 1 --2- 172.21.15.186:0/3057701427 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f401003e9a0 0x7f4010040eb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f402000fa40 tx=0x7f4020005a90 comp rx=0 tx=0).stop 2024-05-09T04:42:06.905 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.905+0000 7f4034f19700 1 -- 172.21.15.186:0/3057701427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f403019a080 msgr2=0x7f403019a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:06.905 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.905+0000 7f4034f19700 1 --2- 172.21.15.186:0/3057701427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f403019a080 0x7f403019a490 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f402800ba50 tx=0x7f402800be10 comp rx=0 tx=0).stop 2024-05-09T04:42:06.905 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.905+0000 7f4034f19700 1 -- 172.21.15.186:0/3057701427 shutdown_connections 2024-05-09T04:42:06.905 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.905+0000 7f4034f19700 1 --2- 172.21.15.186:0/3057701427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f403019a080 0x7f403019a490 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:06.905 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.905+0000 7f4034f19700 1 --2- 172.21.15.186:0/3057701427 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40180062f0 0x7f403019f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:06.905 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.905+0000 7f4034f19700 1 --2- 172.21.15.186:0/3057701427 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f401003e9a0 0x7f4010040eb0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:06.905 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.905+0000 7f4034f19700 1 --2- 172.21.15.186:0/3057701427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f403019fef0 0x7f4030199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:06.906 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.905+0000 7f4034f19700 1 -- 172.21.15.186:0/3057701427 >> 172.21.15.186:0/3057701427 conn(0x7f40300ff7a0 msgr2=0x7f4030106fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:42:06.906 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.905+0000 7f4034f19700 1 -- 172.21.15.186:0/3057701427 shutdown_connections 2024-05-09T04:42:06.906 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:06.905+0000 7f4034f19700 1 -- 172.21.15.186:0/3057701427 wait complete. 2024-05-09T04:42:06.906 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 3 2024-05-09T04:42:06.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:06 smithi006 ceph-mon[29835]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:06.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:06 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:06.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:06 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:06.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:06 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:06.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:06 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:42:07.700 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-09T04:42:07.700 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph config generate-minimal-conf 2024-05-09T04:42:07.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:07 smithi169 ceph-mon[34075]: mon.c calling monitor election 2024-05-09T04:42:07.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:07 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:42:07.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:07 smithi169 ceph-mon[34075]: mon.a calling monitor election 2024-05-09T04:42:07.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:07 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:42:07.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:07 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:42:07.868 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:07 smithi169 ceph-mon[34075]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:07.868 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:07 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:42:07.868 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:07 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:42:07.868 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:07 smithi169 ceph-mon[34075]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:07.868 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:07 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:42:07.868 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:07 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:42:07.868 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:07 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:42:07.869 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:07 smithi169 ceph-mon[34075]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T04:42:07.869 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:07 smithi169 ceph-mon[34075]: monmap epoch 3 2024-05-09T04:42:07.869 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:07 smithi169 ceph-mon[34075]: fsid f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:42:07.869 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:07 smithi169 ceph-mon[34075]: last_changed 2024-05-09T04:42:00.423052+0000 2024-05-09T04:42:07.869 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:07 smithi169 ceph-mon[34075]: created 2024-05-09T04:39:30.984541+0000 2024-05-09T04:42:07.869 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:07 smithi169 ceph-mon[34075]: min_mon_release 19 (squid) 2024-05-09T04:42:07.869 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:07 smithi169 ceph-mon[34075]: election_strategy: 1 2024-05-09T04:42:07.869 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:07 smithi169 ceph-mon[34075]: 0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-09T04:42:07.870 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:07 smithi169 ceph-mon[34075]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-05-09T04:42:07.870 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:07 smithi169 ceph-mon[34075]: 2: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.b 2024-05-09T04:42:07.870 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:07 smithi169 ceph-mon[34075]: fsmap 2024-05-09T04:42:07.870 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:07 smithi169 ceph-mon[34075]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T04:42:07.870 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:07 smithi169 ceph-mon[34075]: mgrmap e13: x(active, since 70s) 2024-05-09T04:42:07.870 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:07 smithi169 ceph-mon[34075]: overall HEALTH_OK 2024-05-09T04:42:07.870 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:07 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:07.870 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:07 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:07.870 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:07 smithi169 ceph-mon[34075]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:07.870 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:07 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:07.870 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:07 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:07.870 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:07 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:07.870 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:07 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:42:08.145 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:42:08.446 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:08 smithi006 ceph-mon[29835]: mon.b calling monitor election 2024-05-09T04:42:08.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:08 smithi006 ceph-mon[29835]: Updating smithi006:/etc/ceph/ceph.conf 2024-05-09T04:42:08.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:08 smithi006 ceph-mon[29835]: Updating smithi169:/etc/ceph/ceph.conf 2024-05-09T04:42:08.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:08 smithi006 ceph-mon[29835]: Updating smithi186:/etc/ceph/ceph.conf 2024-05-09T04:42:08.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:08 smithi006 ceph-mon[29835]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:08.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:08 smithi006 ceph-mon[29835]: mon.b calling monitor election 2024-05-09T04:42:08.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:08 smithi006 ceph-mon[29835]: mon.c calling monitor election 2024-05-09T04:42:08.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:08 smithi006 ceph-mon[29835]: mon.a calling monitor election 2024-05-09T04:42:08.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:08 smithi006 ceph-mon[29835]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-09T04:42:08.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:08 smithi006 ceph-mon[29835]: monmap epoch 3 2024-05-09T04:42:08.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:08 smithi006 ceph-mon[29835]: fsid f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:42:08.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:08 smithi006 ceph-mon[29835]: last_changed 2024-05-09T04:42:00.423052+0000 2024-05-09T04:42:08.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:08 smithi006 ceph-mon[29835]: created 2024-05-09T04:39:30.984541+0000 2024-05-09T04:42:08.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:08 smithi006 ceph-mon[29835]: min_mon_release 19 (squid) 2024-05-09T04:42:08.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:08 smithi006 ceph-mon[29835]: election_strategy: 1 2024-05-09T04:42:08.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:08 smithi006 ceph-mon[29835]: 0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-09T04:42:08.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:08 smithi006 ceph-mon[29835]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-05-09T04:42:08.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:08 smithi006 ceph-mon[29835]: 2: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.b 2024-05-09T04:42:08.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:08 smithi006 ceph-mon[29835]: fsmap 2024-05-09T04:42:08.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:08 smithi006 ceph-mon[29835]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T04:42:08.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:08 smithi006 ceph-mon[29835]: mgrmap e13: x(active, since 72s) 2024-05-09T04:42:08.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:08 smithi006 ceph-mon[29835]: overall HEALTH_OK 2024-05-09T04:42:08.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:08 smithi006 ceph-mon[29835]: Updating smithi186:/var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:42:08.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:08 smithi006 ceph-mon[29835]: Updating smithi169:/var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:42:08.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:08 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:08.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:08 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:08.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:08 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:08.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:08 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:08.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:08 smithi186 ceph-mon[34668]: mon.b calling monitor election 2024-05-09T04:42:08.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:08 smithi186 ceph-mon[34668]: Updating smithi006:/etc/ceph/ceph.conf 2024-05-09T04:42:08.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:08 smithi186 ceph-mon[34668]: Updating smithi169:/etc/ceph/ceph.conf 2024-05-09T04:42:08.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:08 smithi186 ceph-mon[34668]: Updating smithi186:/etc/ceph/ceph.conf 2024-05-09T04:42:08.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:08 smithi186 ceph-mon[34668]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:08.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:08 smithi186 ceph-mon[34668]: mon.b calling monitor election 2024-05-09T04:42:08.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:08 smithi186 ceph-mon[34668]: mon.c calling monitor election 2024-05-09T04:42:08.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:08 smithi186 ceph-mon[34668]: mon.a calling monitor election 2024-05-09T04:42:08.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:08 smithi186 ceph-mon[34668]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-09T04:42:08.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:08 smithi186 ceph-mon[34668]: monmap epoch 3 2024-05-09T04:42:08.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:08 smithi186 ceph-mon[34668]: fsid f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:42:08.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:08 smithi186 ceph-mon[34668]: last_changed 2024-05-09T04:42:00.423052+0000 2024-05-09T04:42:08.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:08 smithi186 ceph-mon[34668]: created 2024-05-09T04:39:30.984541+0000 2024-05-09T04:42:08.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:08 smithi186 ceph-mon[34668]: min_mon_release 19 (squid) 2024-05-09T04:42:08.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:08 smithi186 ceph-mon[34668]: election_strategy: 1 2024-05-09T04:42:08.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:08 smithi186 ceph-mon[34668]: 0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-09T04:42:08.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:08 smithi186 ceph-mon[34668]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-05-09T04:42:08.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:08 smithi186 ceph-mon[34668]: 2: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.b 2024-05-09T04:42:08.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:08 smithi186 ceph-mon[34668]: fsmap 2024-05-09T04:42:08.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:08 smithi186 ceph-mon[34668]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T04:42:08.601 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:08 smithi186 ceph-mon[34668]: mgrmap e13: x(active, since 72s) 2024-05-09T04:42:08.601 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:08 smithi186 ceph-mon[34668]: overall HEALTH_OK 2024-05-09T04:42:08.601 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:08 smithi186 ceph-mon[34668]: Updating smithi186:/var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:42:08.601 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:08 smithi186 ceph-mon[34668]: Updating smithi169:/var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:42:08.601 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:08 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:08.601 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:08 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:08.601 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:08 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:08.601 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:08 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:08.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:08 smithi169 ceph-mon[34075]: mon.b calling monitor election 2024-05-09T04:42:08.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:08 smithi169 ceph-mon[34075]: Updating smithi006:/etc/ceph/ceph.conf 2024-05-09T04:42:08.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:08 smithi169 ceph-mon[34075]: Updating smithi169:/etc/ceph/ceph.conf 2024-05-09T04:42:08.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:08 smithi169 ceph-mon[34075]: Updating smithi186:/etc/ceph/ceph.conf 2024-05-09T04:42:08.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:08 smithi169 ceph-mon[34075]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:08.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:08 smithi169 ceph-mon[34075]: mon.b calling monitor election 2024-05-09T04:42:08.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:08 smithi169 ceph-mon[34075]: mon.c calling monitor election 2024-05-09T04:42:08.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:08 smithi169 ceph-mon[34075]: mon.a calling monitor election 2024-05-09T04:42:08.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:08 smithi169 ceph-mon[34075]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-09T04:42:08.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:08 smithi169 ceph-mon[34075]: monmap epoch 3 2024-05-09T04:42:08.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:08 smithi169 ceph-mon[34075]: fsid f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:42:08.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:08 smithi169 ceph-mon[34075]: last_changed 2024-05-09T04:42:00.423052+0000 2024-05-09T04:42:08.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:08 smithi169 ceph-mon[34075]: created 2024-05-09T04:39:30.984541+0000 2024-05-09T04:42:08.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:08 smithi169 ceph-mon[34075]: min_mon_release 19 (squid) 2024-05-09T04:42:08.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:08 smithi169 ceph-mon[34075]: election_strategy: 1 2024-05-09T04:42:08.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:08 smithi169 ceph-mon[34075]: 0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-09T04:42:08.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:08 smithi169 ceph-mon[34075]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-05-09T04:42:08.618 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:08 smithi169 ceph-mon[34075]: 2: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.b 2024-05-09T04:42:08.618 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:08 smithi169 ceph-mon[34075]: fsmap 2024-05-09T04:42:08.618 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:08 smithi169 ceph-mon[34075]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T04:42:08.618 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:08 smithi169 ceph-mon[34075]: mgrmap e13: x(active, since 72s) 2024-05-09T04:42:08.618 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:08 smithi169 ceph-mon[34075]: overall HEALTH_OK 2024-05-09T04:42:08.618 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:08 smithi169 ceph-mon[34075]: Updating smithi186:/var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:42:08.618 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:08 smithi169 ceph-mon[34075]: Updating smithi169:/var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:42:08.618 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:08 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:08.618 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:08 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:08.618 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:08 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:08.618 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:08 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:09.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:09 smithi186 ceph-mon[34668]: Updating smithi006:/var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:42:09.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:09 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:42:09.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:09 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:09.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:09 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:09.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:09 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:09.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:09 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:09.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:09 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:09.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:09 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:09.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:09 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:09.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:09 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T04:42:09.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:09 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T04:42:09.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:09 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:09.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:09 smithi006 ceph-mon[29835]: Updating smithi006:/var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:42:09.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:09 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:42:09.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:09 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:09.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:09 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:09.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:09 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:09.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:09 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:09.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:09 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:09.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:09 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:09.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:09 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:09.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:09 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T04:42:09.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:09 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T04:42:09.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:09 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:09.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:09 smithi169 ceph-mon[34075]: Updating smithi006:/var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:42:09.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:09 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:42:09.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:09 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:09.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:09 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:09.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:09 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:09.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:09 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:09.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:09 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:09.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:09 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:09.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:09 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:09.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:09 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T04:42:09.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:09 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T04:42:09.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:09 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:10.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:10 smithi186 ceph-mon[34668]: Reconfiguring mon.a (unknown last config time)... 2024-05-09T04:42:10.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:10 smithi186 ceph-mon[34668]: Reconfiguring daemon mon.a on smithi006 2024-05-09T04:42:10.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:10 smithi186 ceph-mon[34668]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:10.360 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:10 smithi006 ceph-mon[29835]: Reconfiguring mon.a (unknown last config time)... 2024-05-09T04:42:10.360 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:10 smithi006 ceph-mon[29835]: Reconfiguring daemon mon.a on smithi006 2024-05-09T04:42:10.360 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:10 smithi006 ceph-mon[29835]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:10.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:10 smithi169 ceph-mon[34075]: Reconfiguring mon.a (unknown last config time)... 2024-05-09T04:42:10.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:10 smithi169 ceph-mon[34075]: Reconfiguring daemon mon.a on smithi006 2024-05-09T04:42:10.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:10 smithi169 ceph-mon[34075]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:11.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.599+0000 7fad7ca00700 1 -- 172.21.15.6:0/594778330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad78103df0 msgr2=0x7fad78104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:11.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.599+0000 7fad7ca00700 1 --2- 172.21.15.6:0/594778330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad78103df0 0x7fad78104200 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fad68009910 tx=0x7fad68009c20 comp rx=0 tx=0).stop 2024-05-09T04:42:11.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.599+0000 7fad7ca00700 1 -- 172.21.15.6:0/594778330 shutdown_connections 2024-05-09T04:42:11.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.599+0000 7fad7ca00700 1 --2- 172.21.15.6:0/594778330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad78103df0 0x7fad78104200 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:11.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.599+0000 7fad7ca00700 1 -- 172.21.15.6:0/594778330 >> 172.21.15.6:0/594778330 conn(0x7fad780ff6a0 msgr2=0x7fad78101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:42:11.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.600+0000 7fad7ca00700 1 -- 172.21.15.6:0/594778330 shutdown_connections 2024-05-09T04:42:11.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.600+0000 7fad7ca00700 1 -- 172.21.15.6:0/594778330 wait complete. 2024-05-09T04:42:11.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.601+0000 7fad7ca00700 1 Processor -- start 2024-05-09T04:42:11.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.601+0000 7fad7ca00700 1 -- start start 2024-05-09T04:42:11.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.601+0000 7fad7ca00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad78103df0 0x7fad7819f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:42:11.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.601+0000 7fad7ca00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad7819fd50 0x7fad78199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:42:11.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.601+0000 7fad7ca00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad78199f70 0x7fad7819a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:42:11.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.601+0000 7fad7ca00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fad7810e0e0 con 0x7fad7819fd50 2024-05-09T04:42:11.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.601+0000 7fad7ca00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fad7810df60 con 0x7fad78199f70 2024-05-09T04:42:11.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.601+0000 7fad7ca00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fad7810e260 con 0x7fad78103df0 2024-05-09T04:42:11.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.602+0000 7fad6fe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad7819fd50 0x7fad78199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:42:11.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.602+0000 7fad6fe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad7819fd50 0x7fad78199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56616/0 (socket says 172.21.15.6:56616) 2024-05-09T04:42:11.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.602+0000 7fad6fe00700 1 -- 172.21.15.6:0/2044014439 learned_addr learned my addr 172.21.15.6:0/2044014439 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:42:11.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.602+0000 7fad75e00700 1 --2- 172.21.15.6:0/2044014439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad78103df0 0x7fad7819f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:42:11.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.602+0000 7fad76800700 1 --2- 172.21.15.6:0/2044014439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad78199f70 0x7fad7819a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:42:11.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.602+0000 7fad6fe00700 1 -- 172.21.15.6:0/2044014439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad78103df0 msgr2=0x7fad7819f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:11.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.602+0000 7fad6fe00700 1 --2- 172.21.15.6:0/2044014439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad78103df0 0x7fad7819f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:11.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.602+0000 7fad6fe00700 1 -- 172.21.15.6:0/2044014439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad78199f70 msgr2=0x7fad7819a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:11.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.602+0000 7fad6fe00700 1 --2- 172.21.15.6:0/2044014439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad78199f70 0x7fad7819a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:11.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.602+0000 7fad6fe00700 1 -- 172.21.15.6:0/2044014439 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad781a66a0 con 0x7fad7819fd50 2024-05-09T04:42:11.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.603+0000 7fad6fe00700 1 --2- 172.21.15.6:0/2044014439 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad7819fd50 0x7fad78199a30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fad6000b780 tx=0x7fad6000ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:42:11.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.603+0000 7fad6ea00700 1 -- 172.21.15.6:0/2044014439 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad60016070 con 0x7fad7819fd50 2024-05-09T04:42:11.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.603+0000 7fad6ea00700 1 -- 172.21.15.6:0/2044014439 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad600036a0 con 0x7fad7819fd50 2024-05-09T04:42:11.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.603+0000 7fad7ca00700 1 -- 172.21.15.6:0/2044014439 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad781a6930 con 0x7fad7819fd50 2024-05-09T04:42:11.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.603+0000 7fad6ea00700 1 -- 172.21.15.6:0/2044014439 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad600119c0 con 0x7fad7819fd50 2024-05-09T04:42:11.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.603+0000 7fad7ca00700 1 -- 172.21.15.6:0/2044014439 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad781a6e90 con 0x7fad7819fd50 2024-05-09T04:42:11.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.604+0000 7fad7ca00700 1 -- 172.21.15.6:0/2044014439 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad78064e20 con 0x7fad7819fd50 2024-05-09T04:42:11.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.605+0000 7fad6ea00700 1 -- 172.21.15.6:0/2044014439 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7fad60019070 con 0x7fad7819fd50 2024-05-09T04:42:11.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.605+0000 7fad6ea00700 1 --2- 172.21.15.6:0/2044014439 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fad5c03e9a0 0x7fad5c040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:42:11.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.605+0000 7fad6ea00700 1 -- 172.21.15.6:0/2044014439 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fad60051360 con 0x7fad7819fd50 2024-05-09T04:42:11.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.606+0000 7fad75e00700 1 --2- 172.21.15.6:0/2044014439 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fad5c03e9a0 0x7fad5c040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:42:11.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.607+0000 7fad75e00700 1 --2- 172.21.15.6:0/2044014439 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fad5c03e9a0 0x7fad5c040eb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fad68005e00 tx=0x7fad6801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:42:11.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.612+0000 7fad6ea00700 1 -- 172.21.15.6:0/2044014439 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fad60019360 con 0x7fad7819fd50 2024-05-09T04:42:11.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.854+0000 7fad7ca00700 1 -- 172.21.15.6:0/2044014439 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fad781a7180 con 0x7fad7819fd50 2024-05-09T04:42:11.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.855+0000 7fad6ea00700 1 -- 172.21.15.6:0/2044014439 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7fad6001d030 con 0x7fad7819fd50 2024-05-09T04:42:11.006 INFO:teuthology.orchestra.run.smithi006.stdout:# minimal ceph.conf for f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:42:11.006 INFO:teuthology.orchestra.run.smithi006.stdout:[global] 2024-05-09T04:42:11.006 INFO:teuthology.orchestra.run.smithi006.stdout: fsid = f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:42:11.006 INFO:teuthology.orchestra.run.smithi006.stdout: mon_host = [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] 2024-05-09T04:42:11.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.858+0000 7fad7ca00700 1 -- 172.21.15.6:0/2044014439 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fad5c03e9a0 msgr2=0x7fad5c040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:11.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.858+0000 7fad7ca00700 1 --2- 172.21.15.6:0/2044014439 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fad5c03e9a0 0x7fad5c040eb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fad68005e00 tx=0x7fad6801a040 comp rx=0 tx=0).stop 2024-05-09T04:42:11.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.858+0000 7fad7ca00700 1 -- 172.21.15.6:0/2044014439 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad7819fd50 msgr2=0x7fad78199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:11.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.858+0000 7fad7ca00700 1 --2- 172.21.15.6:0/2044014439 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad7819fd50 0x7fad78199a30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fad6000b780 tx=0x7fad6000ba90 comp rx=0 tx=0).stop 2024-05-09T04:42:11.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.858+0000 7fad7ca00700 1 -- 172.21.15.6:0/2044014439 shutdown_connections 2024-05-09T04:42:11.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.858+0000 7fad7ca00700 1 --2- 172.21.15.6:0/2044014439 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fad5c03e9a0 0x7fad5c040eb0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:11.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.858+0000 7fad7ca00700 1 --2- 172.21.15.6:0/2044014439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad78199f70 0x7fad7819a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:11.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.858+0000 7fad7ca00700 1 --2- 172.21.15.6:0/2044014439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad78103df0 0x7fad7819f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:11.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.858+0000 7fad7ca00700 1 --2- 172.21.15.6:0/2044014439 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad7819fd50 0x7fad78199a30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:11.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.858+0000 7fad7ca00700 1 -- 172.21.15.6:0/2044014439 >> 172.21.15.6:0/2044014439 conn(0x7fad780ff6a0 msgr2=0x7fad78109920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:42:11.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.859+0000 7fad7ca00700 1 -- 172.21.15.6:0/2044014439 shutdown_connections 2024-05-09T04:42:11.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:10.859+0000 7fad7ca00700 1 -- 172.21.15.6:0/2044014439 wait complete. 2024-05-09T04:42:11.655 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:11 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:11.655 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:11 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/2044014439' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:11.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:11 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:11.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:11 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/2044014439' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:11.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:11 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:11.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:11 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/2044014439' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:12.223 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-09T04:42:12.223 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T04:42:12.224 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/ceph/ceph.conf 2024-05-09T04:42:12.323 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T04:42:12.323 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T04:42:12.451 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-09T04:42:12.451 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/ceph/ceph.conf 2024-05-09T04:42:12.477 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-09T04:42:12.478 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T04:42:12.543 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-05-09T04:42:12.543 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/ceph/ceph.conf 2024-05-09T04:42:12.577 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-05-09T04:42:12.577 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T04:42:12.645 INFO:tasks.cephadm:Adding mgr.x on smithi006 2024-05-09T04:42:12.645 INFO:tasks.cephadm:Adding mgr.y on smithi169 2024-05-09T04:42:12.645 INFO:tasks.cephadm:Adding mgr.z on smithi186 2024-05-09T04:42:12.646 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph orch apply mgr '3;smithi006=x;smithi169=y;smithi186=z' 2024-05-09T04:42:12.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:12 smithi186 ceph-mon[34668]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:12.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:12 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:12.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:12 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:12.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:12 smithi186 ceph-mon[34668]: Reconfiguring mon.b (monmap changed)... 2024-05-09T04:42:12.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:12 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T04:42:12.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:12 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T04:42:12.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:12 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:12.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:12 smithi186 ceph-mon[34668]: Reconfiguring daemon mon.b on smithi169 2024-05-09T04:42:12.923 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T04:42:13.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:12 smithi169 ceph-mon[34075]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:13.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:12 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:13.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:12 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:13.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:12 smithi169 ceph-mon[34075]: Reconfiguring mon.b (monmap changed)... 2024-05-09T04:42:13.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:12 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T04:42:13.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:12 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T04:42:13.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:12 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:13.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:12 smithi169 ceph-mon[34075]: Reconfiguring daemon mon.b on smithi169 2024-05-09T04:42:13.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:12 smithi006 ceph-mon[29835]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:13.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:12 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:13.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:12 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:13.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:12 smithi006 ceph-mon[29835]: Reconfiguring mon.b (monmap changed)... 2024-05-09T04:42:13.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:12 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T04:42:13.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:12 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T04:42:13.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:12 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:13.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:12 smithi006 ceph-mon[29835]: Reconfiguring daemon mon.b on smithi169 2024-05-09T04:42:14.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:14 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:14.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:14 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:14.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:14 smithi186 ceph-mon[34668]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:14.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:14 smithi186 ceph-mon[34668]: Reconfiguring mon.c (monmap changed)... 2024-05-09T04:42:14.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:14 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T04:42:14.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:14 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T04:42:14.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:14 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:14.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:14 smithi186 ceph-mon[34668]: Reconfiguring daemon mon.c on smithi186 2024-05-09T04:42:14.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:14 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:14.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:14 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:14.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:14 smithi169 ceph-mon[34075]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:14.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:14 smithi169 ceph-mon[34075]: Reconfiguring mon.c (monmap changed)... 2024-05-09T04:42:14.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:14 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T04:42:14.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:14 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T04:42:14.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:14 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:14.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:14 smithi169 ceph-mon[34075]: Reconfiguring daemon mon.c on smithi186 2024-05-09T04:42:14.696 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:14 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:14.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:14 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:14.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:14 smithi006 ceph-mon[29835]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:14.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:14 smithi006 ceph-mon[29835]: Reconfiguring mon.c (monmap changed)... 2024-05-09T04:42:14.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:14 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T04:42:14.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:14 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T04:42:14.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:14 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:14.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:14 smithi006 ceph-mon[29835]: Reconfiguring daemon mon.c on smithi186 2024-05-09T04:42:15.496 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.494+0000 7ff6fb505700 1 -- 172.21.15.186:0/57044267 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6f4103d10 msgr2=0x7ff6f4104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:15.496 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.494+0000 7ff6fb505700 1 --2- 172.21.15.186:0/57044267 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6f4103d10 0x7ff6f4104120 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff6f000b0a0 tx=0x7ff6f000b3b0 comp rx=0 tx=0).stop 2024-05-09T04:42:15.496 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.494+0000 7ff6fb505700 1 -- 172.21.15.186:0/57044267 shutdown_connections 2024-05-09T04:42:15.496 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.494+0000 7ff6fb505700 1 --2- 172.21.15.186:0/57044267 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6f4103d10 0x7ff6f4104120 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:15.497 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.494+0000 7ff6fb505700 1 -- 172.21.15.186:0/57044267 >> 172.21.15.186:0/57044267 conn(0x7ff6f40ff350 msgr2=0x7ff6f4101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:42:15.497 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.495+0000 7ff6fb505700 1 -- 172.21.15.186:0/57044267 shutdown_connections 2024-05-09T04:42:15.497 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.495+0000 7ff6fb505700 1 -- 172.21.15.186:0/57044267 wait complete. 2024-05-09T04:42:15.497 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.495+0000 7ff6fb505700 1 Processor -- start 2024-05-09T04:42:15.497 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.495+0000 7ff6fb505700 1 -- start start 2024-05-09T04:42:15.497 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.495+0000 7ff6fb505700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6f4103d10 0x7ff6f41999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:42:15.497 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.495+0000 7ff6fb505700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6f4199ef0 0x7ff6f419ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:42:15.497 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.495+0000 7ff6fb505700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6f419f3c0 0x7ff6f41a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:42:15.497 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.495+0000 7ff6fb505700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f4109c90 con 0x7ff6f4199ef0 2024-05-09T04:42:15.497 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.495+0000 7ff6fb505700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f4109b10 con 0x7ff6f4103d10 2024-05-09T04:42:15.497 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.495+0000 7ff6fb505700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f4109e10 con 0x7ff6f419f3c0 2024-05-09T04:42:15.497 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.496+0000 7ff6f9aa2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6f419f3c0 0x7ff6f41a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:42:15.497 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.496+0000 7ff6f9aa2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6f419f3c0 0x7ff6f41a17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:59566/0 (socket says 172.21.15.186:59566) 2024-05-09T04:42:15.498 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.496+0000 7ff6f9aa2700 1 -- 172.21.15.186:0/2078523432 learned_addr learned my addr 172.21.15.186:0/2078523432 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-05-09T04:42:15.498 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.496+0000 7ff6f92a1700 1 --2- 172.21.15.186:0/2078523432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6f4103d10 0x7ff6f41999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:42:15.498 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.496+0000 7ff6f8aa0700 1 --2- 172.21.15.186:0/2078523432 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6f4199ef0 0x7ff6f419ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:42:15.498 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.496+0000 7ff6f9aa2700 1 -- 172.21.15.186:0/2078523432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6f4103d10 msgr2=0x7ff6f41999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:15.498 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.496+0000 7ff6f9aa2700 1 --2- 172.21.15.186:0/2078523432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6f4103d10 0x7ff6f41999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:15.498 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.496+0000 7ff6f9aa2700 1 -- 172.21.15.186:0/2078523432 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6f4199ef0 msgr2=0x7ff6f419ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:15.498 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.496+0000 7ff6f9aa2700 1 --2- 172.21.15.186:0/2078523432 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6f4199ef0 0x7ff6f419ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:15.498 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.496+0000 7ff6f9aa2700 1 -- 172.21.15.186:0/2078523432 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6f41a1d50 con 0x7ff6f419f3c0 2024-05-09T04:42:15.498 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.496+0000 7ff6f9aa2700 1 --2- 172.21.15.186:0/2078523432 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6f419f3c0 0x7ff6f41a17e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec00b550 tx=0x7ff6ec00b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:42:15.498 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.496+0000 7ff6ea7fc700 1 -- 172.21.15.186:0/2078523432 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec015020 con 0x7ff6f419f3c0 2024-05-09T04:42:15.498 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.496+0000 7ff6ea7fc700 1 -- 172.21.15.186:0/2078523432 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6ec011ce0 con 0x7ff6f419f3c0 2024-05-09T04:42:15.498 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.496+0000 7ff6fb505700 1 -- 172.21.15.186:0/2078523432 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6f41a1fe0 con 0x7ff6f419f3c0 2024-05-09T04:42:15.498 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.496+0000 7ff6ea7fc700 1 -- 172.21.15.186:0/2078523432 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec010db0 con 0x7ff6f419f3c0 2024-05-09T04:42:15.499 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.496+0000 7ff6fb505700 1 -- 172.21.15.186:0/2078523432 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6f41a24c0 con 0x7ff6f419f3c0 2024-05-09T04:42:15.499 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.497+0000 7ff6ea7fc700 1 -- 172.21.15.186:0/2078523432 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7ff6ec018020 con 0x7ff6f419f3c0 2024-05-09T04:42:15.500 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.497+0000 7ff6fb505700 1 -- 172.21.15.186:0/2078523432 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6f4064e20 con 0x7ff6f419f3c0 2024-05-09T04:42:15.500 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.498+0000 7ff6ea7fc700 1 --2- 172.21.15.186:0/2078523432 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff6e003e9a0 0x7ff6e0040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:42:15.500 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.498+0000 7ff6ea7fc700 1 -- 172.21.15.186:0/2078523432 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff6ec0520c0 con 0x7ff6f419f3c0 2024-05-09T04:42:15.500 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.498+0000 7ff6f92a1700 1 --2- 172.21.15.186:0/2078523432 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff6e003e9a0 0x7ff6e0040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:42:15.500 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.499+0000 7ff6f92a1700 1 --2- 172.21.15.186:0/2078523432 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff6e003e9a0 0x7ff6e0040eb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0000f80 tx=0x7ff6f0005040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:42:15.502 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.503+0000 7ff6ea7fc700 1 -- 172.21.15.186:0/2078523432 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff6ec01f020 con 0x7ff6f419f3c0 2024-05-09T04:42:15.720 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.721+0000 7ff6fb505700 1 -- 172.21.15.186:0/2078523432 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi006=x;smithi169=y;smithi186=z", "target": ["mon-mgr", ""]}) -- 0x7ff6f41011c0 con 0x7ff6e003e9a0 2024-05-09T04:42:15.736 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.737+0000 7ff6ea7fc700 1 -- 172.21.15.186:0/2078523432 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff6f41011c0 con 0x7ff6e003e9a0 2024-05-09T04:42:15.737 INFO:teuthology.orchestra.run.smithi186.stdout:Scheduled mgr update... 2024-05-09T04:42:15.739 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.739+0000 7ff6fb505700 1 -- 172.21.15.186:0/2078523432 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff6e003e9a0 msgr2=0x7ff6e0040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:15.739 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.739+0000 7ff6fb505700 1 --2- 172.21.15.186:0/2078523432 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff6e003e9a0 0x7ff6e0040eb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0000f80 tx=0x7ff6f0005040 comp rx=0 tx=0).stop 2024-05-09T04:42:15.739 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.739+0000 7ff6fb505700 1 -- 172.21.15.186:0/2078523432 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6f419f3c0 msgr2=0x7ff6f41a17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:15.739 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.739+0000 7ff6fb505700 1 --2- 172.21.15.186:0/2078523432 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6f419f3c0 0x7ff6f41a17e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec00b550 tx=0x7ff6ec00b860 comp rx=0 tx=0).stop 2024-05-09T04:42:15.739 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.739+0000 7ff6fb505700 1 -- 172.21.15.186:0/2078523432 shutdown_connections 2024-05-09T04:42:15.739 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.739+0000 7ff6fb505700 1 --2- 172.21.15.186:0/2078523432 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6f419f3c0 0x7ff6f41a17e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:15.740 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.739+0000 7ff6fb505700 1 --2- 172.21.15.186:0/2078523432 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff6e003e9a0 0x7ff6e0040eb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:15.740 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.739+0000 7ff6fb505700 1 --2- 172.21.15.186:0/2078523432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6f4103d10 0x7ff6f41999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:15.740 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.739+0000 7ff6fb505700 1 --2- 172.21.15.186:0/2078523432 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6f4199ef0 0x7ff6f419ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:15.740 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.739+0000 7ff6fb505700 1 -- 172.21.15.186:0/2078523432 >> 172.21.15.186:0/2078523432 conn(0x7ff6f40ff350 msgr2=0x7ff6f4100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:42:15.740 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.739+0000 7ff6fb505700 1 -- 172.21.15.186:0/2078523432 shutdown_connections 2024-05-09T04:42:15.740 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:42:15.739+0000 7ff6fb505700 1 -- 172.21.15.186:0/2078523432 wait complete. 2024-05-09T04:42:16.317 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:16.317 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:16.317 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:16.317 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:16.317 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:16.317 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:16.317 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:16 smithi186 ceph-mon[34668]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:16.317 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:16 smithi186 ceph-mon[34668]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi006=x;smithi169=y;smithi186=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:42:16.318 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:16 smithi186 ceph-mon[34668]: Saving service mgr spec with placement smithi006=x;smithi169=y;smithi186=z;count:3 2024-05-09T04:42:16.318 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:16.318 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:16.318 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:16.318 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:16.318 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:16.318 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T04:42:16.318 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T04:42:16.318 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T04:42:16.318 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:16.318 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:16 smithi186 ceph-mon[34668]: Deploying daemon mgr.z on smithi186 2024-05-09T04:42:16.347 DEBUG:teuthology.orchestra.run.smithi169:mgr.y> sudo journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mgr.y.service 2024-05-09T04:42:16.350 DEBUG:teuthology.orchestra.run.smithi186:mgr.z> sudo journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mgr.z.service 2024-05-09T04:42:16.353 INFO:tasks.cephadm:Deploying OSDs... 2024-05-09T04:42:16.353 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T04:42:16.353 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T04:42:16.413 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T04:42:16.413 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_1 2024-05-09T04:42:16.466 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T04:42:16.466 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T04:42:16.466 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-05-09T04:42:16.466 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T04:42:16.466 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T04:42:16.466 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2024-05-09 04:41:09.140412976 +0000 2024-05-09T04:42:16.466 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2024-05-09 04:35:21.765754495 +0000 2024-05-09T04:42:16.466 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2024-05-09 04:35:21.765754495 +0000 2024-05-09T04:42:16.467 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2024-05-09 04:35:21.765754495 +0000 2024-05-09T04:42:16.467 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T04:42:16.541 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:16.541 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:16.542 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:16.542 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:16.542 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:16.542 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:16.542 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:16 smithi006 ceph-mon[29835]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:16.542 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:16 smithi006 ceph-mon[29835]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi006=x;smithi169=y;smithi186=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:42:16.542 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:16 smithi006 ceph-mon[29835]: Saving service mgr spec with placement smithi006=x;smithi169=y;smithi186=z;count:3 2024-05-09T04:42:16.542 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:16.542 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:16.542 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:16.542 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:16.542 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:16.542 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T04:42:16.543 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T04:42:16.543 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T04:42:16.543 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:16.543 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:16 smithi006 ceph-mon[29835]: Deploying daemon mgr.z on smithi186 2024-05-09T04:42:16.551 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2024-05-09T04:42:16.551 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2024-05-09T04:42:16.551 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000391365 s, 1.3 MB/s 2024-05-09T04:42:16.555 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T04:42:16.615 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_2 2024-05-09T04:42:16.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:16.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:16.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:16.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:16.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:16.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:16.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:16 smithi169 ceph-mon[34075]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:16.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:16 smithi169 ceph-mon[34075]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi006=x;smithi169=y;smithi186=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:42:16.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:16 smithi169 ceph-mon[34075]: Saving service mgr spec with placement smithi006=x;smithi169=y;smithi186=z;count:3 2024-05-09T04:42:16.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:16.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:16.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:16.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:16.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:16.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T04:42:16.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T04:42:16.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T04:42:16.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:16.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:16 smithi169 ceph-mon[34075]: Deploying daemon mgr.z on smithi186 2024-05-09T04:42:16.668 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T04:42:16.668 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T04:42:16.668 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-05-09T04:42:16.668 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T04:42:16.668 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T04:42:16.668 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2024-05-09 04:41:09.216412123 +0000 2024-05-09T04:42:16.668 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2024-05-09 04:35:22.392747459 +0000 2024-05-09T04:42:16.668 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2024-05-09 04:35:22.392747459 +0000 2024-05-09T04:42:16.669 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2024-05-09 04:35:22.392747459 +0000 2024-05-09T04:42:16.669 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T04:42:16.755 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2024-05-09T04:42:16.755 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2024-05-09T04:42:16.755 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.00035356 s, 1.4 MB/s 2024-05-09T04:42:16.758 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T04:42:16.812 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_3 2024-05-09T04:42:16.861 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T04:42:16.861 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T04:42:16.861 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-05-09T04:42:16.862 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T04:42:16.862 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T04:42:16.862 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2024-05-09 04:41:09.300411179 +0000 2024-05-09T04:42:16.862 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2024-05-09 04:35:23.027740333 +0000 2024-05-09T04:42:16.862 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2024-05-09 04:35:23.027740333 +0000 2024-05-09T04:42:16.862 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2024-05-09 04:35:23.027740333 +0000 2024-05-09T04:42:16.862 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T04:42:16.946 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2024-05-09T04:42:16.947 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2024-05-09T04:42:16.947 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000372301 s, 1.4 MB/s 2024-05-09T04:42:16.951 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T04:42:17.015 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_4 2024-05-09T04:42:17.070 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T04:42:17.070 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T04:42:17.070 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-05-09T04:42:17.070 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T04:42:17.070 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T04:42:17.070 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2024-05-09 04:41:09.362410483 +0000 2024-05-09T04:42:17.070 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2024-05-09 04:35:23.635733511 +0000 2024-05-09T04:42:17.070 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2024-05-09 04:35:23.635733511 +0000 2024-05-09T04:42:17.070 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2024-05-09 04:35:23.635733511 +0000 2024-05-09T04:42:17.070 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T04:42:17.147 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2024-05-09T04:42:17.147 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2024-05-09T04:42:17.147 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000550445 s, 930 kB/s 2024-05-09T04:42:17.150 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T04:42:17.201 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-09T04:42:17.201 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T04:42:17.217 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T04:42:17.217 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_1 2024-05-09T04:42:17.274 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T04:42:17.274 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T04:42:17.274 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-09T04:42:17.274 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T04:42:17.274 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T04:42:17.274 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-05-09 04:41:59.853262606 +0000 2024-05-09T04:42:17.274 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-05-09 04:34:17.664403941 +0000 2024-05-09T04:42:17.274 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-05-09 04:34:17.664403941 +0000 2024-05-09T04:42:17.275 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-05-09 04:34:17.664403941 +0000 2024-05-09T04:42:17.275 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T04:42:17.336 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-05-09T04:42:17.337 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-05-09T04:42:17.337 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000271725 s, 1.9 MB/s 2024-05-09T04:42:17.338 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T04:42:17.394 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_2 2024-05-09T04:42:17.448 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T04:42:17.448 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T04:42:17.448 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-09T04:42:17.448 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T04:42:17.448 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T04:42:17.448 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-05-09 04:41:59.853262606 +0000 2024-05-09T04:42:17.448 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-05-09 04:34:18.016396290 +0000 2024-05-09T04:42:17.448 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-05-09 04:34:18.016396290 +0000 2024-05-09T04:42:17.449 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-05-09 04:34:18.016396290 +0000 2024-05-09T04:42:17.449 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T04:42:17.511 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-05-09T04:42:17.511 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-05-09T04:42:17.511 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000191797 s, 2.7 MB/s 2024-05-09T04:42:17.512 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T04:42:17.568 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_3 2024-05-09T04:42:17.623 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T04:42:17.623 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T04:42:17.623 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-09T04:42:17.623 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T04:42:17.623 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T04:42:17.623 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-05-09 04:41:59.854262585 +0000 2024-05-09T04:42:17.624 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-05-09 04:34:18.366388683 +0000 2024-05-09T04:42:17.624 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-05-09 04:34:18.366388683 +0000 2024-05-09T04:42:17.624 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-05-09 04:34:18.366388683 +0000 2024-05-09T04:42:17.624 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T04:42:17.686 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-05-09T04:42:17.686 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-05-09T04:42:17.686 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000162069 s, 3.2 MB/s 2024-05-09T04:42:17.687 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T04:42:17.743 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_4 2024-05-09T04:42:17.798 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T04:42:17.798 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T04:42:17.798 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-05-09T04:42:17.798 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T04:42:17.798 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T04:42:17.798 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-05-09 04:41:59.854262585 +0000 2024-05-09T04:42:17.799 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-05-09 04:34:18.721380967 +0000 2024-05-09T04:42:17.799 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-05-09 04:34:18.721380967 +0000 2024-05-09T04:42:17.799 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-05-09 04:34:18.721380967 +0000 2024-05-09T04:42:17.799 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T04:42:17.862 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-05-09T04:42:17.862 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-05-09T04:42:17.863 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000207264 s, 2.5 MB/s 2024-05-09T04:42:17.863 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T04:42:17.918 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-05-09T04:42:17.918 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T04:42:17.936 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T04:42:17.936 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_1 2024-05-09T04:42:17.991 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T04:42:17.992 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T04:42:17.992 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-09T04:42:17.992 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T04:42:17.992 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T04:42:17.992 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-05-09 04:42:17.751476862 +0000 2024-05-09T04:42:17.992 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-05-09 04:34:05.301852972 +0000 2024-05-09T04:42:17.992 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-05-09 04:34:05.301852972 +0000 2024-05-09T04:42:17.992 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2024-05-09 04:34:05.301852972 +0000 2024-05-09T04:42:17.992 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T04:42:18.028 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:42:17 smithi186 systemd[1]: Starting Ceph mgr.z for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T04:42:18.051 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-05-09T04:42:18.051 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-05-09T04:42:18.051 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000190938 s, 2.7 MB/s 2024-05-09T04:42:18.052 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T04:42:18.108 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_2 2024-05-09T04:42:18.163 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T04:42:18.164 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T04:42:18.164 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-09T04:42:18.164 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T04:42:18.164 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T04:42:18.164 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-05-09 04:42:17.752476843 +0000 2024-05-09T04:42:18.164 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-05-09 04:34:05.665845965 +0000 2024-05-09T04:42:18.164 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-05-09 04:34:05.665845965 +0000 2024-05-09T04:42:18.164 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2024-05-09 04:34:05.665845965 +0000 2024-05-09T04:42:18.164 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T04:42:18.254 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-05-09T04:42:18.255 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-05-09T04:42:18.255 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000218775 s, 2.3 MB/s 2024-05-09T04:42:18.259 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T04:42:18.296 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:42:18 smithi186 podman[36326]: 2024-05-09 04:42:17.931895891 +0000 UTC m=+0.019029867 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T04:42:18.296 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:42:18 smithi186 podman[36326]: 2024-05-09 04:42:18.067733737 +0000 UTC m=+0.154867700 container create 2681ae679c05200f527fc4a7aa2b9c66ca6055e77311b07abc1b30aa0ed4e6b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-09T04:42:18.296 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:42:18 smithi186 podman[36326]: 2024-05-09 04:42:18.297222623 +0000 UTC m=+0.384356586 container init 2681ae679c05200f527fc4a7aa2b9c66ca6055e77311b07abc1b30aa0ed4e6b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-z, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS) 2024-05-09T04:42:18.311 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_3 2024-05-09T04:42:18.366 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T04:42:18.366 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T04:42:18.366 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-05-09T04:42:18.366 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T04:42:18.366 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T04:42:18.366 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-05-09 04:42:17.752476843 +0000 2024-05-09T04:42:18.366 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-05-09 04:34:05.994839631 +0000 2024-05-09T04:42:18.366 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-05-09 04:34:05.994839631 +0000 2024-05-09T04:42:18.367 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2024-05-09 04:34:05.994839631 +0000 2024-05-09T04:42:18.367 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T04:42:18.431 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-05-09T04:42:18.431 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-05-09T04:42:18.431 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000233518 s, 2.2 MB/s 2024-05-09T04:42:18.433 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T04:42:18.489 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_4 2024-05-09T04:42:18.545 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T04:42:18.546 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T04:42:18.546 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-05-09T04:42:18.546 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T04:42:18.546 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T04:42:18.546 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-05-09 04:42:17.752476843 +0000 2024-05-09T04:42:18.546 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-05-09 04:34:06.333833106 +0000 2024-05-09T04:42:18.546 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-05-09 04:34:06.333833106 +0000 2024-05-09T04:42:18.546 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2024-05-09 04:34:06.333833106 +0000 2024-05-09T04:42:18.546 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T04:42:18.598 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:18 smithi186 ceph-mon[34668]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:18.599 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:42:18 smithi186 podman[36326]: 2024-05-09 04:42:18.299912231 +0000 UTC m=+0.387046205 container start 2681ae679c05200f527fc4a7aa2b9c66ca6055e77311b07abc1b30aa0ed4e6b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T04:42:18.599 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:42:18 smithi186 ceph-mgr[36449]: -- 172.21.15.186:0/3375930925 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d7f79e65a0 con 0x55d7fc770900 2024-05-09T04:42:18.599 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:42:18 smithi186 bash[36326]: 2681ae679c05200f527fc4a7aa2b9c66ca6055e77311b07abc1b30aa0ed4e6b3 2024-05-09T04:42:18.599 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:42:18 smithi186 systemd[1]: Started Ceph mgr.z for f74d813c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:42:18.599 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:42:18 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-z[36426]: 2024-05-09T04:42:18.473+0000 7fc9740bc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T04:42:18.609 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-05-09T04:42:18.609 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-05-09T04:42:18.609 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000219714 s, 2.3 MB/s 2024-05-09T04:42:18.610 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T04:42:18.666 INFO:tasks.cephadm:Deploying osd.0 on smithi006 with /dev/vg_nvme/lv_4... 2024-05-09T04:42:18.666 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-09T04:42:18.696 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:18 smithi006 ceph-mon[29835]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:18.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:18 smithi169 ceph-mon[34075]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:19.086 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:42:18 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-z[36426]: 2024-05-09T04:42:18.673+0000 7fc9740bc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T04:42:19.086 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:42:18 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-z[36426]: 2024-05-09T04:42:18.776+0000 7fc9740bc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T04:42:19.086 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:42:19 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-z[36426]: 2024-05-09T04:42:19.085+0000 7fc9740bc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T04:42:19.136 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:42:19.349 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:42:19 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-z[36426]: 2024-05-09T04:42:19.170+0000 7fc9740bc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T04:42:19.546 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:19 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:19.547 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:19 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:19.547 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:19 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:19.547 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:19 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T04:42:19.547 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:19 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T04:42:19.547 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:19 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T04:42:19.547 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:19 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:19.548 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:19 smithi006 ceph-mon[29835]: Deploying daemon mgr.y on smithi169 2024-05-09T04:42:19.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:19 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:19.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:19 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:19.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:19 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:19.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:19 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T04:42:19.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:19 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T04:42:19.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:19 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T04:42:19.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:19 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:19.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:19 smithi186 ceph-mon[34668]: Deploying daemon mgr.y on smithi169 2024-05-09T04:42:19.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:19 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:19.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:19 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:19.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:19 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:19.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:19 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T04:42:19.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:19 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T04:42:19.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:19 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T04:42:19.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:19 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:19.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:19 smithi169 ceph-mon[34075]: Deploying daemon mgr.y on smithi169 2024-05-09T04:42:20.685 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:20 smithi006 ceph-mon[29835]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:20.711 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:20 smithi186 ceph-mon[34668]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:20.711 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:42:20 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-z[36426]: 2024-05-09T04:42:20.366+0000 7fc9740bc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T04:42:20.711 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:42:20 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-z[36426]: 2024-05-09T04:42:20.537+0000 7fc9740bc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T04:42:20.711 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:42:20 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-z[36426]: 2024-05-09T04:42:20.711+0000 7fc9740bc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T04:42:20.748 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:20 smithi169 ceph-mon[34075]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:21.087 INFO:journalctl@ceph.mgr.y.smithi169.stdout:May 09 04:42:20 smithi169 systemd[1]: Started Ceph mgr.y for f74d813c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:42:21.295 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:42:20 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-z[36426]: 2024-05-09T04:42:20.998+0000 7fc9740bc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T04:42:21.295 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:42:21 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-z[36426]: 2024-05-09T04:42:21.091+0000 7fc9740bc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T04:42:21.295 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:42:21 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-z[36426]: 2024-05-09T04:42:21.295+0000 7fc9740bc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T04:42:21.366 INFO:journalctl@ceph.mgr.y.smithi169.stdout:May 09 04:42:21 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-y[35508]: 2024-05-09T04:42:21.087+0000 7f3e25773200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T04:42:21.366 INFO:journalctl@ceph.mgr.y.smithi169.stdout:May 09 04:42:21 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-y[35508]: 2024-05-09T04:42:21.274+0000 7f3e25773200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T04:42:21.599 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:42:21 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-z[36426]: 2024-05-09T04:42:21.494+0000 7fc9740bc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T04:42:21.706 INFO:journalctl@ceph.mgr.y.smithi169.stdout:May 09 04:42:21 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-y[35508]: 2024-05-09T04:42:21.377+0000 7f3e25773200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T04:42:21.706 INFO:journalctl@ceph.mgr.y.smithi169.stdout:May 09 04:42:21 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-y[35508]: 2024-05-09T04:42:21.706+0000 7f3e25773200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T04:42:21.953 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-09T04:42:21.953 INFO:teuthology.orchestra.run.smithi006.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-09T04:42:21.953 INFO:teuthology.orchestra.run.smithi006.stderr: stderr: 10+0 records in 2024-05-09T04:42:21.953 INFO:teuthology.orchestra.run.smithi006.stderr:10+0 records out 2024-05-09T04:42:21.954 INFO:teuthology.orchestra.run.smithi006.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.033146 s, 316 MB/s 2024-05-09T04:42:21.954 INFO:teuthology.orchestra.run.smithi006.stderr: stderr: 2024-05-09T04:42:21.954 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping successful for: 2024-05-09T04:42:22.049 INFO:journalctl@ceph.mgr.y.smithi169.stdout:May 09 04:42:21 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-y[35508]: 2024-05-09T04:42:21.796+0000 7f3e25773200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T04:42:22.349 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:42:22 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-z[36426]: 2024-05-09T04:42:22.135+0000 7fc9740bc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T04:42:22.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:22.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:22.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:22.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:22.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:22.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:22 smithi186 ceph-mon[34668]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:22.352 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:22.352 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:22.352 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:22.352 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:22.352 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:22.352 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:22 smithi006 ceph-mon[29835]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:22.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:22.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:22.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:22.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:22.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:22.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:22 smithi169 ceph-mon[34075]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:22.751 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi006:vg_nvme/lv_4 2024-05-09T04:42:22.807 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:42:22 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-z[36426]: 2024-05-09T04:42:22.370+0000 7fc9740bc200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T04:42:23.099 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:42:22 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-z[36426]: 2024-05-09T04:42:22.807+0000 7fc9740bc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T04:42:23.099 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:42:22 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-z[36426]: 2024-05-09T04:42:22.905+0000 7fc9740bc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T04:42:23.237 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:42:23.366 INFO:journalctl@ceph.mgr.y.smithi169.stdout:May 09 04:42:23 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-y[35508]: 2024-05-09T04:42:23.072+0000 7f3e25773200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T04:42:23.366 INFO:journalctl@ceph.mgr.y.smithi169.stdout:May 09 04:42:23 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-y[35508]: 2024-05-09T04:42:23.241+0000 7f3e25773200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T04:42:23.602 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:42:23 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-z[36426]: 2024-05-09T04:42:23.278+0000 7fc9740bc200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T04:42:23.697 INFO:journalctl@ceph.mgr.y.smithi169.stdout:May 09 04:42:23 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-y[35508]: 2024-05-09T04:42:23.412+0000 7f3e25773200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T04:42:23.697 INFO:journalctl@ceph.mgr.y.smithi169.stdout:May 09 04:42:23 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-y[35508]: 2024-05-09T04:42:23.697+0000 7f3e25773200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T04:42:23.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:23 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:23.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:23 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:23.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:23 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:23.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:23 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:23.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:23 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:23.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:23 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:23.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:23 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:23.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:23 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T04:42:23.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:23 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T04:42:23.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:23 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:23.968 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:42:23 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-z[36426]: 2024-05-09T04:42:23.633+0000 7fc9740bc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T04:42:23.968 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:42:23 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-z[36426]: 2024-05-09T04:42:23.725+0000 7fc9740bc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T04:42:23.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:23 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:23.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:23 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:23.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:23 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:23.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:23 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:23.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:23 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:23.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:23 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:23.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:23 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:23.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:23 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T04:42:23.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:23 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T04:42:23.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:23 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:23.995 INFO:journalctl@ceph.mgr.y.smithi169.stdout:May 09 04:42:23 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-y[35508]: 2024-05-09T04:42:23.790+0000 7f3e25773200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T04:42:23.996 INFO:journalctl@ceph.mgr.y.smithi169.stdout:May 09 04:42:23 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-y[35508]: 2024-05-09T04:42:23.995+0000 7f3e25773200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T04:42:23.996 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:23 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:23.996 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:23 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:23.996 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:23 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:23.996 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:23 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:23.996 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:23 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:23.997 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:23 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:23.997 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:23 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:23.997 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:23 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T04:42:23.997 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:23 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T04:42:23.997 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:23 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:24.349 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:42:23 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-z[36426]: 2024-05-09T04:42:23.968+0000 7fc9740bc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T04:42:24.349 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:42:24 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-z[36426]: 2024-05-09T04:42:24.101+0000 7fc9740bc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T04:42:24.349 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:42:24 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-z[36426]: 2024-05-09T04:42:24.195+0000 7fc9740bc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T04:42:24.366 INFO:journalctl@ceph.mgr.y.smithi169.stdout:May 09 04:42:24 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-y[35508]: 2024-05-09T04:42:24.186+0000 7f3e25773200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T04:42:24.796 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:42:24 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-z[36426]: 2024-05-09T04:42:24.523+0000 7fc9740bc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T04:42:24.796 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:42:24 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-z[36426]: 2024-05-09T04:42:24.614+0000 7fc9740bc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T04:42:25.064 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:24 smithi006 ceph-mon[29835]: Reconfiguring mgr.x (unknown last config time)... 2024-05-09T04:42:25.065 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:24 smithi006 ceph-mon[29835]: Reconfiguring daemon mgr.x on smithi006 2024-05-09T04:42:25.065 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:24 smithi006 ceph-mon[29835]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:25.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:24 smithi186 ceph-mon[34668]: Reconfiguring mgr.x (unknown last config time)... 2024-05-09T04:42:25.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:24 smithi186 ceph-mon[34668]: Reconfiguring daemon mgr.x on smithi006 2024-05-09T04:42:25.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:24 smithi186 ceph-mon[34668]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:25.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:24 smithi169 ceph-mon[34075]: Reconfiguring mgr.x (unknown last config time)... 2024-05-09T04:42:25.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:24 smithi169 ceph-mon[34075]: Reconfiguring daemon mgr.x on smithi006 2024-05-09T04:42:25.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:24 smithi169 ceph-mon[34075]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:25.116 INFO:journalctl@ceph.mgr.y.smithi169.stdout:May 09 04:42:24 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-y[35508]: 2024-05-09T04:42:24.794+0000 7f3e25773200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T04:42:25.116 INFO:journalctl@ceph.mgr.y.smithi169.stdout:May 09 04:42:25 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-y[35508]: 2024-05-09T04:42:25.016+0000 7f3e25773200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T04:42:25.599 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:42:25 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-z[36426]: 2024-05-09T04:42:25.216+0000 7fc9740bc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T04:42:25.599 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:42:25 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-z[36426]: 2024-05-09T04:42:25.429+0000 7fc9740bc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T04:42:25.810 INFO:journalctl@ceph.mgr.y.smithi169.stdout:May 09 04:42:25 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-y[35508]: 2024-05-09T04:42:25.437+0000 7f3e25773200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T04:42:25.810 INFO:journalctl@ceph.mgr.y.smithi169.stdout:May 09 04:42:25 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-y[35508]: 2024-05-09T04:42:25.535+0000 7f3e25773200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T04:42:25.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:25 smithi006 ceph-mon[29835]: from='mgr.? 172.21.15.186:0/55803628' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T04:42:25.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:25 smithi006 ceph-mon[29835]: from='mgr.? 172.21.15.186:0/55803628' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T04:42:25.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:25 smithi006 ceph-mon[29835]: Standby manager daemon z started 2024-05-09T04:42:25.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:25 smithi006 ceph-mon[29835]: from='mgr.? 172.21.15.186:0/55803628' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T04:42:25.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:25 smithi006 ceph-mon[29835]: from='mgr.? 172.21.15.186:0/55803628' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T04:42:25.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:25 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:26.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.527+0000 7f7ccbe00700 1 -- 172.21.15.6:0/2355018277 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7cc410a960 msgr2=0x7f7cc410ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:26.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.527+0000 7f7ccbe00700 1 --2- 172.21.15.6:0/2355018277 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7cc410a960 0x7f7cc410ad70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8009990 tx=0x7f7cb8009ca0 comp rx=0 tx=0).stop 2024-05-09T04:42:26.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.527+0000 7f7ccbe00700 1 -- 172.21.15.6:0/2355018277 shutdown_connections 2024-05-09T04:42:26.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.527+0000 7f7ccbe00700 1 --2- 172.21.15.6:0/2355018277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7cc410c430 0x7f7cc4112ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:26.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.527+0000 7f7ccbe00700 1 --2- 172.21.15.6:0/2355018277 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7cc410a960 0x7f7cc410ad70 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:26.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.527+0000 7f7ccbe00700 1 --2- 172.21.15.6:0/2355018277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cc410ba90 0x7f7cc410bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:26.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.527+0000 7f7ccbe00700 1 -- 172.21.15.6:0/2355018277 >> 172.21.15.6:0/2355018277 conn(0x7f7cc40799d0 msgr2=0x7f7cc41085e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:42:26.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.527+0000 7f7ccbe00700 1 -- 172.21.15.6:0/2355018277 shutdown_connections 2024-05-09T04:42:26.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.528+0000 7f7ccbe00700 1 -- 172.21.15.6:0/2355018277 wait complete. 2024-05-09T04:42:26.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.529+0000 7f7ccbe00700 1 Processor -- start 2024-05-09T04:42:26.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.529+0000 7f7ccbe00700 1 -- start start 2024-05-09T04:42:26.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.530+0000 7f7ccbe00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7cc410a960 0x7f7cc41a0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:42:26.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.530+0000 7f7ccbe00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cc410ba90 0x7f7cc41a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:42:26.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.530+0000 7f7ccbe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7cc410c430 0x7f7cc41a8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:42:26.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.530+0000 7f7ccbe00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc41163f0 con 0x7f7cc410c430 2024-05-09T04:42:26.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.530+0000 7f7ccbe00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc4116270 con 0x7f7cc410ba90 2024-05-09T04:42:26.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.530+0000 7f7ccbe00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc4116570 con 0x7f7cc410a960 2024-05-09T04:42:26.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.530+0000 7f7cc9400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7cc410a960 0x7f7cc41a0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:42:26.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.530+0000 7f7cc8a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cc410ba90 0x7f7cc41a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:42:26.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.530+0000 7f7cc9400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7cc410a960 0x7f7cc41a0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:53186/0 (socket says 172.21.15.6:53186) 2024-05-09T04:42:26.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.530+0000 7f7cc9400700 1 -- 172.21.15.6:0/3556487332 learned_addr learned my addr 172.21.15.6:0/3556487332 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:42:26.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.530+0000 7f7cc8a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cc410ba90 0x7f7cc41a1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:32842/0 (socket says 172.21.15.6:32842) 2024-05-09T04:42:26.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.530+0000 7f7cc9e00700 1 --2- 172.21.15.6:0/3556487332 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7cc410c430 0x7f7cc41a8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:42:26.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.530+0000 7f7cc8a00700 1 -- 172.21.15.6:0/3556487332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7cc410a960 msgr2=0x7f7cc41a0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:26.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.531+0000 7f7cc8a00700 1 --2- 172.21.15.6:0/3556487332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7cc410a960 0x7f7cc41a0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:26.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.531+0000 7f7cc8a00700 1 -- 172.21.15.6:0/3556487332 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7cc410c430 msgr2=0x7f7cc41a8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:26.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.531+0000 7f7cc8a00700 1 --2- 172.21.15.6:0/3556487332 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7cc410c430 0x7f7cc41a8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:26.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.531+0000 7f7cc8a00700 1 -- 172.21.15.6:0/3556487332 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc41a8aa0 con 0x7f7cc410ba90 2024-05-09T04:42:26.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.531+0000 7f7cc8a00700 1 --2- 172.21.15.6:0/3556487332 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cc410ba90 0x7f7cc41a1480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7cb000e9f0 tx=0x7f7cb000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:42:26.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.531+0000 7f7cc2000700 1 -- 172.21.15.6:0/3556487332 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb000c9c0 con 0x7f7cc410ba90 2024-05-09T04:42:26.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.531+0000 7f7cc2000700 1 -- 172.21.15.6:0/3556487332 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cb000cb60 con 0x7f7cc410ba90 2024-05-09T04:42:26.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.531+0000 7f7cc2000700 1 -- 172.21.15.6:0/3556487332 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0018680 con 0x7f7cc410ba90 2024-05-09T04:42:26.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.531+0000 7f7ccbe00700 1 -- 172.21.15.6:0/3556487332 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cc41a8d90 con 0x7f7cc410ba90 2024-05-09T04:42:26.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.531+0000 7f7ccbe00700 1 -- 172.21.15.6:0/3556487332 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cc41a9240 con 0x7f7cc410ba90 2024-05-09T04:42:26.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.533+0000 7f7ccbe00700 1 -- 172.21.15.6:0/3556487332 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cc40512c0 con 0x7f7cc410ba90 2024-05-09T04:42:26.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.533+0000 7f7cc2000700 1 -- 172.21.15.6:0/3556487332 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f7cb0010430 con 0x7f7cc410ba90 2024-05-09T04:42:26.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.533+0000 7f7cc2000700 1 --2- 172.21.15.6:0/3556487332 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f7cac03e750 0x7f7cac040c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:42:26.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.534+0000 7f7cc2000700 1 -- 172.21.15.6:0/3556487332 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7cb0052af0 con 0x7f7cc410ba90 2024-05-09T04:42:26.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.534+0000 7f7cc9400700 1 --2- 172.21.15.6:0/3556487332 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f7cac03e750 0x7f7cac040c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:42:26.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.534+0000 7f7cc9400700 1 --2- 172.21.15.6:0/3556487332 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f7cac03e750 0x7f7cac040c60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800b2e0 tx=0x7f7cb8009590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:42:26.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.539+0000 7f7cc2000700 1 -- 172.21.15.6:0/3556487332 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7cb0017e20 con 0x7f7cc410ba90 2024-05-09T04:42:26.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.793+0000 7f7ccbe00700 1 -- 172.21.15.6:0/3556487332 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f7cc4063c00 con 0x7f7cac03e750 2024-05-09T04:42:26.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:25.815+0000 7f7cc2000700 1 -- 172.21.15.6:0/3556487332 <== mon.2 v2:172.21.15.169:3300/0 7 ==== mgrmap(e 14) ==== 99936+0+0 (secure 0 0 0) 0x7f7cb0016d80 con 0x7f7cc410ba90 2024-05-09T04:42:26.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:25 smithi186 ceph-mon[34668]: from='mgr.? 172.21.15.186:0/55803628' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T04:42:26.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:25 smithi186 ceph-mon[34668]: from='mgr.? 172.21.15.186:0/55803628' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T04:42:26.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:25 smithi186 ceph-mon[34668]: Standby manager daemon z started 2024-05-09T04:42:26.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:25 smithi186 ceph-mon[34668]: from='mgr.? 172.21.15.186:0/55803628' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T04:42:26.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:25 smithi186 ceph-mon[34668]: from='mgr.? 172.21.15.186:0/55803628' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T04:42:26.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:25 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:26.116 INFO:journalctl@ceph.mgr.y.smithi169.stdout:May 09 04:42:25 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-y[35508]: 2024-05-09T04:42:25.910+0000 7f3e25773200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T04:42:26.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:25 smithi169 ceph-mon[34075]: from='mgr.? 172.21.15.186:0/55803628' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T04:42:26.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:25 smithi169 ceph-mon[34075]: from='mgr.? 172.21.15.186:0/55803628' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T04:42:26.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:25 smithi169 ceph-mon[34075]: Standby manager daemon z started 2024-05-09T04:42:26.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:25 smithi169 ceph-mon[34075]: from='mgr.? 172.21.15.186:0/55803628' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T04:42:26.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:25 smithi169 ceph-mon[34075]: from='mgr.? 172.21.15.186:0/55803628' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T04:42:26.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:25 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:26.615 INFO:journalctl@ceph.mgr.y.smithi169.stdout:May 09 04:42:26 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-y[35508]: 2024-05-09T04:42:26.264+0000 7f3e25773200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T04:42:26.615 INFO:journalctl@ceph.mgr.y.smithi169.stdout:May 09 04:42:26 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-y[35508]: 2024-05-09T04:42:26.359+0000 7f3e25773200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T04:42:26.866 INFO:journalctl@ceph.mgr.y.smithi169.stdout:May 09 04:42:26 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-y[35508]: 2024-05-09T04:42:26.615+0000 7f3e25773200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T04:42:26.866 INFO:journalctl@ceph.mgr.y.smithi169.stdout:May 09 04:42:26 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-y[35508]: 2024-05-09T04:42:26.755+0000 7f3e25773200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T04:42:26.866 INFO:journalctl@ceph.mgr.y.smithi169.stdout:May 09 04:42:26 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-y[35508]: 2024-05-09T04:42:26.858+0000 7f3e25773200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T04:42:26.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:26 smithi169 ceph-mon[34075]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:26.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:26 smithi169 ceph-mon[34075]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:42:26.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:26 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:42:26.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:26 smithi169 ceph-mon[34075]: mgrmap e14: x(active, since 90s), standbys: z 2024-05-09T04:42:26.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:26 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:42:26.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:26 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T04:42:26.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:26 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:26.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:26 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:26.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:26 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:26.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:26 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:27.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:26 smithi186 ceph-mon[34668]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:27.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:26 smithi186 ceph-mon[34668]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:42:27.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:26 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:42:27.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:26 smithi186 ceph-mon[34668]: mgrmap e14: x(active, since 90s), standbys: z 2024-05-09T04:42:27.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:26 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:42:27.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:26 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T04:42:27.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:26 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:27.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:26 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:27.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:26 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:27.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:26 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:27.167 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:26 smithi006 ceph-mon[29835]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:27.167 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:26 smithi006 ceph-mon[29835]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:42:27.167 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:26 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:42:27.167 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:26 smithi006 ceph-mon[29835]: mgrmap e14: x(active, since 90s), standbys: z 2024-05-09T04:42:27.167 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:26 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:42:27.167 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:26 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T04:42:27.168 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:26 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:27.168 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:26 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:27.168 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:26 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:27.168 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:26 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:27.616 INFO:journalctl@ceph.mgr.y.smithi169.stdout:May 09 04:42:27 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-y[35508]: 2024-05-09T04:42:27.211+0000 7f3e25773200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T04:42:27.616 INFO:journalctl@ceph.mgr.y.smithi169.stdout:May 09 04:42:27 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-y[35508]: 2024-05-09T04:42:27.303+0000 7f3e25773200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T04:42:27.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:27 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:27.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:27 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:27.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:27 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:28.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:27 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:28.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:27 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:28.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:27 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:28.116 INFO:journalctl@ceph.mgr.y.smithi169.stdout:May 09 04:42:27 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-y[35508]: 2024-05-09T04:42:27.918+0000 7f3e25773200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T04:42:28.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:27 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:28.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:27 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:28.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:27 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:28.616 INFO:journalctl@ceph.mgr.y.smithi169.stdout:May 09 04:42:28 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-y[35508]: 2024-05-09T04:42:28.135+0000 7f3e25773200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T04:42:28.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:28 smithi006 ceph-mon[29835]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:28.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:28 smithi006 ceph-mon[29835]: from='mgr.? 172.21.15.169:0/683084874' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T04:42:28.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:28 smithi006 ceph-mon[29835]: Standby manager daemon y started 2024-05-09T04:42:28.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:28 smithi006 ceph-mon[29835]: from='mgr.? 172.21.15.169:0/683084874' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T04:42:28.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:28 smithi006 ceph-mon[29835]: from='mgr.? 172.21.15.169:0/683084874' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T04:42:28.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:28 smithi006 ceph-mon[29835]: from='mgr.? 172.21.15.169:0/683084874' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T04:42:29.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:29.037+0000 7f7cc2000700 1 -- 172.21.15.6:0/3556487332 <== mon.2 v2:172.21.15.169:3300/0 8 ==== mgrmap(e 15) ==== 149595+0+0 (secure 0 0 0) 0x7f7cb001cd30 con 0x7f7cc410ba90 2024-05-09T04:42:29.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:28 smithi186 ceph-mon[34668]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:29.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:28 smithi186 ceph-mon[34668]: from='mgr.? 172.21.15.169:0/683084874' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T04:42:29.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:28 smithi186 ceph-mon[34668]: Standby manager daemon y started 2024-05-09T04:42:29.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:28 smithi186 ceph-mon[34668]: from='mgr.? 172.21.15.169:0/683084874' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T04:42:29.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:28 smithi186 ceph-mon[34668]: from='mgr.? 172.21.15.169:0/683084874' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T04:42:29.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:28 smithi186 ceph-mon[34668]: from='mgr.? 172.21.15.169:0/683084874' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T04:42:29.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:28 smithi169 ceph-mon[34075]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:29.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:28 smithi169 ceph-mon[34075]: from='mgr.? 172.21.15.169:0/683084874' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T04:42:29.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:28 smithi169 ceph-mon[34075]: Standby manager daemon y started 2024-05-09T04:42:29.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:28 smithi169 ceph-mon[34075]: from='mgr.? 172.21.15.169:0/683084874' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T04:42:29.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:28 smithi169 ceph-mon[34075]: from='mgr.? 172.21.15.169:0/683084874' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T04:42:29.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:28 smithi169 ceph-mon[34075]: from='mgr.? 172.21.15.169:0/683084874' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T04:42:30.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:30 smithi186 ceph-mon[34668]: mgrmap e15: x(active, since 93s), standbys: z, y 2024-05-09T04:42:30.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:30 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T04:42:30.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:30 smithi186 ceph-mon[34668]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:30.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:30 smithi169 ceph-mon[34075]: mgrmap e15: x(active, since 93s), standbys: z, y 2024-05-09T04:42:30.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:30 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T04:42:30.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:30 smithi169 ceph-mon[34075]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:30.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:30 smithi006 ceph-mon[29835]: mgrmap e15: x(active, since 93s), standbys: z, y 2024-05-09T04:42:30.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:30 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T04:42:30.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:30 smithi006 ceph-mon[29835]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:31.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:31 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/2082134326' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b732fd3-e648-4b0d-aece-083bdaaa5d68"}]: dispatch 2024-05-09T04:42:31.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:31 smithi006 ceph-mon[29835]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b732fd3-e648-4b0d-aece-083bdaaa5d68"}]: dispatch 2024-05-09T04:42:31.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:31 smithi006 ceph-mon[29835]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1b732fd3-e648-4b0d-aece-083bdaaa5d68"}]': finished 2024-05-09T04:42:31.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:31 smithi006 ceph-mon[29835]: osdmap e5: 1 total, 0 up, 1 in 2024-05-09T04:42:31.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:31 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:42:31.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:31 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/2082134326' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b732fd3-e648-4b0d-aece-083bdaaa5d68"}]: dispatch 2024-05-09T04:42:31.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:31 smithi186 ceph-mon[34668]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b732fd3-e648-4b0d-aece-083bdaaa5d68"}]: dispatch 2024-05-09T04:42:31.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:31 smithi186 ceph-mon[34668]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1b732fd3-e648-4b0d-aece-083bdaaa5d68"}]': finished 2024-05-09T04:42:31.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:31 smithi186 ceph-mon[34668]: osdmap e5: 1 total, 0 up, 1 in 2024-05-09T04:42:31.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:31 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:42:31.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:31 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/2082134326' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b732fd3-e648-4b0d-aece-083bdaaa5d68"}]: dispatch 2024-05-09T04:42:31.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:31 smithi169 ceph-mon[34075]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b732fd3-e648-4b0d-aece-083bdaaa5d68"}]: dispatch 2024-05-09T04:42:31.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:31 smithi169 ceph-mon[34075]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1b732fd3-e648-4b0d-aece-083bdaaa5d68"}]': finished 2024-05-09T04:42:31.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:31 smithi169 ceph-mon[34075]: osdmap e5: 1 total, 0 up, 1 in 2024-05-09T04:42:31.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:31 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:42:32.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:32 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/738302088' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-0/activate.monmap"}]: dispatch 2024-05-09T04:42:32.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:32 smithi186 ceph-mon[34668]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:32.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:32 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/738302088' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-0/activate.monmap"}]: dispatch 2024-05-09T04:42:32.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:32 smithi169 ceph-mon[34075]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:32.446 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:32 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/738302088' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-0/activate.monmap"}]: dispatch 2024-05-09T04:42:32.446 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:32 smithi006 ceph-mon[29835]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:34.765 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:34 smithi006 ceph-mon[29835]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:34.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:34 smithi186 ceph-mon[34668]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:34.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:34 smithi169 ceph-mon[34075]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:36.704 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:36 smithi006 ceph-mon[29835]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:36.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:36 smithi186 ceph-mon[34668]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:36.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:36 smithi169 ceph-mon[34075]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:38.696 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:38 smithi006 ceph-mon[29835]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:38.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:38 smithi186 ceph-mon[34668]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:38.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:38 smithi169 ceph-mon[34075]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:40.790 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:40 smithi006 ceph-mon[29835]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:40.790 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:40 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T04:42:40.790 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:40 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:40.790 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:40 smithi006 ceph-mon[29835]: Deploying daemon osd.0 on smithi006 2024-05-09T04:42:40.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:40 smithi186 ceph-mon[34668]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:40.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:40 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T04:42:40.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:40 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:40.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:40 smithi186 ceph-mon[34668]: Deploying daemon osd.0 on smithi006 2024-05-09T04:42:40.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:40 smithi169 ceph-mon[34075]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:40.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:40 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T04:42:40.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:40 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:40.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:40 smithi169 ceph-mon[34075]: Deploying daemon osd.0 on smithi006 2024-05-09T04:42:42.811 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:42 smithi006 ceph-mon[29835]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:42.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:42 smithi186 ceph-mon[34668]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:42.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:42 smithi169 ceph-mon[34075]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:44.581 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:44 smithi006 ceph-mon[29835]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:44.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:44 smithi186 ceph-mon[34668]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:44.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:44 smithi169 ceph-mon[34075]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:46.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:46 smithi186 ceph-mon[34668]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:46.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:46 smithi169 ceph-mon[34075]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:46.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:46 smithi006 ceph-mon[29835]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:48.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:48 smithi186 ceph-mon[34668]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:48.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:48 smithi169 ceph-mon[34075]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:48.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:48 smithi006 ceph-mon[29835]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:49.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:49 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:49.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:49 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:49.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:49 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:50.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:49 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:50.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:49 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:50.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:49 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:50.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:49 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:50.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:49 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:50.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:49 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:50.908 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:50 smithi006 ceph-mon[29835]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:51.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:50 smithi186 ceph-mon[34668]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:51.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:50 smithi169 ceph-mon[34075]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:52.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:52 smithi006 ceph-mon[29835]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:52.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:52.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:52.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:52.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:52.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:53.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:52 smithi186 ceph-mon[34668]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:53.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:53.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:53.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:53.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:53.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:53.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:52 smithi169 ceph-mon[34075]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:53.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:53.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:53.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:53.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:53.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:53.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:53 smithi006 ceph-mon[29835]: from='osd.0 [v2:172.21.15.6:6802/1738750914,v1:172.21.15.6:6803/1738750914]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T04:42:53.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:53 smithi006 ceph-mon[29835]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T04:42:54.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:53 smithi186 ceph-mon[34668]: from='osd.0 [v2:172.21.15.6:6802/1738750914,v1:172.21.15.6:6803/1738750914]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T04:42:54.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:53 smithi186 ceph-mon[34668]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T04:42:54.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:53 smithi169 ceph-mon[34075]: from='osd.0 [v2:172.21.15.6:6802/1738750914,v1:172.21.15.6:6803/1738750914]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T04:42:54.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:53 smithi169 ceph-mon[34075]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T04:42:54.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:54.316+0000 7f7cc2000700 1 -- 172.21.15.6:0/3556487332 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7cc4063c00 con 0x7f7cac03e750 2024-05-09T04:42:54.317 INFO:teuthology.orchestra.run.smithi006.stdout:Created osd(s) 0 on host 'smithi006' 2024-05-09T04:42:54.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:54.320+0000 7f7ccbe00700 1 -- 172.21.15.6:0/3556487332 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f7cac03e750 msgr2=0x7f7cac040c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:54.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:54.320+0000 7f7ccbe00700 1 --2- 172.21.15.6:0/3556487332 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f7cac03e750 0x7f7cac040c60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800b2e0 tx=0x7f7cb8009590 comp rx=0 tx=0).stop 2024-05-09T04:42:54.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:54.320+0000 7f7ccbe00700 1 -- 172.21.15.6:0/3556487332 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cc410ba90 msgr2=0x7f7cc41a1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:54.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:54.320+0000 7f7ccbe00700 1 --2- 172.21.15.6:0/3556487332 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cc410ba90 0x7f7cc41a1480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7cb000e9f0 tx=0x7f7cb000ed00 comp rx=0 tx=0).stop 2024-05-09T04:42:54.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:54.320+0000 7f7ccbe00700 1 -- 172.21.15.6:0/3556487332 shutdown_connections 2024-05-09T04:42:54.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:54.320+0000 7f7ccbe00700 1 --2- 172.21.15.6:0/3556487332 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7cc410c430 0x7f7cc41a8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:54.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:54.320+0000 7f7ccbe00700 1 --2- 172.21.15.6:0/3556487332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7cc410a960 0x7f7cc41a0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:54.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:54.320+0000 7f7ccbe00700 1 --2- 172.21.15.6:0/3556487332 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f7cac03e750 0x7f7cac040c60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:54.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:54.320+0000 7f7ccbe00700 1 --2- 172.21.15.6:0/3556487332 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cc410ba90 0x7f7cc41a1480 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:54.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:54.320+0000 7f7ccbe00700 1 -- 172.21.15.6:0/3556487332 >> 172.21.15.6:0/3556487332 conn(0x7f7cc40799d0 msgr2=0x7f7cc410ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:42:54.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:54.321+0000 7f7ccbe00700 1 -- 172.21.15.6:0/3556487332 shutdown_connections 2024-05-09T04:42:54.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:42:54.321+0000 7f7ccbe00700 1 -- 172.21.15.6:0/3556487332 wait complete. 2024-05-09T04:42:54.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:54 smithi006 ceph-mon[29835]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:54.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:54 smithi006 ceph-mon[29835]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-09T04:42:54.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:54 smithi006 ceph-mon[29835]: from='osd.0 [v2:172.21.15.6:6802/1738750914,v1:172.21.15.6:6803/1738750914]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-05-09T04:42:54.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:54 smithi006 ceph-mon[29835]: osdmap e6: 1 total, 0 up, 1 in 2024-05-09T04:42:54.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:54 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:42:54.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:54 smithi006 ceph-mon[29835]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-05-09T04:42:54.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:54 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:54.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:54 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:54.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:54 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:55.081 DEBUG:teuthology.orchestra.run.smithi006:osd.0> sudo journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.0.service 2024-05-09T04:42:55.086 INFO:tasks.cephadm:Deploying osd.1 on smithi169 with /dev/vg_nvme/lv_4... 2024-05-09T04:42:55.086 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-09T04:42:55.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:54 smithi186 ceph-mon[34668]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:55.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:54 smithi186 ceph-mon[34668]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-09T04:42:55.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:54 smithi186 ceph-mon[34668]: from='osd.0 [v2:172.21.15.6:6802/1738750914,v1:172.21.15.6:6803/1738750914]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-05-09T04:42:55.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:54 smithi186 ceph-mon[34668]: osdmap e6: 1 total, 0 up, 1 in 2024-05-09T04:42:55.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:54 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:42:55.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:54 smithi186 ceph-mon[34668]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-05-09T04:42:55.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:54 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:55.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:54 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:55.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:54 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:55.113 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:54 smithi169 ceph-mon[34075]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:55.113 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:54 smithi169 ceph-mon[34075]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-09T04:42:55.113 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:54 smithi169 ceph-mon[34075]: from='osd.0 [v2:172.21.15.6:6802/1738750914,v1:172.21.15.6:6803/1738750914]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-05-09T04:42:55.113 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:54 smithi169 ceph-mon[34075]: osdmap e6: 1 total, 0 up, 1 in 2024-05-09T04:42:55.113 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:54 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:42:55.113 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:54 smithi169 ceph-mon[34075]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-05-09T04:42:55.113 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:54 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:55.113 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:54 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:55.113 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:54 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:42:55.330 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T04:42:56.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:55 smithi186 ceph-mon[34668]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-05-09T04:42:56.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:55 smithi186 ceph-mon[34668]: osdmap e7: 1 total, 0 up, 1 in 2024-05-09T04:42:56.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:55 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:42:56.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:55 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:42:56.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:55 smithi169 ceph-mon[34075]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-05-09T04:42:56.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:55 smithi169 ceph-mon[34075]: osdmap e7: 1 total, 0 up, 1 in 2024-05-09T04:42:56.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:55 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:42:56.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:55 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:42:56.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:55 smithi006 ceph-mon[29835]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-05-09T04:42:56.232 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:55 smithi006 ceph-mon[29835]: osdmap e7: 1 total, 0 up, 1 in 2024-05-09T04:42:56.232 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:55 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:42:56.232 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:55 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:42:56.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:56 smithi006 ceph-mon[29835]: purged_snaps scrub starts 2024-05-09T04:42:56.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:56 smithi006 ceph-mon[29835]: purged_snaps scrub ok 2024-05-09T04:42:56.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:56 smithi006 ceph-mon[29835]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:56.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:56 smithi006 ceph-mon[29835]: osd.0 [v2:172.21.15.6:6802/1738750914,v1:172.21.15.6:6803/1738750914] boot 2024-05-09T04:42:56.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:56 smithi006 ceph-mon[29835]: osdmap e8: 1 total, 1 up, 1 in 2024-05-09T04:42:56.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:42:57.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:56 smithi186 ceph-mon[34668]: purged_snaps scrub starts 2024-05-09T04:42:57.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:56 smithi186 ceph-mon[34668]: purged_snaps scrub ok 2024-05-09T04:42:57.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:56 smithi186 ceph-mon[34668]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:57.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:56 smithi186 ceph-mon[34668]: osd.0 [v2:172.21.15.6:6802/1738750914,v1:172.21.15.6:6803/1738750914] boot 2024-05-09T04:42:57.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:56 smithi186 ceph-mon[34668]: osdmap e8: 1 total, 1 up, 1 in 2024-05-09T04:42:57.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:42:57.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:56 smithi169 ceph-mon[34075]: purged_snaps scrub starts 2024-05-09T04:42:57.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:56 smithi169 ceph-mon[34075]: purged_snaps scrub ok 2024-05-09T04:42:57.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:56 smithi169 ceph-mon[34075]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:57.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:56 smithi169 ceph-mon[34075]: osd.0 [v2:172.21.15.6:6802/1738750914,v1:172.21.15.6:6803/1738750914] boot 2024-05-09T04:42:57.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:56 smithi169 ceph-mon[34075]: osdmap e8: 1 total, 1 up, 1 in 2024-05-09T04:42:57.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:56 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:42:57.245 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-09T04:42:57.245 INFO:teuthology.orchestra.run.smithi169.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-09T04:42:57.246 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10+0 records in 2024-05-09T04:42:57.246 INFO:teuthology.orchestra.run.smithi169.stderr:10+0 records out 2024-05-09T04:42:57.246 INFO:teuthology.orchestra.run.smithi169.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0125714 s, 834 MB/s 2024-05-09T04:42:57.246 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 2024-05-09T04:42:57.246 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping successful for: 2024-05-09T04:42:57.863 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:57 smithi169 ceph-mon[34075]: osdmap e9: 1 total, 1 up, 1 in 2024-05-09T04:42:57.894 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_4 2024-05-09T04:42:58.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:57 smithi186 ceph-mon[34668]: osdmap e9: 1 total, 1 up, 1 in 2024-05-09T04:42:58.174 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T04:42:58.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:57 smithi006 ceph-mon[29835]: osdmap e9: 1 total, 1 up, 1 in 2024-05-09T04:42:58.969 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:42:58 smithi169 ceph-mon[34075]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:59.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:42:58 smithi186 ceph-mon[34668]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:59.153 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:42:58 smithi006 ceph-mon[29835]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:59.762 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.761+0000 7fe6815b1700 1 -- 172.21.15.169:0/799588429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe67c104cf0 msgr2=0x7fe67c10b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:59.762 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.761+0000 7fe6815b1700 1 --2- 172.21.15.169:0/799588429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe67c104cf0 0x7fe67c10b7c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe67400b0a0 tx=0x7fe67400b3b0 comp rx=0 tx=0).stop 2024-05-09T04:42:59.762 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.761+0000 7fe6815b1700 1 -- 172.21.15.169:0/799588429 shutdown_connections 2024-05-09T04:42:59.763 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.761+0000 7fe6815b1700 1 --2- 172.21.15.169:0/799588429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe67c104cf0 0x7fe67c10b7c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:59.763 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.761+0000 7fe6815b1700 1 --2- 172.21.15.169:0/799588429 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe67c1037f0 0x7fe67c103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:59.763 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.761+0000 7fe6815b1700 1 --2- 172.21.15.169:0/799588429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe67c1041d0 0x7fe67c104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:59.763 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.761+0000 7fe6815b1700 1 -- 172.21.15.169:0/799588429 >> 172.21.15.169:0/799588429 conn(0x7fe67c0fe020 msgr2=0x7fe67c1004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:42:59.763 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.761+0000 7fe6815b1700 1 -- 172.21.15.169:0/799588429 shutdown_connections 2024-05-09T04:42:59.763 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.761+0000 7fe6815b1700 1 -- 172.21.15.169:0/799588429 wait complete. 2024-05-09T04:42:59.764 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.762+0000 7fe6815b1700 1 Processor -- start 2024-05-09T04:42:59.764 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.762+0000 7fe6815b1700 1 -- start start 2024-05-09T04:42:59.764 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.762+0000 7fe6815b1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe67c1037f0 0x7fe67c1997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:42:59.764 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.762+0000 7fe6815b1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe67c1041d0 0x7fe67c199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:42:59.764 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.763+0000 7fe6815b1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe67c104cf0 0x7fe67c1a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:42:59.764 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.763+0000 7fe6815b1700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe67c10ed40 con 0x7fe67c1041d0 2024-05-09T04:42:59.765 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.763+0000 7fe6815b1700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe67c10ebc0 con 0x7fe67c104cf0 2024-05-09T04:42:59.765 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.763+0000 7fe6815b1700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe67c10eec0 con 0x7fe67c1037f0 2024-05-09T04:42:59.765 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.763+0000 7fe67b7fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe67c104cf0 0x7fe67c1a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:42:59.765 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.763+0000 7fe67b7fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe67c104cf0 0x7fe67c1a0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48418/0 (socket says 172.21.15.169:48418) 2024-05-09T04:42:59.765 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.763+0000 7fe67b7fe700 1 -- 172.21.15.169:0/3640191513 learned_addr learned my addr 172.21.15.169:0/3640191513 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-05-09T04:42:59.765 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.763+0000 7fe67affd700 1 --2- 172.21.15.169:0/3640191513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe67c1037f0 0x7fe67c1997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:42:59.765 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.763+0000 7fe67b7fe700 1 -- 172.21.15.169:0/3640191513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe67c1037f0 msgr2=0x7fe67c1997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:59.765 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.763+0000 7fe67b7fe700 1 --2- 172.21.15.169:0/3640191513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe67c1037f0 0x7fe67c1997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:59.765 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.763+0000 7fe6727fc700 1 --2- 172.21.15.169:0/3640191513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe67c1041d0 0x7fe67c199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:42:59.765 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.763+0000 7fe67b7fe700 1 -- 172.21.15.169:0/3640191513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe67c1041d0 msgr2=0x7fe67c199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:59.765 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.763+0000 7fe67b7fe700 1 --2- 172.21.15.169:0/3640191513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe67c1041d0 0x7fe67c199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:59.765 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.763+0000 7fe67b7fe700 1 -- 172.21.15.169:0/3640191513 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe67c1a1320 con 0x7fe67c104cf0 2024-05-09T04:42:59.765 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.763+0000 7fe67b7fe700 1 --2- 172.21.15.169:0/3640191513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe67c104cf0 0x7fe67c1a0ca0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe67400b070 tx=0x7fe674008ff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:42:59.766 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.765+0000 7fe678ff9700 1 -- 172.21.15.169:0/3640191513 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe674007c80 con 0x7fe67c104cf0 2024-05-09T04:42:59.766 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.765+0000 7fe678ff9700 1 -- 172.21.15.169:0/3640191513 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe67401a970 con 0x7fe67c104cf0 2024-05-09T04:42:59.766 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.765+0000 7fe678ff9700 1 -- 172.21.15.169:0/3640191513 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6740055b0 con 0x7fe67c104cf0 2024-05-09T04:42:59.766 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.765+0000 7fe6815b1700 1 -- 172.21.15.169:0/3640191513 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe67c102450 con 0x7fe67c104cf0 2024-05-09T04:42:59.766 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.765+0000 7fe6815b1700 1 -- 172.21.15.169:0/3640191513 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe67c102a00 con 0x7fe67c104cf0 2024-05-09T04:42:59.767 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.766+0000 7fe6815b1700 1 -- 172.21.15.169:0/3640191513 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe67c0512c0 con 0x7fe67c104cf0 2024-05-09T04:42:59.768 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.768+0000 7fe678ff9700 1 -- 172.21.15.169:0/3640191513 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 15) ==== 149595+0+0 (secure 0 0 0) 0x7fe67402b020 con 0x7fe67c104cf0 2024-05-09T04:42:59.769 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.768+0000 7fe678ff9700 1 --2- 172.21.15.169:0/3640191513 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe6680b2870 0x7fe6680b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:42:59.769 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.768+0000 7fe678ff9700 1 -- 172.21.15.169:0/3640191513 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fe67400f020 con 0x7fe67c104cf0 2024-05-09T04:42:59.769 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.769+0000 7fe67affd700 1 --2- 172.21.15.169:0/3640191513 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe6680b2870 0x7fe6680b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:42:59.771 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.770+0000 7fe67affd700 1 --2- 172.21.15.169:0/3640191513 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe6680b2870 0x7fe6680b4d80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe67c06c040 tx=0x7fe66c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:42:59.774 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.773+0000 7fe678ff9700 1 -- 172.21.15.169:0/3640191513 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe674060df0 con 0x7fe67c104cf0 2024-05-09T04:42:59.986 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:42:59.985+0000 7fe6815b1700 1 -- 172.21.15.169:0/3640191513 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe67c102cc0 con 0x7fe6680b2870 2024-05-09T04:43:00.942 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:00 smithi169 ceph-mon[34075]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:00.942 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:43:00.942 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:43:00.942 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:00.943 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:00.943 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:00.943 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:43:00.943 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:00.943 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:00.943 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:43:00.943 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:00 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:01.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:00 smithi186 ceph-mon[34668]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:01.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:00 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:43:01.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:00 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:43:01.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:00 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:01.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:00 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:01.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:00 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:01.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:00 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:43:01.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:00 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:01.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:00 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:01.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:00 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:43:01.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:00 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:01.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:00 smithi006 ceph-mon[29835]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:01.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:00 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:43:01.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:00 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:43:01.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:00 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:01.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:00 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:01.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:00 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:01.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:00 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:43:01.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:00 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:01.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:00 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:01.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:00 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:43:01.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:00 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:02.101 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:01 smithi186 ceph-mon[34668]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:43:02.102 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:01 smithi186 ceph-mon[34668]: Detected new or changed devices on smithi006 2024-05-09T04:43:02.102 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:01 smithi186 ceph-mon[34668]: Adjusting osd_memory_target on smithi006 to 17094M 2024-05-09T04:43:02.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:01 smithi169 ceph-mon[34075]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:43:02.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:01 smithi169 ceph-mon[34075]: Detected new or changed devices on smithi006 2024-05-09T04:43:02.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:01 smithi169 ceph-mon[34075]: Adjusting osd_memory_target on smithi006 to 17094M 2024-05-09T04:43:02.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:01 smithi006 ceph-mon[29835]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:43:02.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:01 smithi006 ceph-mon[29835]: Detected new or changed devices on smithi006 2024-05-09T04:43:02.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:01 smithi006 ceph-mon[29835]: Adjusting osd_memory_target on smithi006 to 17094M 2024-05-09T04:43:03.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:02 smithi186 ceph-mon[34668]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:03.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:02 smithi186 ceph-mon[34668]: from='client.? 172.21.15.169:0/3794875739' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9770c07d-4243-43bf-a0cd-9d2ac558556c"}]: dispatch 2024-05-09T04:43:03.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:02 smithi186 ceph-mon[34668]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9770c07d-4243-43bf-a0cd-9d2ac558556c"}]: dispatch 2024-05-09T04:43:03.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:02 smithi186 ceph-mon[34668]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9770c07d-4243-43bf-a0cd-9d2ac558556c"}]': finished 2024-05-09T04:43:03.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:02 smithi186 ceph-mon[34668]: osdmap e10: 2 total, 1 up, 2 in 2024-05-09T04:43:03.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:02 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:43:03.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:02 smithi169 ceph-mon[34075]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:03.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:02 smithi169 ceph-mon[34075]: from='client.? 172.21.15.169:0/3794875739' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9770c07d-4243-43bf-a0cd-9d2ac558556c"}]: dispatch 2024-05-09T04:43:03.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:02 smithi169 ceph-mon[34075]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9770c07d-4243-43bf-a0cd-9d2ac558556c"}]: dispatch 2024-05-09T04:43:03.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:02 smithi169 ceph-mon[34075]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9770c07d-4243-43bf-a0cd-9d2ac558556c"}]': finished 2024-05-09T04:43:03.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:02 smithi169 ceph-mon[34075]: osdmap e10: 2 total, 1 up, 2 in 2024-05-09T04:43:03.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:02 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:43:03.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:02 smithi006 ceph-mon[29835]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:03.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:02 smithi006 ceph-mon[29835]: from='client.? 172.21.15.169:0/3794875739' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9770c07d-4243-43bf-a0cd-9d2ac558556c"}]: dispatch 2024-05-09T04:43:03.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:02 smithi006 ceph-mon[29835]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9770c07d-4243-43bf-a0cd-9d2ac558556c"}]: dispatch 2024-05-09T04:43:03.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:02 smithi006 ceph-mon[29835]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9770c07d-4243-43bf-a0cd-9d2ac558556c"}]': finished 2024-05-09T04:43:03.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:02 smithi006 ceph-mon[29835]: osdmap e10: 2 total, 1 up, 2 in 2024-05-09T04:43:03.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:02 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:43:04.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:03 smithi186 ceph-mon[34668]: from='client.? 172.21.15.169:0/2115575737' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-1/activate.monmap"}]: dispatch 2024-05-09T04:43:04.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:03 smithi169 ceph-mon[34075]: from='client.? 172.21.15.169:0/2115575737' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-1/activate.monmap"}]: dispatch 2024-05-09T04:43:04.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:03 smithi006 ceph-mon[29835]: from='client.? 172.21.15.169:0/2115575737' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-1/activate.monmap"}]: dispatch 2024-05-09T04:43:05.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:04 smithi186 ceph-mon[34668]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:05.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:04 smithi169 ceph-mon[34075]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:05.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:04 smithi006 ceph-mon[29835]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:06.928 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:06 smithi169 ceph-mon[34075]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:07.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:06 smithi186 ceph-mon[34668]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:07.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:06 smithi006 ceph-mon[29835]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:09.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:08 smithi186 ceph-mon[34668]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:09.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:08 smithi169 ceph-mon[34075]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:09.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:08 smithi006 ceph-mon[29835]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:11.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:10 smithi186 ceph-mon[34668]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:11.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:10 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T04:43:11.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:10 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:11.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:10 smithi169 ceph-mon[34075]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:11.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:10 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T04:43:11.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:10 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:11.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:10 smithi006 ceph-mon[29835]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:11.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:10 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T04:43:11.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:10 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:12.045 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:11 smithi169 ceph-mon[34075]: Deploying daemon osd.1 on smithi169 2024-05-09T04:43:12.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:11 smithi186 ceph-mon[34668]: Deploying daemon osd.1 on smithi169 2024-05-09T04:43:12.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:11 smithi006 ceph-mon[29835]: Deploying daemon osd.1 on smithi169 2024-05-09T04:43:13.073 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:12 smithi169 ceph-mon[34075]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:13.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:12 smithi186 ceph-mon[34668]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:13.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:12 smithi006 ceph-mon[29835]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:15.048 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:14 smithi169 ceph-mon[34075]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:15.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:14 smithi186 ceph-mon[34668]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:15.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:14 smithi006 ceph-mon[29835]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:16.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:16.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:43:16.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:16.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:16 smithi186 ceph-mon[34668]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:16.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:16.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:43:16.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:16.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:16 smithi169 ceph-mon[34075]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:16.696 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:16.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:43:16.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:16.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:16 smithi006 ceph-mon[29835]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:18.624 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 1 on host 'smithi169' 2024-05-09T04:43:18.624 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:43:18.624+0000 7fe678ff9700 1 -- 172.21.15.169:0/3640191513 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe67c102cc0 con 0x7fe6680b2870 2024-05-09T04:43:18.626 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:43:18.626+0000 7fe6815b1700 1 -- 172.21.15.169:0/3640191513 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe6680b2870 msgr2=0x7fe6680b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:43:18.626 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:43:18.626+0000 7fe6815b1700 1 --2- 172.21.15.169:0/3640191513 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe6680b2870 0x7fe6680b4d80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe67c06c040 tx=0x7fe66c009190 comp rx=0 tx=0).stop 2024-05-09T04:43:18.626 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:43:18.626+0000 7fe6815b1700 1 -- 172.21.15.169:0/3640191513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe67c104cf0 msgr2=0x7fe67c1a0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:43:18.627 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:43:18.626+0000 7fe6815b1700 1 --2- 172.21.15.169:0/3640191513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe67c104cf0 0x7fe67c1a0ca0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe67400b070 tx=0x7fe674008ff0 comp rx=0 tx=0).stop 2024-05-09T04:43:18.627 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:43:18.626+0000 7fe6815b1700 1 -- 172.21.15.169:0/3640191513 shutdown_connections 2024-05-09T04:43:18.627 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:43:18.626+0000 7fe6815b1700 1 --2- 172.21.15.169:0/3640191513 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe6680b2870 0x7fe6680b4d80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:18.627 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:43:18.626+0000 7fe6815b1700 1 --2- 172.21.15.169:0/3640191513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe67c104cf0 0x7fe67c1a0ca0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:18.627 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:43:18.626+0000 7fe6815b1700 1 --2- 172.21.15.169:0/3640191513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe67c1037f0 0x7fe67c1997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:18.627 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:43:18.626+0000 7fe6815b1700 1 --2- 172.21.15.169:0/3640191513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe67c1041d0 0x7fe67c199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:18.627 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:43:18.626+0000 7fe6815b1700 1 -- 172.21.15.169:0/3640191513 >> 172.21.15.169:0/3640191513 conn(0x7fe67c0fe020 msgr2=0x7fe67c1077a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:43:18.627 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:43:18.627+0000 7fe6815b1700 1 -- 172.21.15.169:0/3640191513 shutdown_connections 2024-05-09T04:43:18.627 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:43:18.627+0000 7fe6815b1700 1 -- 172.21.15.169:0/3640191513 wait complete. 2024-05-09T04:43:18.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:18 smithi186 ceph-mon[34668]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:18.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:18 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:18.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:18 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:18.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:18 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:18.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:18 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:43:18.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:18 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:18.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:18 smithi169 ceph-mon[34075]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:18.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:18 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:18.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:18 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:18.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:18 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:18.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:18 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:43:18.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:18 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:18.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:18 smithi006 ceph-mon[29835]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:18.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:18 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:18.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:18 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:18.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:18 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:18.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:18 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:43:18.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:18 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:19.280 DEBUG:teuthology.orchestra.run.smithi169:osd.1> sudo journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.1.service 2024-05-09T04:43:19.286 INFO:tasks.cephadm:Deploying osd.2 on smithi186 with /dev/vg_nvme/lv_4... 2024-05-09T04:43:19.287 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-09T04:43:19.487 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:19 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:43:19.487 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:19 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:19.487 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:19 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:19.487 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:19 smithi169 ceph-mon[34075]: from='osd.1 [v2:172.21.15.169:6800/2552207737,v1:172.21.15.169:6801/2552207737]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T04:43:19.487 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:19 smithi169 ceph-mon[34075]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T04:43:19.534 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T04:43:19.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:19 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:43:19.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:19 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:19.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:19 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:19.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:19 smithi186 ceph-mon[34668]: from='osd.1 [v2:172.21.15.169:6800/2552207737,v1:172.21.15.169:6801/2552207737]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T04:43:19.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:19 smithi186 ceph-mon[34668]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T04:43:19.891 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:19 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:43:19.891 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:19 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:19.892 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:19 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:19.892 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:19 smithi006 ceph-mon[29835]: from='osd.1 [v2:172.21.15.169:6800/2552207737,v1:172.21.15.169:6801/2552207737]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T04:43:19.892 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:19 smithi006 ceph-mon[29835]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T04:43:20.770 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:20 smithi169 ceph-mon[34075]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:20.770 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:20 smithi169 ceph-mon[34075]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-09T04:43:20.770 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:20 smithi169 ceph-mon[34075]: from='osd.1 [v2:172.21.15.169:6800/2552207737,v1:172.21.15.169:6801/2552207737]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T04:43:20.770 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:20 smithi169 ceph-mon[34075]: osdmap e11: 2 total, 1 up, 2 in 2024-05-09T04:43:20.770 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:20 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:43:20.770 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:20 smithi169 ceph-mon[34075]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T04:43:20.771 INFO:journalctl@ceph.osd.1.smithi169.stdout:May 09 04:43:20 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-1[38703]: 2024-05-09T04:43:20.592+0000 7f2ec5672700 -1 osd.1 0 waiting for initial osdmap 2024-05-09T04:43:20.771 INFO:journalctl@ceph.osd.1.smithi169.stdout:May 09 04:43:20 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-1[38703]: 2024-05-09T04:43:20.600+0000 7f2ec1427700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T04:43:20.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:20 smithi186 ceph-mon[34668]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:20.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:20 smithi186 ceph-mon[34668]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-09T04:43:20.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:20 smithi186 ceph-mon[34668]: from='osd.1 [v2:172.21.15.169:6800/2552207737,v1:172.21.15.169:6801/2552207737]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T04:43:20.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:20 smithi186 ceph-mon[34668]: osdmap e11: 2 total, 1 up, 2 in 2024-05-09T04:43:20.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:20 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:43:20.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:20 smithi186 ceph-mon[34668]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T04:43:20.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:20 smithi006 ceph-mon[29835]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:20.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:20 smithi006 ceph-mon[29835]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-09T04:43:20.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:20 smithi006 ceph-mon[29835]: from='osd.1 [v2:172.21.15.169:6800/2552207737,v1:172.21.15.169:6801/2552207737]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T04:43:20.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:20 smithi006 ceph-mon[29835]: osdmap e11: 2 total, 1 up, 2 in 2024-05-09T04:43:20.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:20 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:43:20.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:20 smithi006 ceph-mon[29835]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T04:43:21.490 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-09T04:43:21.491 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-09T04:43:21.491 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2024-05-09T04:43:21.491 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2024-05-09T04:43:21.491 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115425 s, 908 MB/s 2024-05-09T04:43:21.491 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2024-05-09T04:43:21.824 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:21 smithi169 ceph-mon[34075]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-05-09T04:43:21.825 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:21 smithi169 ceph-mon[34075]: osdmap e12: 2 total, 1 up, 2 in 2024-05-09T04:43:21.825 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:21 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:43:21.825 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:21 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:43:21.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:21 smithi006 ceph-mon[29835]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-05-09T04:43:21.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:21 smithi006 ceph-mon[29835]: osdmap e12: 2 total, 1 up, 2 in 2024-05-09T04:43:21.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:21 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:43:21.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:21 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:43:22.005 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:21 smithi186 ceph-mon[34668]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-05-09T04:43:22.005 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:21 smithi186 ceph-mon[34668]: osdmap e12: 2 total, 1 up, 2 in 2024-05-09T04:43:22.005 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:21 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:43:22.005 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:21 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:43:22.033 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi186:vg_nvme/lv_4 2024-05-09T04:43:22.282 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T04:43:22.701 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:22 smithi186 ceph-mon[34668]: purged_snaps scrub starts 2024-05-09T04:43:22.701 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:22 smithi186 ceph-mon[34668]: purged_snaps scrub ok 2024-05-09T04:43:22.701 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:22 smithi186 ceph-mon[34668]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:22.701 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:43:22.701 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:22 smithi186 ceph-mon[34668]: osd.1 [v2:172.21.15.169:6800/2552207737,v1:172.21.15.169:6801/2552207737] boot 2024-05-09T04:43:22.701 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:22 smithi186 ceph-mon[34668]: osdmap e13: 2 total, 2 up, 2 in 2024-05-09T04:43:22.701 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:43:22.701 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:22.701 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:22.701 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:43:22.701 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:22.701 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:22.702 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:43:22.702 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:22.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:22 smithi169 ceph-mon[34075]: purged_snaps scrub starts 2024-05-09T04:43:22.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:22 smithi169 ceph-mon[34075]: purged_snaps scrub ok 2024-05-09T04:43:22.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:22 smithi169 ceph-mon[34075]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:22.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:43:22.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:22 smithi169 ceph-mon[34075]: osd.1 [v2:172.21.15.169:6800/2552207737,v1:172.21.15.169:6801/2552207737] boot 2024-05-09T04:43:22.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:22 smithi169 ceph-mon[34075]: osdmap e13: 2 total, 2 up, 2 in 2024-05-09T04:43:22.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:43:22.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:22.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:22.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:43:22.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:22.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:22.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:43:22.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:22.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:22 smithi006 ceph-mon[29835]: purged_snaps scrub starts 2024-05-09T04:43:22.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:22 smithi006 ceph-mon[29835]: purged_snaps scrub ok 2024-05-09T04:43:22.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:22 smithi006 ceph-mon[29835]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:43:22.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:43:22.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:22 smithi006 ceph-mon[29835]: osd.1 [v2:172.21.15.169:6800/2552207737,v1:172.21.15.169:6801/2552207737] boot 2024-05-09T04:43:22.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:22 smithi006 ceph-mon[29835]: osdmap e13: 2 total, 2 up, 2 in 2024-05-09T04:43:22.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:43:22.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:22.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:22.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:43:22.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:22.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:22.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:43:22.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:23.831 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.831+0000 7f1260eed700 1 -- 172.21.15.186:0/1748799581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f125c1077b0 msgr2=0x7f125c109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:43:23.831 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.831+0000 7f1260eed700 1 --2- 172.21.15.186:0/1748799581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f125c1077b0 0x7f125c109bd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1244009990 tx=0x7f1244009ca0 comp rx=0 tx=0).stop 2024-05-09T04:43:23.831 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.831+0000 7f1260eed700 1 -- 172.21.15.186:0/1748799581 shutdown_connections 2024-05-09T04:43:23.831 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.831+0000 7f1260eed700 1 --2- 172.21.15.186:0/1748799581 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f125c10a110 0x7f125c10c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:23.831 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.831+0000 7f1260eed700 1 --2- 172.21.15.186:0/1748799581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f125c104e50 0x7f125c107270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:23.831 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.831+0000 7f1260eed700 1 --2- 172.21.15.186:0/1748799581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f125c1077b0 0x7f125c109bd0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:23.831 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.831+0000 7f1260eed700 1 -- 172.21.15.186:0/1748799581 >> 172.21.15.186:0/1748799581 conn(0x7f125c0fe870 msgr2=0x7f125c100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:43:23.831 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.831+0000 7f1260eed700 1 -- 172.21.15.186:0/1748799581 shutdown_connections 2024-05-09T04:43:23.832 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.831+0000 7f1260eed700 1 -- 172.21.15.186:0/1748799581 wait complete. 2024-05-09T04:43:23.834 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.832+0000 7f1260eed700 1 Processor -- start 2024-05-09T04:43:23.834 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.832+0000 7f1260eed700 1 -- start start 2024-05-09T04:43:23.834 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.832+0000 7f1260eed700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f125c104e50 0x7f125c19a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:43:23.834 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.832+0000 7f1260eed700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f125c1077b0 0x7f125c19ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:43:23.834 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.832+0000 7f1260eed700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f125c10a110 0x7f125c1a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:43:23.834 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.832+0000 7f1260eed700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f125c10fe20 con 0x7f125c10a110 2024-05-09T04:43:23.834 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.833+0000 7f1260eed700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f125c10fca0 con 0x7f125c104e50 2024-05-09T04:43:23.834 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.833+0000 7f1260eed700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f125c10ffa0 con 0x7f125c1077b0 2024-05-09T04:43:23.834 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.833+0000 7f1251d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f125c1077b0 0x7f125c19ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:43:23.834 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.833+0000 7f1251d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f125c1077b0 0x7f125c19ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:52242/0 (socket says 172.21.15.186:52242) 2024-05-09T04:43:23.834 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.833+0000 7f1251d9b700 1 -- 172.21.15.186:0/596076177 learned_addr learned my addr 172.21.15.186:0/596076177 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-05-09T04:43:23.834 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.833+0000 7f125a59c700 1 --2- 172.21.15.186:0/596076177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f125c104e50 0x7f125c19a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:43:23.835 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.833+0000 7f1251d9b700 1 -- 172.21.15.186:0/596076177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f125c104e50 msgr2=0x7f125c19a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:43:23.835 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.833+0000 7f1251d9b700 1 --2- 172.21.15.186:0/596076177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f125c104e50 0x7f125c19a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:23.835 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.833+0000 7f1251d9b700 1 -- 172.21.15.186:0/596076177 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f125c10a110 msgr2=0x7f125c1a1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:43:23.835 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.833+0000 7f1251d9b700 1 --2- 172.21.15.186:0/596076177 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f125c10a110 0x7f125c1a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:23.835 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.833+0000 7f1251d9b700 1 -- 172.21.15.186:0/596076177 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f125c1a2410 con 0x7f125c1077b0 2024-05-09T04:43:23.835 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.833+0000 7f1251d9b700 1 --2- 172.21.15.186:0/596076177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f125c1077b0 0x7f125c19ae80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1244006010 tx=0x7f1244005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:43:23.835 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.833+0000 7f1253fff700 1 -- 172.21.15.186:0/596076177 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1244027070 con 0x7f125c1077b0 2024-05-09T04:43:23.835 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.833+0000 7f1253fff700 1 -- 172.21.15.186:0/596076177 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1244004100 con 0x7f125c1077b0 2024-05-09T04:43:23.835 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.833+0000 7f1253fff700 1 -- 172.21.15.186:0/596076177 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1244022de0 con 0x7f125c1077b0 2024-05-09T04:43:23.835 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.833+0000 7f1260eed700 1 -- 172.21.15.186:0/596076177 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f125c1a26a0 con 0x7f125c1077b0 2024-05-09T04:43:23.835 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.833+0000 7f1260eed700 1 -- 172.21.15.186:0/596076177 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f125c1a2b70 con 0x7f125c1077b0 2024-05-09T04:43:23.835 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.834+0000 7f1260eed700 1 -- 172.21.15.186:0/596076177 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f125c0512c0 con 0x7f125c1077b0 2024-05-09T04:43:23.836 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.836+0000 7f1253fff700 1 -- 172.21.15.186:0/596076177 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 15) ==== 149595+0+0 (secure 0 0 0) 0x7f124402a070 con 0x7f125c1077b0 2024-05-09T04:43:23.836 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.836+0000 7f1253fff700 1 --2- 172.21.15.186:0/596076177 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f12480b27f0 0x7f12480b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:43:23.836 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.836+0000 7f1253fff700 1 -- 172.21.15.186:0/596076177 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f12440e20e0 con 0x7f125c1077b0 2024-05-09T04:43:23.837 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.837+0000 7f125a59c700 1 --2- 172.21.15.186:0/596076177 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f12480b27f0 0x7f12480b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:43:23.837 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.838+0000 7f125a59c700 1 --2- 172.21.15.186:0/596076177 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f12480b27f0 0x7f12480b4d00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f124c005ea0 tx=0x7f124c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:43:23.840 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:23.840+0000 7f1253fff700 1 -- 172.21.15.186:0/596076177 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1244031080 con 0x7f125c1077b0 2024-05-09T04:43:23.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:23 smithi186 ceph-mon[34668]: Detected new or changed devices on smithi169 2024-05-09T04:43:23.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:23 smithi186 ceph-mon[34668]: Adjusting osd_memory_target on smithi169 to 17088M 2024-05-09T04:43:23.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:23 smithi186 ceph-mon[34668]: osdmap e14: 2 total, 2 up, 2 in 2024-05-09T04:43:23.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:23 smithi169 ceph-mon[34075]: Detected new or changed devices on smithi169 2024-05-09T04:43:23.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:23 smithi169 ceph-mon[34075]: Adjusting osd_memory_target on smithi169 to 17088M 2024-05-09T04:43:23.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:23 smithi169 ceph-mon[34075]: osdmap e14: 2 total, 2 up, 2 in 2024-05-09T04:43:23.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:23 smithi006 ceph-mon[29835]: Detected new or changed devices on smithi169 2024-05-09T04:43:23.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:23 smithi006 ceph-mon[29835]: Adjusting osd_memory_target on smithi169 to 17088M 2024-05-09T04:43:23.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:23 smithi006 ceph-mon[29835]: osdmap e14: 2 total, 2 up, 2 in 2024-05-09T04:43:24.049 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:24.050+0000 7f1260eed700 1 -- 172.21.15.186:0/596076177 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f125c063c00 con 0x7f12480b27f0 2024-05-09T04:43:24.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:24 smithi169 ceph-mon[34075]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:24.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:24 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:43:24.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:24 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:43:24.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:24 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:24.886 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:24 smithi186 ceph-mon[34668]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:24.886 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:24 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:43:24.886 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:24 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:43:24.886 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:24 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:24.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:24 smithi006 ceph-mon[29835]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:24.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:24 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:43:24.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:24 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:43:24.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:24 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:25.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:25 smithi186 ceph-mon[34668]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:43:25.893 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:25 smithi006 ceph-mon[29835]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:43:26.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:25 smithi169 ceph-mon[34075]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:43:26.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:26 smithi006 ceph-mon[29835]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:26.984 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:26 smithi186 ceph-mon[34668]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:27.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:26 smithi169 ceph-mon[34075]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:27.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:27 smithi006 ceph-mon[29835]: from='client.? 172.21.15.186:0/3026268700' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f50ed89-b2a8-415e-b78d-8510d98d9368"}]: dispatch 2024-05-09T04:43:27.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:27 smithi006 ceph-mon[29835]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f50ed89-b2a8-415e-b78d-8510d98d9368"}]: dispatch 2024-05-09T04:43:27.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:27 smithi006 ceph-mon[29835]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f50ed89-b2a8-415e-b78d-8510d98d9368"}]': finished 2024-05-09T04:43:27.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:27 smithi006 ceph-mon[29835]: osdmap e15: 3 total, 2 up, 3 in 2024-05-09T04:43:27.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:27 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:43:27.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:27 smithi006 ceph-mon[29835]: from='client.? 172.21.15.186:0/1461715856' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-2/activate.monmap"}]: dispatch 2024-05-09T04:43:28.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:27 smithi186 ceph-mon[34668]: from='client.? 172.21.15.186:0/3026268700' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f50ed89-b2a8-415e-b78d-8510d98d9368"}]: dispatch 2024-05-09T04:43:28.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:27 smithi186 ceph-mon[34668]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f50ed89-b2a8-415e-b78d-8510d98d9368"}]: dispatch 2024-05-09T04:43:28.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:27 smithi186 ceph-mon[34668]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f50ed89-b2a8-415e-b78d-8510d98d9368"}]': finished 2024-05-09T04:43:28.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:27 smithi186 ceph-mon[34668]: osdmap e15: 3 total, 2 up, 3 in 2024-05-09T04:43:28.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:27 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:43:28.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:27 smithi186 ceph-mon[34668]: from='client.? 172.21.15.186:0/1461715856' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-2/activate.monmap"}]: dispatch 2024-05-09T04:43:28.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:27 smithi169 ceph-mon[34075]: from='client.? 172.21.15.186:0/3026268700' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f50ed89-b2a8-415e-b78d-8510d98d9368"}]: dispatch 2024-05-09T04:43:28.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:27 smithi169 ceph-mon[34075]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f50ed89-b2a8-415e-b78d-8510d98d9368"}]: dispatch 2024-05-09T04:43:28.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:27 smithi169 ceph-mon[34075]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f50ed89-b2a8-415e-b78d-8510d98d9368"}]': finished 2024-05-09T04:43:28.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:27 smithi169 ceph-mon[34075]: osdmap e15: 3 total, 2 up, 3 in 2024-05-09T04:43:28.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:27 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:43:28.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:27 smithi169 ceph-mon[34075]: from='client.? 172.21.15.186:0/1461715856' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-2/activate.monmap"}]: dispatch 2024-05-09T04:43:28.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:28 smithi006 ceph-mon[29835]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:29.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:28 smithi186 ceph-mon[34668]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:29.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:28 smithi169 ceph-mon[34075]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:30.931 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:30 smithi186 ceph-mon[34668]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:30.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:30 smithi006 ceph-mon[29835]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:31.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:30 smithi169 ceph-mon[34075]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:32.762 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:32 smithi186 ceph-mon[34668]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:32.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:32 smithi006 ceph-mon[29835]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:33.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:32 smithi169 ceph-mon[34075]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:34.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:34 smithi186 ceph-mon[34668]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:34.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:34 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T04:43:34.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:34 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:34.722 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:34 smithi186 ceph-mon[34668]: Deploying daemon osd.2 on smithi186 2024-05-09T04:43:34.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:34 smithi006 ceph-mon[29835]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:34.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:34 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T04:43:34.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:34 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:34.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:34 smithi006 ceph-mon[29835]: Deploying daemon osd.2 on smithi186 2024-05-09T04:43:35.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:34 smithi169 ceph-mon[34075]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:35.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:34 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T04:43:35.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:34 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:35.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:34 smithi169 ceph-mon[34075]: Deploying daemon osd.2 on smithi186 2024-05-09T04:43:36.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:36 smithi186 ceph-mon[34668]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:36.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:36 smithi006 ceph-mon[29835]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:37.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:36 smithi169 ceph-mon[34075]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:38.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:38 smithi006 ceph-mon[29835]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:38.956 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:38 smithi186 ceph-mon[34668]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:39.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:38 smithi169 ceph-mon[34075]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:40.446 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:40 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:40.446 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:40 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:40.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:40 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:43:40.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:40 smithi006 ceph-mon[29835]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:40.475 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:40 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:40.476 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:40 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:40.476 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:40 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:43:40.476 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:40 smithi186 ceph-mon[34668]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:40.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:40 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:40.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:40 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:40.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:40 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:43:40.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:40 smithi169 ceph-mon[34075]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:42.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:42 smithi186 ceph-mon[34668]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:42.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:42 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:42.601 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:42 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:42.601 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:42 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:42.601 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:42 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:43:42.602 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:42 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:42.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:42 smithi169 ceph-mon[34075]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:42.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:42 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:42.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:42 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:42.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:42 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:42.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:42 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:43:42.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:42 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:42.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:42 smithi006 ceph-mon[29835]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:42.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:42 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:42.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:42 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:42.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:42 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:42.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:42 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:43:42.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:42 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:43.105 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 2 on host 'smithi186' 2024-05-09T04:43:43.105 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:43.104+0000 7f1253fff700 1 -- 172.21.15.186:0/596076177 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f125c063c00 con 0x7f12480b27f0 2024-05-09T04:43:43.106 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:43.107+0000 7f1260eed700 1 -- 172.21.15.186:0/596076177 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f12480b27f0 msgr2=0x7f12480b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:43:43.107 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:43.107+0000 7f1260eed700 1 --2- 172.21.15.186:0/596076177 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f12480b27f0 0x7f12480b4d00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f124c005ea0 tx=0x7f124c005e30 comp rx=0 tx=0).stop 2024-05-09T04:43:43.107 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:43.107+0000 7f1260eed700 1 -- 172.21.15.186:0/596076177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f125c1077b0 msgr2=0x7f125c19ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:43:43.107 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:43.107+0000 7f1260eed700 1 --2- 172.21.15.186:0/596076177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f125c1077b0 0x7f125c19ae80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1244006010 tx=0x7f1244005e80 comp rx=0 tx=0).stop 2024-05-09T04:43:43.107 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:43.107+0000 7f1260eed700 1 -- 172.21.15.186:0/596076177 shutdown_connections 2024-05-09T04:43:43.107 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:43.107+0000 7f1260eed700 1 --2- 172.21.15.186:0/596076177 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f125c10a110 0x7f125c1a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:43.107 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:43.107+0000 7f1260eed700 1 --2- 172.21.15.186:0/596076177 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f12480b27f0 0x7f12480b4d00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:43.107 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:43.107+0000 7f1260eed700 1 --2- 172.21.15.186:0/596076177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f125c104e50 0x7f125c19a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:43.107 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:43.107+0000 7f1260eed700 1 --2- 172.21.15.186:0/596076177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f125c1077b0 0x7f125c19ae80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:43.107 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:43.107+0000 7f1260eed700 1 -- 172.21.15.186:0/596076177 >> 172.21.15.186:0/596076177 conn(0x7f125c0fe870 msgr2=0x7f125c105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:43:43.107 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:43.107+0000 7f1260eed700 1 -- 172.21.15.186:0/596076177 shutdown_connections 2024-05-09T04:43:43.107 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:43:43.107+0000 7f1260eed700 1 -- 172.21.15.186:0/596076177 wait complete. 2024-05-09T04:43:43.773 DEBUG:teuthology.orchestra.run.smithi186:osd.2> sudo journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.2.service 2024-05-09T04:43:43.775 INFO:tasks.cephadm:Deploying osd.3 on smithi006 with /dev/vg_nvme/lv_3... 2024-05-09T04:43:43.775 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-09T04:43:44.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:44 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:44.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:44 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:43:44.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:44 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:44.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:44 smithi006 ceph-mon[29835]: from='osd.2 [v2:172.21.15.186:6800/231435105,v1:172.21.15.186:6801/231435105]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T04:43:44.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:44 smithi006 ceph-mon[29835]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T04:43:44.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:44 smithi006 ceph-mon[29835]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:44.235 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:43:44.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:44 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:44.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:44 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:43:44.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:44 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:44.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:44 smithi186 ceph-mon[34668]: from='osd.2 [v2:172.21.15.186:6800/231435105,v1:172.21.15.186:6801/231435105]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T04:43:44.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:44 smithi186 ceph-mon[34668]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T04:43:44.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:44 smithi186 ceph-mon[34668]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:44.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:44 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:44.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:44 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:43:44.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:44 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:44.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:44 smithi169 ceph-mon[34075]: from='osd.2 [v2:172.21.15.186:6800/231435105,v1:172.21.15.186:6801/231435105]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T04:43:44.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:44 smithi169 ceph-mon[34075]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T04:43:44.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:44 smithi169 ceph-mon[34075]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:45.313 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:45 smithi186 ceph-mon[34668]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-09T04:43:45.313 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:45 smithi186 ceph-mon[34668]: from='osd.2 [v2:172.21.15.186:6800/231435105,v1:172.21.15.186:6801/231435105]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-09T04:43:45.313 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:45 smithi186 ceph-mon[34668]: osdmap e16: 3 total, 2 up, 3 in 2024-05-09T04:43:45.313 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:45 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:43:45.313 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:45 smithi186 ceph-mon[34668]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-09T04:43:45.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:45 smithi169 ceph-mon[34075]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-09T04:43:45.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:45 smithi169 ceph-mon[34075]: from='osd.2 [v2:172.21.15.186:6800/231435105,v1:172.21.15.186:6801/231435105]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-09T04:43:45.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:45 smithi169 ceph-mon[34075]: osdmap e16: 3 total, 2 up, 3 in 2024-05-09T04:43:45.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:45 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:43:45.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:45 smithi169 ceph-mon[34075]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-09T04:43:45.406 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:45 smithi006 ceph-mon[29835]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-09T04:43:45.406 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:45 smithi006 ceph-mon[29835]: from='osd.2 [v2:172.21.15.186:6800/231435105,v1:172.21.15.186:6801/231435105]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-09T04:43:45.406 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:45 smithi006 ceph-mon[29835]: osdmap e16: 3 total, 2 up, 3 in 2024-05-09T04:43:45.406 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:45 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:43:45.406 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:45 smithi006 ceph-mon[29835]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-09T04:43:46.097 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 09 04:43:45 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-2[39854]: 2024-05-09T04:43:45.807+0000 7f8b7647a700 -1 osd.2 0 waiting for initial osdmap 2024-05-09T04:43:46.097 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 09 04:43:45 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-2[39854]: 2024-05-09T04:43:45.813+0000 7f8b6e227700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T04:43:46.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:46 smithi186 ceph-mon[34668]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-05-09T04:43:46.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:46 smithi186 ceph-mon[34668]: osdmap e17: 3 total, 2 up, 3 in 2024-05-09T04:43:46.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:46 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:43:46.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:46 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:43:46.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:46 smithi186 ceph-mon[34668]: pgmap v92: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:46.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:46 smithi186 ceph-mon[34668]: from='osd.2 ' entity='osd.2' 2024-05-09T04:43:46.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:46 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:43:46.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:46 smithi169 ceph-mon[34075]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-05-09T04:43:46.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:46 smithi169 ceph-mon[34075]: osdmap e17: 3 total, 2 up, 3 in 2024-05-09T04:43:46.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:46 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:43:46.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:46 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:43:46.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:46 smithi169 ceph-mon[34075]: pgmap v92: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:46.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:46 smithi169 ceph-mon[34075]: from='osd.2 ' entity='osd.2' 2024-05-09T04:43:46.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:46 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:43:46.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:46 smithi006 ceph-mon[29835]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-05-09T04:43:46.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:46 smithi006 ceph-mon[29835]: osdmap e17: 3 total, 2 up, 3 in 2024-05-09T04:43:46.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:46 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:43:46.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:46 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:43:46.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:46 smithi006 ceph-mon[29835]: pgmap v92: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:43:46.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:46 smithi006 ceph-mon[29835]: from='osd.2 ' entity='osd.2' 2024-05-09T04:43:46.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:46 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:43:47.102 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-09T04:43:47.102 INFO:teuthology.orchestra.run.smithi006.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-09T04:43:47.102 INFO:teuthology.orchestra.run.smithi006.stderr: stderr: 10+0 records in 2024-05-09T04:43:47.102 INFO:teuthology.orchestra.run.smithi006.stderr:10+0 records out 2024-05-09T04:43:47.102 INFO:teuthology.orchestra.run.smithi006.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0427823 s, 245 MB/s 2024-05-09T04:43:47.102 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping successful for: 2024-05-09T04:43:47.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:47 smithi186 ceph-mon[34668]: purged_snaps scrub starts 2024-05-09T04:43:47.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:47 smithi186 ceph-mon[34668]: purged_snaps scrub ok 2024-05-09T04:43:47.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:47 smithi186 ceph-mon[34668]: osd.2 [v2:172.21.15.186:6800/231435105,v1:172.21.15.186:6801/231435105] boot 2024-05-09T04:43:47.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:47 smithi186 ceph-mon[34668]: osdmap e18: 3 total, 3 up, 3 in 2024-05-09T04:43:47.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:47 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:43:47.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:47 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:47.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:47 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:47.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:47 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:43:47.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:47 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:47.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:47 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:47.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:47 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:43:47.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:47 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:47.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:47 smithi169 ceph-mon[34075]: purged_snaps scrub starts 2024-05-09T04:43:47.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:47 smithi169 ceph-mon[34075]: purged_snaps scrub ok 2024-05-09T04:43:47.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:47 smithi169 ceph-mon[34075]: osd.2 [v2:172.21.15.186:6800/231435105,v1:172.21.15.186:6801/231435105] boot 2024-05-09T04:43:47.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:47 smithi169 ceph-mon[34075]: osdmap e18: 3 total, 3 up, 3 in 2024-05-09T04:43:47.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:47 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:43:47.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:47 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:47.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:47 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:47.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:47 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:43:47.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:47 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:47.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:47 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:47.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:47 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:43:47.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:47 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:47.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:47 smithi006 ceph-mon[29835]: purged_snaps scrub starts 2024-05-09T04:43:47.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:47 smithi006 ceph-mon[29835]: purged_snaps scrub ok 2024-05-09T04:43:47.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:47 smithi006 ceph-mon[29835]: osd.2 [v2:172.21.15.186:6800/231435105,v1:172.21.15.186:6801/231435105] boot 2024-05-09T04:43:47.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:47 smithi006 ceph-mon[29835]: osdmap e18: 3 total, 3 up, 3 in 2024-05-09T04:43:47.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:47 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:43:47.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:47 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:47.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:47 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:47.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:47 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:43:47.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:47 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:47.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:47 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:47.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:47 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:43:47.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:47 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:47.942 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi006:vg_nvme/lv_3 2024-05-09T04:43:48.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:48 smithi169 ceph-mon[34075]: Detected new or changed devices on smithi186 2024-05-09T04:43:48.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:48 smithi169 ceph-mon[34075]: Adjusting osd_memory_target on smithi186 to 17088M 2024-05-09T04:43:48.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:48 smithi169 ceph-mon[34075]: pgmap v94: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:48.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:48 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-09T04:43:48.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:48 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-09T04:43:48.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:48 smithi169 ceph-mon[34075]: osdmap e19: 3 total, 3 up, 3 in 2024-05-09T04:43:48.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:48 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T04:43:48.403 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:43:48.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:48 smithi006 ceph-mon[29835]: Detected new or changed devices on smithi186 2024-05-09T04:43:48.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:48 smithi006 ceph-mon[29835]: Adjusting osd_memory_target on smithi186 to 17088M 2024-05-09T04:43:48.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:48 smithi006 ceph-mon[29835]: pgmap v94: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:48.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:48 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-09T04:43:48.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:48 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-09T04:43:48.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:48 smithi006 ceph-mon[29835]: osdmap e19: 3 total, 3 up, 3 in 2024-05-09T04:43:48.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:48 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T04:43:48.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:48 smithi186 ceph-mon[34668]: Detected new or changed devices on smithi186 2024-05-09T04:43:48.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:48 smithi186 ceph-mon[34668]: Adjusting osd_memory_target on smithi186 to 17088M 2024-05-09T04:43:48.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:48 smithi186 ceph-mon[34668]: pgmap v94: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:48.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:48 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-09T04:43:48.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:48 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-09T04:43:48.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:48 smithi186 ceph-mon[34668]: osdmap e19: 3 total, 3 up, 3 in 2024-05-09T04:43:48.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:48 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T04:43:50.079 INFO:journalctl@ceph.osd.0.smithi006.stdout:May 09 04:43:49 smithi006 sudo[46419]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T04:43:50.079 INFO:journalctl@ceph.osd.0.smithi006.stdout:May 09 04:43:49 smithi006 sudo[46419]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T04:43:50.079 INFO:journalctl@ceph.osd.0.smithi006.stdout:May 09 04:43:49 smithi006 sudo[46419]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T04:43:50.079 INFO:journalctl@ceph.osd.0.smithi006.stdout:May 09 04:43:49 smithi006 sudo[46419]: pam_unix(sudo:session): session closed for user root 2024-05-09T04:43:50.079 INFO:journalctl@ceph.osd.0.smithi006.stdout:May 09 04:43:49 smithi006 sudo[46422]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T04:43:50.079 INFO:journalctl@ceph.osd.0.smithi006.stdout:May 09 04:43:49 smithi006 sudo[46422]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T04:43:50.079 INFO:journalctl@ceph.osd.0.smithi006.stdout:May 09 04:43:49 smithi006 sudo[46422]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T04:43:50.079 INFO:journalctl@ceph.osd.0.smithi006.stdout:May 09 04:43:49 smithi006 sudo[46422]: pam_unix(sudo:session): session closed for user root 2024-05-09T04:43:50.166 INFO:journalctl@ceph.osd.1.smithi169.stdout:May 09 04:43:49 smithi169 sudo[42502]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T04:43:50.166 INFO:journalctl@ceph.osd.1.smithi169.stdout:May 09 04:43:49 smithi169 sudo[42502]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T04:43:50.166 INFO:journalctl@ceph.osd.1.smithi169.stdout:May 09 04:43:49 smithi169 sudo[42502]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T04:43:50.167 INFO:journalctl@ceph.osd.1.smithi169.stdout:May 09 04:43:49 smithi169 sudo[42502]: pam_unix(sudo:session): session closed for user root 2024-05-09T04:43:50.167 INFO:journalctl@ceph.osd.1.smithi169.stdout:May 09 04:43:49 smithi169 sudo[42505]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T04:43:50.167 INFO:journalctl@ceph.osd.1.smithi169.stdout:May 09 04:43:49 smithi169 sudo[42505]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T04:43:50.167 INFO:journalctl@ceph.osd.1.smithi169.stdout:May 09 04:43:49 smithi169 sudo[42505]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T04:43:50.167 INFO:journalctl@ceph.osd.1.smithi169.stdout:May 09 04:43:49 smithi169 sudo[42505]: pam_unix(sudo:session): session closed for user root 2024-05-09T04:43:50.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:50 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-09T04:43:50.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:50 smithi186 ceph-mon[34668]: osdmap e20: 3 total, 3 up, 3 in 2024-05-09T04:43:50.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:50 smithi186 ceph-mon[34668]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:50.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:50 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:50.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:50 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:50.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:50 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:50.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:50 smithi186 sudo[43766]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T04:43:50.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:50 smithi186 sudo[43766]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T04:43:50.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:50 smithi186 sudo[43766]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T04:43:50.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:50 smithi186 sudo[43766]: pam_unix(sudo:session): session closed for user root 2024-05-09T04:43:50.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:50 smithi186 sudo[43769]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T04:43:50.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:50 smithi186 sudo[43769]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T04:43:50.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:50 smithi186 sudo[43769]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T04:43:50.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:50 smithi186 sudo[43769]: pam_unix(sudo:session): session closed for user root 2024-05-09T04:43:50.350 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 09 04:43:49 smithi186 sudo[43760]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T04:43:50.350 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 09 04:43:49 smithi186 sudo[43760]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T04:43:50.350 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 09 04:43:49 smithi186 sudo[43760]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T04:43:50.351 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 09 04:43:49 smithi186 sudo[43760]: pam_unix(sudo:session): session closed for user root 2024-05-09T04:43:50.351 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 09 04:43:50 smithi186 sudo[43763]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T04:43:50.351 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 09 04:43:50 smithi186 sudo[43763]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T04:43:50.351 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 09 04:43:50 smithi186 sudo[43763]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T04:43:50.351 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 09 04:43:50 smithi186 sudo[43763]: pam_unix(sudo:session): session closed for user root 2024-05-09T04:43:50.402 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:50 smithi006 sudo[46445]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T04:43:50.403 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:50 smithi006 sudo[46445]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T04:43:50.403 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:50 smithi006 sudo[46445]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T04:43:50.403 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:50 smithi006 sudo[46445]: pam_unix(sudo:session): session closed for user root 2024-05-09T04:43:50.403 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:50 smithi006 sudo[46448]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T04:43:50.403 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:50 smithi006 sudo[46448]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T04:43:50.403 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:50 smithi006 sudo[46448]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T04:43:50.403 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:50 smithi006 sudo[46448]: pam_unix(sudo:session): session closed for user root 2024-05-09T04:43:50.403 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:50 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-09T04:43:50.403 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:50 smithi006 ceph-mon[29835]: osdmap e20: 3 total, 3 up, 3 in 2024-05-09T04:43:50.403 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:50 smithi006 ceph-mon[29835]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:50.403 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:50 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:50.403 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:50 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:50.403 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:50 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:50.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:50 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-09T04:43:50.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:50 smithi169 ceph-mon[34075]: osdmap e20: 3 total, 3 up, 3 in 2024-05-09T04:43:50.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:50 smithi169 ceph-mon[34075]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:50.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:50 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:50.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:50 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:50.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:50 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:43:50.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:50 smithi169 sudo[42508]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T04:43:50.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:50 smithi169 sudo[42508]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T04:43:50.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:50 smithi169 sudo[42508]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T04:43:50.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:50 smithi169 sudo[42508]: pam_unix(sudo:session): session closed for user root 2024-05-09T04:43:50.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:50 smithi169 sudo[42511]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T04:43:50.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:50 smithi169 sudo[42511]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T04:43:50.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:50 smithi169 sudo[42511]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T04:43:50.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:50 smithi169 sudo[42511]: pam_unix(sudo:session): session closed for user root 2024-05-09T04:43:50.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.672+0000 7f4200000700 1 -- 172.21.15.6:0/3805815111 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41f8104d10 msgr2=0x7f41f810b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:43:50.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.672+0000 7f4200000700 1 --2- 172.21.15.6:0/3805815111 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41f8104d10 0x7f41f810b5c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f41f000b0a0 tx=0x7f41f000b3b0 comp rx=0 tx=0).stop 2024-05-09T04:43:50.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.672+0000 7f4200000700 1 -- 172.21.15.6:0/3805815111 shutdown_connections 2024-05-09T04:43:50.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.672+0000 7f4200000700 1 --2- 172.21.15.6:0/3805815111 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41f8104d10 0x7f41f810b5c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:50.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.672+0000 7f4200000700 1 --2- 172.21.15.6:0/3805815111 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41f8103240 0x7f41f8103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:50.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.672+0000 7f4200000700 1 --2- 172.21.15.6:0/3805815111 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41f8104370 0x7f41f8104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:50.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.672+0000 7f4200000700 1 -- 172.21.15.6:0/3805815111 >> 172.21.15.6:0/3805815111 conn(0x7f41f80fe880 msgr2=0x7f41f8100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:43:50.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.672+0000 7f4200000700 1 -- 172.21.15.6:0/3805815111 shutdown_connections 2024-05-09T04:43:50.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.672+0000 7f4200000700 1 -- 172.21.15.6:0/3805815111 wait complete. 2024-05-09T04:43:50.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.674+0000 7f4200000700 1 Processor -- start 2024-05-09T04:43:50.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.674+0000 7f4200000700 1 -- start start 2024-05-09T04:43:50.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.674+0000 7f4200000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41f8103240 0x7f41f819cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:43:50.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.674+0000 7f4200000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41f8104370 0x7f41f819d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:43:50.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.674+0000 7f4200000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41f8104d10 0x7f41f81a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:43:50.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.674+0000 7f4200000700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f41f810ecf0 con 0x7f41f8103240 2024-05-09T04:43:50.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.675+0000 7f4200000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f41f810eb70 con 0x7f41f8104d10 2024-05-09T04:43:50.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.675+0000 7f4200000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f41f810ee70 con 0x7f41f8104370 2024-05-09T04:43:50.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.675+0000 7f41fcc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41f8104370 0x7f41f819d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:43:50.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.675+0000 7f41fd600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41f8103240 0x7f41f819cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:43:50.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.675+0000 7f41fcc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41f8104370 0x7f41f819d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:54696/0 (socket says 172.21.15.6:54696) 2024-05-09T04:43:50.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.675+0000 7f41fd600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41f8103240 0x7f41f819cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38410/0 (socket says 172.21.15.6:38410) 2024-05-09T04:43:50.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.675+0000 7f41fcc00700 1 -- 172.21.15.6:0/3496638564 learned_addr learned my addr 172.21.15.6:0/3496638564 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:43:50.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.675+0000 7f41fe000700 1 --2- 172.21.15.6:0/3496638564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41f8104d10 0x7f41f81a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:43:50.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.675+0000 7f41fd600700 1 -- 172.21.15.6:0/3496638564 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41f8104370 msgr2=0x7f41f819d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:43:50.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.675+0000 7f41fd600700 1 --2- 172.21.15.6:0/3496638564 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41f8104370 0x7f41f819d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:50.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.675+0000 7f41fd600700 1 -- 172.21.15.6:0/3496638564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41f8104d10 msgr2=0x7f41f81a3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:43:50.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.675+0000 7f41fd600700 1 --2- 172.21.15.6:0/3496638564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41f8104d10 0x7f41f81a3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:50.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.675+0000 7f41fd600700 1 -- 172.21.15.6:0/3496638564 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f81a4640 con 0x7f41f8103240 2024-05-09T04:43:50.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.676+0000 7f41fd600700 1 --2- 172.21.15.6:0/3496638564 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41f8103240 0x7f41f819cae0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f41f8102f40 tx=0x7f41ec00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:43:50.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.676+0000 7f41f6000700 1 -- 172.21.15.6:0/3496638564 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec004220 con 0x7f41f8103240 2024-05-09T04:43:50.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.676+0000 7f4200000700 1 -- 172.21.15.6:0/3496638564 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41f81a4930 con 0x7f41f8103240 2024-05-09T04:43:50.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.676+0000 7f41f6000700 1 -- 172.21.15.6:0/3496638564 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41ec0043c0 con 0x7f41f8103240 2024-05-09T04:43:50.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.676+0000 7f41f6000700 1 -- 172.21.15.6:0/3496638564 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec003be0 con 0x7f41f8103240 2024-05-09T04:43:50.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.676+0000 7f4200000700 1 -- 172.21.15.6:0/3496638564 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41f81a4e70 con 0x7f41f8103240 2024-05-09T04:43:50.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.678+0000 7f4200000700 1 -- 172.21.15.6:0/3496638564 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41f80512c0 con 0x7f41f8103240 2024-05-09T04:43:50.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.680+0000 7f41f6000700 1 -- 172.21.15.6:0/3496638564 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 15) ==== 149595+0+0 (secure 0 0 0) 0x7f41ec010700 con 0x7f41f8103240 2024-05-09T04:43:50.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.684+0000 7f41f6000700 1 --2- 172.21.15.6:0/3496638564 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f41e00b27f0 0x7f41e00b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:43:50.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.684+0000 7f41f6000700 1 -- 172.21.15.6:0/3496638564 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7f41ec0e1280 con 0x7f41f8103240 2024-05-09T04:43:50.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.684+0000 7f41fcc00700 1 --2- 172.21.15.6:0/3496638564 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f41e00b27f0 0x7f41e00b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:43:50.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.685+0000 7f41fcc00700 1 --2- 172.21.15.6:0/3496638564 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f41e00b27f0 0x7f41e00b4d00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f41f819db90 tx=0x7f41e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:43:50.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.688+0000 7f41f6000700 1 -- 172.21.15.6:0/3496638564 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f41ec0aa440 con 0x7f41f8103240 2024-05-09T04:43:50.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:50.937+0000 7f4200000700 1 -- 172.21.15.6:0/3496638564 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f41f81a5160 con 0x7f41e00b27f0 2024-05-09T04:43:51.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:51 smithi006 ceph-mon[29835]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T04:43:51.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:51 smithi006 ceph-mon[29835]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T04:43:51.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:51 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:43:51.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:51 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:43:51.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:51 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:43:51.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:51 smithi006 ceph-mon[29835]: osdmap e21: 3 total, 3 up, 3 in 2024-05-09T04:43:51.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:51 smithi006 ceph-mon[29835]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T04:43:51.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:51 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:43:51.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:51 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:43:51.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:51 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:43:51.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:51 smithi006 ceph-mon[29835]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T04:43:51.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:51 smithi006 ceph-mon[29835]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T04:43:51.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:51 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:43:51.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:51 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:43:51.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:51 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:43:51.199 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:51 smithi006 ceph-mon[29835]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T04:43:51.199 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:51 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:43:51.199 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:51 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:43:51.199 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:51 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:51.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:51 smithi186 ceph-mon[34668]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T04:43:51.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:51 smithi186 ceph-mon[34668]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T04:43:51.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:51 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:43:51.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:51 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:43:51.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:51 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:43:51.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:51 smithi186 ceph-mon[34668]: osdmap e21: 3 total, 3 up, 3 in 2024-05-09T04:43:51.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:51 smithi186 ceph-mon[34668]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T04:43:51.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:51 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:43:51.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:51 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:43:51.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:51 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:43:51.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:51 smithi186 ceph-mon[34668]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T04:43:51.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:51 smithi186 ceph-mon[34668]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T04:43:51.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:51 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:43:51.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:51 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:43:51.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:51 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:43:51.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:51 smithi186 ceph-mon[34668]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T04:43:51.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:51 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:43:51.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:51 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:43:51.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:51 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:51.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:51 smithi169 ceph-mon[34075]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T04:43:51.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:51 smithi169 ceph-mon[34075]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T04:43:51.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:51 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:43:51.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:51 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:43:51.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:51 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:43:51.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:51 smithi169 ceph-mon[34075]: osdmap e21: 3 total, 3 up, 3 in 2024-05-09T04:43:51.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:51 smithi169 ceph-mon[34075]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T04:43:51.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:51 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:43:51.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:51 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:43:51.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:51 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:43:51.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:51 smithi169 ceph-mon[34075]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T04:43:51.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:51 smithi169 ceph-mon[34075]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T04:43:51.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:51 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:43:51.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:51 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:43:51.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:51 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:43:51.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:51 smithi169 ceph-mon[34075]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T04:43:51.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:51 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:43:51.618 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:51 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:43:51.618 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:51 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:52.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:43:52.194+0000 7f41f6000700 1 -- 172.21.15.6:0/3496638564 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f41ec0a5fd0 con 0x7f41f8103240 2024-05-09T04:43:52.300 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:52 smithi006 ceph-mon[29835]: from='client.14247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:43:52.300 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:52 smithi006 ceph-mon[29835]: pgmap v99: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:52.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:52 smithi186 ceph-mon[34668]: from='client.14247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:43:52.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:52 smithi186 ceph-mon[34668]: pgmap v99: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:52.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:52 smithi169 ceph-mon[34075]: from='client.14247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:43:52.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:52 smithi169 ceph-mon[34075]: pgmap v99: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:53.324 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:53 smithi006 ceph-mon[29835]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-09T04:43:53.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:53 smithi186 ceph-mon[34668]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-09T04:43:53.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:53 smithi169 ceph-mon[34075]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-09T04:43:54.446 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:54 smithi006 ceph-mon[29835]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:54.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:54 smithi186 ceph-mon[34668]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:54.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:54 smithi169 ceph-mon[34075]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:56.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:56 smithi006 ceph-mon[29835]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:56.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:56 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1523717310' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "afcbc36b-5669-4044-8dc5-2ebfe7970774"}]: dispatch 2024-05-09T04:43:56.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:56 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1523717310' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "afcbc36b-5669-4044-8dc5-2ebfe7970774"}]': finished 2024-05-09T04:43:56.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:56 smithi006 ceph-mon[29835]: osdmap e22: 4 total, 3 up, 4 in 2024-05-09T04:43:56.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:43:56.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:56 smithi186 ceph-mon[34668]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:56.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:56 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/1523717310' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "afcbc36b-5669-4044-8dc5-2ebfe7970774"}]: dispatch 2024-05-09T04:43:56.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:56 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/1523717310' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "afcbc36b-5669-4044-8dc5-2ebfe7970774"}]': finished 2024-05-09T04:43:56.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:56 smithi186 ceph-mon[34668]: osdmap e22: 4 total, 3 up, 4 in 2024-05-09T04:43:56.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:43:56.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:56 smithi169 ceph-mon[34075]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:56.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:56 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/1523717310' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "afcbc36b-5669-4044-8dc5-2ebfe7970774"}]: dispatch 2024-05-09T04:43:56.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:56 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/1523717310' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "afcbc36b-5669-4044-8dc5-2ebfe7970774"}]': finished 2024-05-09T04:43:56.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:56 smithi169 ceph-mon[34075]: osdmap e22: 4 total, 3 up, 4 in 2024-05-09T04:43:56.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:56 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:43:57.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:57 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/1934854212' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-3/activate.monmap"}]: dispatch 2024-05-09T04:43:57.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:57 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/1934854212' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-3/activate.monmap"}]: dispatch 2024-05-09T04:43:57.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:57 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1934854212' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-3/activate.monmap"}]: dispatch 2024-05-09T04:43:58.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:43:58 smithi186 ceph-mon[34668]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:58.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:43:58 smithi169 ceph-mon[34075]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:58.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:43:58 smithi006 ceph-mon[29835]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:00.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:00 smithi186 ceph-mon[34668]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:00.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:00 smithi169 ceph-mon[34075]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:00.941 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:00 smithi006 ceph-mon[29835]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:02.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:02 smithi006 ceph-mon[29835]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:02.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:02 smithi186 ceph-mon[34668]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:02.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:02 smithi169 ceph-mon[34075]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:04.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:04 smithi186 ceph-mon[34668]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:04.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:04 smithi169 ceph-mon[34075]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:04.927 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:04 smithi006 ceph-mon[29835]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:05.696 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:05 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T04:44:05.696 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:05 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:05.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:05 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T04:44:05.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:05 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:05.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:05 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T04:44:05.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:05 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:06.696 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:06 smithi006 ceph-mon[29835]: Deploying daemon osd.3 on smithi006 2024-05-09T04:44:06.696 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:06 smithi006 ceph-mon[29835]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:06.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:06 smithi186 ceph-mon[34668]: Deploying daemon osd.3 on smithi006 2024-05-09T04:44:06.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:06 smithi186 ceph-mon[34668]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:06.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:06 smithi169 ceph-mon[34075]: Deploying daemon osd.3 on smithi006 2024-05-09T04:44:06.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:06 smithi169 ceph-mon[34075]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:08.840 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:08 smithi006 ceph-mon[29835]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:08.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:08 smithi186 ceph-mon[34668]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:08.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:08 smithi169 ceph-mon[34075]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:10.787 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:10 smithi006 ceph-mon[29835]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:10.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:10 smithi186 ceph-mon[34668]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:10.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:10 smithi169 ceph-mon[34075]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:12.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:12 smithi186 ceph-mon[34668]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:12.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:12 smithi169 ceph-mon[34075]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:12.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:12 smithi006 ceph-mon[29835]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:14.696 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:14 smithi006 ceph-mon[29835]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:14.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:14 smithi186 ceph-mon[34668]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:14.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:14 smithi169 ceph-mon[34075]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:16.630 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:16.630 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:16.630 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:44:16.630 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:16 smithi006 ceph-mon[29835]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:16.668 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:16.669 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:16.669 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:44:16.669 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:16 smithi169 ceph-mon[34075]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:16.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:16.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:16.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:44:16.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:16 smithi186 ceph-mon[34668]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:18.741 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:18 smithi006 ceph-mon[29835]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:18.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:18 smithi186 ceph-mon[34668]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:18.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:18 smithi169 ceph-mon[34075]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:20.286 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:20 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:20.286 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:20 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:20.287 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:20 smithi006 ceph-mon[29835]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:20.287 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:20 smithi006 ceph-mon[29835]: from='osd.3 [v2:172.21.15.6:6810/2846613703,v1:172.21.15.6:6811/2846613703]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T04:44:20.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:20 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:20.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:20 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:20.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:20 smithi186 ceph-mon[34668]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:20.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:20 smithi186 ceph-mon[34668]: from='osd.3 [v2:172.21.15.6:6810/2846613703,v1:172.21.15.6:6811/2846613703]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T04:44:20.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:20 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:20.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:20 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:20.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:20 smithi169 ceph-mon[34075]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:20.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:20 smithi169 ceph-mon[34075]: from='osd.3 [v2:172.21.15.6:6810/2846613703,v1:172.21.15.6:6811/2846613703]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T04:44:21.312 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:21 smithi006 ceph-mon[29835]: from='osd.3 [v2:172.21.15.6:6810/2846613703,v1:172.21.15.6:6811/2846613703]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-09T04:44:21.312 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:21 smithi006 ceph-mon[29835]: osdmap e23: 4 total, 3 up, 4 in 2024-05-09T04:44:21.312 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:21 smithi006 ceph-mon[29835]: from='osd.3 [v2:172.21.15.6:6810/2846613703,v1:172.21.15.6:6811/2846613703]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-05-09T04:44:21.312 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:21 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:44:21.312 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:21 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:21.312 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:21 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:44:21.312 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:21 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:21.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:21 smithi186 ceph-mon[34668]: from='osd.3 [v2:172.21.15.6:6810/2846613703,v1:172.21.15.6:6811/2846613703]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-09T04:44:21.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:21 smithi186 ceph-mon[34668]: osdmap e23: 4 total, 3 up, 4 in 2024-05-09T04:44:21.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:21 smithi186 ceph-mon[34668]: from='osd.3 [v2:172.21.15.6:6810/2846613703,v1:172.21.15.6:6811/2846613703]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-05-09T04:44:21.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:21 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:44:21.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:21 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:21.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:21 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:44:21.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:21 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:21.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:21 smithi169 ceph-mon[34075]: from='osd.3 [v2:172.21.15.6:6810/2846613703,v1:172.21.15.6:6811/2846613703]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-09T04:44:21.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:21 smithi169 ceph-mon[34075]: osdmap e23: 4 total, 3 up, 4 in 2024-05-09T04:44:21.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:21 smithi169 ceph-mon[34075]: from='osd.3 [v2:172.21.15.6:6810/2846613703,v1:172.21.15.6:6811/2846613703]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-05-09T04:44:21.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:21 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:44:21.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:21 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:21.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:21 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:44:21.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:21 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:21.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:44:21.446+0000 7f41f6000700 1 -- 172.21.15.6:0/3496638564 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f41f81a5160 con 0x7f41e00b27f0 2024-05-09T04:44:21.447 INFO:teuthology.orchestra.run.smithi006.stdout:Created osd(s) 3 on host 'smithi006' 2024-05-09T04:44:21.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:44:21.450+0000 7f4200000700 1 -- 172.21.15.6:0/3496638564 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f41e00b27f0 msgr2=0x7f41e00b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:44:21.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:44:21.450+0000 7f4200000700 1 --2- 172.21.15.6:0/3496638564 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f41e00b27f0 0x7f41e00b4d00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f41f819db90 tx=0x7f41e4009260 comp rx=0 tx=0).stop 2024-05-09T04:44:21.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:44:21.450+0000 7f4200000700 1 -- 172.21.15.6:0/3496638564 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41f8103240 msgr2=0x7f41f819cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:44:21.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:44:21.450+0000 7f4200000700 1 --2- 172.21.15.6:0/3496638564 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41f8103240 0x7f41f819cae0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f41f8102f40 tx=0x7f41ec00eca0 comp rx=0 tx=0).stop 2024-05-09T04:44:21.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:44:21.450+0000 7f4200000700 1 -- 172.21.15.6:0/3496638564 shutdown_connections 2024-05-09T04:44:21.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:44:21.450+0000 7f4200000700 1 --2- 172.21.15.6:0/3496638564 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f41e00b27f0 0x7f41e00b4d00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:21.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:44:21.450+0000 7f4200000700 1 --2- 172.21.15.6:0/3496638564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41f8104d10 0x7f41f81a3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:21.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:44:21.450+0000 7f4200000700 1 --2- 172.21.15.6:0/3496638564 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f41f8103240 0x7f41f819cae0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:21.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:44:21.450+0000 7f4200000700 1 --2- 172.21.15.6:0/3496638564 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41f8104370 0x7f41f819d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:21.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:44:21.450+0000 7f4200000700 1 -- 172.21.15.6:0/3496638564 >> 172.21.15.6:0/3496638564 conn(0x7f41f80fe880 msgr2=0x7f41f8100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:44:21.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:44:21.451+0000 7f4200000700 1 -- 172.21.15.6:0/3496638564 shutdown_connections 2024-05-09T04:44:21.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:44:21.451+0000 7f4200000700 1 -- 172.21.15.6:0/3496638564 wait complete. 2024-05-09T04:44:22.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:22 smithi006 ceph-mon[29835]: from='osd.3 [v2:172.21.15.6:6810/2846613703,v1:172.21.15.6:6811/2846613703]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-05-09T04:44:22.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:22 smithi006 ceph-mon[29835]: osdmap e24: 4 total, 3 up, 4 in 2024-05-09T04:44:22.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:44:22.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:44:22.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:44:22.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:22.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:22.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:22 smithi006 ceph-mon[29835]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:22.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:22 smithi006 ceph-mon[29835]: osdmap e25: 4 total, 3 up, 4 in 2024-05-09T04:44:22.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:44:22.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:22 smithi006 ceph-mon[29835]: from='osd.3 [v2:172.21.15.6:6810/2846613703,v1:172.21.15.6:6811/2846613703]' entity='osd.3' 2024-05-09T04:44:22.337 DEBUG:teuthology.orchestra.run.smithi006:osd.3> sudo journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.3.service 2024-05-09T04:44:22.341 INFO:tasks.cephadm:Deploying osd.4 on smithi169 with /dev/vg_nvme/lv_3... 2024-05-09T04:44:22.342 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-09T04:44:22.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:22 smithi186 ceph-mon[34668]: from='osd.3 [v2:172.21.15.6:6810/2846613703,v1:172.21.15.6:6811/2846613703]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-05-09T04:44:22.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:22 smithi186 ceph-mon[34668]: osdmap e24: 4 total, 3 up, 4 in 2024-05-09T04:44:22.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:44:22.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:44:22.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:44:22.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:22.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:22.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:22 smithi186 ceph-mon[34668]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:22.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:22 smithi186 ceph-mon[34668]: osdmap e25: 4 total, 3 up, 4 in 2024-05-09T04:44:22.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:44:22.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:22 smithi186 ceph-mon[34668]: from='osd.3 [v2:172.21.15.6:6810/2846613703,v1:172.21.15.6:6811/2846613703]' entity='osd.3' 2024-05-09T04:44:22.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:22 smithi169 ceph-mon[34075]: from='osd.3 [v2:172.21.15.6:6810/2846613703,v1:172.21.15.6:6811/2846613703]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-05-09T04:44:22.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:22 smithi169 ceph-mon[34075]: osdmap e24: 4 total, 3 up, 4 in 2024-05-09T04:44:22.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:44:22.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:44:22.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:44:22.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:22.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:22.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:22 smithi169 ceph-mon[34075]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:44:22.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:22 smithi169 ceph-mon[34075]: osdmap e25: 4 total, 3 up, 4 in 2024-05-09T04:44:22.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:44:22.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:22 smithi169 ceph-mon[34075]: from='osd.3 [v2:172.21.15.6:6810/2846613703,v1:172.21.15.6:6811/2846613703]' entity='osd.3' 2024-05-09T04:44:22.591 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T04:44:23.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:23 smithi006 ceph-mon[29835]: purged_snaps scrub starts 2024-05-09T04:44:23.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:23 smithi006 ceph-mon[29835]: purged_snaps scrub ok 2024-05-09T04:44:23.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:23 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:44:23.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:23 smithi006 ceph-mon[29835]: osd.3 [v2:172.21.15.6:6810/2846613703,v1:172.21.15.6:6811/2846613703] boot 2024-05-09T04:44:23.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:23 smithi006 ceph-mon[29835]: osdmap e26: 4 total, 4 up, 4 in 2024-05-09T04:44:23.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:23 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:44:23.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:23 smithi186 ceph-mon[34668]: purged_snaps scrub starts 2024-05-09T04:44:23.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:23 smithi186 ceph-mon[34668]: purged_snaps scrub ok 2024-05-09T04:44:23.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:23 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:44:23.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:23 smithi186 ceph-mon[34668]: osd.3 [v2:172.21.15.6:6810/2846613703,v1:172.21.15.6:6811/2846613703] boot 2024-05-09T04:44:23.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:23 smithi186 ceph-mon[34668]: osdmap e26: 4 total, 4 up, 4 in 2024-05-09T04:44:23.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:23 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:44:23.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:23 smithi169 ceph-mon[34075]: purged_snaps scrub starts 2024-05-09T04:44:23.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:23 smithi169 ceph-mon[34075]: purged_snaps scrub ok 2024-05-09T04:44:23.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:23 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:44:23.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:23 smithi169 ceph-mon[34075]: osd.3 [v2:172.21.15.6:6810/2846613703,v1:172.21.15.6:6811/2846613703] boot 2024-05-09T04:44:23.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:23 smithi169 ceph-mon[34075]: osdmap e26: 4 total, 4 up, 4 in 2024-05-09T04:44:23.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:23 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:44:24.088 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:24 smithi169 ceph-mon[34075]: pgmap v120: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:44:24.088 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:24 smithi169 ceph-mon[34075]: osdmap e27: 4 total, 4 up, 4 in 2024-05-09T04:44:24.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:24 smithi186 ceph-mon[34668]: pgmap v120: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:44:24.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:24 smithi186 ceph-mon[34668]: osdmap e27: 4 total, 4 up, 4 in 2024-05-09T04:44:24.356 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:24 smithi006 ceph-mon[29835]: pgmap v120: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:44:24.356 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:24 smithi006 ceph-mon[29835]: osdmap e27: 4 total, 4 up, 4 in 2024-05-09T04:44:24.651 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-09T04:44:24.651 INFO:teuthology.orchestra.run.smithi169.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-09T04:44:24.651 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10+0 records in 2024-05-09T04:44:24.651 INFO:teuthology.orchestra.run.smithi169.stderr:10+0 records out 2024-05-09T04:44:24.651 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126949 s, 826 MB/s 2024-05-09T04:44:24.651 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping successful for: 2024-05-09T04:44:25.272 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_3 2024-05-09T04:44:25.548 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T04:44:25.923 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:25 smithi169 ceph-mon[34075]: osdmap e28: 4 total, 4 up, 4 in 2024-05-09T04:44:25.923 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:25 smithi169 ceph-mon[34075]: pgmap v123: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:44:26.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:25 smithi006 ceph-mon[29835]: osdmap e28: 4 total, 4 up, 4 in 2024-05-09T04:44:26.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:25 smithi006 ceph-mon[29835]: pgmap v123: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:44:26.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:25 smithi186 ceph-mon[34668]: osdmap e28: 4 total, 4 up, 4 in 2024-05-09T04:44:26.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:25 smithi186 ceph-mon[34668]: pgmap v123: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:44:27.204 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.202+0000 7fbd6af4f700 1 -- 172.21.15.169:0/3620417285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd64105e40 msgr2=0x7fbd6410c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:44:27.205 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.202+0000 7fbd6af4f700 1 --2- 172.21.15.169:0/3620417285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd64105e40 0x7fbd6410c6f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c00b0a0 tx=0x7fbd5c00b3b0 comp rx=0 tx=0).stop 2024-05-09T04:44:27.205 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.202+0000 7fbd6af4f700 1 -- 172.21.15.169:0/3620417285 shutdown_connections 2024-05-09T04:44:27.205 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.202+0000 7fbd6af4f700 1 --2- 172.21.15.169:0/3620417285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd64105e40 0x7fbd6410c6f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:27.205 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.202+0000 7fbd6af4f700 1 --2- 172.21.15.169:0/3620417285 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd64104370 0x7fbd64104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:27.205 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.202+0000 7fbd6af4f700 1 --2- 172.21.15.169:0/3620417285 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbd641054a0 0x7fbd641058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:27.205 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.202+0000 7fbd6af4f700 1 -- 172.21.15.169:0/3620417285 >> 172.21.15.169:0/3620417285 conn(0x7fbd640ff9d0 msgr2=0x7fbd64101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:44:27.205 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.202+0000 7fbd6af4f700 1 -- 172.21.15.169:0/3620417285 shutdown_connections 2024-05-09T04:44:27.205 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.202+0000 7fbd6af4f700 1 -- 172.21.15.169:0/3620417285 wait complete. 2024-05-09T04:44:27.205 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.203+0000 7fbd6af4f700 1 Processor -- start 2024-05-09T04:44:27.205 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.203+0000 7fbd6af4f700 1 -- start start 2024-05-09T04:44:27.205 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.203+0000 7fbd6af4f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd64104370 0x7fbd6419a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:44:27.205 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.203+0000 7fbd6af4f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd641054a0 0x7fbd6419ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:44:27.206 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.203+0000 7fbd6af4f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbd64105e40 0x7fbd641a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:44:27.206 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.203+0000 7fbd6af4f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6410fe70 con 0x7fbd64105e40 2024-05-09T04:44:27.206 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.203+0000 7fbd6af4f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6410fcf0 con 0x7fbd64104370 2024-05-09T04:44:27.206 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.203+0000 7fbd6af4f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6410fff0 con 0x7fbd641054a0 2024-05-09T04:44:27.206 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.203+0000 7fbd68ceb700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd64104370 0x7fbd6419a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:44:27.206 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.203+0000 7fbd68ceb700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd64104370 0x7fbd6419a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53772/0 (socket says 172.21.15.169:53772) 2024-05-09T04:44:27.206 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.203+0000 7fbd68ceb700 1 -- 172.21.15.169:0/2245470262 learned_addr learned my addr 172.21.15.169:0/2245470262 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-05-09T04:44:27.206 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.203+0000 7fbd63fff700 1 --2- 172.21.15.169:0/2245470262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd641054a0 0x7fbd6419ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:44:27.206 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.203+0000 7fbd68ceb700 1 -- 172.21.15.169:0/2245470262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd641054a0 msgr2=0x7fbd6419ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:44:27.206 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.203+0000 7fbd68ceb700 1 --2- 172.21.15.169:0/2245470262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd641054a0 0x7fbd6419ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:27.206 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.203+0000 7fbd68ceb700 1 -- 172.21.15.169:0/2245470262 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbd64105e40 msgr2=0x7fbd641a1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:44:27.206 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.203+0000 7fbd68ceb700 1 --2- 172.21.15.169:0/2245470262 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbd64105e40 0x7fbd641a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:27.207 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.203+0000 7fbd68ceb700 1 -- 172.21.15.169:0/2245470262 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd641a2460 con 0x7fbd64104370 2024-05-09T04:44:27.207 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.204+0000 7fbd63fff700 1 --2- 172.21.15.169:0/2245470262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd641054a0 0x7fbd6419ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:44:27.207 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.204+0000 7fbd68ceb700 1 --2- 172.21.15.169:0/2245470262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd64104370 0x7fbd6419a900 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbd5800c8c0 tx=0x7fbd5800cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:44:27.207 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.204+0000 7fbd61ffb700 1 -- 172.21.15.169:0/2245470262 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd58004230 con 0x7fbd64104370 2024-05-09T04:44:27.207 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.204+0000 7fbd61ffb700 1 -- 172.21.15.169:0/2245470262 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd58007db0 con 0x7fbd64104370 2024-05-09T04:44:27.207 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.204+0000 7fbd6af4f700 1 -- 172.21.15.169:0/2245470262 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd641a2750 con 0x7fbd64104370 2024-05-09T04:44:27.207 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.204+0000 7fbd61ffb700 1 -- 172.21.15.169:0/2245470262 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd58005020 con 0x7fbd64104370 2024-05-09T04:44:27.207 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.204+0000 7fbd6af4f700 1 -- 172.21.15.169:0/2245470262 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd641a2d10 con 0x7fbd64104370 2024-05-09T04:44:27.208 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.205+0000 7fbd6af4f700 1 -- 172.21.15.169:0/2245470262 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd640512c0 con 0x7fbd64104370 2024-05-09T04:44:27.208 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.206+0000 7fbd61ffb700 1 -- 172.21.15.169:0/2245470262 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fbd58007900 con 0x7fbd64104370 2024-05-09T04:44:27.208 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.207+0000 7fbd61ffb700 1 --2- 172.21.15.169:0/2245470262 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fbd540b2900 0x7fbd540b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:44:27.208 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.207+0000 7fbd61ffb700 1 -- 172.21.15.169:0/2245470262 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7fbd580e0c80 con 0x7fbd64104370 2024-05-09T04:44:27.208 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.207+0000 7fbd63fff700 1 --2- 172.21.15.169:0/2245470262 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fbd540b2900 0x7fbd540b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:44:27.209 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.208+0000 7fbd63fff700 1 --2- 172.21.15.169:0/2245470262 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fbd540b2900 0x7fbd540b4e10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbd6419b9b0 tx=0x7fbd50008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:44:27.212 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.212+0000 7fbd61ffb700 1 -- 172.21.15.169:0/2245470262 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbd58014070 con 0x7fbd64104370 2024-05-09T04:44:27.421 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:27.421+0000 7fbd6af4f700 1 -- 172.21.15.169:0/2245470262 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fbd64063c00 con 0x7fbd540b2900 2024-05-09T04:44:28.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:28 smithi006 ceph-mon[29835]: Detected new or changed devices on smithi006 2024-05-09T04:44:28.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:28 smithi006 ceph-mon[29835]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:44:28.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:28 smithi006 ceph-mon[29835]: pgmap v124: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:44:28.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:28 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:28.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:28 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:44:28.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:28 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:28.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:28 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:44:28.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:28 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:44:28.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:28 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:28.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:28 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:44:28.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:28 smithi006 ceph-mon[29835]: Adjusting osd_memory_target on smithi006 to 8547M 2024-05-09T04:44:28.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:28 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:28.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:28 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:28.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:28 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:44:28.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:28 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:28.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:28 smithi186 ceph-mon[34668]: Detected new or changed devices on smithi006 2024-05-09T04:44:28.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:28 smithi186 ceph-mon[34668]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:44:28.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:28 smithi186 ceph-mon[34668]: pgmap v124: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:44:28.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:28 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:28.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:28 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:44:28.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:28 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:28.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:28 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:44:28.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:28 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:44:28.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:28 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:28.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:28 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:44:28.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:28 smithi186 ceph-mon[34668]: Adjusting osd_memory_target on smithi006 to 8547M 2024-05-09T04:44:28.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:28 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:28.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:28 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:28.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:28 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:44:28.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:28 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:28.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:28 smithi169 ceph-mon[34075]: Detected new or changed devices on smithi006 2024-05-09T04:44:28.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:28 smithi169 ceph-mon[34075]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:44:28.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:28 smithi169 ceph-mon[34075]: pgmap v124: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:44:28.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:28 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:28.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:28 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:44:28.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:28 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:28.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:28 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:44:28.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:28 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:44:28.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:28 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:28.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:28 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:44:28.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:28 smithi169 ceph-mon[34075]: Adjusting osd_memory_target on smithi006 to 8547M 2024-05-09T04:44:28.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:28 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:28.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:28 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:28.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:28 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:44:28.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:28 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:30.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:30 smithi006 ceph-mon[29835]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-09T04:44:30.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:30 smithi006 ceph-mon[29835]: from='client.? 172.21.15.169:0/3969925542' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb84cc5e-0297-4e23-be50-7ee2f7e59665"}]: dispatch 2024-05-09T04:44:30.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:30 smithi006 ceph-mon[29835]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb84cc5e-0297-4e23-be50-7ee2f7e59665"}]: dispatch 2024-05-09T04:44:30.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:30 smithi006 ceph-mon[29835]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb84cc5e-0297-4e23-be50-7ee2f7e59665"}]': finished 2024-05-09T04:44:30.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:30 smithi006 ceph-mon[29835]: osdmap e29: 5 total, 4 up, 5 in 2024-05-09T04:44:30.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:30 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T04:44:30.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:30 smithi186 ceph-mon[34668]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-09T04:44:30.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:30 smithi186 ceph-mon[34668]: from='client.? 172.21.15.169:0/3969925542' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb84cc5e-0297-4e23-be50-7ee2f7e59665"}]: dispatch 2024-05-09T04:44:30.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:30 smithi186 ceph-mon[34668]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb84cc5e-0297-4e23-be50-7ee2f7e59665"}]: dispatch 2024-05-09T04:44:30.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:30 smithi186 ceph-mon[34668]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb84cc5e-0297-4e23-be50-7ee2f7e59665"}]': finished 2024-05-09T04:44:30.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:30 smithi186 ceph-mon[34668]: osdmap e29: 5 total, 4 up, 5 in 2024-05-09T04:44:30.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:30 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T04:44:30.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:30 smithi169 ceph-mon[34075]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-09T04:44:30.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:30 smithi169 ceph-mon[34075]: from='client.? 172.21.15.169:0/3969925542' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb84cc5e-0297-4e23-be50-7ee2f7e59665"}]: dispatch 2024-05-09T04:44:30.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:30 smithi169 ceph-mon[34075]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb84cc5e-0297-4e23-be50-7ee2f7e59665"}]: dispatch 2024-05-09T04:44:30.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:30 smithi169 ceph-mon[34075]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb84cc5e-0297-4e23-be50-7ee2f7e59665"}]': finished 2024-05-09T04:44:30.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:30 smithi169 ceph-mon[34075]: osdmap e29: 5 total, 4 up, 5 in 2024-05-09T04:44:30.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:30 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T04:44:31.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:31 smithi186 ceph-mon[34668]: from='client.? 172.21.15.169:0/2555943156' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-4/activate.monmap"}]: dispatch 2024-05-09T04:44:31.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:31 smithi169 ceph-mon[34075]: from='client.? 172.21.15.169:0/2555943156' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-4/activate.monmap"}]: dispatch 2024-05-09T04:44:31.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:31 smithi006 ceph-mon[29835]: from='client.? 172.21.15.169:0/2555943156' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-4/activate.monmap"}]: dispatch 2024-05-09T04:44:32.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:32 smithi186 ceph-mon[34668]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-05-09T04:44:32.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:32 smithi169 ceph-mon[34075]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-05-09T04:44:32.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:32 smithi006 ceph-mon[29835]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-05-09T04:44:34.613 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:34 smithi169 ceph-mon[34075]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-09T04:44:34.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:34 smithi186 ceph-mon[34668]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-09T04:44:34.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:34 smithi006 ceph-mon[29835]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-09T04:44:36.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:36 smithi186 ceph-mon[34668]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:44:36.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:36 smithi169 ceph-mon[34075]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:44:36.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:36 smithi006 ceph-mon[29835]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:44:38.565 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:38 smithi169 ceph-mon[34075]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:44:38.565 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:38 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T04:44:38.565 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:38 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:38.565 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:38 smithi169 ceph-mon[34075]: Deploying daemon osd.4 on smithi169 2024-05-09T04:44:38.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:38 smithi186 ceph-mon[34668]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:44:38.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:38 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T04:44:38.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:38 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:38.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:38 smithi186 ceph-mon[34668]: Deploying daemon osd.4 on smithi169 2024-05-09T04:44:38.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:38 smithi006 ceph-mon[29835]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:44:38.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:38 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T04:44:38.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:38 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:38.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:38 smithi006 ceph-mon[29835]: Deploying daemon osd.4 on smithi169 2024-05-09T04:44:40.584 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:40 smithi169 ceph-mon[34075]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:44:40.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:40 smithi186 ceph-mon[34668]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:44:40.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:40 smithi006 ceph-mon[29835]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:44:42.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:42 smithi186 ceph-mon[34668]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:44:42.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:42 smithi169 ceph-mon[34075]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:44:42.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:42 smithi006 ceph-mon[29835]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:44:44.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:44 smithi186 ceph-mon[34668]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:44:44.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:44 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:44.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:44 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:44.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:44 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:44:44.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:44 smithi169 ceph-mon[34075]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:44:44.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:44 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:44.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:44 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:44.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:44 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:44:44.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:44 smithi006 ceph-mon[29835]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:44:44.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:44 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:44.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:44 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:44.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:44 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:44:46.676 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:46 smithi169 ceph-mon[34075]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:44:46.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:46 smithi186 ceph-mon[34668]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:44:46.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:46 smithi006 ceph-mon[29835]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:44:47.074 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 4 on host 'smithi169' 2024-05-09T04:44:47.074 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:47.074+0000 7fbd61ffb700 1 -- 172.21.15.169:0/2245470262 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbd64063c00 con 0x7fbd540b2900 2024-05-09T04:44:47.077 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:47.076+0000 7fbd6af4f700 1 -- 172.21.15.169:0/2245470262 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fbd540b2900 msgr2=0x7fbd540b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:44:47.077 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:47.076+0000 7fbd6af4f700 1 --2- 172.21.15.169:0/2245470262 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fbd540b2900 0x7fbd540b4e10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbd6419b9b0 tx=0x7fbd50008040 comp rx=0 tx=0).stop 2024-05-09T04:44:47.077 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:47.077+0000 7fbd6af4f700 1 -- 172.21.15.169:0/2245470262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd64104370 msgr2=0x7fbd6419a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:44:47.077 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:47.077+0000 7fbd6af4f700 1 --2- 172.21.15.169:0/2245470262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd64104370 0x7fbd6419a900 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbd5800c8c0 tx=0x7fbd5800cc80 comp rx=0 tx=0).stop 2024-05-09T04:44:47.077 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:47.077+0000 7fbd6af4f700 1 -- 172.21.15.169:0/2245470262 shutdown_connections 2024-05-09T04:44:47.077 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:47.077+0000 7fbd6af4f700 1 --2- 172.21.15.169:0/2245470262 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbd64105e40 0x7fbd641a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:47.077 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:47.077+0000 7fbd6af4f700 1 --2- 172.21.15.169:0/2245470262 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fbd540b2900 0x7fbd540b4e10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:47.077 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:47.077+0000 7fbd6af4f700 1 --2- 172.21.15.169:0/2245470262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd64104370 0x7fbd6419a900 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:47.077 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:47.077+0000 7fbd6af4f700 1 --2- 172.21.15.169:0/2245470262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd641054a0 0x7fbd6419ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:47.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:47.077+0000 7fbd6af4f700 1 -- 172.21.15.169:0/2245470262 >> 172.21.15.169:0/2245470262 conn(0x7fbd640ff9d0 msgr2=0x7fbd641086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:44:47.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:47.077+0000 7fbd6af4f700 1 -- 172.21.15.169:0/2245470262 shutdown_connections 2024-05-09T04:44:47.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:44:47.077+0000 7fbd6af4f700 1 -- 172.21.15.169:0/2245470262 wait complete. 2024-05-09T04:44:47.780 DEBUG:teuthology.orchestra.run.smithi169:osd.4> sudo journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.4.service 2024-05-09T04:44:47.782 INFO:tasks.cephadm:Deploying osd.5 on smithi186 with /dev/vg_nvme/lv_3... 2024-05-09T04:44:47.783 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-09T04:44:48.027 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T04:44:48.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:48 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:48.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:48 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:48.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:48 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:48.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:48 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:48.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:48 smithi186 ceph-mon[34668]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:44:48.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:48 smithi186 ceph-mon[34668]: from='osd.4 [v2:172.21.15.169:6808/1928722453,v1:172.21.15.169:6809/1928722453]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T04:44:48.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:48 smithi186 ceph-mon[34668]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T04:44:48.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:48 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:48.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:48 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:48.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:48 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:48.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:48 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:48.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:48 smithi169 ceph-mon[34075]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:44:48.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:48 smithi169 ceph-mon[34075]: from='osd.4 [v2:172.21.15.169:6808/1928722453,v1:172.21.15.169:6809/1928722453]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T04:44:48.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:48 smithi169 ceph-mon[34075]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T04:44:48.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:48 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:48.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:48 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:48.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:48 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:48.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:48 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:48.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:48 smithi006 ceph-mon[29835]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:44:48.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:48 smithi006 ceph-mon[29835]: from='osd.4 [v2:172.21.15.169:6808/1928722453,v1:172.21.15.169:6809/1928722453]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T04:44:48.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:48 smithi006 ceph-mon[29835]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T04:44:49.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:49 smithi006 ceph-mon[29835]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-09T04:44:49.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:49 smithi006 ceph-mon[29835]: from='osd.4 [v2:172.21.15.169:6808/1928722453,v1:172.21.15.169:6809/1928722453]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T04:44:49.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:49 smithi006 ceph-mon[29835]: osdmap e30: 5 total, 4 up, 5 in 2024-05-09T04:44:49.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:49 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T04:44:49.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:49 smithi006 ceph-mon[29835]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T04:44:49.454 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:49 smithi186 ceph-mon[34668]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-09T04:44:49.454 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:49 smithi186 ceph-mon[34668]: from='osd.4 [v2:172.21.15.169:6808/1928722453,v1:172.21.15.169:6809/1928722453]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T04:44:49.454 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:49 smithi186 ceph-mon[34668]: osdmap e30: 5 total, 4 up, 5 in 2024-05-09T04:44:49.454 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:49 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T04:44:49.455 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:49 smithi186 ceph-mon[34668]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T04:44:49.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:49 smithi169 ceph-mon[34075]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-09T04:44:49.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:49 smithi169 ceph-mon[34075]: from='osd.4 [v2:172.21.15.169:6808/1928722453,v1:172.21.15.169:6809/1928722453]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T04:44:49.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:49 smithi169 ceph-mon[34075]: osdmap e30: 5 total, 4 up, 5 in 2024-05-09T04:44:49.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:49 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T04:44:49.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:49 smithi169 ceph-mon[34075]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T04:44:50.158 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-09T04:44:50.159 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-09T04:44:50.159 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2024-05-09T04:44:50.159 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2024-05-09T04:44:50.159 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115212 s, 910 MB/s 2024-05-09T04:44:50.159 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2024-05-09T04:44:50.204 INFO:journalctl@ceph.osd.4.smithi169.stdout:May 09 04:44:49 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-4[45231]: 2024-05-09T04:44:49.891+0000 7f4d642b8700 -1 osd.4 0 waiting for initial osdmap 2024-05-09T04:44:50.205 INFO:journalctl@ceph.osd.4.smithi169.stdout:May 09 04:44:49 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-4[45231]: 2024-05-09T04:44:49.898+0000 7f4d5c065700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T04:44:50.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:50 smithi006 ceph-mon[29835]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-05-09T04:44:50.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:50 smithi006 ceph-mon[29835]: osdmap e31: 5 total, 4 up, 5 in 2024-05-09T04:44:50.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:50 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T04:44:50.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:50 smithi006 ceph-mon[29835]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:44:50.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:50 smithi006 ceph-mon[29835]: from='osd.4 ' entity='osd.4' 2024-05-09T04:44:50.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:50 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T04:44:50.454 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:50 smithi186 ceph-mon[34668]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-05-09T04:44:50.454 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:50 smithi186 ceph-mon[34668]: osdmap e31: 5 total, 4 up, 5 in 2024-05-09T04:44:50.455 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:50 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T04:44:50.455 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:50 smithi186 ceph-mon[34668]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:44:50.455 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:50 smithi186 ceph-mon[34668]: from='osd.4 ' entity='osd.4' 2024-05-09T04:44:50.455 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:50 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T04:44:50.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:50 smithi169 ceph-mon[34075]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-05-09T04:44:50.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:50 smithi169 ceph-mon[34075]: osdmap e31: 5 total, 4 up, 5 in 2024-05-09T04:44:50.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:50 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T04:44:50.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:50 smithi169 ceph-mon[34075]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:44:50.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:50 smithi169 ceph-mon[34075]: from='osd.4 ' entity='osd.4' 2024-05-09T04:44:50.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:50 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T04:44:50.859 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi186:vg_nvme/lv_3 2024-05-09T04:44:51.100 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T04:44:51.484 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:51 smithi186 ceph-mon[34668]: purged_snaps scrub starts 2024-05-09T04:44:51.484 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:51 smithi186 ceph-mon[34668]: purged_snaps scrub ok 2024-05-09T04:44:51.484 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:51 smithi186 ceph-mon[34668]: osd.4 [v2:172.21.15.169:6808/1928722453,v1:172.21.15.169:6809/1928722453] boot 2024-05-09T04:44:51.484 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:51 smithi186 ceph-mon[34668]: osdmap e32: 5 total, 5 up, 5 in 2024-05-09T04:44:51.484 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:51 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T04:44:51.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:51 smithi169 ceph-mon[34075]: purged_snaps scrub starts 2024-05-09T04:44:51.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:51 smithi169 ceph-mon[34075]: purged_snaps scrub ok 2024-05-09T04:44:51.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:51 smithi169 ceph-mon[34075]: osd.4 [v2:172.21.15.169:6808/1928722453,v1:172.21.15.169:6809/1928722453] boot 2024-05-09T04:44:51.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:51 smithi169 ceph-mon[34075]: osdmap e32: 5 total, 5 up, 5 in 2024-05-09T04:44:51.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:51 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T04:44:51.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:51 smithi006 ceph-mon[29835]: purged_snaps scrub starts 2024-05-09T04:44:51.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:51 smithi006 ceph-mon[29835]: purged_snaps scrub ok 2024-05-09T04:44:51.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:51 smithi006 ceph-mon[29835]: osd.4 [v2:172.21.15.169:6808/1928722453,v1:172.21.15.169:6809/1928722453] boot 2024-05-09T04:44:51.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:51 smithi006 ceph-mon[29835]: osdmap e32: 5 total, 5 up, 5 in 2024-05-09T04:44:51.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:51 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T04:44:52.824 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.822+0000 7fbfc6760700 1 -- 172.21.15.186:0/29276909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfc010a000 msgr2=0x7fbfc010c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:44:52.824 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.822+0000 7fbfc6760700 1 --2- 172.21.15.186:0/29276909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfc010a000 0x7fbfc010c510 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800b0a0 tx=0x7fbfb800b3b0 comp rx=0 tx=0).stop 2024-05-09T04:44:52.825 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.822+0000 7fbfc6760700 1 -- 172.21.15.186:0/29276909 shutdown_connections 2024-05-09T04:44:52.825 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.822+0000 7fbfc6760700 1 --2- 172.21.15.186:0/29276909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfc010a000 0x7fbfc010c510 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:52.825 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.822+0000 7fbfc6760700 1 --2- 172.21.15.186:0/29276909 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfc0076690 0x7fbfc0106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:52.825 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.822+0000 7fbfc6760700 1 --2- 172.21.15.186:0/29276909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfc01074b0 0x7fbfc0109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:52.825 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.822+0000 7fbfc6760700 1 -- 172.21.15.186:0/29276909 >> 172.21.15.186:0/29276909 conn(0x7fbfc00fe890 msgr2=0x7fbfc0100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:44:52.825 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.822+0000 7fbfc6760700 1 -- 172.21.15.186:0/29276909 shutdown_connections 2024-05-09T04:44:52.825 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.822+0000 7fbfc6760700 1 -- 172.21.15.186:0/29276909 wait complete. 2024-05-09T04:44:52.825 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.822+0000 7fbfc6760700 1 Processor -- start 2024-05-09T04:44:52.825 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.822+0000 7fbfc6760700 1 -- start start 2024-05-09T04:44:52.825 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.822+0000 7fbfc6760700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfc0076690 0x7fbfc019a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:44:52.825 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.822+0000 7fbfc6760700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfc01074b0 0x7fbfc019adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:44:52.826 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.822+0000 7fbfc6760700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfc010a000 0x7fbfc01a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:44:52.826 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.822+0000 7fbfc6760700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc010fe20 con 0x7fbfc010a000 2024-05-09T04:44:52.826 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.822+0000 7fbfc6760700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc010fca0 con 0x7fbfc01074b0 2024-05-09T04:44:52.826 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.823+0000 7fbfc6760700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc010ffa0 con 0x7fbfc0076690 2024-05-09T04:44:52.826 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.823+0000 7fbfbffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfc0076690 0x7fbfc019a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:44:52.826 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.823+0000 7fbfbffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfc0076690 0x7fbfc019a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:36470/0 (socket says 172.21.15.186:36470) 2024-05-09T04:44:52.826 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.823+0000 7fbfbffff700 1 -- 172.21.15.186:0/3750814327 learned_addr learned my addr 172.21.15.186:0/3750814327 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-05-09T04:44:52.826 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.823+0000 7fbfb7fff700 1 --2- 172.21.15.186:0/3750814327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfc01074b0 0x7fbfc019adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:44:52.826 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.823+0000 7fbfbffff700 1 -- 172.21.15.186:0/3750814327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfc01074b0 msgr2=0x7fbfc019adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:44:52.826 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.823+0000 7fbfbffff700 1 --2- 172.21.15.186:0/3750814327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfc01074b0 0x7fbfc019adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:52.826 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.823+0000 7fbfbffff700 1 -- 172.21.15.186:0/3750814327 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfc010a000 msgr2=0x7fbfc01a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:44:52.827 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.823+0000 7fbfbffff700 1 --2- 172.21.15.186:0/3750814327 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfc010a000 0x7fbfc01a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:52.827 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.823+0000 7fbfbffff700 1 -- 172.21.15.186:0/3750814327 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc01a2410 con 0x7fbfc0076690 2024-05-09T04:44:52.827 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.823+0000 7fbfb7fff700 1 --2- 172.21.15.186:0/3750814327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfc01074b0 0x7fbfc019adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:44:52.827 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.823+0000 7fbfbffff700 1 --2- 172.21.15.186:0/3750814327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfc0076690 0x7fbfc019a8b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000b500 tx=0x7fbfb000b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:44:52.827 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.823+0000 7fbfbdffb700 1 -- 172.21.15.186:0/3750814327 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0015020 con 0x7fbfc0076690 2024-05-09T04:44:52.827 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.823+0000 7fbfbdffb700 1 -- 172.21.15.186:0/3750814327 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfb0011ce0 con 0x7fbfc0076690 2024-05-09T04:44:52.827 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.823+0000 7fbfc6760700 1 -- 172.21.15.186:0/3750814327 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfc01a2700 con 0x7fbfc0076690 2024-05-09T04:44:52.827 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.823+0000 7fbfbdffb700 1 -- 172.21.15.186:0/3750814327 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0010d40 con 0x7fbfc0076690 2024-05-09T04:44:52.827 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.823+0000 7fbfc6760700 1 -- 172.21.15.186:0/3750814327 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfc01a2d10 con 0x7fbfc0076690 2024-05-09T04:44:52.827 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.824+0000 7fbfc6760700 1 -- 172.21.15.186:0/3750814327 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfc00512c0 con 0x7fbfc0076690 2024-05-09T04:44:52.829 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.826+0000 7fbfbdffb700 1 -- 172.21.15.186:0/3750814327 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fbfb0018020 con 0x7fbfc0076690 2024-05-09T04:44:52.829 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.826+0000 7fbfbdffb700 1 --2- 172.21.15.186:0/3750814327 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fbfac0b2900 0x7fbfac0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:44:52.829 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.827+0000 7fbfbdffb700 1 -- 172.21.15.186:0/3750814327 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4101+0+0 (secure 0 0 0) 0x7fbfb00e0ed0 con 0x7fbfc0076690 2024-05-09T04:44:52.829 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.827+0000 7fbfb7fff700 1 --2- 172.21.15.186:0/3750814327 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fbfac0b2900 0x7fbfac0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:44:52.829 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.828+0000 7fbfb7fff700 1 --2- 172.21.15.186:0/3750814327 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fbfac0b2900 0x7fbfac0b4e10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbfc019b960 tx=0x7fbfa8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:44:52.830 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:52.831+0000 7fbfbdffb700 1 -- 172.21.15.186:0/3750814327 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbfb001f020 con 0x7fbfc0076690 2024-05-09T04:44:53.042 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:44:53.042+0000 7fbfc6760700 1 -- 172.21.15.186:0/3750814327 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fbfc0063c00 con 0x7fbfac0b2900 2024-05-09T04:44:53.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:52 smithi186 ceph-mon[34668]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:44:53.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:52 smithi186 ceph-mon[34668]: Detected new or changed devices on smithi169 2024-05-09T04:44:53.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:53.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:53.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:44:53.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:44:53.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:52 smithi186 ceph-mon[34668]: Adjusting osd_memory_target on smithi169 to 8544M 2024-05-09T04:44:53.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:53.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:53.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:44:53.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:53.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:44:53.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:53.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:44:53.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:53.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:52 smithi186 ceph-mon[34668]: osdmap e33: 5 total, 5 up, 5 in 2024-05-09T04:44:53.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:52 smithi169 ceph-mon[34075]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:44:53.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:52 smithi169 ceph-mon[34075]: Detected new or changed devices on smithi169 2024-05-09T04:44:53.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:53.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:53.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:44:53.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:44:53.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:52 smithi169 ceph-mon[34075]: Adjusting osd_memory_target on smithi169 to 8544M 2024-05-09T04:44:53.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:53.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:53.118 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:44:53.118 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:53.118 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:44:53.118 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:53.118 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:44:53.118 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:53.118 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:52 smithi169 ceph-mon[34075]: osdmap e33: 5 total, 5 up, 5 in 2024-05-09T04:44:53.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:52 smithi006 ceph-mon[29835]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:44:53.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:52 smithi006 ceph-mon[29835]: Detected new or changed devices on smithi169 2024-05-09T04:44:53.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:53.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:53.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:44:53.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:44:53.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:52 smithi006 ceph-mon[29835]: Adjusting osd_memory_target on smithi169 to 8544M 2024-05-09T04:44:53.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:53.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:53.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:44:53.199 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:53.199 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:44:53.199 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:53.199 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:44:53.199 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:44:53.199 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:52 smithi006 ceph-mon[29835]: osdmap e33: 5 total, 5 up, 5 in 2024-05-09T04:44:54.002 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:53 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:44:54.003 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:53 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:44:54.003 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:53 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:54.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:53 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:44:54.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:53 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:44:54.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:53 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:54.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:53 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:44:54.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:53 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:44:54.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:53 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:54.953 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:54 smithi186 ceph-mon[34668]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:44:54.953 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:54 smithi186 ceph-mon[34668]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:44:55.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:54 smithi169 ceph-mon[34075]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:44:55.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:54 smithi169 ceph-mon[34075]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:44:55.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:54 smithi006 ceph-mon[29835]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:44:55.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:54 smithi006 ceph-mon[29835]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:44:56.118 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:55 smithi169 ceph-mon[34075]: from='client.? 172.21.15.186:0/1802447033' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "830dc7f3-352e-4b01-8d4c-a9b68dcb40d9"}]: dispatch 2024-05-09T04:44:56.118 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:55 smithi169 ceph-mon[34075]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "830dc7f3-352e-4b01-8d4c-a9b68dcb40d9"}]: dispatch 2024-05-09T04:44:56.118 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:55 smithi169 ceph-mon[34075]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "830dc7f3-352e-4b01-8d4c-a9b68dcb40d9"}]': finished 2024-05-09T04:44:56.118 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:55 smithi169 ceph-mon[34075]: osdmap e34: 6 total, 5 up, 6 in 2024-05-09T04:44:56.118 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:55 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T04:44:56.164 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:55 smithi186 ceph-mon[34668]: from='client.? 172.21.15.186:0/1802447033' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "830dc7f3-352e-4b01-8d4c-a9b68dcb40d9"}]: dispatch 2024-05-09T04:44:56.164 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:55 smithi186 ceph-mon[34668]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "830dc7f3-352e-4b01-8d4c-a9b68dcb40d9"}]: dispatch 2024-05-09T04:44:56.164 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:55 smithi186 ceph-mon[34668]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "830dc7f3-352e-4b01-8d4c-a9b68dcb40d9"}]': finished 2024-05-09T04:44:56.164 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:55 smithi186 ceph-mon[34668]: osdmap e34: 6 total, 5 up, 6 in 2024-05-09T04:44:56.164 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:55 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T04:44:56.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:55 smithi006 ceph-mon[29835]: from='client.? 172.21.15.186:0/1802447033' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "830dc7f3-352e-4b01-8d4c-a9b68dcb40d9"}]: dispatch 2024-05-09T04:44:56.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:55 smithi006 ceph-mon[29835]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "830dc7f3-352e-4b01-8d4c-a9b68dcb40d9"}]: dispatch 2024-05-09T04:44:56.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:55 smithi006 ceph-mon[29835]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "830dc7f3-352e-4b01-8d4c-a9b68dcb40d9"}]': finished 2024-05-09T04:44:56.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:55 smithi006 ceph-mon[29835]: osdmap e34: 6 total, 5 up, 6 in 2024-05-09T04:44:56.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:55 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T04:44:57.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:56 smithi169 ceph-mon[34075]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:44:57.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:56 smithi169 ceph-mon[34075]: from='client.? 172.21.15.186:0/3372688589' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-5/activate.monmap"}]: dispatch 2024-05-09T04:44:57.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:56 smithi006 ceph-mon[29835]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:44:57.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:56 smithi006 ceph-mon[29835]: from='client.? 172.21.15.186:0/3372688589' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-5/activate.monmap"}]: dispatch 2024-05-09T04:44:57.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:56 smithi186 ceph-mon[34668]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:44:57.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:56 smithi186 ceph-mon[34668]: from='client.? 172.21.15.186:0/3372688589' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-5/activate.monmap"}]: dispatch 2024-05-09T04:44:59.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:44:58 smithi169 ceph-mon[34075]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:44:59.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:44:58 smithi006 ceph-mon[29835]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:44:59.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:44:58 smithi186 ceph-mon[34668]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:45:01.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:00 smithi186 ceph-mon[34668]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:45:01.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:00 smithi006 ceph-mon[29835]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:45:01.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:00 smithi169 ceph-mon[34075]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:45:02.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:01 smithi186 ceph-mon[34668]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:45:02.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:01 smithi006 ceph-mon[29835]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:45:02.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:01 smithi169 ceph-mon[34075]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:45:03.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:03 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T04:45:03.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:03 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:03.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:03 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T04:45:03.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:03 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:03.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:03 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T04:45:03.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:03 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:04.536 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:04 smithi186 ceph-mon[34668]: Deploying daemon osd.5 on smithi186 2024-05-09T04:45:04.536 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:04 smithi186 ceph-mon[34668]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:45:04.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:04 smithi169 ceph-mon[34075]: Deploying daemon osd.5 on smithi186 2024-05-09T04:45:04.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:04 smithi169 ceph-mon[34075]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:45:04.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:04 smithi006 ceph-mon[29835]: Deploying daemon osd.5 on smithi186 2024-05-09T04:45:04.704 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:04 smithi006 ceph-mon[29835]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:45:06.802 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:06 smithi186 ceph-mon[34668]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:45:06.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:06 smithi169 ceph-mon[34075]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:45:06.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:06 smithi006 ceph-mon[29835]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:45:08.602 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:08 smithi186 ceph-mon[34668]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:45:08.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:08 smithi169 ceph-mon[34075]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:45:08.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:08 smithi006 ceph-mon[29835]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:45:10.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:09 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:10.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:09 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:45:10.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:09 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:10.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:09 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:10.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:09 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:45:10.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:09 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:10.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:09 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:10.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:09 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:45:10.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:09 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:11.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:10 smithi186 ceph-mon[34668]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:45:11.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:10 smithi169 ceph-mon[34075]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:45:11.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:10 smithi006 ceph-mon[29835]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:45:12.679 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 5 on host 'smithi186' 2024-05-09T04:45:12.680 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:45:12.678+0000 7fbfbdffb700 1 -- 172.21.15.186:0/3750814327 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbfc0063c00 con 0x7fbfac0b2900 2024-05-09T04:45:12.680 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:45:12.681+0000 7fbfc6760700 1 -- 172.21.15.186:0/3750814327 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fbfac0b2900 msgr2=0x7fbfac0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:45:12.681 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:45:12.681+0000 7fbfc6760700 1 --2- 172.21.15.186:0/3750814327 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fbfac0b2900 0x7fbfac0b4e10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbfc019b960 tx=0x7fbfa8009260 comp rx=0 tx=0).stop 2024-05-09T04:45:12.681 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:45:12.681+0000 7fbfc6760700 1 -- 172.21.15.186:0/3750814327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfc0076690 msgr2=0x7fbfc019a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:45:12.681 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:45:12.681+0000 7fbfc6760700 1 --2- 172.21.15.186:0/3750814327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfc0076690 0x7fbfc019a8b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000b500 tx=0x7fbfb000b8c0 comp rx=0 tx=0).stop 2024-05-09T04:45:12.681 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:45:12.681+0000 7fbfc6760700 1 -- 172.21.15.186:0/3750814327 shutdown_connections 2024-05-09T04:45:12.681 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:45:12.681+0000 7fbfc6760700 1 --2- 172.21.15.186:0/3750814327 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfc010a000 0x7fbfc01a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:12.681 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:45:12.681+0000 7fbfc6760700 1 --2- 172.21.15.186:0/3750814327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfc0076690 0x7fbfc019a8b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:12.681 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:45:12.681+0000 7fbfc6760700 1 --2- 172.21.15.186:0/3750814327 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fbfac0b2900 0x7fbfac0b4e10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:12.681 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:45:12.681+0000 7fbfc6760700 1 --2- 172.21.15.186:0/3750814327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfc01074b0 0x7fbfc019adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:12.681 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:45:12.681+0000 7fbfc6760700 1 -- 172.21.15.186:0/3750814327 >> 172.21.15.186:0/3750814327 conn(0x7fbfc00fe890 msgr2=0x7fbfc0105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:45:12.681 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:45:12.681+0000 7fbfc6760700 1 -- 172.21.15.186:0/3750814327 shutdown_connections 2024-05-09T04:45:12.681 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:45:12.681+0000 7fbfc6760700 1 -- 172.21.15.186:0/3750814327 wait complete. 2024-05-09T04:45:12.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:12 smithi186 ceph-mon[34668]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:45:12.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:12 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:12.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:12 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:12.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:12 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:12.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:12 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:45:12.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:12 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:12.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:12 smithi006 ceph-mon[29835]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:45:12.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:12 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:12.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:12 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:12.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:12 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:12.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:12 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:45:12.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:12 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:13.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:12 smithi169 ceph-mon[34075]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:45:13.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:12 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:13.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:12 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:13.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:12 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:13.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:12 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:45:13.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:12 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:13.370 DEBUG:teuthology.orchestra.run.smithi186:osd.5> sudo journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.5.service 2024-05-09T04:45:13.373 INFO:tasks.cephadm:Deploying osd.6 on smithi006 with /dev/vg_nvme/lv_2... 2024-05-09T04:45:13.373 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-09T04:45:13.848 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:45:13.879 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:13 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:45:13.880 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:13 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:13.880 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:13 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:13.880 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:13 smithi186 ceph-mon[34668]: from='osd.5 [v2:172.21.15.186:6808/3644445439,v1:172.21.15.186:6809/3644445439]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T04:45:13.880 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:13 smithi186 ceph-mon[34668]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T04:45:13.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:13 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:45:13.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:13 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:13.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:13 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:13.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:13 smithi006 ceph-mon[29835]: from='osd.5 [v2:172.21.15.186:6808/3644445439,v1:172.21.15.186:6809/3644445439]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T04:45:13.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:13 smithi006 ceph-mon[29835]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T04:45:14.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:13 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:45:14.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:13 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:14.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:13 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:14.118 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:13 smithi169 ceph-mon[34075]: from='osd.5 [v2:172.21.15.186:6808/3644445439,v1:172.21.15.186:6809/3644445439]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T04:45:14.118 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:13 smithi169 ceph-mon[34075]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T04:45:14.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:14 smithi006 ceph-mon[29835]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:45:14.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:14 smithi006 ceph-mon[29835]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-09T04:45:14.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:14 smithi006 ceph-mon[29835]: from='osd.5 [v2:172.21.15.186:6808/3644445439,v1:172.21.15.186:6809/3644445439]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-09T04:45:14.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:14 smithi006 ceph-mon[29835]: osdmap e35: 6 total, 5 up, 6 in 2024-05-09T04:45:14.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:14 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T04:45:14.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:14 smithi006 ceph-mon[29835]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-09T04:45:14.974 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:14 smithi186 ceph-mon[34668]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:45:14.974 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:14 smithi186 ceph-mon[34668]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-09T04:45:14.974 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:14 smithi186 ceph-mon[34668]: from='osd.5 [v2:172.21.15.186:6808/3644445439,v1:172.21.15.186:6809/3644445439]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-09T04:45:14.975 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:14 smithi186 ceph-mon[34668]: osdmap e35: 6 total, 5 up, 6 in 2024-05-09T04:45:14.975 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:14 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T04:45:14.975 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:14 smithi186 ceph-mon[34668]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-09T04:45:14.975 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 04:45:14 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T04:45:14.669+0000 7fce47d18700 -1 osd.5 0 waiting for initial osdmap 2024-05-09T04:45:14.975 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 04:45:14 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T04:45:14.676+0000 7fce43acd700 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T04:45:15.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:14 smithi169 ceph-mon[34075]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:45:15.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:14 smithi169 ceph-mon[34075]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-09T04:45:15.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:14 smithi169 ceph-mon[34075]: from='osd.5 [v2:172.21.15.186:6808/3644445439,v1:172.21.15.186:6809/3644445439]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-09T04:45:15.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:14 smithi169 ceph-mon[34075]: osdmap e35: 6 total, 5 up, 6 in 2024-05-09T04:45:15.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:14 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T04:45:15.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:14 smithi169 ceph-mon[34075]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-09T04:45:15.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:15 smithi006 ceph-mon[29835]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-05-09T04:45:15.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:15 smithi006 ceph-mon[29835]: osdmap e36: 6 total, 5 up, 6 in 2024-05-09T04:45:15.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:15 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T04:45:15.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:15 smithi006 ceph-mon[29835]: osd.5 [v2:172.21.15.186:6808/3644445439,v1:172.21.15.186:6809/3644445439] boot 2024-05-09T04:45:15.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:15 smithi006 ceph-mon[29835]: osdmap e37: 6 total, 6 up, 6 in 2024-05-09T04:45:15.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:15 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T04:45:16.018 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:15 smithi169 ceph-mon[34075]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-05-09T04:45:16.018 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:15 smithi169 ceph-mon[34075]: osdmap e36: 6 total, 5 up, 6 in 2024-05-09T04:45:16.018 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:15 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T04:45:16.018 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:15 smithi169 ceph-mon[34075]: osd.5 [v2:172.21.15.186:6808/3644445439,v1:172.21.15.186:6809/3644445439] boot 2024-05-09T04:45:16.018 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:15 smithi169 ceph-mon[34075]: osdmap e37: 6 total, 6 up, 6 in 2024-05-09T04:45:16.018 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:15 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T04:45:16.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:15 smithi186 ceph-mon[34668]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-05-09T04:45:16.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:15 smithi186 ceph-mon[34668]: osdmap e36: 6 total, 5 up, 6 in 2024-05-09T04:45:16.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:15 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T04:45:16.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:15 smithi186 ceph-mon[34668]: osd.5 [v2:172.21.15.186:6808/3644445439,v1:172.21.15.186:6809/3644445439] boot 2024-05-09T04:45:16.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:15 smithi186 ceph-mon[34668]: osdmap e37: 6 total, 6 up, 6 in 2024-05-09T04:45:16.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:15 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T04:45:16.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:16 smithi186 ceph-mon[34668]: purged_snaps scrub starts 2024-05-09T04:45:16.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:16 smithi186 ceph-mon[34668]: purged_snaps scrub ok 2024-05-09T04:45:16.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:16 smithi186 ceph-mon[34668]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:45:16.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:16.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:16.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:16.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:16.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:16.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:16.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:45:16.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:16.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:16 smithi186 ceph-mon[34668]: osdmap e38: 6 total, 6 up, 6 in 2024-05-09T04:45:16.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:16 smithi006 ceph-mon[29835]: purged_snaps scrub starts 2024-05-09T04:45:16.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:16 smithi006 ceph-mon[29835]: purged_snaps scrub ok 2024-05-09T04:45:16.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:16 smithi006 ceph-mon[29835]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:45:16.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:16.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:16.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:16.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:16.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:16.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:16.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:45:16.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:16.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:16 smithi006 ceph-mon[29835]: osdmap e38: 6 total, 6 up, 6 in 2024-05-09T04:45:17.099 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-09T04:45:17.099 INFO:teuthology.orchestra.run.smithi006.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-09T04:45:17.100 INFO:teuthology.orchestra.run.smithi006.stderr: stderr: 10+0 records in 2024-05-09T04:45:17.100 INFO:teuthology.orchestra.run.smithi006.stderr:10+0 records out 2024-05-09T04:45:17.100 INFO:teuthology.orchestra.run.smithi006.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0336364 s, 312 MB/s 2024-05-09T04:45:17.100 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping successful for: 2024-05-09T04:45:17.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:16 smithi169 ceph-mon[34075]: purged_snaps scrub starts 2024-05-09T04:45:17.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:16 smithi169 ceph-mon[34075]: purged_snaps scrub ok 2024-05-09T04:45:17.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:16 smithi169 ceph-mon[34075]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:45:17.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:17.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:17.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:17.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:17.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:17.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:17.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:45:17.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:17.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:16 smithi169 ceph-mon[34075]: osdmap e38: 6 total, 6 up, 6 in 2024-05-09T04:45:17.919 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi006:vg_nvme/lv_2 2024-05-09T04:45:17.982 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:17 smithi006 ceph-mon[29835]: Detected new or changed devices on smithi186 2024-05-09T04:45:17.983 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:17 smithi006 ceph-mon[29835]: Adjusting osd_memory_target on smithi186 to 8544M 2024-05-09T04:45:17.983 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:17 smithi006 ceph-mon[29835]: osdmap e39: 6 total, 6 up, 6 in 2024-05-09T04:45:18.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:17 smithi186 ceph-mon[34668]: Detected new or changed devices on smithi186 2024-05-09T04:45:18.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:17 smithi186 ceph-mon[34668]: Adjusting osd_memory_target on smithi186 to 8544M 2024-05-09T04:45:18.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:17 smithi186 ceph-mon[34668]: osdmap e39: 6 total, 6 up, 6 in 2024-05-09T04:45:18.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:17 smithi169 ceph-mon[34075]: Detected new or changed devices on smithi186 2024-05-09T04:45:18.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:17 smithi169 ceph-mon[34075]: Adjusting osd_memory_target on smithi186 to 8544M 2024-05-09T04:45:18.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:17 smithi169 ceph-mon[34075]: osdmap e39: 6 total, 6 up, 6 in 2024-05-09T04:45:18.350 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:45:18.938 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:18 smithi006 ceph-mon[29835]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:19.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:18 smithi186 ceph-mon[34668]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:19.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:18 smithi169 ceph-mon[34075]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:20.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.687+0000 7f9e77800700 1 -- 172.21.15.6:0/4068140216 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e70105470 msgr2=0x7f9e70105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:45:20.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.687+0000 7f9e77800700 1 --2- 172.21.15.6:0/4068140216 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e70105470 0x7f9e70105880 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9e60009990 tx=0x7f9e60009ca0 comp rx=0 tx=0).stop 2024-05-09T04:45:20.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.687+0000 7f9e77800700 1 -- 172.21.15.6:0/4068140216 shutdown_connections 2024-05-09T04:45:20.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.687+0000 7f9e77800700 1 --2- 172.21.15.6:0/4068140216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e70105e10 0x7f9e7010c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:20.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.687+0000 7f9e77800700 1 --2- 172.21.15.6:0/4068140216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e70104340 0x7f9e70104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:20.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.687+0000 7f9e77800700 1 --2- 172.21.15.6:0/4068140216 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e70105470 0x7f9e70105880 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:20.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.687+0000 7f9e77800700 1 -- 172.21.15.6:0/4068140216 >> 172.21.15.6:0/4068140216 conn(0x7f9e700ff9c0 msgr2=0x7f9e70101e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:45:20.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.687+0000 7f9e77800700 1 -- 172.21.15.6:0/4068140216 shutdown_connections 2024-05-09T04:45:20.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.687+0000 7f9e77800700 1 -- 172.21.15.6:0/4068140216 wait complete. 2024-05-09T04:45:20.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.689+0000 7f9e77800700 1 Processor -- start 2024-05-09T04:45:20.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.689+0000 7f9e77800700 1 -- start start 2024-05-09T04:45:20.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.690+0000 7f9e77800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e70104340 0x7f9e7019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:45:20.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.690+0000 7f9e77800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e70105470 0x7f9e7019d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:45:20.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.690+0000 7f9e77800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e70105e10 0x7f9e701a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:45:20.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.690+0000 7f9e77800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7010fda0 con 0x7f9e70105470 2024-05-09T04:45:20.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.690+0000 7f9e77800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7010fc20 con 0x7f9e70104340 2024-05-09T04:45:20.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.690+0000 7f9e77800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7010ff20 con 0x7f9e70105e10 2024-05-09T04:45:20.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.690+0000 7f9e74e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e70104340 0x7f9e7019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:45:20.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.690+0000 7f9e74e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e70104340 0x7f9e7019cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:36706/0 (socket says 172.21.15.6:36706) 2024-05-09T04:45:20.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.690+0000 7f9e74e00700 1 -- 172.21.15.6:0/3337822483 learned_addr learned my addr 172.21.15.6:0/3337822483 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:45:20.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.690+0000 7f9e67e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e70105470 0x7f9e7019d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:45:20.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.690+0000 7f9e75800700 1 --2- 172.21.15.6:0/3337822483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e70105e10 0x7f9e701a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:45:20.693 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.691+0000 7f9e74e00700 1 -- 172.21.15.6:0/3337822483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e70105e10 msgr2=0x7f9e701a4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:45:20.693 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.691+0000 7f9e74e00700 1 --2- 172.21.15.6:0/3337822483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e70105e10 0x7f9e701a4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:20.693 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.691+0000 7f9e74e00700 1 -- 172.21.15.6:0/3337822483 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e70105470 msgr2=0x7f9e7019d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:45:20.693 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.691+0000 7f9e74e00700 1 --2- 172.21.15.6:0/3337822483 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e70105470 0x7f9e7019d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:20.693 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.691+0000 7f9e74e00700 1 -- 172.21.15.6:0/3337822483 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e701a46d0 con 0x7f9e70104340 2024-05-09T04:45:20.693 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.691+0000 7f9e74e00700 1 --2- 172.21.15.6:0/3337822483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e70104340 0x7f9e7019cb70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9e5800e940 tx=0x7f9e5800ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:45:20.693 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.691+0000 7f9e6e000700 1 -- 172.21.15.6:0/3337822483 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e58004220 con 0x7f9e70104340 2024-05-09T04:45:20.693 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.691+0000 7f9e6e000700 1 -- 172.21.15.6:0/3337822483 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e580043c0 con 0x7f9e70104340 2024-05-09T04:45:20.693 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.691+0000 7f9e6e000700 1 -- 172.21.15.6:0/3337822483 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e58003c60 con 0x7f9e70104340 2024-05-09T04:45:20.693 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.692+0000 7f9e77800700 1 -- 172.21.15.6:0/3337822483 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e701a49c0 con 0x7f9e70104340 2024-05-09T04:45:20.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.692+0000 7f9e77800700 1 -- 172.21.15.6:0/3337822483 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e701a4eb0 con 0x7f9e70104340 2024-05-09T04:45:20.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.693+0000 7f9e77800700 1 -- 172.21.15.6:0/3337822483 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e700512c0 con 0x7f9e70104340 2024-05-09T04:45:20.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.695+0000 7f9e6e000700 1 -- 172.21.15.6:0/3337822483 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f9e5800c7e0 con 0x7f9e70104340 2024-05-09T04:45:20.697 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.697+0000 7f9e6e000700 1 --2- 172.21.15.6:0/3337822483 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9e500b2880 0x7f9e500b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:45:20.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.697+0000 7f9e6e000700 1 -- 172.21.15.6:0/3337822483 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4533+0+0 (secure 0 0 0) 0x7f9e580e15c0 con 0x7f9e70104340 2024-05-09T04:45:20.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.697+0000 7f9e67e00700 1 --2- 172.21.15.6:0/3337822483 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9e500b2880 0x7f9e500b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:45:20.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.698+0000 7f9e67e00700 1 --2- 172.21.15.6:0/3337822483 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9e500b2880 0x7f9e500b4d90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9e6000b760 tx=0x7f9e6000b680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:45:20.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.702+0000 7f9e6e000700 1 -- 172.21.15.6:0/3337822483 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9e580aa100 con 0x7f9e70104340 2024-05-09T04:45:20.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:20 smithi006 ceph-mon[29835]: pgmap v161: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:20.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:20.951+0000 7f9e77800700 1 -- 172.21.15.6:0/3337822483 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f9e70077110 con 0x7f9e500b2880 2024-05-09T04:45:20.996 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:20 smithi186 ceph-mon[34668]: pgmap v161: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:21.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:20 smithi169 ceph-mon[34075]: pgmap v161: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:22.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:21 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:45:22.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:21 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:45:22.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:21 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:22.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:21 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:45:22.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:21 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:45:22.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:21 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:22.144 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:21 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:45:22.144 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:21 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:45:22.144 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:21 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:23.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:22 smithi186 ceph-mon[34668]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:45:23.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:22 smithi186 ceph-mon[34668]: pgmap v162: 1 pgs: 1 peering; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:23.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:22 smithi169 ceph-mon[34075]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:45:23.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:22 smithi169 ceph-mon[34075]: pgmap v162: 1 pgs: 1 peering; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:23.170 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:22 smithi006 ceph-mon[29835]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:45:23.170 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:22 smithi006 ceph-mon[29835]: pgmap v162: 1 pgs: 1 peering; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:25.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:24 smithi186 ceph-mon[34668]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-09T04:45:25.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:24 smithi169 ceph-mon[34075]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-09T04:45:25.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:24 smithi006 ceph-mon[29835]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-09T04:45:26.023 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:25 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1935432167' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2e8478cb-c8c9-47ce-a3a5-48ffc959ab24"}]: dispatch 2024-05-09T04:45:26.024 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:25 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1935432167' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2e8478cb-c8c9-47ce-a3a5-48ffc959ab24"}]': finished 2024-05-09T04:45:26.024 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:25 smithi006 ceph-mon[29835]: osdmap e40: 7 total, 6 up, 7 in 2024-05-09T04:45:26.024 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:25 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:45:26.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:25 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/1935432167' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2e8478cb-c8c9-47ce-a3a5-48ffc959ab24"}]: dispatch 2024-05-09T04:45:26.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:25 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/1935432167' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2e8478cb-c8c9-47ce-a3a5-48ffc959ab24"}]': finished 2024-05-09T04:45:26.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:25 smithi186 ceph-mon[34668]: osdmap e40: 7 total, 6 up, 7 in 2024-05-09T04:45:26.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:25 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:45:26.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:25 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/1935432167' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2e8478cb-c8c9-47ce-a3a5-48ffc959ab24"}]: dispatch 2024-05-09T04:45:26.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:25 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/1935432167' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2e8478cb-c8c9-47ce-a3a5-48ffc959ab24"}]': finished 2024-05-09T04:45:26.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:25 smithi169 ceph-mon[34075]: osdmap e40: 7 total, 6 up, 7 in 2024-05-09T04:45:26.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:25 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:45:27.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:26 smithi186 ceph-mon[34668]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T04:45:27.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:26 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/4272974394' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-6/activate.monmap"}]: dispatch 2024-05-09T04:45:27.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:26 smithi169 ceph-mon[34075]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T04:45:27.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:26 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/4272974394' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-6/activate.monmap"}]: dispatch 2024-05-09T04:45:27.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:26 smithi006 ceph-mon[29835]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T04:45:27.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:26 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/4272974394' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-6/activate.monmap"}]: dispatch 2024-05-09T04:45:29.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:28 smithi186 ceph-mon[34668]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-09T04:45:29.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:28 smithi169 ceph-mon[34075]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-09T04:45:29.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:28 smithi006 ceph-mon[29835]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-09T04:45:31.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:30 smithi186 ceph-mon[34668]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:45:31.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:30 smithi169 ceph-mon[34075]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:45:31.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:30 smithi006 ceph-mon[29835]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:45:33.035 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:32 smithi006 ceph-mon[29835]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:45:33.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:32 smithi186 ceph-mon[34668]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:45:33.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:32 smithi169 ceph-mon[34075]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:45:34.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:34 smithi006 ceph-mon[29835]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:35.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:34 smithi186 ceph-mon[34668]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:35.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:34 smithi169 ceph-mon[34075]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:36.092 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:35 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-09T04:45:36.093 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:35 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:36.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:35 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-09T04:45:36.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:35 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:36.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:35 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-09T04:45:36.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:35 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:37.019 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:36 smithi006 ceph-mon[29835]: Deploying daemon osd.6 on smithi006 2024-05-09T04:45:37.020 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:36 smithi006 ceph-mon[29835]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:37.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:36 smithi186 ceph-mon[34668]: Deploying daemon osd.6 on smithi006 2024-05-09T04:45:37.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:36 smithi186 ceph-mon[34668]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:37.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:36 smithi169 ceph-mon[34075]: Deploying daemon osd.6 on smithi006 2024-05-09T04:45:37.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:36 smithi169 ceph-mon[34075]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:38.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:38 smithi006 ceph-mon[29835]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:39.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:38 smithi186 ceph-mon[34668]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:39.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:38 smithi169 ceph-mon[34075]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:40.916 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:40 smithi006 ceph-mon[29835]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:41.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:40 smithi186 ceph-mon[34668]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:41.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:40 smithi169 ceph-mon[34075]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:43.076 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:42 smithi006 ceph-mon[29835]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:43.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:42 smithi186 ceph-mon[34668]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:43.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:42 smithi169 ceph-mon[34075]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:44.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:44 smithi006 ceph-mon[29835]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:45.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:44 smithi186 ceph-mon[34668]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:45.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:44 smithi169 ceph-mon[34075]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:46.184 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:45 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:46.185 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:45 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:46.185 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:45 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:45:46.185 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:45 smithi006 ceph-mon[29835]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:46.300 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:45 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:46.300 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:45 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:46.300 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:45 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:45:46.300 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:45 smithi169 ceph-mon[34075]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:46.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:45 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:46.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:45 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:46.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:45 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:45:46.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:45 smithi186 ceph-mon[34668]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:48.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:48 smithi006 ceph-mon[29835]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:48.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:48 smithi186 ceph-mon[34668]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:48.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:48 smithi169 ceph-mon[34075]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:50.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:49 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:50.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:49 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:50.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:49 smithi186 ceph-mon[34668]: from='osd.6 [v2:172.21.15.6:6818/3068252891,v1:172.21.15.6:6819/3068252891]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T04:45:50.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:49 smithi186 ceph-mon[34668]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T04:45:50.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:49 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:50.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:49 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:50.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:49 smithi169 ceph-mon[34075]: from='osd.6 [v2:172.21.15.6:6818/3068252891,v1:172.21.15.6:6819/3068252891]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T04:45:50.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:49 smithi169 ceph-mon[34075]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T04:45:50.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:49 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:50.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:49 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:50.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:49 smithi006 ceph-mon[29835]: from='osd.6 [v2:172.21.15.6:6818/3068252891,v1:172.21.15.6:6819/3068252891]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T04:45:50.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:49 smithi006 ceph-mon[29835]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T04:45:50.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:50.811+0000 7f9e6e000700 1 -- 172.21.15.6:0/3337822483 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9e70077110 con 0x7f9e500b2880 2024-05-09T04:45:50.813 INFO:teuthology.orchestra.run.smithi006.stdout:Created osd(s) 6 on host 'smithi006' 2024-05-09T04:45:50.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:50.817+0000 7f9e77800700 1 -- 172.21.15.6:0/3337822483 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9e500b2880 msgr2=0x7f9e500b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:45:50.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:50.817+0000 7f9e77800700 1 --2- 172.21.15.6:0/3337822483 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9e500b2880 0x7f9e500b4d90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9e6000b760 tx=0x7f9e6000b680 comp rx=0 tx=0).stop 2024-05-09T04:45:50.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:50.818+0000 7f9e77800700 1 -- 172.21.15.6:0/3337822483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e70104340 msgr2=0x7f9e7019cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:45:50.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:50.818+0000 7f9e77800700 1 --2- 172.21.15.6:0/3337822483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e70104340 0x7f9e7019cb70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9e5800e940 tx=0x7f9e5800ec50 comp rx=0 tx=0).stop 2024-05-09T04:45:50.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:50.818+0000 7f9e77800700 1 -- 172.21.15.6:0/3337822483 shutdown_connections 2024-05-09T04:45:50.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:50.818+0000 7f9e77800700 1 --2- 172.21.15.6:0/3337822483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e70105e10 0x7f9e701a4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:50.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:50.818+0000 7f9e77800700 1 --2- 172.21.15.6:0/3337822483 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9e500b2880 0x7f9e500b4d90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:50.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:50.818+0000 7f9e77800700 1 --2- 172.21.15.6:0/3337822483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e70104340 0x7f9e7019cb70 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:50.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:50.818+0000 7f9e77800700 1 --2- 172.21.15.6:0/3337822483 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e70105470 0x7f9e7019d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:50.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:50.818+0000 7f9e77800700 1 -- 172.21.15.6:0/3337822483 >> 172.21.15.6:0/3337822483 conn(0x7f9e700ff9c0 msgr2=0x7f9e700759f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:45:50.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:50.818+0000 7f9e77800700 1 -- 172.21.15.6:0/3337822483 shutdown_connections 2024-05-09T04:45:50.820 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:45:50.818+0000 7f9e77800700 1 -- 172.21.15.6:0/3337822483 wait complete. 2024-05-09T04:45:51.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:50 smithi186 ceph-mon[34668]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:51.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:50 smithi186 ceph-mon[34668]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-09T04:45:51.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:50 smithi186 ceph-mon[34668]: osdmap e41: 7 total, 6 up, 7 in 2024-05-09T04:45:51.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:50 smithi186 ceph-mon[34668]: from='osd.6 [v2:172.21.15.6:6818/3068252891,v1:172.21.15.6:6819/3068252891]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-05-09T04:45:51.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:50 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:45:51.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:50 smithi186 ceph-mon[34668]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-05-09T04:45:51.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:50 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:51.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:50 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:45:51.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:50 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:51.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:50 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:45:51.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:50 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:51.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:50 smithi169 ceph-mon[34075]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:51.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:50 smithi169 ceph-mon[34075]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-09T04:45:51.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:50 smithi169 ceph-mon[34075]: osdmap e41: 7 total, 6 up, 7 in 2024-05-09T04:45:51.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:50 smithi169 ceph-mon[34075]: from='osd.6 [v2:172.21.15.6:6818/3068252891,v1:172.21.15.6:6819/3068252891]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-05-09T04:45:51.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:50 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:45:51.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:50 smithi169 ceph-mon[34075]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-05-09T04:45:51.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:50 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:51.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:50 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:45:51.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:50 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:51.118 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:50 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:45:51.118 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:50 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:51.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:50 smithi006 ceph-mon[29835]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:51.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:50 smithi006 ceph-mon[29835]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-09T04:45:51.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:50 smithi006 ceph-mon[29835]: osdmap e41: 7 total, 6 up, 7 in 2024-05-09T04:45:51.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:50 smithi006 ceph-mon[29835]: from='osd.6 [v2:172.21.15.6:6818/3068252891,v1:172.21.15.6:6819/3068252891]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-05-09T04:45:51.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:50 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:45:51.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:50 smithi006 ceph-mon[29835]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-05-09T04:45:51.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:50 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:51.199 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:50 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:45:51.199 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:50 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:51.199 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:50 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:45:51.199 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:50 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:51.663 DEBUG:teuthology.orchestra.run.smithi006:osd.6> sudo journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.6.service 2024-05-09T04:45:51.670 INFO:tasks.cephadm:Deploying osd.7 on smithi169 with /dev/vg_nvme/lv_2... 2024-05-09T04:45:51.670 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-09T04:45:51.902 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:51 smithi006 ceph-mon[29835]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-05-09T04:45:51.902 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:51 smithi006 ceph-mon[29835]: osdmap e42: 7 total, 6 up, 7 in 2024-05-09T04:45:51.902 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:51 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:45:51.902 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:51 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:51.902 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:51 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:45:51.905 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T04:45:52.027 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:51 smithi169 ceph-mon[34075]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-05-09T04:45:52.027 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:51 smithi169 ceph-mon[34075]: osdmap e42: 7 total, 6 up, 7 in 2024-05-09T04:45:52.028 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:51 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:45:52.028 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:51 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:52.028 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:51 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:45:52.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:51 smithi186 ceph-mon[34668]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-05-09T04:45:52.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:51 smithi186 ceph-mon[34668]: osdmap e42: 7 total, 6 up, 7 in 2024-05-09T04:45:52.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:51 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:45:52.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:51 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:52.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:51 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:45:52.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:52 smithi006 ceph-mon[29835]: purged_snaps scrub starts 2024-05-09T04:45:52.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:52 smithi006 ceph-mon[29835]: purged_snaps scrub ok 2024-05-09T04:45:52.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:52 smithi006 ceph-mon[29835]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:52.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:52 smithi006 ceph-mon[29835]: osd.6 [v2:172.21.15.6:6818/3068252891,v1:172.21.15.6:6819/3068252891] boot 2024-05-09T04:45:52.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:52 smithi006 ceph-mon[29835]: osdmap e43: 7 total, 7 up, 7 in 2024-05-09T04:45:52.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:45:53.091 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:52 smithi169 ceph-mon[34075]: purged_snaps scrub starts 2024-05-09T04:45:53.091 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:52 smithi169 ceph-mon[34075]: purged_snaps scrub ok 2024-05-09T04:45:53.091 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:52 smithi169 ceph-mon[34075]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:53.092 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:52 smithi169 ceph-mon[34075]: osd.6 [v2:172.21.15.6:6818/3068252891,v1:172.21.15.6:6819/3068252891] boot 2024-05-09T04:45:53.092 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:52 smithi169 ceph-mon[34075]: osdmap e43: 7 total, 7 up, 7 in 2024-05-09T04:45:53.092 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:45:53.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:52 smithi186 ceph-mon[34668]: purged_snaps scrub starts 2024-05-09T04:45:53.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:52 smithi186 ceph-mon[34668]: purged_snaps scrub ok 2024-05-09T04:45:53.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:52 smithi186 ceph-mon[34668]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:45:53.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:52 smithi186 ceph-mon[34668]: osd.6 [v2:172.21.15.6:6818/3068252891,v1:172.21.15.6:6819/3068252891] boot 2024-05-09T04:45:53.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:52 smithi186 ceph-mon[34668]: osdmap e43: 7 total, 7 up, 7 in 2024-05-09T04:45:53.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:45:53.999 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-09T04:45:53.999 INFO:teuthology.orchestra.run.smithi169.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-09T04:45:53.999 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10+0 records in 2024-05-09T04:45:53.999 INFO:teuthology.orchestra.run.smithi169.stderr:10+0 records out 2024-05-09T04:45:53.999 INFO:teuthology.orchestra.run.smithi169.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0127161 s, 825 MB/s 2024-05-09T04:45:53.999 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 2024-05-09T04:45:53.999 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping successful for: 2024-05-09T04:45:54.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:53 smithi186 ceph-mon[34668]: osdmap e44: 7 total, 7 up, 7 in 2024-05-09T04:45:54.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:53 smithi169 ceph-mon[34075]: osdmap e44: 7 total, 7 up, 7 in 2024-05-09T04:45:54.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:53 smithi006 ceph-mon[29835]: osdmap e44: 7 total, 7 up, 7 in 2024-05-09T04:45:54.636 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_2 2024-05-09T04:45:54.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:54 smithi169 ceph-mon[34075]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:45:54.873 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T04:45:55.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:54 smithi186 ceph-mon[34668]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:45:55.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:54 smithi006 ceph-mon[29835]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:45:56.493 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.491+0000 7f126e11d700 1 -- 172.21.15.169:0/3188073694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1268109fe0 msgr2=0x7f126810c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:45:56.493 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.491+0000 7f126e11d700 1 --2- 172.21.15.169:0/3188073694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1268109fe0 0x7f126810c4f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f126000b0a0 tx=0x7f126000b3b0 comp rx=0 tx=0).stop 2024-05-09T04:45:56.493 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.491+0000 7f126e11d700 1 -- 172.21.15.169:0/3188073694 shutdown_connections 2024-05-09T04:45:56.493 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.491+0000 7f126e11d700 1 --2- 172.21.15.169:0/3188073694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1268109fe0 0x7f126810c4f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:56.493 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.491+0000 7f126e11d700 1 --2- 172.21.15.169:0/3188073694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f126806ccc0 0x7f1268106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:56.493 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.491+0000 7f126e11d700 1 --2- 172.21.15.169:0/3188073694 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12681074b0 0x7f1268109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:56.493 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.491+0000 7f126e11d700 1 -- 172.21.15.169:0/3188073694 >> 172.21.15.169:0/3188073694 conn(0x7f12680fe870 msgr2=0x7f1268100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:45:56.493 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.491+0000 7f126e11d700 1 -- 172.21.15.169:0/3188073694 shutdown_connections 2024-05-09T04:45:56.493 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.491+0000 7f126e11d700 1 -- 172.21.15.169:0/3188073694 wait complete. 2024-05-09T04:45:56.493 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.492+0000 7f126e11d700 1 Processor -- start 2024-05-09T04:45:56.494 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.492+0000 7f126e11d700 1 -- start start 2024-05-09T04:45:56.494 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.492+0000 7f126e11d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f126806ccc0 0x7f126819a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:45:56.494 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.492+0000 7f126e11d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12681074b0 0x7f126819ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:45:56.494 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.492+0000 7f126e11d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1268109fe0 0x7f12681a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:45:56.494 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.492+0000 7f126e11d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f126810fe00 con 0x7f1268109fe0 2024-05-09T04:45:56.494 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.492+0000 7f126e11d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f126810fc80 con 0x7f12681074b0 2024-05-09T04:45:56.494 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.492+0000 7f126e11d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f126810ff80 con 0x7f126806ccc0 2024-05-09T04:45:56.494 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.492+0000 7f1266ffd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12681074b0 0x7f126819ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:45:56.494 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.492+0000 7f1266ffd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12681074b0 0x7f126819ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:32940/0 (socket says 172.21.15.169:32940) 2024-05-09T04:45:56.494 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.492+0000 7f1266ffd700 1 -- 172.21.15.169:0/339098962 learned_addr learned my addr 172.21.15.169:0/339098962 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-05-09T04:45:56.494 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.492+0000 7f12677fe700 1 --2- 172.21.15.169:0/339098962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f126806ccc0 0x7f126819a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:45:56.494 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.493+0000 7f1266ffd700 1 -- 172.21.15.169:0/339098962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f126806ccc0 msgr2=0x7f126819a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:45:56.494 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.493+0000 7f1266ffd700 1 --2- 172.21.15.169:0/339098962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f126806ccc0 0x7f126819a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:56.495 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.493+0000 7f1266ffd700 1 -- 172.21.15.169:0/339098962 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1268109fe0 msgr2=0x7f12681a1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:45:56.495 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.493+0000 7f1266ffd700 1 --2- 172.21.15.169:0/339098962 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1268109fe0 0x7f12681a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:56.495 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.493+0000 7f1266ffd700 1 -- 172.21.15.169:0/339098962 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12681a2420 con 0x7f12681074b0 2024-05-09T04:45:56.495 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.493+0000 7f1266ffd700 1 --2- 172.21.15.169:0/339098962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12681074b0 0x7f126819ae00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f125c00b530 tx=0x7f125c00b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:45:56.495 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.493+0000 7f1264ff9700 1 -- 172.21.15.169:0/339098962 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c015020 con 0x7f12681074b0 2024-05-09T04:45:56.495 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.493+0000 7f1264ff9700 1 -- 172.21.15.169:0/339098962 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f125c011ce0 con 0x7f12681074b0 2024-05-09T04:45:56.495 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.493+0000 7f126e11d700 1 -- 172.21.15.169:0/339098962 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12681a2710 con 0x7f12681074b0 2024-05-09T04:45:56.495 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.493+0000 7f1264ff9700 1 -- 172.21.15.169:0/339098962 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c010e40 con 0x7f12681074b0 2024-05-09T04:45:56.495 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.493+0000 7f126e11d700 1 -- 172.21.15.169:0/339098962 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12681a2cd0 con 0x7f12681074b0 2024-05-09T04:45:56.496 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.494+0000 7f126e11d700 1 -- 172.21.15.169:0/339098962 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12680512c0 con 0x7f12681074b0 2024-05-09T04:45:56.496 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.495+0000 7f1264ff9700 1 -- 172.21.15.169:0/339098962 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f125c018020 con 0x7f12681074b0 2024-05-09T04:45:56.496 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.496+0000 7f1264ff9700 1 --2- 172.21.15.169:0/339098962 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f12500b2880 0x7f12500b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:45:56.496 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.496+0000 7f1264ff9700 1 -- 172.21.15.169:0/339098962 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4965+0+0 (secure 0 0 0) 0x7f125c0e12f0 con 0x7f12681074b0 2024-05-09T04:45:56.496 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.497+0000 7f12677fe700 1 --2- 172.21.15.169:0/339098962 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f12500b2880 0x7f12500b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:45:56.497 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.498+0000 7f12677fe700 1 --2- 172.21.15.169:0/339098962 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f12500b2880 0x7f12500b4d90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f126806c9c0 tx=0x7f1258009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:45:56.500 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.500+0000 7f1264ff9700 1 -- 172.21.15.169:0/339098962 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f125c01f020 con 0x7f12681074b0 2024-05-09T04:45:56.711 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:45:56.711+0000 7f126e11d700 1 -- 172.21.15.169:0/339098962 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f1268063c00 con 0x7f12500b2880 2024-05-09T04:45:56.838 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:56 smithi006 ceph-mon[29835]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:45:56.838 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:56.838 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:45:56.839 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:56.839 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:45:56.839 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:56.839 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:56.839 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:56.839 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:56.839 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:56.839 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:56.839 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:45:56.839 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:57.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:56 smithi186 ceph-mon[34668]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:45:57.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:57.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:45:57.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:57.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:45:57.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:57.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:57.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:57.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:57.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:57.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:57.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:45:57.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:57.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:56 smithi169 ceph-mon[34075]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:45:57.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:56 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:57.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:56 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:45:57.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:56 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:57.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:56 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:45:57.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:56 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:57.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:56 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:57.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:56 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:57.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:56 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:57.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:56 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:57.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:56 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:57.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:56 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:45:57.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:56 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:45:58.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:57 smithi186 ceph-mon[34668]: Detected new or changed devices on smithi006 2024-05-09T04:45:58.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:57 smithi186 ceph-mon[34668]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:45:58.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:57 smithi186 ceph-mon[34668]: Adjusting osd_memory_target on smithi006 to 5698M 2024-05-09T04:45:58.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:57 smithi169 ceph-mon[34075]: Detected new or changed devices on smithi006 2024-05-09T04:45:58.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:57 smithi169 ceph-mon[34075]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:45:58.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:57 smithi169 ceph-mon[34075]: Adjusting osd_memory_target on smithi006 to 5698M 2024-05-09T04:45:58.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:57 smithi006 ceph-mon[29835]: Detected new or changed devices on smithi006 2024-05-09T04:45:58.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:57 smithi006 ceph-mon[29835]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:45:58.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:57 smithi006 ceph-mon[29835]: Adjusting osd_memory_target on smithi006 to 5698M 2024-05-09T04:45:59.006 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:58 smithi169 ceph-mon[34075]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:45:59.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:58 smithi186 ceph-mon[34668]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:45:59.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:58 smithi006 ceph-mon[29835]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:46:00.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:59 smithi169 ceph-mon[34075]: from='client.? 172.21.15.169:0/1938967492' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f432ee2-42a4-4efa-9233-8e5a2fd65d10"}]: dispatch 2024-05-09T04:46:00.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:59 smithi169 ceph-mon[34075]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f432ee2-42a4-4efa-9233-8e5a2fd65d10"}]: dispatch 2024-05-09T04:46:00.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:59 smithi169 ceph-mon[34075]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f432ee2-42a4-4efa-9233-8e5a2fd65d10"}]': finished 2024-05-09T04:46:00.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:59 smithi169 ceph-mon[34075]: osdmap e45: 8 total, 7 up, 8 in 2024-05-09T04:46:00.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:45:59 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T04:46:00.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:59 smithi006 ceph-mon[29835]: from='client.? 172.21.15.169:0/1938967492' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f432ee2-42a4-4efa-9233-8e5a2fd65d10"}]: dispatch 2024-05-09T04:46:00.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:59 smithi006 ceph-mon[29835]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f432ee2-42a4-4efa-9233-8e5a2fd65d10"}]: dispatch 2024-05-09T04:46:00.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:59 smithi006 ceph-mon[29835]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f432ee2-42a4-4efa-9233-8e5a2fd65d10"}]': finished 2024-05-09T04:46:00.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:59 smithi006 ceph-mon[29835]: osdmap e45: 8 total, 7 up, 8 in 2024-05-09T04:46:00.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:45:59 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T04:46:00.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:59 smithi186 ceph-mon[34668]: from='client.? 172.21.15.169:0/1938967492' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f432ee2-42a4-4efa-9233-8e5a2fd65d10"}]: dispatch 2024-05-09T04:46:00.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:59 smithi186 ceph-mon[34668]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f432ee2-42a4-4efa-9233-8e5a2fd65d10"}]: dispatch 2024-05-09T04:46:00.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:59 smithi186 ceph-mon[34668]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f432ee2-42a4-4efa-9233-8e5a2fd65d10"}]': finished 2024-05-09T04:46:00.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:59 smithi186 ceph-mon[34668]: osdmap e45: 8 total, 7 up, 8 in 2024-05-09T04:46:00.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:45:59 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T04:46:01.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:00 smithi169 ceph-mon[34075]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:46:01.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:00 smithi169 ceph-mon[34075]: from='client.? 172.21.15.169:0/2089044788' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-7/activate.monmap"}]: dispatch 2024-05-09T04:46:01.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:00 smithi006 ceph-mon[29835]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:46:01.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:00 smithi006 ceph-mon[29835]: from='client.? 172.21.15.169:0/2089044788' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-7/activate.monmap"}]: dispatch 2024-05-09T04:46:01.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:00 smithi186 ceph-mon[34668]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:46:01.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:00 smithi186 ceph-mon[34668]: from='client.? 172.21.15.169:0/2089044788' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-7/activate.monmap"}]: dispatch 2024-05-09T04:46:03.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:02 smithi006 ceph-mon[29835]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:46:03.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:02 smithi186 ceph-mon[34668]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:46:03.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:02 smithi169 ceph-mon[34075]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:46:05.058 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:04 smithi169 ceph-mon[34075]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:46:05.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:04 smithi006 ceph-mon[29835]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:46:05.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:04 smithi186 ceph-mon[34668]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:46:07.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:06 smithi006 ceph-mon[29835]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:46:07.245 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:06 smithi169 ceph-mon[34075]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:46:07.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:06 smithi186 ceph-mon[34668]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:46:08.157 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:07 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-09T04:46:08.157 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:07 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:08.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:07 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-09T04:46:08.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:07 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:08.250 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:07 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-09T04:46:08.251 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:07 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:08.970 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:08 smithi169 ceph-mon[34075]: Deploying daemon osd.7 on smithi169 2024-05-09T04:46:08.970 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:08 smithi169 ceph-mon[34075]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:46:09.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:08 smithi006 ceph-mon[29835]: Deploying daemon osd.7 on smithi169 2024-05-09T04:46:09.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:08 smithi006 ceph-mon[29835]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:46:09.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:08 smithi186 ceph-mon[34668]: Deploying daemon osd.7 on smithi169 2024-05-09T04:46:09.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:08 smithi186 ceph-mon[34668]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:46:10.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:09 smithi169 ceph-mon[34075]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:46:10.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:09 smithi006 ceph-mon[29835]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:46:10.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:09 smithi186 ceph-mon[34668]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:46:12.791 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:12 smithi169 ceph-mon[34075]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:46:12.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:12 smithi186 ceph-mon[34668]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:46:12.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:12 smithi006 ceph-mon[29835]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:46:14.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:14 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:14.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:14 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:14.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:14 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:46:14.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:14 smithi006 ceph-mon[29835]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:46:14.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:14 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:14.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:14 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:14.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:14 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:46:14.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:14 smithi186 ceph-mon[34668]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:46:14.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:14 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:14.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:14 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:14.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:14 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:46:14.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:14 smithi169 ceph-mon[34075]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:46:16.512 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 7 on host 'smithi169' 2024-05-09T04:46:16.512 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:46:16.511+0000 7f1264ff9700 1 -- 172.21.15.169:0/339098962 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1268063c00 con 0x7f12500b2880 2024-05-09T04:46:16.514 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:46:16.514+0000 7f126e11d700 1 -- 172.21.15.169:0/339098962 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f12500b2880 msgr2=0x7f12500b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:16.514 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:46:16.514+0000 7f126e11d700 1 --2- 172.21.15.169:0/339098962 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f12500b2880 0x7f12500b4d90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f126806c9c0 tx=0x7f1258009260 comp rx=0 tx=0).stop 2024-05-09T04:46:16.514 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:46:16.514+0000 7f126e11d700 1 -- 172.21.15.169:0/339098962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12681074b0 msgr2=0x7f126819ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:16.514 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:46:16.514+0000 7f126e11d700 1 --2- 172.21.15.169:0/339098962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12681074b0 0x7f126819ae00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f125c00b530 tx=0x7f125c00b840 comp rx=0 tx=0).stop 2024-05-09T04:46:16.514 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:46:16.514+0000 7f126e11d700 1 -- 172.21.15.169:0/339098962 shutdown_connections 2024-05-09T04:46:16.514 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:46:16.514+0000 7f126e11d700 1 --2- 172.21.15.169:0/339098962 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1268109fe0 0x7f12681a1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:16.514 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:46:16.514+0000 7f126e11d700 1 --2- 172.21.15.169:0/339098962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f126806ccc0 0x7f126819a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:16.514 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:46:16.514+0000 7f126e11d700 1 --2- 172.21.15.169:0/339098962 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f12500b2880 0x7f12500b4d90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:16.514 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:46:16.514+0000 7f126e11d700 1 --2- 172.21.15.169:0/339098962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12681074b0 0x7f126819ae00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:16.514 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:46:16.514+0000 7f126e11d700 1 -- 172.21.15.169:0/339098962 >> 172.21.15.169:0/339098962 conn(0x7f12680fe870 msgr2=0x7f1268105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:46:16.514 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:46:16.514+0000 7f126e11d700 1 -- 172.21.15.169:0/339098962 shutdown_connections 2024-05-09T04:46:16.515 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:46:16.514+0000 7f126e11d700 1 -- 172.21.15.169:0/339098962 wait complete. 2024-05-09T04:46:16.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:16 smithi186 ceph-mon[34668]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:46:16.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:16.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:16.855 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:16 smithi169 ceph-mon[34075]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:46:16.855 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:16.855 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:16.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:16 smithi006 ceph-mon[29835]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:46:16.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:16.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:17.495 DEBUG:teuthology.orchestra.run.smithi169:osd.7> sudo journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.7.service 2024-05-09T04:46:17.498 INFO:tasks.cephadm:Deploying osd.8 on smithi186 with /dev/vg_nvme/lv_2... 2024-05-09T04:46:17.498 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-09T04:46:17.744 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T04:46:17.789 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:17 smithi169 ceph-mon[34075]: from='osd.7 [v2:172.21.15.169:6816/1303584711,v1:172.21.15.169:6817/1303584711]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T04:46:17.789 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:17 smithi169 ceph-mon[34075]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T04:46:17.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:17 smithi186 ceph-mon[34668]: from='osd.7 [v2:172.21.15.169:6816/1303584711,v1:172.21.15.169:6817/1303584711]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T04:46:17.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:17 smithi186 ceph-mon[34668]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T04:46:17.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:17 smithi006 ceph-mon[29835]: from='osd.7 [v2:172.21.15.169:6816/1303584711,v1:172.21.15.169:6817/1303584711]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T04:46:17.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:17 smithi006 ceph-mon[29835]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T04:46:18.696 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:18 smithi169 ceph-mon[34075]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:46:18.696 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:18 smithi169 ceph-mon[34075]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-09T04:46:18.696 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:18 smithi169 ceph-mon[34075]: from='osd.7 [v2:172.21.15.169:6816/1303584711,v1:172.21.15.169:6817/1303584711]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T04:46:18.696 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:18 smithi169 ceph-mon[34075]: osdmap e46: 8 total, 7 up, 8 in 2024-05-09T04:46:18.696 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:18 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T04:46:18.696 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:18 smithi169 ceph-mon[34075]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T04:46:18.696 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:18 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:18.696 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:18 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:18.697 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 04:46:18 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T04:46:18.544+0000 7f0a4532f700 -1 osd.7 0 waiting for initial osdmap 2024-05-09T04:46:18.697 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 04:46:18 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T04:46:18.557+0000 7f0a410e4700 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T04:46:18.829 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:18 smithi186 ceph-mon[34668]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:46:18.829 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:18 smithi186 ceph-mon[34668]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-09T04:46:18.829 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:18 smithi186 ceph-mon[34668]: from='osd.7 [v2:172.21.15.169:6816/1303584711,v1:172.21.15.169:6817/1303584711]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T04:46:18.830 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:18 smithi186 ceph-mon[34668]: osdmap e46: 8 total, 7 up, 8 in 2024-05-09T04:46:18.830 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:18 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T04:46:18.830 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:18 smithi186 ceph-mon[34668]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T04:46:18.830 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:18 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:18.830 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:18 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:18.849 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:18 smithi006 ceph-mon[29835]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:46:18.850 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:18 smithi006 ceph-mon[29835]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-09T04:46:18.850 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:18 smithi006 ceph-mon[29835]: from='osd.7 [v2:172.21.15.169:6816/1303584711,v1:172.21.15.169:6817/1303584711]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T04:46:18.850 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:18 smithi006 ceph-mon[29835]: osdmap e46: 8 total, 7 up, 8 in 2024-05-09T04:46:18.851 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:18 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T04:46:18.851 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:18 smithi006 ceph-mon[29835]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T04:46:18.851 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:18 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:18.851 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:18 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:19.728 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-09T04:46:19.729 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-09T04:46:19.729 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2024-05-09T04:46:19.729 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2024-05-09T04:46:19.729 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126591 s, 828 MB/s 2024-05-09T04:46:19.729 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2024-05-09T04:46:19.773 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:19 smithi169 ceph-mon[34075]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-05-09T04:46:19.773 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:19 smithi169 ceph-mon[34075]: osdmap e47: 8 total, 7 up, 8 in 2024-05-09T04:46:19.773 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:19 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T04:46:19.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:19 smithi186 ceph-mon[34668]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-05-09T04:46:19.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:19 smithi186 ceph-mon[34668]: osdmap e47: 8 total, 7 up, 8 in 2024-05-09T04:46:19.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:19 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T04:46:19.883 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:19 smithi006 ceph-mon[29835]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-05-09T04:46:19.883 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:19 smithi006 ceph-mon[29835]: osdmap e47: 8 total, 7 up, 8 in 2024-05-09T04:46:19.883 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:19 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T04:46:20.429 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi186:vg_nvme/lv_2 2024-05-09T04:46:20.684 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T04:46:20.724 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:20 smithi186 ceph-mon[34668]: purged_snaps scrub starts 2024-05-09T04:46:20.724 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:20 smithi186 ceph-mon[34668]: purged_snaps scrub ok 2024-05-09T04:46:20.724 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:20 smithi186 ceph-mon[34668]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:46:20.724 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:20 smithi186 ceph-mon[34668]: osd.7 [v2:172.21.15.169:6816/1303584711,v1:172.21.15.169:6817/1303584711] boot 2024-05-09T04:46:20.725 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:20 smithi186 ceph-mon[34668]: osdmap e48: 8 total, 8 up, 8 in 2024-05-09T04:46:20.725 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:20 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T04:46:20.725 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:20 smithi186 ceph-mon[34668]: osdmap e49: 8 total, 8 up, 8 in 2024-05-09T04:46:20.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:20 smithi169 ceph-mon[34075]: purged_snaps scrub starts 2024-05-09T04:46:20.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:20 smithi169 ceph-mon[34075]: purged_snaps scrub ok 2024-05-09T04:46:20.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:20 smithi169 ceph-mon[34075]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:46:20.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:20 smithi169 ceph-mon[34075]: osd.7 [v2:172.21.15.169:6816/1303584711,v1:172.21.15.169:6817/1303584711] boot 2024-05-09T04:46:20.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:20 smithi169 ceph-mon[34075]: osdmap e48: 8 total, 8 up, 8 in 2024-05-09T04:46:20.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:20 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T04:46:20.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:20 smithi169 ceph-mon[34075]: osdmap e49: 8 total, 8 up, 8 in 2024-05-09T04:46:20.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:20 smithi006 ceph-mon[29835]: purged_snaps scrub starts 2024-05-09T04:46:20.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:20 smithi006 ceph-mon[29835]: purged_snaps scrub ok 2024-05-09T04:46:20.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:20 smithi006 ceph-mon[29835]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:46:20.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:20 smithi006 ceph-mon[29835]: osd.7 [v2:172.21.15.169:6816/1303584711,v1:172.21.15.169:6817/1303584711] boot 2024-05-09T04:46:20.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:20 smithi006 ceph-mon[29835]: osdmap e48: 8 total, 8 up, 8 in 2024-05-09T04:46:20.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:20 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T04:46:20.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:20 smithi006 ceph-mon[29835]: osdmap e49: 8 total, 8 up, 8 in 2024-05-09T04:46:22.364 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.362+0000 7f0501556700 1 -- 172.21.15.186:0/3300459951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04fc103240 msgr2=0x7f04fc103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:22.364 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.362+0000 7f0501556700 1 --2- 172.21.15.186:0/3300459951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04fc103240 0x7f04fc103650 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f04ec009990 tx=0x7f04ec009ca0 comp rx=0 tx=0).stop 2024-05-09T04:46:22.364 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.362+0000 7f0501556700 1 -- 172.21.15.186:0/3300459951 shutdown_connections 2024-05-09T04:46:22.364 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.362+0000 7f0501556700 1 --2- 172.21.15.186:0/3300459951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04fc104d10 0x7f04fc10b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:22.364 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.362+0000 7f0501556700 1 --2- 172.21.15.186:0/3300459951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04fc103240 0x7f04fc103650 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:22.364 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.362+0000 7f0501556700 1 --2- 172.21.15.186:0/3300459951 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04fc104370 0x7f04fc104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:22.364 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.362+0000 7f0501556700 1 -- 172.21.15.186:0/3300459951 >> 172.21.15.186:0/3300459951 conn(0x7f04fc0fe880 msgr2=0x7f04fc100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:46:22.364 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.362+0000 7f0501556700 1 -- 172.21.15.186:0/3300459951 shutdown_connections 2024-05-09T04:46:22.364 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.362+0000 7f0501556700 1 -- 172.21.15.186:0/3300459951 wait complete. 2024-05-09T04:46:22.364 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.363+0000 7f0501556700 1 Processor -- start 2024-05-09T04:46:22.364 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.363+0000 7f0501556700 1 -- start start 2024-05-09T04:46:22.365 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.363+0000 7f0501556700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04fc103240 0x7f04fc19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:46:22.365 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.363+0000 7f0501556700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04fc104370 0x7f04fc19aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:46:22.365 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.363+0000 7f0501556700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04fc104d10 0x7f04fc1a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:46:22.365 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.363+0000 7f0501556700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc10ed00 con 0x7f04fc104d10 2024-05-09T04:46:22.365 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.363+0000 7f0501556700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc10eb80 con 0x7f04fc103240 2024-05-09T04:46:22.365 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.363+0000 7f0501556700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc10ee80 con 0x7f04fc104370 2024-05-09T04:46:22.365 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.363+0000 7f04fa7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04fc104370 0x7f04fc19aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:46:22.365 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.363+0000 7f04fa7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04fc104370 0x7f04fc19aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:47052/0 (socket says 172.21.15.186:47052) 2024-05-09T04:46:22.365 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.363+0000 7f04fa7fc700 1 -- 172.21.15.186:0/2862764806 learned_addr learned my addr 172.21.15.186:0/2862764806 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-05-09T04:46:22.365 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.363+0000 7f04faffd700 1 --2- 172.21.15.186:0/2862764806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04fc103240 0x7f04fc19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:46:22.365 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.363+0000 7f04fa7fc700 1 -- 172.21.15.186:0/2862764806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04fc103240 msgr2=0x7f04fc19a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:22.365 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.363+0000 7f04fa7fc700 1 --2- 172.21.15.186:0/2862764806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04fc103240 0x7f04fc19a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:22.365 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.363+0000 7f04fa7fc700 1 -- 172.21.15.186:0/2862764806 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04fc104d10 msgr2=0x7f04fc1a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:22.365 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.363+0000 7f04fa7fc700 1 --2- 172.21.15.186:0/2862764806 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04fc104d10 0x7f04fc1a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:22.365 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.363+0000 7f04fa7fc700 1 -- 172.21.15.186:0/2862764806 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04fc1a24f0 con 0x7f04fc104370 2024-05-09T04:46:22.366 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.364+0000 7f04fa7fc700 1 --2- 172.21.15.186:0/2862764806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04fc104370 0x7f04fc19aed0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f04f000b530 tx=0x7f04f000b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:46:22.366 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.364+0000 7f04e3fff700 1 -- 172.21.15.186:0/2862764806 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0015020 con 0x7f04fc104370 2024-05-09T04:46:22.366 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.364+0000 7f04e3fff700 1 -- 172.21.15.186:0/2862764806 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04f0011ce0 con 0x7f04fc104370 2024-05-09T04:46:22.366 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.364+0000 7f0501556700 1 -- 172.21.15.186:0/2862764806 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04fc1a27e0 con 0x7f04fc104370 2024-05-09T04:46:22.366 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.364+0000 7f04e3fff700 1 -- 172.21.15.186:0/2862764806 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0010d40 con 0x7f04fc104370 2024-05-09T04:46:22.366 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.364+0000 7f0501556700 1 -- 172.21.15.186:0/2862764806 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04fc1a2da0 con 0x7f04fc104370 2024-05-09T04:46:22.366 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.365+0000 7f0501556700 1 -- 172.21.15.186:0/2862764806 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04fc0512c0 con 0x7f04fc104370 2024-05-09T04:46:22.367 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.366+0000 7f04e3fff700 1 -- 172.21.15.186:0/2862764806 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f04f0018020 con 0x7f04fc104370 2024-05-09T04:46:22.367 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.367+0000 7f04e3fff700 1 --2- 172.21.15.186:0/2862764806 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f04e40b2880 0x7f04e40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:46:22.367 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.367+0000 7f04e3fff700 1 -- 172.21.15.186:0/2862764806 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5397+0+0 (secure 0 0 0) 0x7f04f00e13d0 con 0x7f04fc104370 2024-05-09T04:46:22.367 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.368+0000 7f04faffd700 1 --2- 172.21.15.186:0/2862764806 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f04e40b2880 0x7f04e40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:46:22.369 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.369+0000 7f04faffd700 1 --2- 172.21.15.186:0/2862764806 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f04e40b2880 0x7f04e40b4d90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f04ec00b6c0 tx=0x7f04ec00b720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:46:22.371 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.371+0000 7f04e3fff700 1 -- 172.21.15.186:0/2862764806 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f04f001f020 con 0x7f04fc104370 2024-05-09T04:46:22.582 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:22.583+0000 7f0501556700 1 -- 172.21.15.186:0/2862764806 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f04fc100940 con 0x7f04e40b2880 2024-05-09T04:46:22.583 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:22 smithi186 ceph-mon[34668]: Detected new or changed devices on smithi169 2024-05-09T04:46:22.583 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:22.583 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:22.583 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:22.583 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:22.583 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:22.583 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:22 smithi186 ceph-mon[34668]: Adjusting osd_memory_target on smithi169 to 5696M 2024-05-09T04:46:22.583 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:22.583 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:22.583 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:46:22.583 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:22.583 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:22 smithi186 ceph-mon[34668]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:46:22.583 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:22 smithi186 ceph-mon[34668]: osdmap e50: 8 total, 8 up, 8 in 2024-05-09T04:46:22.584 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:46:22.584 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:22.584 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:46:22.584 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:22.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:22 smithi169 ceph-mon[34075]: Detected new or changed devices on smithi169 2024-05-09T04:46:22.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:22.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:22.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:22.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:22.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:22.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:22 smithi169 ceph-mon[34075]: Adjusting osd_memory_target on smithi169 to 5696M 2024-05-09T04:46:22.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:22.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:22.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:46:22.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:22.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:22 smithi169 ceph-mon[34075]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:46:22.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:22 smithi169 ceph-mon[34075]: osdmap e50: 8 total, 8 up, 8 in 2024-05-09T04:46:22.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:46:22.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:22.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:46:22.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:22.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:22 smithi006 ceph-mon[29835]: Detected new or changed devices on smithi169 2024-05-09T04:46:22.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:22.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:22.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:22.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:22.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:22.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:22 smithi006 ceph-mon[29835]: Adjusting osd_memory_target on smithi169 to 5696M 2024-05-09T04:46:22.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:22.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:22.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:46:22.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:22.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:22 smithi006 ceph-mon[29835]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:46:22.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:22 smithi006 ceph-mon[29835]: osdmap e50: 8 total, 8 up, 8 in 2024-05-09T04:46:22.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:46:22.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:22.699 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:46:22.699 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:23.485 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:23 smithi186 ceph-mon[34668]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:46:23.486 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:23 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:46:23.486 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:23 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:46:23.486 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:23 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:23.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:23 smithi006 ceph-mon[29835]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:46:23.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:23 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:46:23.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:23 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:46:23.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:23 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:23.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:23 smithi169 ceph-mon[34075]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:46:23.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:23 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:46:23.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:23 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:46:23.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:23 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:24.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:24 smithi186 ceph-mon[34668]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:46:24.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:24 smithi006 ceph-mon[29835]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:46:24.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:24 smithi169 ceph-mon[34075]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:46:25.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:25 smithi006 ceph-mon[29835]: from='client.? 172.21.15.186:0/1447565769' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf5dca81-131b-48e7-9417-d01d4be47316"}]: dispatch 2024-05-09T04:46:25.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:25 smithi006 ceph-mon[29835]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf5dca81-131b-48e7-9417-d01d4be47316"}]: dispatch 2024-05-09T04:46:25.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:25 smithi006 ceph-mon[29835]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf5dca81-131b-48e7-9417-d01d4be47316"}]': finished 2024-05-09T04:46:25.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:25 smithi186 ceph-mon[34668]: from='client.? 172.21.15.186:0/1447565769' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf5dca81-131b-48e7-9417-d01d4be47316"}]: dispatch 2024-05-09T04:46:25.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:25 smithi186 ceph-mon[34668]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf5dca81-131b-48e7-9417-d01d4be47316"}]: dispatch 2024-05-09T04:46:25.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:25 smithi186 ceph-mon[34668]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf5dca81-131b-48e7-9417-d01d4be47316"}]': finished 2024-05-09T04:46:25.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:25 smithi169 ceph-mon[34075]: from='client.? 172.21.15.186:0/1447565769' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf5dca81-131b-48e7-9417-d01d4be47316"}]: dispatch 2024-05-09T04:46:25.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:25 smithi169 ceph-mon[34075]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf5dca81-131b-48e7-9417-d01d4be47316"}]: dispatch 2024-05-09T04:46:25.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:25 smithi169 ceph-mon[34075]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf5dca81-131b-48e7-9417-d01d4be47316"}]': finished 2024-05-09T04:46:26.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:26 smithi006 ceph-mon[29835]: osdmap e51: 9 total, 8 up, 9 in 2024-05-09T04:46:26.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:26 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T04:46:26.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:26 smithi006 ceph-mon[29835]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-05-09T04:46:26.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:26 smithi006 ceph-mon[29835]: from='client.? 172.21.15.186:0/3861049290' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-8/activate.monmap"}]: dispatch 2024-05-09T04:46:26.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:26 smithi186 ceph-mon[34668]: osdmap e51: 9 total, 8 up, 9 in 2024-05-09T04:46:26.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:26 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T04:46:26.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:26 smithi186 ceph-mon[34668]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-05-09T04:46:26.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:26 smithi186 ceph-mon[34668]: from='client.? 172.21.15.186:0/3861049290' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-8/activate.monmap"}]: dispatch 2024-05-09T04:46:26.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:26 smithi169 ceph-mon[34075]: osdmap e51: 9 total, 8 up, 9 in 2024-05-09T04:46:26.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:26 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T04:46:26.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:26 smithi169 ceph-mon[34075]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-05-09T04:46:26.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:26 smithi169 ceph-mon[34075]: from='client.? 172.21.15.186:0/3861049290' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-8/activate.monmap"}]: dispatch 2024-05-09T04:46:28.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:28 smithi186 ceph-mon[34668]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-05-09T04:46:28.854 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:28 smithi169 ceph-mon[34075]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-05-09T04:46:28.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:28 smithi006 ceph-mon[29835]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-05-09T04:46:30.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:30 smithi186 ceph-mon[34668]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T04:46:30.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:30 smithi169 ceph-mon[34075]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T04:46:30.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:30 smithi006 ceph-mon[29835]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T04:46:32.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:32 smithi186 ceph-mon[34668]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:46:32.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:32 smithi169 ceph-mon[34075]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:46:32.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:32 smithi006 ceph-mon[29835]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:46:33.780 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:33 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-09T04:46:33.780 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:33 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:33.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:33 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-09T04:46:33.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:33 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:33.950 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:33 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-09T04:46:33.950 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:33 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:34.823 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:34 smithi186 ceph-mon[34668]: Deploying daemon osd.8 on smithi186 2024-05-09T04:46:34.823 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:34 smithi186 ceph-mon[34668]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:46:34.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:34 smithi169 ceph-mon[34075]: Deploying daemon osd.8 on smithi186 2024-05-09T04:46:34.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:34 smithi169 ceph-mon[34075]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:46:34.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:34 smithi006 ceph-mon[29835]: Deploying daemon osd.8 on smithi186 2024-05-09T04:46:34.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:34 smithi006 ceph-mon[29835]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:46:36.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:36 smithi186 ceph-mon[34668]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:46:36.890 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:36 smithi169 ceph-mon[34075]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:46:36.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:36 smithi006 ceph-mon[29835]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:46:38.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:38 smithi186 ceph-mon[34668]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:46:38.870 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:38 smithi169 ceph-mon[34075]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:46:38.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:38 smithi006 ceph-mon[29835]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:46:40.055 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:39 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:40.055 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:39 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:46:40.055 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:39 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:40.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:39 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:40.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:39 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:46:40.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:39 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:40.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:39 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:40.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:39 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:46:40.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:39 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:41.002 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:40 smithi186 ceph-mon[34668]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:46:41.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:40 smithi169 ceph-mon[34075]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:46:41.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:40 smithi006 ceph-mon[29835]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:46:42.575 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:42.574+0000 7f04e3fff700 1 -- 172.21.15.186:0/2862764806 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f04fc100940 con 0x7f04e40b2880 2024-05-09T04:46:42.575 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 8 on host 'smithi186' 2024-05-09T04:46:42.577 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:42.577+0000 7f0501556700 1 -- 172.21.15.186:0/2862764806 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f04e40b2880 msgr2=0x7f04e40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:42.577 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:42.577+0000 7f0501556700 1 --2- 172.21.15.186:0/2862764806 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f04e40b2880 0x7f04e40b4d90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f04ec00b6c0 tx=0x7f04ec00b720 comp rx=0 tx=0).stop 2024-05-09T04:46:42.578 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:42.577+0000 7f0501556700 1 -- 172.21.15.186:0/2862764806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04fc104370 msgr2=0x7f04fc19aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:42.578 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:42.577+0000 7f0501556700 1 --2- 172.21.15.186:0/2862764806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04fc104370 0x7f04fc19aed0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f04f000b530 tx=0x7f04f000b840 comp rx=0 tx=0).stop 2024-05-09T04:46:42.578 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:42.577+0000 7f0501556700 1 -- 172.21.15.186:0/2862764806 shutdown_connections 2024-05-09T04:46:42.578 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:42.577+0000 7f0501556700 1 --2- 172.21.15.186:0/2862764806 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04fc104d10 0x7f04fc1a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:42.578 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:42.577+0000 7f0501556700 1 --2- 172.21.15.186:0/2862764806 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f04e40b2880 0x7f04e40b4d90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:42.578 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:42.577+0000 7f0501556700 1 --2- 172.21.15.186:0/2862764806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04fc103240 0x7f04fc19a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:42.578 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:42.577+0000 7f0501556700 1 --2- 172.21.15.186:0/2862764806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04fc104370 0x7f04fc19aed0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:42.578 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:42.577+0000 7f0501556700 1 -- 172.21.15.186:0/2862764806 >> 172.21.15.186:0/2862764806 conn(0x7f04fc0fe880 msgr2=0x7f04fc100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:46:42.578 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:42.578+0000 7f0501556700 1 -- 172.21.15.186:0/2862764806 shutdown_connections 2024-05-09T04:46:42.578 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:46:42.578+0000 7f0501556700 1 -- 172.21.15.186:0/2862764806 wait complete. 2024-05-09T04:46:43.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:42 smithi186 ceph-mon[34668]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:46:43.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:42 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:43.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:42 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:43.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:42 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:43.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:42 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:43.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:42 smithi186 ceph-mon[34668]: from='osd.8 [v2:172.21.15.186:6816/3475163771,v1:172.21.15.186:6817/3475163771]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T04:46:43.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:42 smithi186 ceph-mon[34668]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T04:46:43.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:42 smithi169 ceph-mon[34075]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:46:43.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:42 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:43.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:42 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:43.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:42 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:43.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:42 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:43.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:42 smithi169 ceph-mon[34075]: from='osd.8 [v2:172.21.15.186:6816/3475163771,v1:172.21.15.186:6817/3475163771]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T04:46:43.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:42 smithi169 ceph-mon[34075]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T04:46:43.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:42 smithi006 ceph-mon[29835]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:46:43.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:42 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:43.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:42 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:43.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:42 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:43.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:42 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:43.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:42 smithi006 ceph-mon[29835]: from='osd.8 [v2:172.21.15.186:6816/3475163771,v1:172.21.15.186:6817/3475163771]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T04:46:43.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:42 smithi006 ceph-mon[29835]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T04:46:43.319 DEBUG:teuthology.orchestra.run.smithi186:osd.8> sudo journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.8.service 2024-05-09T04:46:43.322 INFO:tasks.cephadm:Deploying osd.9 on smithi006 with /dev/vg_nvme/lv_1... 2024-05-09T04:46:43.322 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-09T04:46:43.809 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:46:44.927 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:44 smithi186 ceph-mon[34668]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:46:44.927 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:44 smithi186 ceph-mon[34668]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-09T04:46:44.927 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:44 smithi186 ceph-mon[34668]: from='osd.8 [v2:172.21.15.186:6816/3475163771,v1:172.21.15.186:6817/3475163771]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-09T04:46:44.927 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:44 smithi186 ceph-mon[34668]: osdmap e52: 9 total, 8 up, 9 in 2024-05-09T04:46:44.927 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:44 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T04:46:44.927 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:44 smithi186 ceph-mon[34668]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-09T04:46:44.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:44 smithi006 ceph-mon[29835]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:46:44.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:44 smithi006 ceph-mon[29835]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-09T04:46:44.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:44 smithi006 ceph-mon[29835]: from='osd.8 [v2:172.21.15.186:6816/3475163771,v1:172.21.15.186:6817/3475163771]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-09T04:46:44.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:44 smithi006 ceph-mon[29835]: osdmap e52: 9 total, 8 up, 9 in 2024-05-09T04:46:44.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:44 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T04:46:44.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:44 smithi006 ceph-mon[29835]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-09T04:46:45.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:44 smithi169 ceph-mon[34075]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:46:45.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:44 smithi169 ceph-mon[34075]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-09T04:46:45.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:44 smithi169 ceph-mon[34075]: from='osd.8 [v2:172.21.15.186:6816/3475163771,v1:172.21.15.186:6817/3475163771]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-09T04:46:45.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:44 smithi169 ceph-mon[34075]: osdmap e52: 9 total, 8 up, 9 in 2024-05-09T04:46:45.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:44 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T04:46:45.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:44 smithi169 ceph-mon[34075]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-09T04:46:45.599 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 04:46:45 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T04:46:45.406+0000 7fc4f8bbd700 -1 osd.8 0 waiting for initial osdmap 2024-05-09T04:46:45.599 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 04:46:45 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T04:46:45.417+0000 7fc4f096a700 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T04:46:45.915 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:45 smithi006 ceph-mon[29835]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-05-09T04:46:45.915 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:45 smithi006 ceph-mon[29835]: osdmap e53: 9 total, 8 up, 9 in 2024-05-09T04:46:45.915 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:45 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T04:46:45.916 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:45 smithi006 ceph-mon[29835]: from='osd.8 ' entity='osd.8' 2024-05-09T04:46:45.916 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:45 smithi006 ceph-mon[29835]: osd.8 [v2:172.21.15.186:6816/3475163771,v1:172.21.15.186:6817/3475163771] boot 2024-05-09T04:46:45.916 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:45 smithi006 ceph-mon[29835]: osdmap e54: 9 total, 9 up, 9 in 2024-05-09T04:46:45.916 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:45 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T04:46:46.018 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:45 smithi169 ceph-mon[34075]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-05-09T04:46:46.018 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:45 smithi169 ceph-mon[34075]: osdmap e53: 9 total, 8 up, 9 in 2024-05-09T04:46:46.018 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:45 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T04:46:46.018 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:45 smithi169 ceph-mon[34075]: from='osd.8 ' entity='osd.8' 2024-05-09T04:46:46.018 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:45 smithi169 ceph-mon[34075]: osd.8 [v2:172.21.15.186:6816/3475163771,v1:172.21.15.186:6817/3475163771] boot 2024-05-09T04:46:46.018 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:45 smithi169 ceph-mon[34075]: osdmap e54: 9 total, 9 up, 9 in 2024-05-09T04:46:46.018 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:45 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T04:46:46.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:45 smithi186 ceph-mon[34668]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-05-09T04:46:46.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:45 smithi186 ceph-mon[34668]: osdmap e53: 9 total, 8 up, 9 in 2024-05-09T04:46:46.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:45 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T04:46:46.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:45 smithi186 ceph-mon[34668]: from='osd.8 ' entity='osd.8' 2024-05-09T04:46:46.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:45 smithi186 ceph-mon[34668]: osd.8 [v2:172.21.15.186:6816/3475163771,v1:172.21.15.186:6817/3475163771] boot 2024-05-09T04:46:46.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:45 smithi186 ceph-mon[34668]: osdmap e54: 9 total, 9 up, 9 in 2024-05-09T04:46:46.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:45 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T04:46:46.881 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-09T04:46:46.881 INFO:teuthology.orchestra.run.smithi006.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-09T04:46:46.881 INFO:teuthology.orchestra.run.smithi006.stderr: stderr: 10+0 records in 2024-05-09T04:46:46.881 INFO:teuthology.orchestra.run.smithi006.stderr:10+0 records out 2024-05-09T04:46:46.881 INFO:teuthology.orchestra.run.smithi006.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0349886 s, 300 MB/s 2024-05-09T04:46:46.881 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping successful for: 2024-05-09T04:46:46.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:46 smithi006 ceph-mon[29835]: purged_snaps scrub starts 2024-05-09T04:46:46.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:46 smithi006 ceph-mon[29835]: purged_snaps scrub ok 2024-05-09T04:46:46.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:46 smithi006 ceph-mon[29835]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:46:46.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:46 smithi006 ceph-mon[29835]: osdmap e55: 9 total, 9 up, 9 in 2024-05-09T04:46:47.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:46 smithi186 ceph-mon[34668]: purged_snaps scrub starts 2024-05-09T04:46:47.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:46 smithi186 ceph-mon[34668]: purged_snaps scrub ok 2024-05-09T04:46:47.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:46 smithi186 ceph-mon[34668]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:46:47.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:46 smithi186 ceph-mon[34668]: osdmap e55: 9 total, 9 up, 9 in 2024-05-09T04:46:47.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:46 smithi169 ceph-mon[34075]: purged_snaps scrub starts 2024-05-09T04:46:47.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:46 smithi169 ceph-mon[34075]: purged_snaps scrub ok 2024-05-09T04:46:47.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:46 smithi169 ceph-mon[34075]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:46:47.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:46 smithi169 ceph-mon[34075]: osdmap e55: 9 total, 9 up, 9 in 2024-05-09T04:46:47.874 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi006:vg_nvme/lv_1 2024-05-09T04:46:48.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:47 smithi006 ceph-mon[29835]: Detected new or changed devices on smithi186 2024-05-09T04:46:48.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:47 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:48.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:47 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:48.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:47 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:48.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:47 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:48.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:47 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:48.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:47 smithi006 ceph-mon[29835]: Adjusting osd_memory_target on smithi186 to 5696M 2024-05-09T04:46:48.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:47 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:48.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:47 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:48.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:47 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:46:48.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:47 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:48.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:47 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:46:48.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:47 smithi006 ceph-mon[29835]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 419 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:46:48.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:47 smithi006 ceph-mon[29835]: osdmap e56: 9 total, 9 up, 9 in 2024-05-09T04:46:48.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:47 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:48.199 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:47 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:46:48.199 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:47 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:48.321 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:46:48.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:47 smithi186 ceph-mon[34668]: Detected new or changed devices on smithi186 2024-05-09T04:46:48.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:47 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:48.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:47 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:48.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:47 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:48.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:47 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:48.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:47 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:48.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:47 smithi186 ceph-mon[34668]: Adjusting osd_memory_target on smithi186 to 5696M 2024-05-09T04:46:48.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:47 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:48.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:47 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:48.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:47 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:46:48.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:47 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:48.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:47 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:46:48.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:47 smithi186 ceph-mon[34668]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 419 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:46:48.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:47 smithi186 ceph-mon[34668]: osdmap e56: 9 total, 9 up, 9 in 2024-05-09T04:46:48.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:47 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:48.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:47 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:46:48.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:47 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:48.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:47 smithi169 ceph-mon[34075]: Detected new or changed devices on smithi186 2024-05-09T04:46:48.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:47 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:48.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:47 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:48.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:47 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:48.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:47 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:48.368 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:47 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:48.368 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:47 smithi169 ceph-mon[34075]: Adjusting osd_memory_target on smithi186 to 5696M 2024-05-09T04:46:48.368 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:47 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:48.368 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:47 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:48.368 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:47 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:46:48.369 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:47 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:48.369 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:47 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:46:48.369 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:47 smithi169 ceph-mon[34075]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 419 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:46:48.369 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:47 smithi169 ceph-mon[34075]: osdmap e56: 9 total, 9 up, 9 in 2024-05-09T04:46:48.369 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:47 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:48.369 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:47 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:46:48.370 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:47 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:46:50.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:50 smithi186 ceph-mon[34668]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 419 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:46:50.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.648+0000 7fefa3800700 1 -- 172.21.15.6:0/1511839761 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef9c10a100 msgr2=0x7fef9c10c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:50.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.648+0000 7fefa3800700 1 --2- 172.21.15.6:0/1511839761 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef9c10a100 0x7fef9c10c520 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fef9400b0a0 tx=0x7fef9400b3b0 comp rx=0 tx=0).stop 2024-05-09T04:46:50.649 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:50 smithi006 ceph-mon[29835]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 419 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:46:50.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.648+0000 7fefa3800700 1 -- 172.21.15.6:0/1511839761 shutdown_connections 2024-05-09T04:46:50.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.648+0000 7fefa3800700 1 --2- 172.21.15.6:0/1511839761 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef9c10a100 0x7fef9c10c520 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:50.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.648+0000 7fefa3800700 1 --2- 172.21.15.6:0/1511839761 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef9c104e40 0x7fef9c107260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:50.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.648+0000 7fefa3800700 1 --2- 172.21.15.6:0/1511839761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef9c1077a0 0x7fef9c109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:50.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.648+0000 7fefa3800700 1 -- 172.21.15.6:0/1511839761 >> 172.21.15.6:0/1511839761 conn(0x7fef9c0fe880 msgr2=0x7fef9c100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:46:50.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.648+0000 7fefa3800700 1 -- 172.21.15.6:0/1511839761 shutdown_connections 2024-05-09T04:46:50.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.648+0000 7fefa3800700 1 -- 172.21.15.6:0/1511839761 wait complete. 2024-05-09T04:46:50.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.649+0000 7fefa3800700 1 Processor -- start 2024-05-09T04:46:50.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.650+0000 7fefa3800700 1 -- start start 2024-05-09T04:46:50.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.650+0000 7fefa3800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef9c104e40 0x7fef9c19caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:46:50.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.650+0000 7fefa3800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef9c1077a0 0x7fef9c19d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:46:50.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.650+0000 7fefa3800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef9c10a100 0x7fef9c1a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:46:50.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.650+0000 7fefa3800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c10fe00 con 0x7fef9c1077a0 2024-05-09T04:46:50.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.650+0000 7fefa3800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c10fc80 con 0x7fef9c10a100 2024-05-09T04:46:50.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.650+0000 7fefa3800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c10ff80 con 0x7fef9c104e40 2024-05-09T04:46:50.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.650+0000 7fefa0e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef9c104e40 0x7fef9c19caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:46:50.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.650+0000 7fefa1800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef9c10a100 0x7fef9c1a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:46:50.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.650+0000 7fef9be00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef9c1077a0 0x7fef9c19d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:46:50.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.650+0000 7fefa0e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef9c104e40 0x7fef9c19caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:53636/0 (socket says 172.21.15.6:53636) 2024-05-09T04:46:50.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.650+0000 7fefa0e00700 1 -- 172.21.15.6:0/2359703038 learned_addr learned my addr 172.21.15.6:0/2359703038 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:46:50.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.651+0000 7fefa1800700 1 -- 172.21.15.6:0/2359703038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef9c104e40 msgr2=0x7fef9c19caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:50.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.651+0000 7fefa1800700 1 --2- 172.21.15.6:0/2359703038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef9c104e40 0x7fef9c19caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:50.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.651+0000 7fefa1800700 1 -- 172.21.15.6:0/2359703038 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef9c1077a0 msgr2=0x7fef9c19d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:50.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.651+0000 7fefa1800700 1 --2- 172.21.15.6:0/2359703038 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef9c1077a0 0x7fef9c19d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:50.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.651+0000 7fefa1800700 1 -- 172.21.15.6:0/2359703038 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9c1a4650 con 0x7fef9c10a100 2024-05-09T04:46:50.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.651+0000 7fefa1800700 1 --2- 172.21.15.6:0/2359703038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef9c10a100 0x7fef9c1a3fd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fef9400b0a0 tx=0x7fef94004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:46:50.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.651+0000 7fef99600700 1 -- 172.21.15.6:0/2359703038 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef94021070 con 0x7fef9c10a100 2024-05-09T04:46:50.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.651+0000 7fef99600700 1 -- 172.21.15.6:0/2359703038 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef94007b80 con 0x7fef9c10a100 2024-05-09T04:46:50.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.651+0000 7fef99600700 1 -- 172.21.15.6:0/2359703038 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef94005040 con 0x7fef9c10a100 2024-05-09T04:46:50.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.651+0000 7fefa3800700 1 -- 172.21.15.6:0/2359703038 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef9c1a48e0 con 0x7fef9c10a100 2024-05-09T04:46:50.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.652+0000 7fefa3800700 1 -- 172.21.15.6:0/2359703038 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef9c1a4dd0 con 0x7fef9c10a100 2024-05-09T04:46:50.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.653+0000 7fefa3800700 1 -- 172.21.15.6:0/2359703038 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef9c0512c0 con 0x7fef9c10a100 2024-05-09T04:46:50.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.655+0000 7fef99600700 1 -- 172.21.15.6:0/2359703038 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fef94024070 con 0x7fef9c10a100 2024-05-09T04:46:50.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.657+0000 7fef99600700 1 --2- 172.21.15.6:0/2359703038 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fef840b2900 0x7fef840b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:46:50.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.657+0000 7fef99600700 1 -- 172.21.15.6:0/2359703038 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5829+0+0 (secure 0 0 0) 0x7fef940e27c0 con 0x7fef9c10a100 2024-05-09T04:46:50.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.657+0000 7fefa0e00700 1 --2- 172.21.15.6:0/2359703038 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fef840b2900 0x7fef840b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:46:50.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.658+0000 7fefa0e00700 1 --2- 172.21.15.6:0/2359703038 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fef840b2900 0x7fef840b4e10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fef90009680 tx=0x7fef90009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:46:50.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.661+0000 7fef99600700 1 -- 172.21.15.6:0/2359703038 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fef940aae80 con 0x7fef9c10a100 2024-05-09T04:46:50.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:50 smithi169 ceph-mon[34075]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 419 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:46:50.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:46:50.914+0000 7fefa3800700 1 -- 172.21.15.6:0/2359703038 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fef9c063c00 con 0x7fef840b2900 2024-05-09T04:46:51.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:51 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:46:51.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:51 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:46:51.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:51 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:51.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:51 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:46:51.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:51 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:46:51.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:51 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:51.897 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:51 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:46:51.898 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:51 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:46:51.898 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:51 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:52.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:52 smithi186 ceph-mon[34668]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:46:52.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:52 smithi186 ceph-mon[34668]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T04:46:52.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:52 smithi169 ceph-mon[34075]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:46:52.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:52 smithi169 ceph-mon[34075]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T04:46:52.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:52 smithi006 ceph-mon[29835]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:46:52.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:52 smithi006 ceph-mon[29835]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T04:46:54.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:54 smithi186 ceph-mon[34668]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-09T04:46:54.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:54 smithi169 ceph-mon[34075]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-09T04:46:54.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:54 smithi006 ceph-mon[29835]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-09T04:46:55.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:55 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/3916138758' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bca7bedd-1a57-4465-9033-064ca468add2"}]: dispatch 2024-05-09T04:46:55.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:55 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/3916138758' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bca7bedd-1a57-4465-9033-064ca468add2"}]': finished 2024-05-09T04:46:55.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:55 smithi186 ceph-mon[34668]: osdmap e57: 10 total, 9 up, 10 in 2024-05-09T04:46:55.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:55 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T04:46:55.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:55 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/3916138758' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bca7bedd-1a57-4465-9033-064ca468add2"}]: dispatch 2024-05-09T04:46:55.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:55 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/3916138758' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bca7bedd-1a57-4465-9033-064ca468add2"}]': finished 2024-05-09T04:46:55.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:55 smithi169 ceph-mon[34075]: osdmap e57: 10 total, 9 up, 10 in 2024-05-09T04:46:55.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:55 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T04:46:55.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:55 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/3916138758' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bca7bedd-1a57-4465-9033-064ca468add2"}]: dispatch 2024-05-09T04:46:55.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:55 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/3916138758' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bca7bedd-1a57-4465-9033-064ca468add2"}]': finished 2024-05-09T04:46:55.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:55 smithi006 ceph-mon[29835]: osdmap e57: 10 total, 9 up, 10 in 2024-05-09T04:46:55.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:55 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T04:46:56.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:56 smithi186 ceph-mon[34668]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T04:46:56.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:56 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/2714880989' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-9/activate.monmap"}]: dispatch 2024-05-09T04:46:56.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:56 smithi169 ceph-mon[34075]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T04:46:56.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:56 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/2714880989' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-9/activate.monmap"}]: dispatch 2024-05-09T04:46:56.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:56 smithi006 ceph-mon[29835]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T04:46:56.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:56 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/2714880989' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-9/activate.monmap"}]: dispatch 2024-05-09T04:46:58.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:46:58 smithi186 ceph-mon[34668]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-09T04:46:58.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:46:58 smithi169 ceph-mon[34075]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-09T04:46:58.878 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:46:58 smithi006 ceph-mon[29835]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-09T04:47:00.696 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:00 smithi006 ceph-mon[29835]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:47:00.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:00 smithi186 ceph-mon[34668]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:47:00.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:00 smithi169 ceph-mon[34075]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:47:02.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:02 smithi186 ceph-mon[34668]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:47:02.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:02 smithi169 ceph-mon[34075]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:47:02.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:02 smithi006 ceph-mon[29835]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:47:04.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:04 smithi186 ceph-mon[34668]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:47:04.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:04 smithi169 ceph-mon[34075]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:47:04.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:04 smithi006 ceph-mon[29835]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:47:05.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:05 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-09T04:47:05.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:05 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:05.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:05 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-09T04:47:05.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:05 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:05.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:05 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-09T04:47:05.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:05 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:06.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:06 smithi186 ceph-mon[34668]: Deploying daemon osd.9 on smithi006 2024-05-09T04:47:06.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:06 smithi186 ceph-mon[34668]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:47:06.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:06 smithi169 ceph-mon[34075]: Deploying daemon osd.9 on smithi006 2024-05-09T04:47:06.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:06 smithi169 ceph-mon[34075]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:47:06.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:06 smithi006 ceph-mon[29835]: Deploying daemon osd.9 on smithi006 2024-05-09T04:47:06.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:06 smithi006 ceph-mon[29835]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:47:08.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:08 smithi186 ceph-mon[34668]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:47:08.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:08 smithi006 ceph-mon[29835]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:47:08.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:08 smithi169 ceph-mon[34075]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:47:10.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:10 smithi006 ceph-mon[29835]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:47:10.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:10 smithi186 ceph-mon[34668]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:47:10.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:10 smithi169 ceph-mon[34075]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:47:12.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:12 smithi186 ceph-mon[34668]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:47:12.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:12 smithi169 ceph-mon[34075]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:47:12.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:12 smithi006 ceph-mon[29835]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:47:14.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:14 smithi169 ceph-mon[34075]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:47:14.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:14 smithi006 ceph-mon[29835]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:47:15.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:14 smithi186 ceph-mon[34668]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:47:16.050 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:15 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:16.051 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:15 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:47:16.051 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:15 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:16.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:15 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:16.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:15 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:47:16.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:15 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:16.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:15 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:16.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:15 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:47:16.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:15 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:16.934 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:16 smithi006 ceph-mon[29835]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:47:17.018 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:16 smithi169 ceph-mon[34075]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:47:17.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:16 smithi186 ceph-mon[34668]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:47:18.879 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:18 smithi006 ceph-mon[29835]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:47:18.879 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:18 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:18.879 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:18 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:19.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:18 smithi186 ceph-mon[34668]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:47:19.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:18 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:19.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:18 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:19.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:18 smithi169 ceph-mon[34075]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:47:19.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:18 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:19.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:18 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:20.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:19 smithi186 ceph-mon[34668]: from='osd.9 [v2:172.21.15.6:6826/3590990973,v1:172.21.15.6:6827/3590990973]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-09T04:47:20.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:19 smithi186 ceph-mon[34668]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-09T04:47:20.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:19 smithi169 ceph-mon[34075]: from='osd.9 [v2:172.21.15.6:6826/3590990973,v1:172.21.15.6:6827/3590990973]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-09T04:47:20.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:19 smithi169 ceph-mon[34075]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-09T04:47:20.194 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:19 smithi006 ceph-mon[29835]: from='osd.9 [v2:172.21.15.6:6826/3590990973,v1:172.21.15.6:6827/3590990973]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-09T04:47:20.194 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:19 smithi006 ceph-mon[29835]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-09T04:47:20.772 INFO:teuthology.orchestra.run.smithi006.stdout:Created osd(s) 9 on host 'smithi006' 2024-05-09T04:47:20.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:47:20.771+0000 7fef99600700 1 -- 172.21.15.6:0/2359703038 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fef9c063c00 con 0x7fef840b2900 2024-05-09T04:47:20.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:47:20.775+0000 7fefa3800700 1 -- 172.21.15.6:0/2359703038 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fef840b2900 msgr2=0x7fef840b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:20.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:47:20.775+0000 7fefa3800700 1 --2- 172.21.15.6:0/2359703038 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fef840b2900 0x7fef840b4e10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fef90009680 tx=0x7fef90009260 comp rx=0 tx=0).stop 2024-05-09T04:47:20.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:47:20.776+0000 7fefa3800700 1 -- 172.21.15.6:0/2359703038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef9c10a100 msgr2=0x7fef9c1a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:20.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:47:20.776+0000 7fefa3800700 1 --2- 172.21.15.6:0/2359703038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef9c10a100 0x7fef9c1a3fd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fef9400b0a0 tx=0x7fef94004c80 comp rx=0 tx=0).stop 2024-05-09T04:47:20.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:47:20.776+0000 7fefa3800700 1 -- 172.21.15.6:0/2359703038 shutdown_connections 2024-05-09T04:47:20.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:47:20.776+0000 7fefa3800700 1 --2- 172.21.15.6:0/2359703038 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fef840b2900 0x7fef840b4e10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:20.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:47:20.776+0000 7fefa3800700 1 --2- 172.21.15.6:0/2359703038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef9c10a100 0x7fef9c1a3fd0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:20.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:47:20.776+0000 7fefa3800700 1 --2- 172.21.15.6:0/2359703038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef9c104e40 0x7fef9c19caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:20.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:47:20.776+0000 7fefa3800700 1 --2- 172.21.15.6:0/2359703038 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef9c1077a0 0x7fef9c19d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:20.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:47:20.776+0000 7fefa3800700 1 -- 172.21.15.6:0/2359703038 >> 172.21.15.6:0/2359703038 conn(0x7fef9c0fe880 msgr2=0x7fef9c108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:47:20.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:47:20.776+0000 7fefa3800700 1 -- 172.21.15.6:0/2359703038 shutdown_connections 2024-05-09T04:47:20.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:47:20.776+0000 7fefa3800700 1 -- 172.21.15.6:0/2359703038 wait complete. 2024-05-09T04:47:20.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:20 smithi006 ceph-mon[29835]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:47:20.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:20 smithi006 ceph-mon[29835]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-09T04:47:20.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:20 smithi006 ceph-mon[29835]: osdmap e58: 10 total, 9 up, 10 in 2024-05-09T04:47:20.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:20 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T04:47:20.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:20 smithi006 ceph-mon[29835]: from='osd.9 [v2:172.21.15.6:6826/3590990973,v1:172.21.15.6:6827/3590990973]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-05-09T04:47:20.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:20 smithi006 ceph-mon[29835]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-05-09T04:47:20.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:20 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:20.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:20 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:47:20.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:20 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:20.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:20 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:20.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:20 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:20.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:20 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:47:21.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:20 smithi186 ceph-mon[34668]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:47:21.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:20 smithi186 ceph-mon[34668]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-09T04:47:21.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:20 smithi186 ceph-mon[34668]: osdmap e58: 10 total, 9 up, 10 in 2024-05-09T04:47:21.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:20 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T04:47:21.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:20 smithi186 ceph-mon[34668]: from='osd.9 [v2:172.21.15.6:6826/3590990973,v1:172.21.15.6:6827/3590990973]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-05-09T04:47:21.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:20 smithi186 ceph-mon[34668]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-05-09T04:47:21.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:20 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:21.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:20 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:47:21.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:20 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:21.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:20 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:21.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:20 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:21.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:20 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:47:21.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:20 smithi169 ceph-mon[34075]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:47:21.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:20 smithi169 ceph-mon[34075]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-09T04:47:21.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:20 smithi169 ceph-mon[34075]: osdmap e58: 10 total, 9 up, 10 in 2024-05-09T04:47:21.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:20 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T04:47:21.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:20 smithi169 ceph-mon[34075]: from='osd.9 [v2:172.21.15.6:6826/3590990973,v1:172.21.15.6:6827/3590990973]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-05-09T04:47:21.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:20 smithi169 ceph-mon[34075]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-05-09T04:47:21.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:20 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:21.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:20 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:47:21.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:20 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:21.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:20 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:21.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:20 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:21.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:20 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:47:21.755 DEBUG:teuthology.orchestra.run.smithi006:osd.9> sudo journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.9.service 2024-05-09T04:47:21.760 INFO:tasks.cephadm:Deploying osd.10 on smithi169 with /dev/vg_nvme/lv_1... 2024-05-09T04:47:21.760 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-09T04:47:21.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:21 smithi006 ceph-mon[29835]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-05-09T04:47:21.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:21 smithi006 ceph-mon[29835]: osdmap e59: 10 total, 9 up, 10 in 2024-05-09T04:47:21.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:21 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T04:47:21.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:21 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T04:47:22.008 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T04:47:22.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:21 smithi186 ceph-mon[34668]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-05-09T04:47:22.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:21 smithi186 ceph-mon[34668]: osdmap e59: 10 total, 9 up, 10 in 2024-05-09T04:47:22.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:21 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T04:47:22.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:21 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T04:47:22.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:21 smithi169 ceph-mon[34075]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-05-09T04:47:22.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:21 smithi169 ceph-mon[34075]: osdmap e59: 10 total, 9 up, 10 in 2024-05-09T04:47:22.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:21 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T04:47:22.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:21 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T04:47:23.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:22 smithi186 ceph-mon[34668]: purged_snaps scrub starts 2024-05-09T04:47:23.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:22 smithi186 ceph-mon[34668]: purged_snaps scrub ok 2024-05-09T04:47:23.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:22 smithi186 ceph-mon[34668]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:47:23.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:22 smithi186 ceph-mon[34668]: osd.9 [v2:172.21.15.6:6826/3590990973,v1:172.21.15.6:6827/3590990973] boot 2024-05-09T04:47:23.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:22 smithi186 ceph-mon[34668]: osdmap e60: 10 total, 10 up, 10 in 2024-05-09T04:47:23.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T04:47:23.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:22 smithi169 ceph-mon[34075]: purged_snaps scrub starts 2024-05-09T04:47:23.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:22 smithi169 ceph-mon[34075]: purged_snaps scrub ok 2024-05-09T04:47:23.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:22 smithi169 ceph-mon[34075]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:47:23.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:22 smithi169 ceph-mon[34075]: osd.9 [v2:172.21.15.6:6826/3590990973,v1:172.21.15.6:6827/3590990973] boot 2024-05-09T04:47:23.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:22 smithi169 ceph-mon[34075]: osdmap e60: 10 total, 10 up, 10 in 2024-05-09T04:47:23.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T04:47:23.177 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:22 smithi006 ceph-mon[29835]: purged_snaps scrub starts 2024-05-09T04:47:23.177 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:22 smithi006 ceph-mon[29835]: purged_snaps scrub ok 2024-05-09T04:47:23.177 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:22 smithi006 ceph-mon[29835]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:47:23.177 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:22 smithi006 ceph-mon[29835]: osd.9 [v2:172.21.15.6:6826/3590990973,v1:172.21.15.6:6827/3590990973] boot 2024-05-09T04:47:23.177 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:22 smithi006 ceph-mon[29835]: osdmap e60: 10 total, 10 up, 10 in 2024-05-09T04:47:23.177 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T04:47:24.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:23 smithi186 ceph-mon[34668]: osdmap e61: 10 total, 10 up, 10 in 2024-05-09T04:47:24.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:23 smithi169 ceph-mon[34075]: osdmap e61: 10 total, 10 up, 10 in 2024-05-09T04:47:24.134 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-09T04:47:24.135 INFO:teuthology.orchestra.run.smithi169.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-09T04:47:24.135 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10+0 records in 2024-05-09T04:47:24.135 INFO:teuthology.orchestra.run.smithi169.stderr:10+0 records out 2024-05-09T04:47:24.135 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125719 s, 834 MB/s 2024-05-09T04:47:24.135 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping successful for: 2024-05-09T04:47:24.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:23 smithi006 ceph-mon[29835]: osdmap e61: 10 total, 10 up, 10 in 2024-05-09T04:47:24.795 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_1 2024-05-09T04:47:25.033 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T04:47:25.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:24 smithi186 ceph-mon[34668]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:25.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:24 smithi169 ceph-mon[34075]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:25.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:24 smithi006 ceph-mon[29835]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:26.793 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.793+0000 7fba6e62c700 1 -- 172.21.15.169:0/1082233735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba68105e40 msgr2=0x7fba6810c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:26.794 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.793+0000 7fba6e62c700 1 --2- 172.21.15.169:0/1082233735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba68105e40 0x7fba6810c6f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fba6000b0a0 tx=0x7fba6000b3b0 comp rx=0 tx=0).stop 2024-05-09T04:47:26.794 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.793+0000 7fba6e62c700 1 -- 172.21.15.169:0/1082233735 shutdown_connections 2024-05-09T04:47:26.794 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.793+0000 7fba6e62c700 1 --2- 172.21.15.169:0/1082233735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba68105e40 0x7fba6810c6f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:26.794 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.793+0000 7fba6e62c700 1 --2- 172.21.15.169:0/1082233735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba68104370 0x7fba68104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:26.794 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.793+0000 7fba6e62c700 1 --2- 172.21.15.169:0/1082233735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba681054a0 0x7fba681058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:26.794 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.793+0000 7fba6e62c700 1 -- 172.21.15.169:0/1082233735 >> 172.21.15.169:0/1082233735 conn(0x7fba680ff9d0 msgr2=0x7fba68101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:47:26.794 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.793+0000 7fba6e62c700 1 -- 172.21.15.169:0/1082233735 shutdown_connections 2024-05-09T04:47:26.794 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.793+0000 7fba6e62c700 1 -- 172.21.15.169:0/1082233735 wait complete. 2024-05-09T04:47:26.797 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.794+0000 7fba6e62c700 1 Processor -- start 2024-05-09T04:47:26.797 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.794+0000 7fba6e62c700 1 -- start start 2024-05-09T04:47:26.797 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.794+0000 7fba6e62c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba68104370 0x7fba6819a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:26.797 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.794+0000 7fba6e62c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba681054a0 0x7fba6819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:26.797 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.794+0000 7fba6e62c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba68105e40 0x7fba681a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:26.797 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.794+0000 7fba6e62c700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fba6810fe50 con 0x7fba68104370 2024-05-09T04:47:26.797 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.794+0000 7fba6e62c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fba6810fcd0 con 0x7fba681054a0 2024-05-09T04:47:26.797 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.794+0000 7fba6e62c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fba6810ffd0 con 0x7fba68105e40 2024-05-09T04:47:26.797 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.794+0000 7fba677fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba681054a0 0x7fba6819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:26.797 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.795+0000 7fba677fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba681054a0 0x7fba6819ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:58490/0 (socket says 172.21.15.169:58490) 2024-05-09T04:47:26.797 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.795+0000 7fba677fe700 1 -- 172.21.15.169:0/1103374015 learned_addr learned my addr 172.21.15.169:0/1103374015 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-05-09T04:47:26.797 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.795+0000 7fba67fff700 1 --2- 172.21.15.169:0/1103374015 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba68104370 0x7fba6819a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:26.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.795+0000 7fba6cbc9700 1 --2- 172.21.15.169:0/1103374015 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba68105e40 0x7fba681a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:26.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.795+0000 7fba677fe700 1 -- 172.21.15.169:0/1103374015 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba68105e40 msgr2=0x7fba681a1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:26.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.795+0000 7fba677fe700 1 --2- 172.21.15.169:0/1103374015 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba68105e40 0x7fba681a1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:26.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.795+0000 7fba677fe700 1 -- 172.21.15.169:0/1103374015 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba68104370 msgr2=0x7fba6819a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:26.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.795+0000 7fba677fe700 1 --2- 172.21.15.169:0/1103374015 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba68104370 0x7fba6819a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:26.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.795+0000 7fba677fe700 1 -- 172.21.15.169:0/1103374015 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba681a2430 con 0x7fba681054a0 2024-05-09T04:47:26.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.795+0000 7fba6cbc9700 1 --2- 172.21.15.169:0/1103374015 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba68105e40 0x7fba681a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:47:26.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.795+0000 7fba67fff700 1 --2- 172.21.15.169:0/1103374015 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba68104370 0x7fba6819a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:47:26.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.795+0000 7fba677fe700 1 --2- 172.21.15.169:0/1103374015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba681054a0 0x7fba6819ae10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fba5000c920 tx=0x7fba5000cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:47:26.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.795+0000 7fba657fa700 1 -- 172.21.15.169:0/1103374015 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba50007b00 con 0x7fba681054a0 2024-05-09T04:47:26.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.795+0000 7fba6e62c700 1 -- 172.21.15.169:0/1103374015 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba681a2720 con 0x7fba681054a0 2024-05-09T04:47:26.799 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.795+0000 7fba657fa700 1 -- 172.21.15.169:0/1103374015 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba50004ce0 con 0x7fba681054a0 2024-05-09T04:47:26.799 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.795+0000 7fba657fa700 1 -- 172.21.15.169:0/1103374015 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba50005650 con 0x7fba681054a0 2024-05-09T04:47:26.799 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.795+0000 7fba6e62c700 1 -- 172.21.15.169:0/1103374015 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba681a2d30 con 0x7fba681054a0 2024-05-09T04:47:26.799 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.796+0000 7fba6e62c700 1 -- 172.21.15.169:0/1103374015 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba680512c0 con 0x7fba681054a0 2024-05-09T04:47:26.800 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.798+0000 7fba657fa700 1 -- 172.21.15.169:0/1103374015 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fba50007ca0 con 0x7fba681054a0 2024-05-09T04:47:26.800 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.798+0000 7fba657fa700 1 --2- 172.21.15.169:0/1103374015 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fba540b2880 0x7fba540b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:26.800 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.799+0000 7fba657fa700 1 -- 172.21.15.169:0/1103374015 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6261+0+0 (secure 0 0 0) 0x7fba500e1a00 con 0x7fba681054a0 2024-05-09T04:47:26.800 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.799+0000 7fba67fff700 1 --2- 172.21.15.169:0/1103374015 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fba540b2880 0x7fba540b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:26.800 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.800+0000 7fba67fff700 1 --2- 172.21.15.169:0/1103374015 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fba540b2880 0x7fba540b4d90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fba681053a0 tx=0x7fba58008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:47:26.802 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:26.802+0000 7fba657fa700 1 -- 172.21.15.169:0/1103374015 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fba50014030 con 0x7fba681054a0 2024-05-09T04:47:27.017 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:27.017+0000 7fba6e62c700 1 -- 172.21.15.169:0/1103374015 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fba68063c00 con 0x7fba540b2880 2024-05-09T04:47:27.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:26 smithi186 ceph-mon[34668]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:27.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:26 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:27.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:26 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:27.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:26 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:47:27.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:26 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:47:27.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:26 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:47:27.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:26 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:47:27.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:26 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:27.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:26 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:27.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:26 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:47:27.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:26 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:27.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:26 smithi169 ceph-mon[34075]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:27.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:26 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:27.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:26 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:27.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:26 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:47:27.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:26 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:47:27.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:26 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:47:27.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:26 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:47:27.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:26 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:27.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:26 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:27.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:26 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:47:27.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:26 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:27.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:26 smithi006 ceph-mon[29835]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:27.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:26 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:27.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:26 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:27.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:26 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:47:27.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:26 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:47:27.199 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:26 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:47:27.199 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:26 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:47:27.199 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:26 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:27.199 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:26 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:27.199 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:26 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:47:27.199 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:26 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:27.995 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:27 smithi169 ceph-mon[34075]: Detected new or changed devices on smithi006 2024-05-09T04:47:27.996 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:27 smithi169 ceph-mon[34075]: Adjusting osd_memory_target on smithi006 to 4273M 2024-05-09T04:47:27.996 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:27 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:47:27.996 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:27 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:47:27.996 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:27 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:28.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:27 smithi006 ceph-mon[29835]: Detected new or changed devices on smithi006 2024-05-09T04:47:28.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:27 smithi006 ceph-mon[29835]: Adjusting osd_memory_target on smithi006 to 4273M 2024-05-09T04:47:28.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:27 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:47:28.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:27 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:47:28.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:27 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:28.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:27 smithi186 ceph-mon[34668]: Detected new or changed devices on smithi006 2024-05-09T04:47:28.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:27 smithi186 ceph-mon[34668]: Adjusting osd_memory_target on smithi006 to 4273M 2024-05-09T04:47:28.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:27 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:47:28.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:27 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:47:28.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:27 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:29.010 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:28 smithi169 ceph-mon[34075]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:47:29.010 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:28 smithi169 ceph-mon[34075]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:29.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:28 smithi006 ceph-mon[29835]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:47:29.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:28 smithi006 ceph-mon[29835]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:29.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:28 smithi186 ceph-mon[34668]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:47:29.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:28 smithi186 ceph-mon[34668]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:31.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:30 smithi169 ceph-mon[34075]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:31.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:30 smithi169 ceph-mon[34075]: from='client.? 172.21.15.169:0/2262904850' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97b0e058-64f8-42ee-9b99-3ef8e39c8a3d"}]: dispatch 2024-05-09T04:47:31.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:30 smithi169 ceph-mon[34075]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97b0e058-64f8-42ee-9b99-3ef8e39c8a3d"}]: dispatch 2024-05-09T04:47:31.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:30 smithi169 ceph-mon[34075]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "97b0e058-64f8-42ee-9b99-3ef8e39c8a3d"}]': finished 2024-05-09T04:47:31.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:30 smithi169 ceph-mon[34075]: osdmap e62: 11 total, 10 up, 11 in 2024-05-09T04:47:31.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:30 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T04:47:31.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:30 smithi169 ceph-mon[34075]: from='client.? 172.21.15.169:0/3814330623' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-10/activate.monmap"}]: dispatch 2024-05-09T04:47:31.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:30 smithi006 ceph-mon[29835]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:31.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:30 smithi006 ceph-mon[29835]: from='client.? 172.21.15.169:0/2262904850' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97b0e058-64f8-42ee-9b99-3ef8e39c8a3d"}]: dispatch 2024-05-09T04:47:31.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:30 smithi006 ceph-mon[29835]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97b0e058-64f8-42ee-9b99-3ef8e39c8a3d"}]: dispatch 2024-05-09T04:47:31.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:30 smithi006 ceph-mon[29835]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "97b0e058-64f8-42ee-9b99-3ef8e39c8a3d"}]': finished 2024-05-09T04:47:31.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:30 smithi006 ceph-mon[29835]: osdmap e62: 11 total, 10 up, 11 in 2024-05-09T04:47:31.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:30 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T04:47:31.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:30 smithi006 ceph-mon[29835]: from='client.? 172.21.15.169:0/3814330623' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-10/activate.monmap"}]: dispatch 2024-05-09T04:47:31.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:30 smithi186 ceph-mon[34668]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:31.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:30 smithi186 ceph-mon[34668]: from='client.? 172.21.15.169:0/2262904850' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97b0e058-64f8-42ee-9b99-3ef8e39c8a3d"}]: dispatch 2024-05-09T04:47:31.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:30 smithi186 ceph-mon[34668]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97b0e058-64f8-42ee-9b99-3ef8e39c8a3d"}]: dispatch 2024-05-09T04:47:31.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:30 smithi186 ceph-mon[34668]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "97b0e058-64f8-42ee-9b99-3ef8e39c8a3d"}]': finished 2024-05-09T04:47:31.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:30 smithi186 ceph-mon[34668]: osdmap e62: 11 total, 10 up, 11 in 2024-05-09T04:47:31.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:30 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T04:47:31.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:30 smithi186 ceph-mon[34668]: from='client.? 172.21.15.169:0/3814330623' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-10/activate.monmap"}]: dispatch 2024-05-09T04:47:33.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:32 smithi169 ceph-mon[34075]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:33.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:32 smithi006 ceph-mon[29835]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:33.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:32 smithi186 ceph-mon[34668]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:35.159 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:34 smithi169 ceph-mon[34075]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:35.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:34 smithi006 ceph-mon[29835]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:35.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:34 smithi186 ceph-mon[34668]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:37.109 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:36 smithi169 ceph-mon[34075]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:37.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:36 smithi006 ceph-mon[29835]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:37.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:36 smithi186 ceph-mon[34668]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:38.231 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:37 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-09T04:47:38.231 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:37 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:38.231 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:37 smithi169 ceph-mon[34075]: Deploying daemon osd.10 on smithi169 2024-05-09T04:47:38.231 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:37 smithi169 ceph-mon[34075]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:38.251 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:37 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-09T04:47:38.251 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:37 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:38.251 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:37 smithi186 ceph-mon[34668]: Deploying daemon osd.10 on smithi169 2024-05-09T04:47:38.251 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:37 smithi186 ceph-mon[34668]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:38.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:37 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-09T04:47:38.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:37 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:38.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:37 smithi006 ceph-mon[29835]: Deploying daemon osd.10 on smithi169 2024-05-09T04:47:38.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:37 smithi006 ceph-mon[29835]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:40.816 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:40 smithi169 ceph-mon[34075]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:40.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:40 smithi186 ceph-mon[34668]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:40.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:40 smithi006 ceph-mon[29835]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:42.727 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:42 smithi169 ceph-mon[34075]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:42.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:42 smithi186 ceph-mon[34668]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:42.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:42 smithi006 ceph-mon[29835]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:44.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:44 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:44.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:44 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:47:44.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:44 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:44.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:44 smithi186 ceph-mon[34668]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:44.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:44 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:44.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:44 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:47:44.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:44 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:44.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:44 smithi169 ceph-mon[34075]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:44.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:44 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:44.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:44 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:47:44.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:44 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:44.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:44 smithi006 ceph-mon[29835]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:46.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:46 smithi186 ceph-mon[34668]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:46.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:46 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:46.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:46 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:46.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:46 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:46.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:46 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:47:46.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:46 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:46.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:46 smithi169 ceph-mon[34075]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:46.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:46 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:46.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:46 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:46.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:46 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:46.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:46 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:47:46.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:46 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:46.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:46 smithi006 ceph-mon[29835]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:46.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:46 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:46.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:46 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:46.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:46 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:46.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:46 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:47:46.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:46 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:47.315 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:47.314+0000 7fba657fa700 1 -- 172.21.15.169:0/1103374015 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fba68063c00 con 0x7fba540b2880 2024-05-09T04:47:47.315 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 10 on host 'smithi169' 2024-05-09T04:47:47.317 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:47.317+0000 7fba6e62c700 1 -- 172.21.15.169:0/1103374015 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fba540b2880 msgr2=0x7fba540b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:47.317 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:47.317+0000 7fba6e62c700 1 --2- 172.21.15.169:0/1103374015 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fba540b2880 0x7fba540b4d90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fba681053a0 tx=0x7fba58008040 comp rx=0 tx=0).stop 2024-05-09T04:47:47.317 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:47.317+0000 7fba6e62c700 1 -- 172.21.15.169:0/1103374015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba681054a0 msgr2=0x7fba6819ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:47.317 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:47.317+0000 7fba6e62c700 1 --2- 172.21.15.169:0/1103374015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba681054a0 0x7fba6819ae10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fba5000c920 tx=0x7fba5000cce0 comp rx=0 tx=0).stop 2024-05-09T04:47:47.317 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:47.317+0000 7fba6e62c700 1 -- 172.21.15.169:0/1103374015 shutdown_connections 2024-05-09T04:47:47.318 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:47.317+0000 7fba6e62c700 1 --2- 172.21.15.169:0/1103374015 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba68105e40 0x7fba681a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:47.318 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:47.317+0000 7fba6e62c700 1 --2- 172.21.15.169:0/1103374015 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba68104370 0x7fba6819a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:47.318 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:47.317+0000 7fba6e62c700 1 --2- 172.21.15.169:0/1103374015 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fba540b2880 0x7fba540b4d90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:47.318 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:47.317+0000 7fba6e62c700 1 --2- 172.21.15.169:0/1103374015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba681054a0 0x7fba6819ae10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:47.318 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:47.317+0000 7fba6e62c700 1 -- 172.21.15.169:0/1103374015 >> 172.21.15.169:0/1103374015 conn(0x7fba680ff9d0 msgr2=0x7fba681086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:47:47.318 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:47.317+0000 7fba6e62c700 1 -- 172.21.15.169:0/1103374015 shutdown_connections 2024-05-09T04:47:47.318 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T04:47:47.317+0000 7fba6e62c700 1 -- 172.21.15.169:0/1103374015 wait complete. 2024-05-09T04:47:48.015 DEBUG:teuthology.orchestra.run.smithi169:osd.10> sudo journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.10.service 2024-05-09T04:47:48.018 INFO:tasks.cephadm:Deploying osd.11 on smithi186 with /dev/vg_nvme/lv_1... 2024-05-09T04:47:48.018 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-09T04:47:48.268 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T04:47:48.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:48 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:48.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:48 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:47:48.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:48 smithi186 ceph-mon[34668]: from='osd.10 [v2:172.21.15.169:6824/61007341,v1:172.21.15.169:6825/61007341]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T04:47:48.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:48 smithi186 ceph-mon[34668]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T04:47:48.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:48 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:48.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:48 smithi186 ceph-mon[34668]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:48.552 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:48 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:48.553 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:48 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:47:48.553 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:48 smithi169 ceph-mon[34075]: from='osd.10 [v2:172.21.15.169:6824/61007341,v1:172.21.15.169:6825/61007341]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T04:47:48.553 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:48 smithi169 ceph-mon[34075]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T04:47:48.553 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:48 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:48.553 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:48 smithi169 ceph-mon[34075]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:48.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:48 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:48.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:48 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:47:48.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:48 smithi006 ceph-mon[29835]: from='osd.10 [v2:172.21.15.169:6824/61007341,v1:172.21.15.169:6825/61007341]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T04:47:48.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:48 smithi006 ceph-mon[29835]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T04:47:48.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:48 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:48.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:48 smithi006 ceph-mon[29835]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:49.575 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:49 smithi169 ceph-mon[34075]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-09T04:47:49.575 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:49 smithi169 ceph-mon[34075]: from='osd.10 [v2:172.21.15.169:6824/61007341,v1:172.21.15.169:6825/61007341]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T04:47:49.575 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:49 smithi169 ceph-mon[34075]: osdmap e63: 11 total, 10 up, 11 in 2024-05-09T04:47:49.575 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:49 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T04:47:49.575 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:49 smithi169 ceph-mon[34075]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T04:47:49.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:49 smithi186 ceph-mon[34668]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-09T04:47:49.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:49 smithi186 ceph-mon[34668]: from='osd.10 [v2:172.21.15.169:6824/61007341,v1:172.21.15.169:6825/61007341]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T04:47:49.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:49 smithi186 ceph-mon[34668]: osdmap e63: 11 total, 10 up, 11 in 2024-05-09T04:47:49.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:49 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T04:47:49.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:49 smithi186 ceph-mon[34668]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T04:47:49.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:49 smithi006 ceph-mon[29835]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-09T04:47:49.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:49 smithi006 ceph-mon[29835]: from='osd.10 [v2:172.21.15.169:6824/61007341,v1:172.21.15.169:6825/61007341]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T04:47:49.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:49 smithi006 ceph-mon[29835]: osdmap e63: 11 total, 10 up, 11 in 2024-05-09T04:47:49.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:49 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T04:47:49.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:49 smithi006 ceph-mon[29835]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T04:47:50.278 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:50 smithi186 ceph-mon[34668]: purged_snaps scrub starts 2024-05-09T04:47:50.278 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:50 smithi186 ceph-mon[34668]: purged_snaps scrub ok 2024-05-09T04:47:50.278 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:50 smithi186 ceph-mon[34668]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-05-09T04:47:50.278 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:50 smithi186 ceph-mon[34668]: osdmap e64: 11 total, 10 up, 11 in 2024-05-09T04:47:50.279 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:50 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T04:47:50.279 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:50 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T04:47:50.279 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:50 smithi186 ceph-mon[34668]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:50.279 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:50 smithi186 ceph-mon[34668]: from='osd.10 ' entity='osd.10' 2024-05-09T04:47:50.279 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:50 smithi186 ceph-mon[34668]: osd.10 [v2:172.21.15.169:6824/61007341,v1:172.21.15.169:6825/61007341] boot 2024-05-09T04:47:50.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:50 smithi169 ceph-mon[34075]: purged_snaps scrub starts 2024-05-09T04:47:50.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:50 smithi169 ceph-mon[34075]: purged_snaps scrub ok 2024-05-09T04:47:50.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:50 smithi169 ceph-mon[34075]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-05-09T04:47:50.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:50 smithi169 ceph-mon[34075]: osdmap e64: 11 total, 10 up, 11 in 2024-05-09T04:47:50.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:50 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T04:47:50.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:50 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T04:47:50.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:50 smithi169 ceph-mon[34075]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:50.368 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:50 smithi169 ceph-mon[34075]: from='osd.10 ' entity='osd.10' 2024-05-09T04:47:50.368 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:50 smithi169 ceph-mon[34075]: osd.10 [v2:172.21.15.169:6824/61007341,v1:172.21.15.169:6825/61007341] boot 2024-05-09T04:47:50.368 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:50 smithi169 ceph-mon[34075]: osdmap e65: 11 total, 11 up, 11 in 2024-05-09T04:47:50.368 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:50 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T04:47:50.368 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 04:47:50 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T04:47:50.044+0000 7fcefaa78700 -1 osd.10 0 waiting for initial osdmap 2024-05-09T04:47:50.368 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 04:47:50 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T04:47:50.059+0000 7fcef2825700 -1 osd.10 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T04:47:50.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:50 smithi006 ceph-mon[29835]: purged_snaps scrub starts 2024-05-09T04:47:50.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:50 smithi006 ceph-mon[29835]: purged_snaps scrub ok 2024-05-09T04:47:50.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:50 smithi006 ceph-mon[29835]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-05-09T04:47:50.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:50 smithi006 ceph-mon[29835]: osdmap e64: 11 total, 10 up, 11 in 2024-05-09T04:47:50.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:50 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T04:47:50.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:50 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T04:47:50.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:50 smithi006 ceph-mon[29835]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:47:50.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:50 smithi006 ceph-mon[29835]: from='osd.10 ' entity='osd.10' 2024-05-09T04:47:50.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:50 smithi006 ceph-mon[29835]: osd.10 [v2:172.21.15.169:6824/61007341,v1:172.21.15.169:6825/61007341] boot 2024-05-09T04:47:50.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:50 smithi006 ceph-mon[29835]: osdmap e65: 11 total, 11 up, 11 in 2024-05-09T04:47:50.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:50 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T04:47:50.544 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:50 smithi186 ceph-mon[34668]: osdmap e65: 11 total, 11 up, 11 in 2024-05-09T04:47:50.545 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:50 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T04:47:50.545 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-09T04:47:50.545 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-09T04:47:50.545 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2024-05-09T04:47:50.545 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2024-05-09T04:47:50.545 INFO:teuthology.orchestra.run.smithi186.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0116851 s, 897 MB/s 2024-05-09T04:47:50.545 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 2024-05-09T04:47:50.545 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2024-05-09T04:47:51.277 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi186:vg_nvme/lv_1 2024-05-09T04:47:51.521 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T04:47:52.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:52 smithi186 ceph-mon[34668]: Detected new or changed devices on smithi169 2024-05-09T04:47:52.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:52.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:52.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:47:52.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:47:52.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:47:52.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:47:52.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:52 smithi186 ceph-mon[34668]: Adjusting osd_memory_target on smithi169 to 4272M 2024-05-09T04:47:52.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:52.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:52.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:47:52.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:52.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:52 smithi186 ceph-mon[34668]: osdmap e66: 11 total, 11 up, 11 in 2024-05-09T04:47:52.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:52 smithi186 ceph-mon[34668]: pgmap v264: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:47:52.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:52 smithi006 ceph-mon[29835]: Detected new or changed devices on smithi169 2024-05-09T04:47:52.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:52.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:52.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:47:52.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:47:52.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:47:52.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:47:52.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:52 smithi006 ceph-mon[29835]: Adjusting osd_memory_target on smithi169 to 4272M 2024-05-09T04:47:52.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:52.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:52.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:47:52.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:52.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:52 smithi006 ceph-mon[29835]: osdmap e66: 11 total, 11 up, 11 in 2024-05-09T04:47:52.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:52 smithi006 ceph-mon[29835]: pgmap v264: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:47:52.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:52 smithi169 ceph-mon[34075]: Detected new or changed devices on smithi169 2024-05-09T04:47:52.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:52.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:52.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:47:52.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:47:52.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:47:52.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:47:52.618 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:52 smithi169 ceph-mon[34075]: Adjusting osd_memory_target on smithi169 to 4272M 2024-05-09T04:47:52.618 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:52.618 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:52.618 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:47:52.618 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:47:52.618 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:52 smithi169 ceph-mon[34075]: osdmap e66: 11 total, 11 up, 11 in 2024-05-09T04:47:52.618 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:52 smithi169 ceph-mon[34075]: pgmap v264: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:47:53.377 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.342+0000 7fa981b2b700 1 -- 172.21.15.186:0/2243353160 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa97c1029e0 msgr2=0x7fa97c102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:53.377 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.342+0000 7fa981b2b700 1 --2- 172.21.15.186:0/2243353160 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa97c1029e0 0x7fa97c102df0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa96c009990 tx=0x7fa96c009ca0 comp rx=0 tx=0).stop 2024-05-09T04:47:53.377 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.342+0000 7fa981b2b700 1 -- 172.21.15.186:0/2243353160 shutdown_connections 2024-05-09T04:47:53.377 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.342+0000 7fa981b2b700 1 --2- 172.21.15.186:0/2243353160 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa97c1044b0 0x7fa97c10ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:53.378 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.342+0000 7fa981b2b700 1 --2- 172.21.15.186:0/2243353160 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa97c1029e0 0x7fa97c102df0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:53.378 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.342+0000 7fa981b2b700 1 --2- 172.21.15.186:0/2243353160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa97c103b10 0x7fa97c103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:53.378 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.342+0000 7fa981b2b700 1 -- 172.21.15.186:0/2243353160 >> 172.21.15.186:0/2243353160 conn(0x7fa97c0fe020 msgr2=0x7fa97c1004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:47:53.378 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.342+0000 7fa981b2b700 1 -- 172.21.15.186:0/2243353160 shutdown_connections 2024-05-09T04:47:53.378 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.342+0000 7fa981b2b700 1 -- 172.21.15.186:0/2243353160 wait complete. 2024-05-09T04:47:53.378 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.343+0000 7fa981b2b700 1 Processor -- start 2024-05-09T04:47:53.378 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.343+0000 7fa981b2b700 1 -- start start 2024-05-09T04:47:53.379 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.343+0000 7fa981b2b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa97c1029e0 0x7fa97c19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:53.379 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.343+0000 7fa981b2b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa97c103b10 0x7fa97c19aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:53.379 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.343+0000 7fa981b2b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa97c1044b0 0x7fa97c1a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:53.379 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.343+0000 7fa981b2b700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c10ed20 con 0x7fa97c1044b0 2024-05-09T04:47:53.379 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.343+0000 7fa981b2b700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c10eba0 con 0x7fa97c103b10 2024-05-09T04:47:53.379 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.343+0000 7fa981b2b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c10eea0 con 0x7fa97c1029e0 2024-05-09T04:47:53.379 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.343+0000 7fa97b7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa97c1029e0 0x7fa97c19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:53.380 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.343+0000 7fa97b7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa97c1029e0 0x7fa97c19a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:48584/0 (socket says 172.21.15.186:48584) 2024-05-09T04:47:53.380 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.343+0000 7fa97b7fe700 1 -- 172.21.15.186:0/3753447697 learned_addr learned my addr 172.21.15.186:0/3753447697 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-05-09T04:47:53.380 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.343+0000 7fa97affd700 1 --2- 172.21.15.186:0/3753447697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa97c103b10 0x7fa97c19aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:53.380 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.343+0000 7fa97b7fe700 1 -- 172.21.15.186:0/3753447697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa97c103b10 msgr2=0x7fa97c19aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:53.380 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.343+0000 7fa97b7fe700 1 --2- 172.21.15.186:0/3753447697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa97c103b10 0x7fa97c19aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:53.380 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.343+0000 7fa97b7fe700 1 -- 172.21.15.186:0/3753447697 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa97c1044b0 msgr2=0x7fa97c1a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:53.380 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.343+0000 7fa97b7fe700 1 --2- 172.21.15.186:0/3753447697 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa97c1044b0 0x7fa97c1a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:53.380 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.343+0000 7fa97b7fe700 1 -- 172.21.15.186:0/3753447697 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa97c1a2510 con 0x7fa97c1029e0 2024-05-09T04:47:53.380 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.344+0000 7fa97b7fe700 1 --2- 172.21.15.186:0/3753447697 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa97c1029e0 0x7fa97c19a9b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa96c0057c0 tx=0x7fa96c0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:47:53.381 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.344+0000 7fa978ff9700 1 -- 172.21.15.186:0/3753447697 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c027070 con 0x7fa97c1029e0 2024-05-09T04:47:53.381 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.344+0000 7fa978ff9700 1 -- 172.21.15.186:0/3753447697 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa96c0040c0 con 0x7fa97c1029e0 2024-05-09T04:47:53.381 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.344+0000 7fa981b2b700 1 -- 172.21.15.186:0/3753447697 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa97c1a27a0 con 0x7fa97c1029e0 2024-05-09T04:47:53.381 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.344+0000 7fa978ff9700 1 -- 172.21.15.186:0/3753447697 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c022cf0 con 0x7fa97c1029e0 2024-05-09T04:47:53.381 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.344+0000 7fa981b2b700 1 -- 172.21.15.186:0/3753447697 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa97c1a2bf0 con 0x7fa97c1029e0 2024-05-09T04:47:53.381 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.345+0000 7fa981b2b700 1 -- 172.21.15.186:0/3753447697 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa97c0512c0 con 0x7fa97c1029e0 2024-05-09T04:47:53.381 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.346+0000 7fa978ff9700 1 -- 172.21.15.186:0/3753447697 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fa96c02a070 con 0x7fa97c1029e0 2024-05-09T04:47:53.381 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.347+0000 7fa978ff9700 1 --2- 172.21.15.186:0/3753447697 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa9640b2880 0x7fa9640b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:53.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.347+0000 7fa978ff9700 1 -- 172.21.15.186:0/3753447697 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6694+0+0 (secure 0 0 0) 0x7fa96c0e3400 con 0x7fa97c1029e0 2024-05-09T04:47:53.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.348+0000 7fa97affd700 1 --2- 172.21.15.186:0/3753447697 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa9640b2880 0x7fa9640b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:53.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.349+0000 7fa97affd700 1 --2- 172.21.15.186:0/3753447697 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa9640b2880 0x7fa9640b4d90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa97c19ba60 tx=0x7fa97000b5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:47:53.382 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.351+0000 7fa978ff9700 1 -- 172.21.15.186:0/3753447697 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa96c031080 con 0x7fa97c1029e0 2024-05-09T04:47:53.569 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:47:53.568+0000 7fa981b2b700 1 -- 172.21.15.186:0/3753447697 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fa97c10bfc0 con 0x7fa9640b2880 2024-05-09T04:47:53.569 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:53 smithi186 ceph-mon[34668]: osdmap e67: 11 total, 11 up, 11 in 2024-05-09T04:47:53.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:53 smithi169 ceph-mon[34075]: osdmap e67: 11 total, 11 up, 11 in 2024-05-09T04:47:53.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:53 smithi006 ceph-mon[29835]: osdmap e67: 11 total, 11 up, 11 in 2024-05-09T04:47:54.470 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:54 smithi186 ceph-mon[34668]: pgmap v266: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:47:54.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:54 smithi186 ceph-mon[34668]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:47:54.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:54 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:47:54.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:54 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:47:54.471 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:54 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:54.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:54 smithi169 ceph-mon[34075]: pgmap v266: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:47:54.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:54 smithi169 ceph-mon[34075]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:47:54.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:54 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:47:54.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:54 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:47:54.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:54 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:54.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:54 smithi006 ceph-mon[29835]: pgmap v266: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:47:54.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:54 smithi006 ceph-mon[29835]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:47:54.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:54 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:47:54.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:54 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:47:54.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:54 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:56.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:56 smithi186 ceph-mon[34668]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T04:47:56.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:56 smithi186 ceph-mon[34668]: from='client.? 172.21.15.186:0/2080439960' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f2220b0-24cf-48b2-95ae-1d66e00e57fc"}]: dispatch 2024-05-09T04:47:56.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:56 smithi186 ceph-mon[34668]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f2220b0-24cf-48b2-95ae-1d66e00e57fc"}]: dispatch 2024-05-09T04:47:56.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:56 smithi186 ceph-mon[34668]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7f2220b0-24cf-48b2-95ae-1d66e00e57fc"}]': finished 2024-05-09T04:47:56.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:56 smithi186 ceph-mon[34668]: osdmap e68: 12 total, 11 up, 12 in 2024-05-09T04:47:56.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:47:56.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:56 smithi169 ceph-mon[34075]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T04:47:56.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:56 smithi169 ceph-mon[34075]: from='client.? 172.21.15.186:0/2080439960' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f2220b0-24cf-48b2-95ae-1d66e00e57fc"}]: dispatch 2024-05-09T04:47:56.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:56 smithi169 ceph-mon[34075]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f2220b0-24cf-48b2-95ae-1d66e00e57fc"}]: dispatch 2024-05-09T04:47:56.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:56 smithi169 ceph-mon[34075]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7f2220b0-24cf-48b2-95ae-1d66e00e57fc"}]': finished 2024-05-09T04:47:56.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:56 smithi169 ceph-mon[34075]: osdmap e68: 12 total, 11 up, 12 in 2024-05-09T04:47:56.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:56 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:47:56.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:56 smithi006 ceph-mon[29835]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T04:47:56.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:56 smithi006 ceph-mon[29835]: from='client.? 172.21.15.186:0/2080439960' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f2220b0-24cf-48b2-95ae-1d66e00e57fc"}]: dispatch 2024-05-09T04:47:56.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:56 smithi006 ceph-mon[29835]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f2220b0-24cf-48b2-95ae-1d66e00e57fc"}]: dispatch 2024-05-09T04:47:56.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:56 smithi006 ceph-mon[29835]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7f2220b0-24cf-48b2-95ae-1d66e00e57fc"}]': finished 2024-05-09T04:47:56.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:56 smithi006 ceph-mon[29835]: osdmap e68: 12 total, 11 up, 12 in 2024-05-09T04:47:56.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:47:57.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:57 smithi186 ceph-mon[34668]: from='client.? 172.21.15.186:0/601123550' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-11/activate.monmap"}]: dispatch 2024-05-09T04:47:57.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:57 smithi169 ceph-mon[34075]: from='client.? 172.21.15.186:0/601123550' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-11/activate.monmap"}]: dispatch 2024-05-09T04:47:57.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:57 smithi006 ceph-mon[29835]: from='client.? 172.21.15.186:0/601123550' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-11/activate.monmap"}]: dispatch 2024-05-09T04:47:58.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:47:58 smithi186 ceph-mon[34668]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-05-09T04:47:58.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:47:58 smithi169 ceph-mon[34075]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-05-09T04:47:58.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:47:58 smithi006 ceph-mon[29835]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-05-09T04:48:00.835 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:00 smithi186 ceph-mon[34668]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T04:48:00.842 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:00 smithi006 ceph-mon[29835]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T04:48:00.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:00 smithi169 ceph-mon[34075]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T04:48:02.841 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:02 smithi186 ceph-mon[34668]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-09T04:48:02.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:02 smithi169 ceph-mon[34075]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-09T04:48:02.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:02 smithi006 ceph-mon[29835]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-09T04:48:04.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:04 smithi186 ceph-mon[34668]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:48:04.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:04 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-09T04:48:04.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:04 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:04.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:04 smithi169 ceph-mon[34075]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:48:04.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:04 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-09T04:48:04.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:04 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:04.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:04 smithi006 ceph-mon[29835]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:48:04.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:04 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-09T04:48:04.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:04 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:05.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:05 smithi186 ceph-mon[34668]: Deploying daemon osd.11 on smithi186 2024-05-09T04:48:05.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:05 smithi169 ceph-mon[34075]: Deploying daemon osd.11 on smithi186 2024-05-09T04:48:05.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:05 smithi006 ceph-mon[29835]: Deploying daemon osd.11 on smithi186 2024-05-09T04:48:06.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:06 smithi186 ceph-mon[34668]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:48:06.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:06 smithi169 ceph-mon[34075]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:48:06.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:06 smithi006 ceph-mon[29835]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:48:08.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:08 smithi186 ceph-mon[34668]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:48:08.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:08 smithi169 ceph-mon[34075]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:48:08.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:08 smithi006 ceph-mon[29835]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:48:10.803 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:10 smithi186 ceph-mon[34668]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:48:10.803 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:10 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:48:10.803 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:10 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:10.804 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:10 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:10.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:10 smithi169 ceph-mon[34075]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:48:10.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:10 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:48:10.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:10 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:10.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:10 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:10.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:10 smithi006 ceph-mon[29835]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:48:10.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:10 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:48:10.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:10 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:10.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:10 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:12.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:12 smithi186 ceph-mon[34668]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:48:12.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:12 smithi169 ceph-mon[34075]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:48:12.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:12 smithi006 ceph-mon[29835]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:48:14.120 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:14.120+0000 7fa978ff9700 1 -- 172.21.15.186:0/3753447697 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fa97c10bfc0 con 0x7fa9640b2880 2024-05-09T04:48:14.120 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 11 on host 'smithi186' 2024-05-09T04:48:14.122 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:14.122+0000 7fa981b2b700 1 -- 172.21.15.186:0/3753447697 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa9640b2880 msgr2=0x7fa9640b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:14.123 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:14.122+0000 7fa981b2b700 1 --2- 172.21.15.186:0/3753447697 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa9640b2880 0x7fa9640b4d90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa97c19ba60 tx=0x7fa97000b5d0 comp rx=0 tx=0).stop 2024-05-09T04:48:14.123 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:14.122+0000 7fa981b2b700 1 -- 172.21.15.186:0/3753447697 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa97c1029e0 msgr2=0x7fa97c19a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:14.123 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:14.122+0000 7fa981b2b700 1 --2- 172.21.15.186:0/3753447697 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa97c1029e0 0x7fa97c19a9b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa96c0057c0 tx=0x7fa96c0057f0 comp rx=0 tx=0).stop 2024-05-09T04:48:14.123 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:14.122+0000 7fa981b2b700 1 -- 172.21.15.186:0/3753447697 shutdown_connections 2024-05-09T04:48:14.123 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:14.122+0000 7fa981b2b700 1 --2- 172.21.15.186:0/3753447697 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa97c1044b0 0x7fa97c1a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:14.123 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:14.122+0000 7fa981b2b700 1 --2- 172.21.15.186:0/3753447697 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa97c1029e0 0x7fa97c19a9b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:14.123 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:14.122+0000 7fa981b2b700 1 --2- 172.21.15.186:0/3753447697 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa9640b2880 0x7fa9640b4d90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:14.123 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:14.122+0000 7fa981b2b700 1 --2- 172.21.15.186:0/3753447697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa97c103b10 0x7fa97c19aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:14.123 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:14.123+0000 7fa981b2b700 1 -- 172.21.15.186:0/3753447697 >> 172.21.15.186:0/3753447697 conn(0x7fa97c0fe020 msgr2=0x7fa97c10b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:14.123 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:14.123+0000 7fa981b2b700 1 -- 172.21.15.186:0/3753447697 shutdown_connections 2024-05-09T04:48:14.123 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:14.123+0000 7fa981b2b700 1 -- 172.21.15.186:0/3753447697 wait complete. 2024-05-09T04:48:14.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:14 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:14.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:14 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:14.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:14 smithi186 ceph-mon[34668]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:48:14.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:14 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:14.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:14 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:48:14.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:14 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:14.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:14 smithi186 ceph-mon[34668]: from='osd.11 [v2:172.21.15.186:6824/1138940441,v1:172.21.15.186:6825/1138940441]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T04:48:14.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:14 smithi186 ceph-mon[34668]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T04:48:14.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:14 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:14.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:14 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:14.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:14 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:14.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:14 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:14.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:14 smithi169 ceph-mon[34075]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:48:14.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:14 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:14.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:14 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:48:14.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:14 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:14.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:14 smithi169 ceph-mon[34075]: from='osd.11 [v2:172.21.15.186:6824/1138940441,v1:172.21.15.186:6825/1138940441]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T04:48:14.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:14 smithi169 ceph-mon[34075]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T04:48:14.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:14 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:14.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:14 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:14.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:14 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:14.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:14 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:14.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:14 smithi006 ceph-mon[29835]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:48:14.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:14 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:14.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:14 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:48:14.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:14 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:14.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:14 smithi006 ceph-mon[29835]: from='osd.11 [v2:172.21.15.186:6824/1138940441,v1:172.21.15.186:6825/1138940441]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T04:48:14.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:14 smithi006 ceph-mon[29835]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T04:48:14.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:14 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:14.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:14 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:14.829 DEBUG:teuthology.orchestra.run.smithi186:osd.11> sudo journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.11.service 2024-05-09T04:48:14.831 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-09T04:48:14.831 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-09T04:48:15.297 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:15.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:15 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:48:15.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:15 smithi186 ceph-mon[34668]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-09T04:48:15.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:15 smithi186 ceph-mon[34668]: from='osd.11 [v2:172.21.15.186:6824/1138940441,v1:172.21.15.186:6825/1138940441]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-09T04:48:15.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:15 smithi186 ceph-mon[34668]: osdmap e69: 12 total, 11 up, 12 in 2024-05-09T04:48:15.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:15 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:48:15.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:15 smithi186 ceph-mon[34668]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-09T04:48:15.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:15 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:48:15.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:15 smithi169 ceph-mon[34075]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-09T04:48:15.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:15 smithi169 ceph-mon[34075]: from='osd.11 [v2:172.21.15.186:6824/1138940441,v1:172.21.15.186:6825/1138940441]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-09T04:48:15.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:15 smithi169 ceph-mon[34075]: osdmap e69: 12 total, 11 up, 12 in 2024-05-09T04:48:15.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:15 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:48:15.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:15 smithi169 ceph-mon[34075]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-09T04:48:15.650 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:15 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:48:15.650 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:15 smithi006 ceph-mon[29835]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-09T04:48:15.650 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:15 smithi006 ceph-mon[29835]: from='osd.11 [v2:172.21.15.186:6824/1138940441,v1:172.21.15.186:6825/1138940441]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-09T04:48:15.650 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:15 smithi006 ceph-mon[29835]: osdmap e69: 12 total, 11 up, 12 in 2024-05-09T04:48:15.650 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:15 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:48:15.651 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:15 smithi006 ceph-mon[29835]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-05-09T04:48:16.099 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 04:48:15 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T04:48:15.927+0000 7f062f13c700 -1 osd.11 0 waiting for initial osdmap 2024-05-09T04:48:16.100 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 04:48:15 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T04:48:15.939+0000 7f062aef1700 -1 osd.11 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T04:48:16.411 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:16 smithi186 ceph-mon[34668]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:48:16.411 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:16 smithi186 ceph-mon[34668]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-05-09T04:48:16.411 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:16 smithi186 ceph-mon[34668]: osdmap e70: 12 total, 11 up, 12 in 2024-05-09T04:48:16.411 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:48:16.411 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:48:16.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:16 smithi006 ceph-mon[29835]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:48:16.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:16 smithi006 ceph-mon[29835]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-05-09T04:48:16.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:16 smithi006 ceph-mon[29835]: osdmap e70: 12 total, 11 up, 12 in 2024-05-09T04:48:16.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:48:16.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:48:16.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:16 smithi169 ceph-mon[34075]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:48:16.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:16 smithi169 ceph-mon[34075]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-05-09T04:48:16.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:16 smithi169 ceph-mon[34075]: osdmap e70: 12 total, 11 up, 12 in 2024-05-09T04:48:16.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:48:16.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:48:17.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.934+0000 7f1e29a00700 1 -- 172.21.15.6:0/3132280098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e24105c20 msgr2=0x7f1e2410c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:17.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.934+0000 7f1e29a00700 1 --2- 172.21.15.6:0/3132280098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e24105c20 0x7f1e2410c4d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c00b0a0 tx=0x7f1e1c00b3b0 comp rx=0 tx=0).stop 2024-05-09T04:48:17.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.934+0000 7f1e29a00700 1 -- 172.21.15.6:0/3132280098 shutdown_connections 2024-05-09T04:48:17.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.934+0000 7f1e29a00700 1 --2- 172.21.15.6:0/3132280098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e24105c20 0x7f1e2410c4d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:17.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.934+0000 7f1e29a00700 1 --2- 172.21.15.6:0/3132280098 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e24104150 0x7f1e24104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:17.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.934+0000 7f1e29a00700 1 --2- 172.21.15.6:0/3132280098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e24105280 0x7f1e24105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:17.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.934+0000 7f1e29a00700 1 -- 172.21.15.6:0/3132280098 >> 172.21.15.6:0/3132280098 conn(0x7f1e240ff7b0 msgr2=0x7f1e24101c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:17.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.934+0000 7f1e29a00700 1 -- 172.21.15.6:0/3132280098 shutdown_connections 2024-05-09T04:48:17.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.935+0000 7f1e29a00700 1 -- 172.21.15.6:0/3132280098 wait complete. 2024-05-09T04:48:17.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.936+0000 7f1e29a00700 1 Processor -- start 2024-05-09T04:48:17.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.936+0000 7f1e29a00700 1 -- start start 2024-05-09T04:48:17.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.937+0000 7f1e29a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e24104150 0x7f1e2419c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:17.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.937+0000 7f1e29a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e24105280 0x7f1e2419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:17.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.937+0000 7f1e29a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e24105c20 0x7f1e241a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:17.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.937+0000 7f1e29a00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2410fc20 con 0x7f1e24105c20 2024-05-09T04:48:17.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.937+0000 7f1e29a00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2410faa0 con 0x7f1e24105280 2024-05-09T04:48:17.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.937+0000 7f1e29a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2410fda0 con 0x7f1e24104150 2024-05-09T04:48:17.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.937+0000 7f1e23e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e24104150 0x7f1e2419c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:17.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.937+0000 7f1e23e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e24104150 0x7f1e2419c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:39010/0 (socket says 172.21.15.6:39010) 2024-05-09T04:48:17.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.937+0000 7f1e23e00700 1 -- 172.21.15.6:0/1034034265 learned_addr learned my addr 172.21.15.6:0/1034034265 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:48:17.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.937+0000 7f1e23400700 1 --2- 172.21.15.6:0/1034034265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e24105280 0x7f1e2419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:17.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.937+0000 7f1e29000700 1 --2- 172.21.15.6:0/1034034265 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e24105c20 0x7f1e241a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:17.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.938+0000 7f1e23e00700 1 -- 172.21.15.6:0/1034034265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e24105280 msgr2=0x7f1e2419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:17.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.938+0000 7f1e23e00700 1 --2- 172.21.15.6:0/1034034265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e24105280 0x7f1e2419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:17.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.938+0000 7f1e23e00700 1 -- 172.21.15.6:0/1034034265 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e24105c20 msgr2=0x7f1e241a3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:17.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.938+0000 7f1e23e00700 1 --2- 172.21.15.6:0/1034034265 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e24105c20 0x7f1e241a3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:17.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.938+0000 7f1e23e00700 1 -- 172.21.15.6:0/1034034265 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e241a4460 con 0x7f1e24104150 2024-05-09T04:48:17.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.938+0000 7f1e23e00700 1 --2- 172.21.15.6:0/1034034265 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e24104150 0x7f1e2419c900 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400e940 tx=0x7f1e1400ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:17.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.939+0000 7f1e20c00700 1 -- 172.21.15.6:0/1034034265 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e14004220 con 0x7f1e24104150 2024-05-09T04:48:17.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.939+0000 7f1e20c00700 1 -- 172.21.15.6:0/1034034265 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e140043c0 con 0x7f1e24104150 2024-05-09T04:48:17.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.939+0000 7f1e29a00700 1 -- 172.21.15.6:0/1034034265 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e241a4750 con 0x7f1e24104150 2024-05-09T04:48:17.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.939+0000 7f1e20c00700 1 -- 172.21.15.6:0/1034034265 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e14003c60 con 0x7f1e24104150 2024-05-09T04:48:17.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.939+0000 7f1e29a00700 1 -- 172.21.15.6:0/1034034265 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e241a4c90 con 0x7f1e24104150 2024-05-09T04:48:17.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.940+0000 7f1e29a00700 1 -- 172.21.15.6:0/1034034265 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e240512c0 con 0x7f1e24104150 2024-05-09T04:48:17.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.942+0000 7f1e20c00700 1 -- 172.21.15.6:0/1034034265 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f1e1400c7e0 con 0x7f1e24104150 2024-05-09T04:48:17.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.945+0000 7f1e20c00700 1 --2- 172.21.15.6:0/1034034265 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f1e080b2880 0x7f1e080b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:17.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.945+0000 7f1e20c00700 1 -- 172.21.15.6:0/1034034265 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f1e140e1fe0 con 0x7f1e24104150 2024-05-09T04:48:17.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.945+0000 7f1e23400700 1 --2- 172.21.15.6:0/1034034265 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f1e080b2880 0x7f1e080b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:17.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.946+0000 7f1e23400700 1 --2- 172.21.15.6:0/1034034265 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f1e080b2880 0x7f1e080b4d90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1e2419d9b0 tx=0x7f1e10009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:17.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:17.948+0000 7f1e20c00700 1 -- 172.21.15.6:0/1034034265 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1e14003e40 con 0x7f1e24104150 2024-05-09T04:48:18.148 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:17 smithi006 ceph-mon[29835]: purged_snaps scrub starts 2024-05-09T04:48:18.148 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:17 smithi006 ceph-mon[29835]: purged_snaps scrub ok 2024-05-09T04:48:18.148 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:17 smithi006 ceph-mon[29835]: osd.11 [v2:172.21.15.186:6824/1138940441,v1:172.21.15.186:6825/1138940441] boot 2024-05-09T04:48:18.148 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:17 smithi006 ceph-mon[29835]: osdmap e71: 12 total, 12 up, 12 in 2024-05-09T04:48:18.148 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:17 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:48:18.148 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:17 smithi006 ceph-mon[29835]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:18.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:18.202+0000 7f1e29a00700 1 -- 172.21.15.6:0/1034034265 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f1e241a4f50 con 0x7f1e24104150 2024-05-09T04:48:18.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:18.203+0000 7f1e20c00700 1 -- 172.21.15.6:0/1034034265 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7f1e140aa120 con 0x7f1e24104150 2024-05-09T04:48:18.204 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:48:18.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:18.206+0000 7f1e29a00700 1 -- 172.21.15.6:0/1034034265 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f1e080b2880 msgr2=0x7f1e080b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:18.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:18.206+0000 7f1e29a00700 1 --2- 172.21.15.6:0/1034034265 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f1e080b2880 0x7f1e080b4d90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1e2419d9b0 tx=0x7f1e10009260 comp rx=0 tx=0).stop 2024-05-09T04:48:18.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:18.206+0000 7f1e29a00700 1 -- 172.21.15.6:0/1034034265 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e24104150 msgr2=0x7f1e2419c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:18.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:18.206+0000 7f1e29a00700 1 --2- 172.21.15.6:0/1034034265 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e24104150 0x7f1e2419c900 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400e940 tx=0x7f1e1400ec50 comp rx=0 tx=0).stop 2024-05-09T04:48:18.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:18.206+0000 7f1e29a00700 1 -- 172.21.15.6:0/1034034265 shutdown_connections 2024-05-09T04:48:18.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:18.206+0000 7f1e29a00700 1 --2- 172.21.15.6:0/1034034265 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e24105c20 0x7f1e241a3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:18.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:18.206+0000 7f1e29a00700 1 --2- 172.21.15.6:0/1034034265 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e24104150 0x7f1e2419c900 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:18.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:18.206+0000 7f1e29a00700 1 --2- 172.21.15.6:0/1034034265 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f1e080b2880 0x7f1e080b4d90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:18.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:18.206+0000 7f1e29a00700 1 --2- 172.21.15.6:0/1034034265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e24105280 0x7f1e2419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:18.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:18.206+0000 7f1e29a00700 1 -- 172.21.15.6:0/1034034265 >> 172.21.15.6:0/1034034265 conn(0x7f1e240ff7b0 msgr2=0x7f1e240759f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:18.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:18.207+0000 7f1e29a00700 1 -- 172.21.15.6:0/1034034265 shutdown_connections 2024-05-09T04:48:18.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:18.207+0000 7f1e29a00700 1 -- 172.21.15.6:0/1034034265 wait complete. 2024-05-09T04:48:18.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:17 smithi186 ceph-mon[34668]: purged_snaps scrub starts 2024-05-09T04:48:18.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:17 smithi186 ceph-mon[34668]: purged_snaps scrub ok 2024-05-09T04:48:18.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:17 smithi186 ceph-mon[34668]: osd.11 [v2:172.21.15.186:6824/1138940441,v1:172.21.15.186:6825/1138940441] boot 2024-05-09T04:48:18.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:17 smithi186 ceph-mon[34668]: osdmap e71: 12 total, 12 up, 12 in 2024-05-09T04:48:18.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:17 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:48:18.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:17 smithi186 ceph-mon[34668]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:18.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:17 smithi169 ceph-mon[34075]: purged_snaps scrub starts 2024-05-09T04:48:18.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:17 smithi169 ceph-mon[34075]: purged_snaps scrub ok 2024-05-09T04:48:18.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:17 smithi169 ceph-mon[34075]: osd.11 [v2:172.21.15.186:6824/1138940441,v1:172.21.15.186:6825/1138940441] boot 2024-05-09T04:48:18.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:17 smithi169 ceph-mon[34075]: osdmap e71: 12 total, 12 up, 12 in 2024-05-09T04:48:18.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:17 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:48:18.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:17 smithi169 ceph-mon[34075]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:19.005 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":72,"num_osds":12,"num_up_osds":12,"osd_up_since":1715230096,"num_in_osds":12,"osd_in_since":1715230076,"num_remapped_pgs":0} 2024-05-09T04:48:19.006 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T04:48:19.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:18 smithi006 ceph-mon[29835]: osdmap e72: 12 total, 12 up, 12 in 2024-05-09T04:48:19.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:18 smithi006 ceph-mon[29835]: Detected new or changed devices on smithi186 2024-05-09T04:48:19.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:18 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:19.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:18 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:19.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:18 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:48:19.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:18 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:48:19.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:18 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:48:19.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:18 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:48:19.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:18 smithi006 ceph-mon[29835]: Adjusting osd_memory_target on smithi186 to 4272M 2024-05-09T04:48:19.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:18 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:19.199 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:18 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:19.199 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:18 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:48:19.199 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:18 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:19.199 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:18 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1034034265' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T04:48:19.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:18 smithi186 ceph-mon[34668]: osdmap e72: 12 total, 12 up, 12 in 2024-05-09T04:48:19.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:18 smithi186 ceph-mon[34668]: Detected new or changed devices on smithi186 2024-05-09T04:48:19.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:18 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:19.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:18 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:19.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:18 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:48:19.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:18 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:48:19.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:18 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:48:19.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:18 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:48:19.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:18 smithi186 ceph-mon[34668]: Adjusting osd_memory_target on smithi186 to 4272M 2024-05-09T04:48:19.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:18 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:19.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:18 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:19.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:18 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:48:19.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:18 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:19.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:18 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/1034034265' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T04:48:19.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:18 smithi169 ceph-mon[34075]: osdmap e72: 12 total, 12 up, 12 in 2024-05-09T04:48:19.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:18 smithi169 ceph-mon[34075]: Detected new or changed devices on smithi186 2024-05-09T04:48:19.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:18 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:19.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:18 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:19.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:18 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:48:19.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:18 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:48:19.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:18 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:48:19.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:18 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:48:19.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:18 smithi169 ceph-mon[34075]: Adjusting osd_memory_target on smithi186 to 4272M 2024-05-09T04:48:19.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:18 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:19.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:18 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:19.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:18 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:48:19.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:18 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:19.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:18 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/1034034265' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T04:48:19.452 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:20.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:19 smithi006 ceph-mon[29835]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:20.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:19 smithi186 ceph-mon[34668]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:20.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:19 smithi169 ceph-mon[34075]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:21.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.776+0000 7fb334e00700 1 -- 172.21.15.6:0/1628189388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb33010c5c0 msgr2=0x7fb33010c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:21.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.776+0000 7fb334e00700 1 --2- 172.21.15.6:0/1628189388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb33010c5c0 0x7fb33010c9d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb320009990 tx=0x7fb320009ca0 comp rx=0 tx=0).stop 2024-05-09T04:48:21.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.777+0000 7fb334e00700 1 -- 172.21.15.6:0/1628189388 shutdown_connections 2024-05-09T04:48:21.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.777+0000 7fb334e00700 1 --2- 172.21.15.6:0/1628189388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb330106ed0 0x7fb330113360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:21.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.777+0000 7fb334e00700 1 --2- 172.21.15.6:0/1628189388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb33010c5c0 0x7fb33010c9d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:21.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.777+0000 7fb334e00700 1 --2- 172.21.15.6:0/1628189388 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb330106580 0x7fb330106990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:21.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.777+0000 7fb334e00700 1 -- 172.21.15.6:0/1628189388 >> 172.21.15.6:0/1628189388 conn(0x7fb330102080 msgr2=0x7fb3301044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:21.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.777+0000 7fb334e00700 1 -- 172.21.15.6:0/1628189388 shutdown_connections 2024-05-09T04:48:21.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.777+0000 7fb334e00700 1 -- 172.21.15.6:0/1628189388 wait complete. 2024-05-09T04:48:21.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.778+0000 7fb334e00700 1 Processor -- start 2024-05-09T04:48:21.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.778+0000 7fb334e00700 1 -- start start 2024-05-09T04:48:21.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.779+0000 7fb334e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb330106580 0x7fb33019d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:21.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.779+0000 7fb334e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb330106ed0 0x7fb33019d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:21.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.779+0000 7fb334e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb33010c5c0 0x7fb3301a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:21.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.779+0000 7fb334e00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb330116720 con 0x7fb330106580 2024-05-09T04:48:21.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.779+0000 7fb334e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb3301165a0 con 0x7fb33010c5c0 2024-05-09T04:48:21.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.779+0000 7fb334e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb3301168a0 con 0x7fb330106ed0 2024-05-09T04:48:21.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.779+0000 7fb32de00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb330106580 0x7fb33019d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:21.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.779+0000 7fb32de00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb330106580 0x7fb33019d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45756/0 (socket says 172.21.15.6:45756) 2024-05-09T04:48:21.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.779+0000 7fb32de00700 1 -- 172.21.15.6:0/3919236500 learned_addr learned my addr 172.21.15.6:0/3919236500 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:48:21.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.779+0000 7fb32d400700 1 --2- 172.21.15.6:0/3919236500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb330106ed0 0x7fb33019d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:21.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.779+0000 7fb32e800700 1 --2- 172.21.15.6:0/3919236500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb33010c5c0 0x7fb3301a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:21.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.780+0000 7fb32d400700 1 -- 172.21.15.6:0/3919236500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb33010c5c0 msgr2=0x7fb3301a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:21.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.780+0000 7fb32d400700 1 --2- 172.21.15.6:0/3919236500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb33010c5c0 0x7fb3301a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:21.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.780+0000 7fb32d400700 1 -- 172.21.15.6:0/3919236500 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb330106580 msgr2=0x7fb33019d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:21.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.780+0000 7fb32d400700 1 --2- 172.21.15.6:0/3919236500 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb330106580 0x7fb33019d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:21.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.780+0000 7fb32d400700 1 -- 172.21.15.6:0/3919236500 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3301a1f80 con 0x7fb330106ed0 2024-05-09T04:48:21.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.780+0000 7fb32de00700 1 --2- 172.21.15.6:0/3919236500 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb330106580 0x7fb33019d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:48:21.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.780+0000 7fb32d400700 1 --2- 172.21.15.6:0/3919236500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb330106ed0 0x7fb33019d650 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb330075b80 tx=0x7fb31c00eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:21.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.780+0000 7fb326a00700 1 -- 172.21.15.6:0/3919236500 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31c010900 con 0x7fb330106ed0 2024-05-09T04:48:21.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.780+0000 7fb326a00700 1 -- 172.21.15.6:0/3919236500 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb31c00c430 con 0x7fb330106ed0 2024-05-09T04:48:21.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.780+0000 7fb334e00700 1 -- 172.21.15.6:0/3919236500 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3301a2270 con 0x7fb330106ed0 2024-05-09T04:48:21.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.781+0000 7fb326a00700 1 -- 172.21.15.6:0/3919236500 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31c018680 con 0x7fb330106ed0 2024-05-09T04:48:21.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.781+0000 7fb334e00700 1 -- 172.21.15.6:0/3919236500 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3301a9b90 con 0x7fb330106ed0 2024-05-09T04:48:21.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.782+0000 7fb334e00700 1 -- 172.21.15.6:0/3919236500 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb33010e980 con 0x7fb330106ed0 2024-05-09T04:48:21.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.785+0000 7fb326a00700 1 -- 172.21.15.6:0/3919236500 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fb31c010aa0 con 0x7fb330106ed0 2024-05-09T04:48:21.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.786+0000 7fb326a00700 1 --2- 172.21.15.6:0/3919236500 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fb3140b2900 0x7fb3140b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:21.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.787+0000 7fb326a00700 1 -- 172.21.15.6:0/3919236500 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fb31c0e20b0 con 0x7fb330106ed0 2024-05-09T04:48:21.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.787+0000 7fb32de00700 1 --2- 172.21.15.6:0/3919236500 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fb3140b2900 0x7fb3140b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:21.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.788+0000 7fb32de00700 1 --2- 172.21.15.6:0/3919236500 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fb3140b2900 0x7fb3140b4e10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb320006010 tx=0x7fb32000b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:21.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:21.788+0000 7fb326a00700 1 -- 172.21.15.6:0/3919236500 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb31c0aa1d0 con 0x7fb330106ed0 2024-05-09T04:48:22.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:22.025+0000 7fb334e00700 1 -- 172.21.15.6:0/3919236500 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb33019e410 con 0x7fb330106ed0 2024-05-09T04:48:22.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:22.026+0000 7fb326a00700 1 -- 172.21.15.6:0/3919236500 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+18397 (secure 0 0 0) 0x7fb31c0a9ad0 con 0x7fb330106ed0 2024-05-09T04:48:22.028 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:48:22.028 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":72,"fsid":"f74d813c-0dbd-11ef-bc97-c7b262605968","created":"2024-05-09T04:39:35.287470+0000","modified":"2024-05-09T04:48:17.923152+0000","last_up_change":"2024-05-09T04:48:16.919842+0000","last_in_change":"2024-05-09T04:47:56.452645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:43:47.538306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b732fd3-e648-4b0d-aece-083bdaaa5d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6803","nonce":1738750914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6805","nonce":1738750914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6809","nonce":1738750914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6807","nonce":1738750914}]},"public_addr":"172.21.15.6:6803/1738750914","cluster_addr":"172.21.15.6:6805/1738750914","heartbeat_back_addr":"172.21.15.6:6809/1738750914","heartbeat_front_addr":"172.21.15.6:6807/1738750914","state":["exists","up"]},{"osd":1,"uuid":"9770c07d-4243-43bf-a0cd-9d2ac558556c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6801","nonce":2552207737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6803","nonce":2552207737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6807","nonce":2552207737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6805","nonce":2552207737}]},"public_addr":"172.21.15.169:6801/2552207737","cluster_addr":"172.21.15.169:6803/2552207737","heartbeat_back_addr":"172.21.15.169:6807/2552207737","heartbeat_front_addr":"172.21.15.169:6805/2552207737","state":["exists","up"]},{"osd":2,"uuid":"3f50ed89-b2a8-415e-b78d-8510d98d9368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6801","nonce":231435105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6803","nonce":231435105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6807","nonce":231435105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6805","nonce":231435105}]},"public_addr":"172.21.15.186:6801/231435105","cluster_addr":"172.21.15.186:6803/231435105","heartbeat_back_addr":"172.21.15.186:6807/231435105","heartbeat_front_addr":"172.21.15.186:6805/231435105","state":["exists","up"]},{"osd":3,"uuid":"afcbc36b-5669-4044-8dc5-2ebfe7970774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6811","nonce":2846613703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6813","nonce":2846613703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6817","nonce":2846613703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6815","nonce":2846613703}]},"public_addr":"172.21.15.6:6811/2846613703","cluster_addr":"172.21.15.6:6813/2846613703","heartbeat_back_addr":"172.21.15.6:6817/2846613703","heartbeat_front_addr":"172.21.15.6:6815/2846613703","state":["exists","up"]},{"osd":4,"uuid":"bb84cc5e-0297-4e23-be50-7ee2f7e59665","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6809","nonce":1928722453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6811","nonce":1928722453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6815","nonce":1928722453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6813","nonce":1928722453}]},"public_addr":"172.21.15.169:6809/1928722453","cluster_addr":"172.21.15.169:6811/1928722453","heartbeat_back_addr":"172.21.15.169:6815/1928722453","heartbeat_front_addr":"172.21.15.169:6813/1928722453","state":["exists","up"]},{"osd":5,"uuid":"830dc7f3-352e-4b01-8d4c-a9b68dcb40d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6809","nonce":3644445439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6811","nonce":3644445439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6815","nonce":3644445439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6813","nonce":3644445439}]},"public_addr":"172.21.15.186:6809/3644445439","cluster_addr":"172.21.15.186:6811/3644445439","heartbeat_back_addr":"172.21.15.186:6815/3644445439","heartbeat_front_addr":"172.21.15.186:6813/3644445439","state":["exists","up"]},{"osd":6,"uuid":"2e8478cb-c8c9-47ce-a3a5-48ffc959ab24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6819","nonce":3068252891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6821","nonce":3068252891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6825","nonce":3068252891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6823","nonce":3068252891}]},"public_addr":"172.21.15.6:6819/3068252891","cluster_addr":"172.21.15.6:6821/3068252891","heartbeat_back_addr":"172.21.15.6:6825/3068252891","heartbeat_front_addr":"172.21.15.6:6823/3068252891","state":["exists","up"]},{"osd":7,"uuid":"0f432ee2-42a4-4efa-9233-8e5a2fd65d10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6817","nonce":1303584711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6819","nonce":1303584711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6823","nonce":1303584711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6821","nonce":1303584711}]},"public_addr":"172.21.15.169:6817/1303584711","cluster_addr":"172.21.15.169:6819/1303584711","heartbeat_back_addr":"172.21.15.169:6823/1303584711","heartbeat_front_addr":"172.21.15.169:6821/1303584711","state":["exists","up"]},{"osd":8,"uuid":"cf5dca81-131b-48e7-9417-d01d4be47316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6817","nonce":3475163771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6819","nonce":3475163771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6823","nonce":3475163771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6821","nonce":3475163771}]},"public_addr":"172.21.15.186:6817/3475163771","cluster_addr":"172.21.15.186:6819/3475163771","heartbeat_back_addr":"172.21.15.186:6823/3475163771","heartbeat_front_addr":"172.21.15.186:6821/3475163771","state":["exists","up"]},{"osd":9,"uuid":"bca7bedd-1a57-4465-9033-064ca468add2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6827","nonce":3590990973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6829","nonce":3590990973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6833","nonce":3590990973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6831","nonce":3590990973}]},"public_addr":"172.21.15.6:6827/3590990973","cluster_addr":"172.21.15.6:6829/3590990973","heartbeat_back_addr":"172.21.15.6:6833/3590990973","heartbeat_front_addr":"172.21.15.6:6831/3590990973","state":["exists","up"]},{"osd":10,"uuid":"97b0e058-64f8-42ee-9b99-3ef8e39c8a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6825","nonce":61007341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6827","nonce":61007341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6831","nonce":61007341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6829","nonce":61007341}]},"public_addr":"172.21.15.169:6825/61007341","cluster_addr":"172.21.15.169:6827/61007341","heartbeat_back_addr":"172.21.15.169:6831/61007341","heartbeat_front_addr":"172.21.15.169:6829/61007341","state":["exists","up"]},{"osd":11,"uuid":"7f2220b0-24cf-48b2-95ae-1d66e00e57fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6825","nonce":1138940441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6827","nonce":1138940441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6831","nonce":1138940441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6829","nonce":1138940441}]},"public_addr":"172.21.15.186:6825/1138940441","cluster_addr":"172.21.15.186:6827/1138940441","heartbeat_back_addr":"172.21.15.186:6831/1138940441","heartbeat_front_addr":"172.21.15.186:6829/1138940441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:53.766295+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:20.195243+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:44.153812+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:20.472086+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:48.415371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:13.795415+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:50.236398+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:18.066623+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:43.781009+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:47:20.084719+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:47:48.219901+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:48:14.965536+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/3742411767":"2024-05-10T04:40:55.357057+0000","172.21.15.6:0/2410771702":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6801/937713898":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6800/937713898":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6801/3021503778":"2024-05-10T04:40:19.639151+0000","172.21.15.6:6800/3021503778":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/2559510850":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/3810474743":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/4133970555":"2024-05-10T04:40:55.357057+0000","172.21.15.6:0/664817743":"2024-05-10T04:40:19.639151+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:48:22.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:22.030+0000 7fb334e00700 1 -- 172.21.15.6:0/3919236500 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fb3140b2900 msgr2=0x7fb3140b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:22.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:22.030+0000 7fb334e00700 1 --2- 172.21.15.6:0/3919236500 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fb3140b2900 0x7fb3140b4e10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb320006010 tx=0x7fb32000b280 comp rx=0 tx=0).stop 2024-05-09T04:48:22.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:22.030+0000 7fb334e00700 1 -- 172.21.15.6:0/3919236500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb330106ed0 msgr2=0x7fb33019d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:22.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:22.030+0000 7fb334e00700 1 --2- 172.21.15.6:0/3919236500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb330106ed0 0x7fb33019d650 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb330075b80 tx=0x7fb31c00eca0 comp rx=0 tx=0).stop 2024-05-09T04:48:22.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:22.030+0000 7fb334e00700 1 -- 172.21.15.6:0/3919236500 shutdown_connections 2024-05-09T04:48:22.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:22.030+0000 7fb334e00700 1 --2- 172.21.15.6:0/3919236500 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fb3140b2900 0x7fb3140b4e10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:22.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:22.030+0000 7fb334e00700 1 --2- 172.21.15.6:0/3919236500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb33010c5c0 0x7fb3301a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:22.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:22.030+0000 7fb334e00700 1 --2- 172.21.15.6:0/3919236500 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb330106580 0x7fb33019d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:22.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:22.030+0000 7fb334e00700 1 --2- 172.21.15.6:0/3919236500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb330106ed0 0x7fb33019d650 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:22.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:22.030+0000 7fb334e00700 1 -- 172.21.15.6:0/3919236500 >> 172.21.15.6:0/3919236500 conn(0x7fb330102080 msgr2=0x7fb330104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:22.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:22.031+0000 7fb334e00700 1 -- 172.21.15.6:0/3919236500 shutdown_connections 2024-05-09T04:48:22.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:22.031+0000 7fb334e00700 1 -- 172.21.15.6:0/3919236500 wait complete. 2024-05-09T04:48:22.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:22 smithi006 ceph-mon[29835]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:22.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:22 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/3919236500' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:48:22.831 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T04:43:47.538306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T04:48:22.831 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-09T04:48:22.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:22 smithi186 ceph-mon[34668]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:22.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:22 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/3919236500' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:48:22.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:22 smithi169 ceph-mon[34075]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:22.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:22 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/3919236500' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:48:23.291 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:24.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:24 smithi006 ceph-mon[29835]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:24.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:24 smithi186 ceph-mon[34668]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:24.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:24 smithi169 ceph-mon[34075]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:25.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.679+0000 7f8c36800700 1 -- 172.21.15.6:0/2395023742 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c30105e50 msgr2=0x7f8c3010e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:25.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.679+0000 7f8c36800700 1 --2- 172.21.15.6:0/2395023742 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c30105e50 0x7f8c3010e3a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8c18009990 tx=0x7f8c18009ca0 comp rx=0 tx=0).stop 2024-05-09T04:48:25.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.679+0000 7f8c36800700 1 -- 172.21.15.6:0/2395023742 shutdown_connections 2024-05-09T04:48:25.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.679+0000 7f8c36800700 1 --2- 172.21.15.6:0/2395023742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c3010e930 0x7f8c30110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:25.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.679+0000 7f8c36800700 1 --2- 172.21.15.6:0/2395023742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c30105500 0x7f8c30105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:25.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.679+0000 7f8c36800700 1 --2- 172.21.15.6:0/2395023742 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c30105e50 0x7f8c3010e3a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:25.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.679+0000 7f8c36800700 1 -- 172.21.15.6:0/2395023742 >> 172.21.15.6:0/2395023742 conn(0x7f8c300fedf0 msgr2=0x7f8c30101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:25.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.679+0000 7f8c36800700 1 -- 172.21.15.6:0/2395023742 shutdown_connections 2024-05-09T04:48:25.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.679+0000 7f8c36800700 1 -- 172.21.15.6:0/2395023742 wait complete. 2024-05-09T04:48:25.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.680+0000 7f8c36800700 1 Processor -- start 2024-05-09T04:48:25.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.681+0000 7f8c36800700 1 -- start start 2024-05-09T04:48:25.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.681+0000 7f8c36800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c30105500 0x7f8c3019d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:25.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.681+0000 7f8c36800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c30105e50 0x7f8c3019d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:25.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.681+0000 7f8c36800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c3010e930 0x7f8c301a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:25.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.681+0000 7f8c36800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8c301144a0 con 0x7f8c30105e50 2024-05-09T04:48:25.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.681+0000 7f8c36800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8c30114320 con 0x7f8c30105500 2024-05-09T04:48:25.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.681+0000 7f8c36800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8c30114620 con 0x7f8c3010e930 2024-05-09T04:48:25.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.681+0000 7f8c27e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c30105e50 0x7f8c3019d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:25.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.681+0000 7f8c2f400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c30105500 0x7f8c3019d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:25.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.682+0000 7f8c27e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c30105e50 0x7f8c3019d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45772/0 (socket says 172.21.15.6:45772) 2024-05-09T04:48:25.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.682+0000 7f8c27e00700 1 -- 172.21.15.6:0/2038878698 learned_addr learned my addr 172.21.15.6:0/2038878698 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:48:25.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.682+0000 7f8c2f400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c30105500 0x7f8c3019d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:33066/0 (socket says 172.21.15.6:33066) 2024-05-09T04:48:25.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.682+0000 7f8c2fe00700 1 --2- 172.21.15.6:0/2038878698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c3010e930 0x7f8c301a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:25.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.682+0000 7f8c2f400700 1 -- 172.21.15.6:0/2038878698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c3010e930 msgr2=0x7f8c301a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:25.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.682+0000 7f8c2f400700 1 --2- 172.21.15.6:0/2038878698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c3010e930 0x7f8c301a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:25.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.682+0000 7f8c2f400700 1 -- 172.21.15.6:0/2038878698 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c30105e50 msgr2=0x7f8c3019d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:25.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.682+0000 7f8c2f400700 1 --2- 172.21.15.6:0/2038878698 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c30105e50 0x7f8c3019d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:25.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.682+0000 7f8c2f400700 1 -- 172.21.15.6:0/2038878698 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c301a1f40 con 0x7f8c30105500 2024-05-09T04:48:25.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.682+0000 7f8c2fe00700 1 --2- 172.21.15.6:0/2038878698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c3010e930 0x7f8c301a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:48:25.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.682+0000 7f8c27e00700 1 --2- 172.21.15.6:0/2038878698 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c30105e50 0x7f8c3019d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:48:25.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.682+0000 7f8c2f400700 1 --2- 172.21.15.6:0/2038878698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c30105500 0x7f8c3019d070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8c2000eee0 tx=0x7f8c2000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:25.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.683+0000 7f8c2cc00700 1 -- 172.21.15.6:0/2038878698 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c200092e0 con 0x7f8c30105500 2024-05-09T04:48:25.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.683+0000 7f8c2cc00700 1 -- 172.21.15.6:0/2038878698 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c2000c9b0 con 0x7f8c30105500 2024-05-09T04:48:25.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.683+0000 7f8c36800700 1 -- 172.21.15.6:0/2038878698 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c301a21d0 con 0x7f8c30105500 2024-05-09T04:48:25.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.683+0000 7f8c2cc00700 1 -- 172.21.15.6:0/2038878698 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c200044c0 con 0x7f8c30105500 2024-05-09T04:48:25.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.683+0000 7f8c36800700 1 -- 172.21.15.6:0/2038878698 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c301a9b50 con 0x7f8c30105500 2024-05-09T04:48:25.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.684+0000 7f8c36800700 1 -- 172.21.15.6:0/2038878698 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c300512c0 con 0x7f8c30105500 2024-05-09T04:48:25.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.687+0000 7f8c2cc00700 1 -- 172.21.15.6:0/2038878698 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f8c2001b030 con 0x7f8c30105500 2024-05-09T04:48:25.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.688+0000 7f8c2cc00700 1 --2- 172.21.15.6:0/2038878698 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f8c140b2880 0x7f8c140b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:25.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.689+0000 7f8c2cc00700 1 -- 172.21.15.6:0/2038878698 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f8c200e26b0 con 0x7f8c30105500 2024-05-09T04:48:25.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.689+0000 7f8c27e00700 1 --2- 172.21.15.6:0/2038878698 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f8c140b2880 0x7f8c140b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:25.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.690+0000 7f8c27e00700 1 --2- 172.21.15.6:0/2038878698 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f8c140b2880 0x7f8c140b4d90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8c301022c0 tx=0x7f8c1800b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:25.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.690+0000 7f8c2cc00700 1 -- 172.21.15.6:0/2038878698 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8c20022020 con 0x7f8c30105500 2024-05-09T04:48:25.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.926+0000 7f8c36800700 1 -- 172.21.15.6:0/2038878698 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8c301a9d20 con 0x7f8c30105500 2024-05-09T04:48:25.928 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2024-05-09T04:48:25.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.927+0000 7f8c2cc00700 1 -- 172.21.15.6:0/2038878698 <== mon.2 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7f8c20010040 con 0x7f8c30105500 2024-05-09T04:48:25.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.930+0000 7f8c36800700 1 -- 172.21.15.6:0/2038878698 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f8c140b2880 msgr2=0x7f8c140b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:25.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.930+0000 7f8c36800700 1 --2- 172.21.15.6:0/2038878698 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f8c140b2880 0x7f8c140b4d90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8c301022c0 tx=0x7f8c1800b280 comp rx=0 tx=0).stop 2024-05-09T04:48:25.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.930+0000 7f8c36800700 1 -- 172.21.15.6:0/2038878698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c30105500 msgr2=0x7f8c3019d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:25.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.930+0000 7f8c36800700 1 --2- 172.21.15.6:0/2038878698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c30105500 0x7f8c3019d070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8c2000eee0 tx=0x7f8c2000c3a0 comp rx=0 tx=0).stop 2024-05-09T04:48:25.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.931+0000 7f8c36800700 1 -- 172.21.15.6:0/2038878698 shutdown_connections 2024-05-09T04:48:25.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.931+0000 7f8c36800700 1 --2- 172.21.15.6:0/2038878698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c3010e930 0x7f8c301a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:25.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.931+0000 7f8c36800700 1 --2- 172.21.15.6:0/2038878698 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f8c140b2880 0x7f8c140b4d90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:25.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.931+0000 7f8c36800700 1 --2- 172.21.15.6:0/2038878698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c30105500 0x7f8c3019d070 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:25.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.931+0000 7f8c36800700 1 --2- 172.21.15.6:0/2038878698 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c30105e50 0x7f8c3019d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:25.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.931+0000 7f8c36800700 1 -- 172.21.15.6:0/2038878698 >> 172.21.15.6:0/2038878698 conn(0x7f8c300fedf0 msgr2=0x7f8c300ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:25.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.931+0000 7f8c36800700 1 -- 172.21.15.6:0/2038878698 shutdown_connections 2024-05-09T04:48:25.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:25.931+0000 7f8c36800700 1 -- 172.21.15.6:0/2038878698 wait complete. 2024-05-09T04:48:26.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:26 smithi006 ceph-mon[29835]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:26.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:26 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/2038878698' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T04:48:26.800 INFO:tasks.cephadm:Adding mds.a on smithi006 2024-05-09T04:48:26.800 INFO:tasks.cephadm:Adding mds.d on smithi006 2024-05-09T04:48:26.800 INFO:tasks.cephadm:Adding mds.g on smithi006 2024-05-09T04:48:26.800 INFO:tasks.cephadm:Adding mds.j on smithi006 2024-05-09T04:48:26.800 INFO:tasks.cephadm:Adding mds.b on smithi169 2024-05-09T04:48:26.800 INFO:tasks.cephadm:Adding mds.e on smithi169 2024-05-09T04:48:26.800 INFO:tasks.cephadm:Adding mds.h on smithi169 2024-05-09T04:48:26.800 INFO:tasks.cephadm:Adding mds.k on smithi169 2024-05-09T04:48:26.800 INFO:tasks.cephadm:Adding mds.c on smithi186 2024-05-09T04:48:26.800 INFO:tasks.cephadm:Adding mds.f on smithi186 2024-05-09T04:48:26.801 INFO:tasks.cephadm:Adding mds.i on smithi186 2024-05-09T04:48:26.801 INFO:tasks.cephadm:Adding mds.l on smithi186 2024-05-09T04:48:26.801 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph orch apply mds all '12;smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi169=b;smithi169=e;smithi169=h;smithi169=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l' 2024-05-09T04:48:26.828 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:26 smithi186 ceph-mon[34668]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:26.829 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:26 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/2038878698' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T04:48:26.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:26 smithi169 ceph-mon[34075]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:26.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:26 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/2038878698' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T04:48:27.052 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T04:48:28.837 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:28 smithi186 ceph-mon[34668]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:28.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:28 smithi169 ceph-mon[34075]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:28.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:28 smithi006 ceph-mon[29835]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:28.952 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.950+0000 7f602f9b6700 1 -- 172.21.15.186:0/1522196294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6028104d00 msgr2=0x7f602810b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:28.952 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.950+0000 7f602f9b6700 1 --2- 172.21.15.186:0/1522196294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6028104d00 0x7f602810b5b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6020009990 tx=0x7f6020009ca0 comp rx=0 tx=0).stop 2024-05-09T04:48:28.952 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.950+0000 7f602f9b6700 1 -- 172.21.15.186:0/1522196294 shutdown_connections 2024-05-09T04:48:28.952 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.950+0000 7f602f9b6700 1 --2- 172.21.15.186:0/1522196294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6028104d00 0x7f602810b5b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:28.952 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.950+0000 7f602f9b6700 1 --2- 172.21.15.186:0/1522196294 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6028103230 0x7f6028103640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:28.953 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.950+0000 7f602f9b6700 1 --2- 172.21.15.186:0/1522196294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6028104360 0x7f6028104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:28.953 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.950+0000 7f602f9b6700 1 -- 172.21.15.186:0/1522196294 >> 172.21.15.186:0/1522196294 conn(0x7f60280fe870 msgr2=0x7f6028100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:28.953 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.950+0000 7f602f9b6700 1 -- 172.21.15.186:0/1522196294 shutdown_connections 2024-05-09T04:48:28.953 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.950+0000 7f602f9b6700 1 -- 172.21.15.186:0/1522196294 wait complete. 2024-05-09T04:48:28.953 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.951+0000 7f602f9b6700 1 Processor -- start 2024-05-09T04:48:28.953 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.951+0000 7f602f9b6700 1 -- start start 2024-05-09T04:48:28.953 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.951+0000 7f602f9b6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6028103230 0x7f602806caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:28.953 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.951+0000 7f602f9b6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6028104360 0x7f60281a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:28.953 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.951+0000 7f602f9b6700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6028104d00 0x7f60281a4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:28.953 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.951+0000 7f602f9b6700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f602810ed60 con 0x7f6028104360 2024-05-09T04:48:28.953 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.951+0000 7f602f9b6700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f602810ebe0 con 0x7f6028104d00 2024-05-09T04:48:28.953 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.951+0000 7f602f9b6700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f602810eee0 con 0x7f6028103230 2024-05-09T04:48:28.953 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.951+0000 7f602d752700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6028103230 0x7f602806caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:28.954 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.951+0000 7f602d752700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6028103230 0x7f602806caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:46020/0 (socket says 172.21.15.186:46020) 2024-05-09T04:48:28.954 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.951+0000 7f602d752700 1 -- 172.21.15.186:0/1847473702 learned_addr learned my addr 172.21.15.186:0/1847473702 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-05-09T04:48:28.954 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.951+0000 7f602df53700 1 --2- 172.21.15.186:0/1847473702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6028104d00 0x7f60281a4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:28.954 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.952+0000 7f602d752700 1 -- 172.21.15.186:0/1847473702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6028104d00 msgr2=0x7f60281a4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:28.954 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.952+0000 7f602d752700 1 --2- 172.21.15.186:0/1847473702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6028104d00 0x7f60281a4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:28.954 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.952+0000 7f602d752700 1 -- 172.21.15.186:0/1847473702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6028104360 msgr2=0x7f60281a1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:28.954 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.952+0000 7f602d752700 1 --2- 172.21.15.186:0/1847473702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6028104360 0x7f60281a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:28.954 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.952+0000 7f602d752700 1 -- 172.21.15.186:0/1847473702 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60281a46f0 con 0x7f6028103230 2024-05-09T04:48:28.954 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.952+0000 7f602d752700 1 --2- 172.21.15.186:0/1847473702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6028103230 0x7f602806caa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f602400b990 tx=0x7f602400bca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:28.954 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.952+0000 7f601e7fc700 1 -- 172.21.15.186:0/1847473702 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6024019070 con 0x7f6028103230 2024-05-09T04:48:28.954 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.952+0000 7f601e7fc700 1 -- 172.21.15.186:0/1847473702 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f602400cde0 con 0x7f6028103230 2024-05-09T04:48:28.954 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.952+0000 7f602f9b6700 1 -- 172.21.15.186:0/1847473702 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60281a49e0 con 0x7f6028103230 2024-05-09T04:48:28.954 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.952+0000 7f601e7fc700 1 -- 172.21.15.186:0/1847473702 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6024012340 con 0x7f6028103230 2024-05-09T04:48:28.955 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.952+0000 7f602f9b6700 1 -- 172.21.15.186:0/1847473702 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60281a4fa0 con 0x7f6028103230 2024-05-09T04:48:28.955 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.953+0000 7f602f9b6700 1 -- 172.21.15.186:0/1847473702 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60280512c0 con 0x7f6028103230 2024-05-09T04:48:28.955 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.955+0000 7f601e7fc700 1 -- 172.21.15.186:0/1847473702 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f602401c070 con 0x7f6028103230 2024-05-09T04:48:28.955 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.955+0000 7f601e7fc700 1 --2- 172.21.15.186:0/1847473702 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f60140b2880 0x7f60140b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:28.956 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.956+0000 7f601e7fc700 1 -- 172.21.15.186:0/1847473702 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f60240e1930 con 0x7f6028103230 2024-05-09T04:48:28.956 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.956+0000 7f602cf51700 1 --2- 172.21.15.186:0/1847473702 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f60140b2880 0x7f60140b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:28.956 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.957+0000 7f602cf51700 1 --2- 172.21.15.186:0/1847473702 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f60140b2880 0x7f60140b4d90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6018005950 tx=0x7f60180058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:28.959 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:28.960+0000 7f601e7fc700 1 -- 172.21.15.186:0/1847473702 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6024023070 con 0x7f6028103230 2024-05-09T04:48:29.173 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:29.173+0000 7f602f9b6700 1 -- 172.21.15.186:0/1847473702 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi169=b;smithi169=e;smithi169=h;smithi169=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}) -- 0x7f6028063dc0 con 0x7f60140b2880 2024-05-09T04:48:29.192 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:29.192+0000 7f601e7fc700 1 -- 172.21.15.186:0/1847473702 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f6028063dc0 con 0x7f60140b2880 2024-05-09T04:48:29.192 INFO:teuthology.orchestra.run.smithi186.stdout:Scheduled mds.all update... 2024-05-09T04:48:29.194 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:29.195+0000 7f602f9b6700 1 -- 172.21.15.186:0/1847473702 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f60140b2880 msgr2=0x7f60140b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:29.195 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:29.195+0000 7f602f9b6700 1 --2- 172.21.15.186:0/1847473702 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f60140b2880 0x7f60140b4d90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6018005950 tx=0x7f60180058e0 comp rx=0 tx=0).stop 2024-05-09T04:48:29.195 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:29.195+0000 7f602f9b6700 1 -- 172.21.15.186:0/1847473702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6028103230 msgr2=0x7f602806caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:29.195 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:29.195+0000 7f602f9b6700 1 --2- 172.21.15.186:0/1847473702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6028103230 0x7f602806caa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f602400b990 tx=0x7f602400bca0 comp rx=0 tx=0).stop 2024-05-09T04:48:29.195 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:29.195+0000 7f602f9b6700 1 -- 172.21.15.186:0/1847473702 shutdown_connections 2024-05-09T04:48:29.195 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:29.195+0000 7f602f9b6700 1 --2- 172.21.15.186:0/1847473702 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f60140b2880 0x7f60140b4d90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:29.195 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:29.195+0000 7f602f9b6700 1 --2- 172.21.15.186:0/1847473702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6028104d00 0x7f60281a4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:29.195 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:29.195+0000 7f602f9b6700 1 --2- 172.21.15.186:0/1847473702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6028103230 0x7f602806caa0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:29.195 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:29.195+0000 7f602f9b6700 1 --2- 172.21.15.186:0/1847473702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6028104360 0x7f60281a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:29.195 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:29.195+0000 7f602f9b6700 1 -- 172.21.15.186:0/1847473702 >> 172.21.15.186:0/1847473702 conn(0x7f60280fe870 msgr2=0x7f6028107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:29.195 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:29.195+0000 7f602f9b6700 1 -- 172.21.15.186:0/1847473702 shutdown_connections 2024-05-09T04:48:29.195 INFO:teuthology.orchestra.run.smithi186.stderr:2024-05-09T04:48:29.195+0000 7f602f9b6700 1 -- 172.21.15.186:0/1847473702 wait complete. 2024-05-09T04:48:29.845 DEBUG:teuthology.orchestra.run.smithi006:mds.a> sudo journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.a.service 2024-05-09T04:48:29.853 DEBUG:teuthology.orchestra.run.smithi006:mds.d> sudo journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.d.service 2024-05-09T04:48:29.900 DEBUG:teuthology.orchestra.run.smithi006:mds.g> sudo journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.g.service 2024-05-09T04:48:29.946 DEBUG:teuthology.orchestra.run.smithi006:mds.j> sudo journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.j.service 2024-05-09T04:48:29.992 DEBUG:teuthology.orchestra.run.smithi169:mds.b> sudo journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.b.service 2024-05-09T04:48:29.995 DEBUG:teuthology.orchestra.run.smithi169:mds.e> sudo journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.e.service 2024-05-09T04:48:30.038 DEBUG:teuthology.orchestra.run.smithi169:mds.h> sudo journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.h.service 2024-05-09T04:48:30.081 DEBUG:teuthology.orchestra.run.smithi169:mds.k> sudo journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.k.service 2024-05-09T04:48:30.124 DEBUG:teuthology.orchestra.run.smithi186:mds.c> sudo journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.c.service 2024-05-09T04:48:30.126 DEBUG:teuthology.orchestra.run.smithi186:mds.f> sudo journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.f.service 2024-05-09T04:48:30.169 DEBUG:teuthology.orchestra.run.smithi186:mds.i> sudo journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.i.service 2024-05-09T04:48:30.213 DEBUG:teuthology.orchestra.run.smithi186:mds.l> sudo journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.l.service 2024-05-09T04:48:30.255 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-09T04:48:30.256 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:48:30.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:30 smithi169 ceph-mon[34075]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi169=b;smithi169=e;smithi169=h;smithi169=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:48:30.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:30 smithi169 ceph-mon[34075]: Saving service mds.all spec with placement smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi169=b;smithi169=e;smithi169=h;smithi169=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2024-05-09T04:48:30.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:30 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:30.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:30 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:48:30.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:30 smithi169 ceph-mon[34075]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:30.437 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:30 smithi186 ceph-mon[34668]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi169=b;smithi169=e;smithi169=h;smithi169=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:48:30.437 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:30 smithi186 ceph-mon[34668]: Saving service mds.all spec with placement smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi169=b;smithi169=e;smithi169=h;smithi169=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2024-05-09T04:48:30.437 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:30 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:30.437 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:30 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:48:30.437 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:30 smithi186 ceph-mon[34668]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:30.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:30 smithi006 ceph-mon[29835]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi169=b;smithi169=e;smithi169=h;smithi169=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:48:30.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:30 smithi006 ceph-mon[29835]: Saving service mds.all spec with placement smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi169=b;smithi169=e;smithi169=h;smithi169=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2024-05-09T04:48:30.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:30 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:30.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:30 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:48:30.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:30 smithi006 ceph-mon[29835]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:30.725 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:31.312 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:31 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:31.312 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:31 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:48:31.312 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:31 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:31.312 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:31 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:48:31.312 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:31 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:48:31.312 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:31 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:31.312 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:31 smithi186 ceph-mon[34668]: Deploying daemon mds.l on smithi186 2024-05-09T04:48:31.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:31 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:31.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:31 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:48:31.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:31 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:31.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:31 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:48:31.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:31 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:48:31.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:31 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:31.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:31 smithi169 ceph-mon[34075]: Deploying daemon mds.l on smithi186 2024-05-09T04:48:31.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:31 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:31.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:31 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:48:31.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:31 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:31.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:31 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:48:31.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:31 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:48:31.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:31 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:31.673 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:31 smithi006 ceph-mon[29835]: Deploying daemon mds.l on smithi186 2024-05-09T04:48:32.338 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:32 smithi006 ceph-mon[29835]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:32.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:32 smithi186 ceph-mon[34668]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:32.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:32 smithi169 ceph-mon[34075]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:32.838 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 09 04:48:32 smithi186 systemd[1]: Starting Ceph mds.l for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T04:48:32.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.946+0000 7f1d69200700 1 -- 172.21.15.6:0/3796796686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d641109b0 msgr2=0x7f1d64112e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:32.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.946+0000 7f1d69200700 1 --2- 172.21.15.6:0/3796796686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d641109b0 0x7f1d64112e10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00b0a0 tx=0x7f1d5c00b3b0 comp rx=0 tx=0).stop 2024-05-09T04:48:32.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.947+0000 7f1d69200700 1 -- 172.21.15.6:0/3796796686 shutdown_connections 2024-05-09T04:48:32.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.947+0000 7f1d69200700 1 --2- 172.21.15.6:0/3796796686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d641109b0 0x7f1d64112e10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:32.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.947+0000 7f1d69200700 1 --2- 172.21.15.6:0/3796796686 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d64075d30 0x7f1d64076140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:32.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.947+0000 7f1d69200700 1 --2- 172.21.15.6:0/3796796686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d64076680 0x7f1d64110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:32.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.947+0000 7f1d69200700 1 -- 172.21.15.6:0/3796796686 >> 172.21.15.6:0/3796796686 conn(0x7f1d64079d10 msgr2=0x7f1d6407a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:32.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.947+0000 7f1d69200700 1 -- 172.21.15.6:0/3796796686 shutdown_connections 2024-05-09T04:48:32.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.947+0000 7f1d69200700 1 -- 172.21.15.6:0/3796796686 wait complete. 2024-05-09T04:48:32.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.948+0000 7f1d69200700 1 Processor -- start 2024-05-09T04:48:32.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.948+0000 7f1d69200700 1 -- start start 2024-05-09T04:48:32.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.949+0000 7f1d69200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d64075d30 0x7f1d6419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:32.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.949+0000 7f1d69200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d64076680 0x7f1d6419d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:32.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.949+0000 7f1d69200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d641109b0 0x7f1d641a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:32.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.949+0000 7f1d69200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1d641164e0 con 0x7f1d641109b0 2024-05-09T04:48:32.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.949+0000 7f1d69200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1d64116360 con 0x7f1d64076680 2024-05-09T04:48:32.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.949+0000 7f1d69200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1d64116660 con 0x7f1d64075d30 2024-05-09T04:48:32.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.949+0000 7f1d63e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d641109b0 0x7f1d641a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:32.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.949+0000 7f1d63400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d64075d30 0x7f1d6419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:32.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.949+0000 7f1d63400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d64075d30 0x7f1d6419ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:36688/0 (socket says 172.21.15.6:36688) 2024-05-09T04:48:32.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.949+0000 7f1d63e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d641109b0 0x7f1d641a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59434/0 (socket says 172.21.15.6:59434) 2024-05-09T04:48:32.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.949+0000 7f1d63400700 1 -- 172.21.15.6:0/1932702156 learned_addr learned my addr 172.21.15.6:0/1932702156 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:48:32.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.949+0000 7f1d62a00700 1 --2- 172.21.15.6:0/1932702156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d64076680 0x7f1d6419d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:32.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.949+0000 7f1d63400700 1 -- 172.21.15.6:0/1932702156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d64076680 msgr2=0x7f1d6419d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:32.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.949+0000 7f1d63400700 1 --2- 172.21.15.6:0/1932702156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d64076680 0x7f1d6419d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:32.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.949+0000 7f1d63400700 1 -- 172.21.15.6:0/1932702156 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d641109b0 msgr2=0x7f1d641a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:32.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.949+0000 7f1d63400700 1 --2- 172.21.15.6:0/1932702156 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d641109b0 0x7f1d641a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:32.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.949+0000 7f1d63400700 1 -- 172.21.15.6:0/1932702156 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d641a1d60 con 0x7f1d64075d30 2024-05-09T04:48:32.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.950+0000 7f1d62a00700 1 --2- 172.21.15.6:0/1932702156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d64076680 0x7f1d6419d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:48:32.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.950+0000 7f1d63400700 1 --2- 172.21.15.6:0/1932702156 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d64075d30 0x7f1d6419ce90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1d5400e9a0 tx=0x7f1d5400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:32.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.950+0000 7f1d5be00700 1 -- 172.21.15.6:0/1932702156 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5400cbd0 con 0x7f1d64075d30 2024-05-09T04:48:32.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.950+0000 7f1d5be00700 1 -- 172.21.15.6:0/1932702156 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d5400cd70 con 0x7f1d64075d30 2024-05-09T04:48:32.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.950+0000 7f1d5be00700 1 -- 172.21.15.6:0/1932702156 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54018680 con 0x7f1d64075d30 2024-05-09T04:48:32.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.950+0000 7f1d69200700 1 -- 172.21.15.6:0/1932702156 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d641a2050 con 0x7f1d64075d30 2024-05-09T04:48:32.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.951+0000 7f1d69200700 1 -- 172.21.15.6:0/1932702156 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d641a9970 con 0x7f1d64075d30 2024-05-09T04:48:32.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.952+0000 7f1d69200700 1 -- 172.21.15.6:0/1932702156 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d640772a0 con 0x7f1d64075d30 2024-05-09T04:48:32.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.954+0000 7f1d5be00700 1 -- 172.21.15.6:0/1932702156 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f1d5404aab0 con 0x7f1d64075d30 2024-05-09T04:48:32.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.956+0000 7f1d5be00700 1 --2- 172.21.15.6:0/1932702156 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f1d480b2900 0x7f1d480b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:32.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.956+0000 7f1d5be00700 1 -- 172.21.15.6:0/1932702156 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f1d540e2090 con 0x7f1d64075d30 2024-05-09T04:48:32.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.956+0000 7f1d62a00700 1 --2- 172.21.15.6:0/1932702156 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f1d480b2900 0x7f1d480b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:32.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.957+0000 7f1d62a00700 1 --2- 172.21.15.6:0/1932702156 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f1d480b2900 0x7f1d480b4e10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1d64112870 tx=0x7f1d50009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:32.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:32.958+0000 7f1d5be00700 1 -- 172.21.15.6:0/1932702156 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1d540aa1b0 con 0x7f1d64075d30 2024-05-09T04:48:33.099 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 09 04:48:32 smithi186 podman[64923]: 2024-05-09 04:48:32.740646319 +0000 UTC m=+0.018009899 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T04:48:33.099 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 09 04:48:32 smithi186 podman[64923]: 2024-05-09 04:48:32.886812607 +0000 UTC m=+0.164176159 container create 1123264bf8e8e376547c91d8ef07d5dbe1c7c5680c7da11ea977381c1f2a8a90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-l, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-09T04:48:33.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:33.248+0000 7f1d69200700 1 -- 172.21.15.6:0/1932702156 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d6419e1f0 con 0x7f1d64075d30 2024-05-09T04:48:33.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:33.249+0000 7f1d5be00700 1 -- 172.21.15.6:0/1932702156 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f1d540b2230 con 0x7f1d64075d30 2024-05-09T04:48:33.250 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:48:33.250 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/2977145206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2977145206},{"type":"v1","addr":"172.21.15.186:6833","nonce":2977145206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-09T04:48:33.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:33.252+0000 7f1d69200700 1 -- 172.21.15.6:0/1932702156 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f1d480b2900 msgr2=0x7f1d480b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:33.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:33.252+0000 7f1d69200700 1 --2- 172.21.15.6:0/1932702156 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f1d480b2900 0x7f1d480b4e10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1d64112870 tx=0x7f1d50009260 comp rx=0 tx=0).stop 2024-05-09T04:48:33.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:33.252+0000 7f1d69200700 1 -- 172.21.15.6:0/1932702156 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d64075d30 msgr2=0x7f1d6419ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:33.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:33.252+0000 7f1d69200700 1 --2- 172.21.15.6:0/1932702156 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d64075d30 0x7f1d6419ce90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1d5400e9a0 tx=0x7f1d5400ecb0 comp rx=0 tx=0).stop 2024-05-09T04:48:33.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:33.252+0000 7f1d69200700 1 -- 172.21.15.6:0/1932702156 shutdown_connections 2024-05-09T04:48:33.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:33.252+0000 7f1d69200700 1 --2- 172.21.15.6:0/1932702156 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d641109b0 0x7f1d641a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:33.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:33.252+0000 7f1d69200700 1 --2- 172.21.15.6:0/1932702156 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d64075d30 0x7f1d6419ce90 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:33.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:33.252+0000 7f1d69200700 1 --2- 172.21.15.6:0/1932702156 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f1d480b2900 0x7f1d480b4e10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:33.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:33.252+0000 7f1d69200700 1 --2- 172.21.15.6:0/1932702156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d64076680 0x7f1d6419d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:33.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:33.252+0000 7f1d69200700 1 -- 172.21.15.6:0/1932702156 >> 172.21.15.6:0/1932702156 conn(0x7f1d64079d10 msgr2=0x7f1d64100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:33.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:33.253+0000 7f1d69200700 1 -- 172.21.15.6:0/1932702156 shutdown_connections 2024-05-09T04:48:33.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:33.253+0000 7f1d69200700 1 -- 172.21.15.6:0/1932702156 wait complete. 2024-05-09T04:48:33.255 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 2 2024-05-09T04:48:33.599 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 09 04:48:33 smithi186 podman[64923]: 2024-05-09 04:48:33.134421998 +0000 UTC m=+0.411785551 container init 1123264bf8e8e376547c91d8ef07d5dbe1c7c5680c7da11ea977381c1f2a8a90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-l, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5) 2024-05-09T04:48:33.599 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 09 04:48:33 smithi186 podman[64923]: 2024-05-09 04:48:33.137367593 +0000 UTC m=+0.414731146 container start 1123264bf8e8e376547c91d8ef07d5dbe1c7c5680c7da11ea977381c1f2a8a90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD) 2024-05-09T04:48:33.599 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 09 04:48:33 smithi186 ceph-mds[64942]: -- 172.21.15.186:0/260852747 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556ad53eda40 con 0x556ad5435180 2024-05-09T04:48:33.599 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 09 04:48:33 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-l[64938]: starting mds.l at 2024-05-09T04:48:33.600 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 09 04:48:33 smithi186 bash[64923]: 1123264bf8e8e376547c91d8ef07d5dbe1c7c5680c7da11ea977381c1f2a8a90 2024-05-09T04:48:33.600 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 09 04:48:33 smithi186 systemd[1]: Started Ceph mds.l for f74d813c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:48:34.333 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:34 smithi169 ceph-mon[34075]: mds.? [v2:172.21.15.186:6832/2977145206,v1:172.21.15.186:6833/2977145206] up:boot 2024-05-09T04:48:34.333 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:34 smithi169 ceph-mon[34075]: fsmap 1 up:standby 2024-05-09T04:48:34.333 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:34 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T04:48:34.333 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:34 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T04:48:34.333 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:34 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/1932702156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:48:34.333 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:34 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:34.334 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:34 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:34.334 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:34 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:34.334 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:34 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:48:34.334 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:34 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:48:34.334 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:34 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:34.334 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:34 smithi169 ceph-mon[34075]: Deploying daemon mds.k on smithi169 2024-05-09T04:48:34.334 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:34 smithi169 ceph-mon[34075]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:34.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:34 smithi006 ceph-mon[29835]: mds.? [v2:172.21.15.186:6832/2977145206,v1:172.21.15.186:6833/2977145206] up:boot 2024-05-09T04:48:34.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:34 smithi006 ceph-mon[29835]: fsmap 1 up:standby 2024-05-09T04:48:34.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:34 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T04:48:34.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:34 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T04:48:34.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:34 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1932702156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:48:34.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:34 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:34.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:34 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:34.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:34 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:34.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:34 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:48:34.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:34 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:48:34.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:34 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:34.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:34 smithi006 ceph-mon[29835]: Deploying daemon mds.k on smithi169 2024-05-09T04:48:34.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:34 smithi006 ceph-mon[29835]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:34.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:34 smithi186 ceph-mon[34668]: mds.? [v2:172.21.15.186:6832/2977145206,v1:172.21.15.186:6833/2977145206] up:boot 2024-05-09T04:48:34.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:34 smithi186 ceph-mon[34668]: fsmap 1 up:standby 2024-05-09T04:48:34.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:34 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T04:48:34.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:34 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T04:48:34.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:34 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/1932702156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:48:34.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:34 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:34.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:34 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:34.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:34 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:34.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:34 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:48:34.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:34 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:48:34.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:34 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:34.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:34 smithi186 ceph-mon[34668]: Deploying daemon mds.k on smithi169 2024-05-09T04:48:34.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:34 smithi186 ceph-mon[34668]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:36.028 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:48:36.117 INFO:journalctl@ceph.mds.k.smithi169.stdout:May 09 04:48:36 smithi169 podman[63972]: 2024-05-09 04:48:36.032883978 +0000 UTC m=+0.146760635 container create 92ceeac7bec680bffe8909eff262ed4a3d8d3aebf19b8b8c8dcc9dbdc378a4d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-k, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, ceph=True) 2024-05-09T04:48:36.493 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:36.568 INFO:journalctl@ceph.mds.k.smithi169.stdout:May 09 04:48:36 smithi169 podman[63972]: 2024-05-09 04:48:36.268796879 +0000 UTC m=+0.382663827 container init 92ceeac7bec680bffe8909eff262ed4a3d8d3aebf19b8b8c8dcc9dbdc378a4d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-k, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-09T04:48:36.568 INFO:journalctl@ceph.mds.k.smithi169.stdout:May 09 04:48:36 smithi169 podman[63972]: 2024-05-09 04:48:36.271375505 +0000 UTC m=+0.385242447 container start 92ceeac7bec680bffe8909eff262ed4a3d8d3aebf19b8b8c8dcc9dbdc378a4d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-k, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240507) 2024-05-09T04:48:36.568 INFO:journalctl@ceph.mds.k.smithi169.stdout:May 09 04:48:36 smithi169 ceph-mds[63991]: -- 172.21.15.169:0/161222154 <== mon.2 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55eb10867a40 con 0x55eb108afa80 2024-05-09T04:48:36.568 INFO:journalctl@ceph.mds.k.smithi169.stdout:May 09 04:48:36 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-k[63987]: starting mds.k at 2024-05-09T04:48:36.568 INFO:journalctl@ceph.mds.k.smithi169.stdout:May 09 04:48:36 smithi169 bash[63972]: 92ceeac7bec680bffe8909eff262ed4a3d8d3aebf19b8b8c8dcc9dbdc378a4d1 2024-05-09T04:48:36.569 INFO:journalctl@ceph.mds.k.smithi169.stdout:May 09 04:48:36 smithi169 systemd[1]: Started Ceph mds.k for f74d813c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:48:36.848 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:36 smithi006 ceph-mon[29835]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:36.848 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:36 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:36.849 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:36 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:36.849 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:36 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:36.849 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:36 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:48:36.849 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:36 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:48:36.849 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:36 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:36.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:36 smithi186 ceph-mon[34668]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:36.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:36 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:36.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:36 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:36.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:36 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:36.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:36 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:48:36.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:36 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:48:36.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:36 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:36.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:36 smithi169 ceph-mon[34075]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:36.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:36 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:36.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:36 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:36.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:36 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:36.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:36 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:48:36.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:36 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:48:36.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:36 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:37.842 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:37 smithi169 ceph-mon[34075]: Deploying daemon mds.j on smithi006 2024-05-09T04:48:37.842 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:37 smithi169 ceph-mon[34075]: mds.? [v2:172.21.15.169:6832/133345948,v1:172.21.15.169:6833/133345948] up:boot 2024-05-09T04:48:37.843 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:37 smithi169 ceph-mon[34075]: fsmap 2 up:standby 2024-05-09T04:48:37.843 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:37 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T04:48:37.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:37 smithi186 ceph-mon[34668]: Deploying daemon mds.j on smithi006 2024-05-09T04:48:37.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:37 smithi186 ceph-mon[34668]: mds.? [v2:172.21.15.169:6832/133345948,v1:172.21.15.169:6833/133345948] up:boot 2024-05-09T04:48:37.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:37 smithi186 ceph-mon[34668]: fsmap 2 up:standby 2024-05-09T04:48:37.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:37 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T04:48:37.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:37 smithi006 ceph-mon[29835]: Deploying daemon mds.j on smithi006 2024-05-09T04:48:37.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:37 smithi006 ceph-mon[29835]: mds.? [v2:172.21.15.169:6832/133345948,v1:172.21.15.169:6833/133345948] up:boot 2024-05-09T04:48:37.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:37 smithi006 ceph-mon[29835]: fsmap 2 up:standby 2024-05-09T04:48:37.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:37 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T04:48:38.642 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:38 smithi006 ceph-mon[29835]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:38.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:38 smithi186 ceph-mon[34668]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:38.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:38 smithi169 ceph-mon[34075]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:39.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.740+0000 7fc356000700 1 -- 172.21.15.6:0/837178348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc35010a7c0 msgr2=0x7fc35010abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:39.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.740+0000 7fc356000700 1 --2- 172.21.15.6:0/837178348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc35010a7c0 0x7fc35010abd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc33c009990 tx=0x7fc33c009ca0 comp rx=0 tx=0).stop 2024-05-09T04:48:39.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.740+0000 7fc356000700 1 -- 172.21.15.6:0/837178348 shutdown_connections 2024-05-09T04:48:39.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.740+0000 7fc356000700 1 --2- 172.21.15.6:0/837178348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc35010b110 0x7fc350117620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:39.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.740+0000 7fc356000700 1 --2- 172.21.15.6:0/837178348 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc35010f6c0 0x7fc35010fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:39.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.740+0000 7fc356000700 1 --2- 172.21.15.6:0/837178348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc35010a7c0 0x7fc35010abd0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:39.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.740+0000 7fc356000700 1 -- 172.21.15.6:0/837178348 >> 172.21.15.6:0/837178348 conn(0x7fc350076b00 msgr2=0x7fc350076f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:39.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.740+0000 7fc356000700 1 -- 172.21.15.6:0/837178348 shutdown_connections 2024-05-09T04:48:39.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.740+0000 7fc356000700 1 -- 172.21.15.6:0/837178348 wait complete. 2024-05-09T04:48:39.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.741+0000 7fc356000700 1 Processor -- start 2024-05-09T04:48:39.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.742+0000 7fc356000700 1 -- start start 2024-05-09T04:48:39.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.742+0000 7fc356000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc35010a7c0 0x7fc3501a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:39.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.742+0000 7fc356000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc35010b110 0x7fc3501a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:39.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.742+0000 7fc356000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc35010f6c0 0x7fc3501a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:39.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.742+0000 7fc356000700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc35011a900 con 0x7fc35010a7c0 2024-05-09T04:48:39.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.742+0000 7fc356000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc35011a780 con 0x7fc35010f6c0 2024-05-09T04:48:39.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.742+0000 7fc356000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc35011aa80 con 0x7fc35010b110 2024-05-09T04:48:39.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.742+0000 7fc354c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc35010a7c0 0x7fc3501a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:39.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.742+0000 7fc354c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc35010a7c0 0x7fc3501a1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60522/0 (socket says 172.21.15.6:60522) 2024-05-09T04:48:39.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.742+0000 7fc354c00700 1 -- 172.21.15.6:0/4002984709 learned_addr learned my addr 172.21.15.6:0/4002984709 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:48:39.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.743+0000 7fc355600700 1 --2- 172.21.15.6:0/4002984709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc35010f6c0 0x7fc3501a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:39.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.743+0000 7fc34fe00700 1 --2- 172.21.15.6:0/4002984709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc35010b110 0x7fc3501a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:39.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.743+0000 7fc354c00700 1 -- 172.21.15.6:0/4002984709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc35010b110 msgr2=0x7fc3501a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:39.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.743+0000 7fc354c00700 1 --2- 172.21.15.6:0/4002984709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc35010b110 0x7fc3501a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:39.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.743+0000 7fc354c00700 1 -- 172.21.15.6:0/4002984709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc35010f6c0 msgr2=0x7fc3501a5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:39.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.743+0000 7fc354c00700 1 --2- 172.21.15.6:0/4002984709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc35010f6c0 0x7fc3501a5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:39.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.743+0000 7fc354c00700 1 -- 172.21.15.6:0/4002984709 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3501a6120 con 0x7fc35010a7c0 2024-05-09T04:48:39.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.743+0000 7fc355600700 1 --2- 172.21.15.6:0/4002984709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc35010f6c0 0x7fc3501a5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:48:39.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.743+0000 7fc354c00700 1 --2- 172.21.15.6:0/4002984709 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc35010a7c0 0x7fc3501a1250 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc34400e9a0 tx=0x7fc34400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:39.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.744+0000 7fc34d600700 1 -- 172.21.15.6:0/4002984709 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34400ca10 con 0x7fc35010a7c0 2024-05-09T04:48:39.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.744+0000 7fc34d600700 1 -- 172.21.15.6:0/4002984709 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc34400cbb0 con 0x7fc35010a7c0 2024-05-09T04:48:39.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.744+0000 7fc356000700 1 -- 172.21.15.6:0/4002984709 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3501a6410 con 0x7fc35010a7c0 2024-05-09T04:48:39.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.744+0000 7fc34d600700 1 -- 172.21.15.6:0/4002984709 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc344018680 con 0x7fc35010a7c0 2024-05-09T04:48:39.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.744+0000 7fc356000700 1 -- 172.21.15.6:0/4002984709 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3501add30 con 0x7fc35010a7c0 2024-05-09T04:48:39.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.746+0000 7fc356000700 1 -- 172.21.15.6:0/4002984709 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc330005330 con 0x7fc35010a7c0 2024-05-09T04:48:39.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.748+0000 7fc34d600700 1 -- 172.21.15.6:0/4002984709 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fc344018820 con 0x7fc35010a7c0 2024-05-09T04:48:39.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.750+0000 7fc34d600700 1 --2- 172.21.15.6:0/4002984709 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fc3380b2900 0x7fc3380b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:39.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.750+0000 7fc34d600700 1 -- 172.21.15.6:0/4002984709 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fc3440e1e00 con 0x7fc35010a7c0 2024-05-09T04:48:39.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.750+0000 7fc34fe00700 1 --2- 172.21.15.6:0/4002984709 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fc3380b2900 0x7fc3380b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:39.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.751+0000 7fc34fe00700 1 --2- 172.21.15.6:0/4002984709 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fc3380b2900 0x7fc3380b4e10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc350077f30 tx=0x7fc33c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:39.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:39.755+0000 7fc34d600700 1 -- 172.21.15.6:0/4002984709 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc3440a9f20 con 0x7fc35010a7c0 2024-05-09T04:48:40.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:40.047+0000 7fc356000700 1 -- 172.21.15.6:0/4002984709 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc330005620 con 0x7fc35010a7c0 2024-05-09T04:48:40.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:40.048+0000 7fc34d600700 1 -- 172.21.15.6:0/4002984709 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7fc3440a9820 con 0x7fc35010a7c0 2024-05-09T04:48:40.049 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:48:40.049 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/2977145206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2977145206},{"type":"v1","addr":"172.21.15.186:6833","nonce":2977145206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/133345948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":133345948},{"type":"v1","addr":"172.21.15.169:6833","nonce":133345948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-09T04:48:40.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:40.051+0000 7fc356000700 1 -- 172.21.15.6:0/4002984709 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fc3380b2900 msgr2=0x7fc3380b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:40.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:40.051+0000 7fc356000700 1 --2- 172.21.15.6:0/4002984709 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fc3380b2900 0x7fc3380b4e10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc350077f30 tx=0x7fc33c005fd0 comp rx=0 tx=0).stop 2024-05-09T04:48:40.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:40.051+0000 7fc356000700 1 -- 172.21.15.6:0/4002984709 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc35010a7c0 msgr2=0x7fc3501a1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:40.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:40.051+0000 7fc356000700 1 --2- 172.21.15.6:0/4002984709 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc35010a7c0 0x7fc3501a1250 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc34400e9a0 tx=0x7fc34400ecb0 comp rx=0 tx=0).stop 2024-05-09T04:48:40.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:40.051+0000 7fc356000700 1 -- 172.21.15.6:0/4002984709 shutdown_connections 2024-05-09T04:48:40.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:40.051+0000 7fc356000700 1 --2- 172.21.15.6:0/4002984709 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fc3380b2900 0x7fc3380b4e10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:40.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:40.051+0000 7fc356000700 1 --2- 172.21.15.6:0/4002984709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc35010f6c0 0x7fc3501a5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:40.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:40.051+0000 7fc356000700 1 --2- 172.21.15.6:0/4002984709 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc35010a7c0 0x7fc3501a1250 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:40.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:40.051+0000 7fc356000700 1 --2- 172.21.15.6:0/4002984709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc35010b110 0x7fc3501a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:40.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:40.051+0000 7fc356000700 1 -- 172.21.15.6:0/4002984709 >> 172.21.15.6:0/4002984709 conn(0x7fc350076b00 msgr2=0x7fc350108860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:40.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:40.051+0000 7fc356000700 1 -- 172.21.15.6:0/4002984709 shutdown_connections 2024-05-09T04:48:40.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:40.052+0000 7fc356000700 1 -- 172.21.15.6:0/4002984709 wait complete. 2024-05-09T04:48:40.053 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 3 2024-05-09T04:48:40.847 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:40 smithi006 ceph-mon[29835]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:40.847 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:40 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/4002984709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:48:40.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:40 smithi186 ceph-mon[34668]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:40.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:40 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/4002984709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:48:40.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:40 smithi169 ceph-mon[34075]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:40.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:40 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/4002984709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:48:41.448 INFO:journalctl@ceph.mds.j.smithi006.stdout:May 09 04:48:41 smithi006 podman[67540]: 2024-05-09 04:48:41.189674154 +0000 UTC m=+0.188867076 container create 443ae74a980ba5014215ebc561068c6a81c08e127d73668c6f64fd387522638a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-j, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-09T04:48:41.947 INFO:journalctl@ceph.mds.j.smithi006.stdout:May 09 04:48:41 smithi006 podman[67540]: 2024-05-09 04:48:41.460107983 +0000 UTC m=+0.459300911 container init 443ae74a980ba5014215ebc561068c6a81c08e127d73668c6f64fd387522638a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-j, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-09T04:48:41.947 INFO:journalctl@ceph.mds.j.smithi006.stdout:May 09 04:48:41 smithi006 podman[67540]: 2024-05-09 04:48:41.470781345 +0000 UTC m=+0.469974274 container start 443ae74a980ba5014215ebc561068c6a81c08e127d73668c6f64fd387522638a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-j, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-09T04:48:41.947 INFO:journalctl@ceph.mds.j.smithi006.stdout:May 09 04:48:41 smithi006 bash[67540]: 443ae74a980ba5014215ebc561068c6a81c08e127d73668c6f64fd387522638a 2024-05-09T04:48:41.947 INFO:journalctl@ceph.mds.j.smithi006.stdout:May 09 04:48:41 smithi006 systemd[1]: Started Ceph mds.j for f74d813c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:48:41.947 INFO:journalctl@ceph.mds.j.smithi006.stdout:May 09 04:48:41 smithi006 ceph-mds[67559]: -- 172.21.15.6:0/1019783290 <== mon.2 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557241c4da40 con 0x557241c95a80 2024-05-09T04:48:41.948 INFO:journalctl@ceph.mds.j.smithi006.stdout:May 09 04:48:41 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-j[67555]: starting mds.j at 2024-05-09T04:48:42.852 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:48:42.935 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:42 smithi006 ceph-mon[29835]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:42.935 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:42 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:42.935 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:42 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:42.935 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:42 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:42.935 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:42 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:48:42.935 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:42 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:48:42.935 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:42 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:42.935 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:42 smithi006 ceph-mon[29835]: Deploying daemon mds.d on smithi006 2024-05-09T04:48:42.935 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:42 smithi006 ceph-mon[29835]: mds.? [v2:172.21.15.6:6834/162094599,v1:172.21.15.6:6835/162094599] up:boot 2024-05-09T04:48:42.935 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:42 smithi006 ceph-mon[29835]: fsmap 3 up:standby 2024-05-09T04:48:42.935 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:42 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T04:48:43.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:42 smithi186 ceph-mon[34668]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:43.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:42 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:43.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:42 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:43.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:42 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:43.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:42 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:48:43.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:42 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:48:43.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:42 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:43.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:42 smithi186 ceph-mon[34668]: Deploying daemon mds.d on smithi006 2024-05-09T04:48:43.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:42 smithi186 ceph-mon[34668]: mds.? [v2:172.21.15.6:6834/162094599,v1:172.21.15.6:6835/162094599] up:boot 2024-05-09T04:48:43.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:42 smithi186 ceph-mon[34668]: fsmap 3 up:standby 2024-05-09T04:48:43.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:42 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T04:48:43.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:42 smithi169 ceph-mon[34075]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:43.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:42 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:43.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:42 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:43.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:42 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:43.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:42 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:48:43.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:42 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:48:43.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:42 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:43.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:42 smithi169 ceph-mon[34075]: Deploying daemon mds.d on smithi006 2024-05-09T04:48:43.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:42 smithi169 ceph-mon[34075]: mds.? [v2:172.21.15.6:6834/162094599,v1:172.21.15.6:6835/162094599] up:boot 2024-05-09T04:48:43.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:42 smithi169 ceph-mon[34075]: fsmap 3 up:standby 2024-05-09T04:48:43.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:42 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T04:48:44.139 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:44.932 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:44 smithi006 ceph-mon[29835]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:45.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:44 smithi186 ceph-mon[34668]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:45.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:44 smithi169 ceph-mon[34075]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:46.548 INFO:journalctl@ceph.mds.d.smithi006.stdout:May 09 04:48:46 smithi006 podman[67938]: 2024-05-09 04:48:46.233744782 +0000 UTC m=+0.159859878 container create 507be399f6da9b932396846fd7b80be24df57057a4c0b09639e9cc3fdda09903 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=) 2024-05-09T04:48:46.548 INFO:journalctl@ceph.mds.d.smithi006.stdout:May 09 04:48:46 smithi006 podman[67938]: 2024-05-09 04:48:46.546425225 +0000 UTC m=+0.472540325 container init 507be399f6da9b932396846fd7b80be24df57057a4c0b09639e9cc3fdda09903 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2) 2024-05-09T04:48:46.939 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:46 smithi006 ceph-mon[29835]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:46.940 INFO:journalctl@ceph.mds.d.smithi006.stdout:May 09 04:48:46 smithi006 podman[67938]: 2024-05-09 04:48:46.558149261 +0000 UTC m=+0.484264328 container start 507be399f6da9b932396846fd7b80be24df57057a4c0b09639e9cc3fdda09903 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-d, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0) 2024-05-09T04:48:46.940 INFO:journalctl@ceph.mds.d.smithi006.stdout:May 09 04:48:46 smithi006 bash[67938]: 507be399f6da9b932396846fd7b80be24df57057a4c0b09639e9cc3fdda09903 2024-05-09T04:48:46.940 INFO:journalctl@ceph.mds.d.smithi006.stdout:May 09 04:48:46 smithi006 systemd[1]: Started Ceph mds.d for f74d813c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:48:46.940 INFO:journalctl@ceph.mds.d.smithi006.stdout:May 09 04:48:46 smithi006 ceph-mds[67972]: -- 172.21.15.6:0/2395022387 <== mon.0 v2:172.21.15.6:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cf021fda40 con 0x55cf02245600 2024-05-09T04:48:46.940 INFO:journalctl@ceph.mds.d.smithi006.stdout:May 09 04:48:46 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-d[67968]: starting mds.d at 2024-05-09T04:48:47.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:46 smithi186 ceph-mon[34668]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:47.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:46 smithi169 ceph-mon[34075]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:47.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.343+0000 7f039da00700 1 -- 172.21.15.6:0/2318963934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0398106ed0 msgr2=0x7f0398113360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:47.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.343+0000 7f039da00700 1 --2- 172.21.15.6:0/2318963934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0398106ed0 0x7f0398113360 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f039000b0a0 tx=0x7f039000b3b0 comp rx=0 tx=0).stop 2024-05-09T04:48:47.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.343+0000 7f039da00700 1 -- 172.21.15.6:0/2318963934 shutdown_connections 2024-05-09T04:48:47.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.343+0000 7f039da00700 1 --2- 172.21.15.6:0/2318963934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0398106ed0 0x7f0398113360 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:47.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.343+0000 7f039da00700 1 --2- 172.21.15.6:0/2318963934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f039810c5c0 0x7f039810c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:47.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.343+0000 7f039da00700 1 --2- 172.21.15.6:0/2318963934 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0398106580 0x7f0398106990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:47.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.343+0000 7f039da00700 1 -- 172.21.15.6:0/2318963934 >> 172.21.15.6:0/2318963934 conn(0x7f0398102080 msgr2=0x7f03981044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:47.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.344+0000 7f039da00700 1 -- 172.21.15.6:0/2318963934 shutdown_connections 2024-05-09T04:48:47.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.344+0000 7f039da00700 1 -- 172.21.15.6:0/2318963934 wait complete. 2024-05-09T04:48:47.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.345+0000 7f039da00700 1 Processor -- start 2024-05-09T04:48:47.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.346+0000 7f039da00700 1 -- start start 2024-05-09T04:48:47.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.347+0000 7f039da00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0398106580 0x7f039819d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:47.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.347+0000 7f039da00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0398106ed0 0x7f039819d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:47.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.347+0000 7f039da00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f039810c5c0 0x7f03981a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:47.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.347+0000 7f039da00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0398116740 con 0x7f039810c5c0 2024-05-09T04:48:47.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.347+0000 7f039da00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f03981165c0 con 0x7f0398106ed0 2024-05-09T04:48:47.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.347+0000 7f039da00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f03981168c0 con 0x7f0398106580 2024-05-09T04:48:47.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.347+0000 7f0397400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f039810c5c0 0x7f03981a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:47.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.347+0000 7f0397400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f039810c5c0 0x7f03981a1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37546/0 (socket says 172.21.15.6:37546) 2024-05-09T04:48:47.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.347+0000 7f0397400700 1 -- 172.21.15.6:0/2618844534 learned_addr learned my addr 172.21.15.6:0/2618844534 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:48:47.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.347+0000 7f0396a00700 1 --2- 172.21.15.6:0/2618844534 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0398106580 0x7f039819d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:47.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.347+0000 7f038e000700 1 --2- 172.21.15.6:0/2618844534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0398106ed0 0x7f039819d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:47.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.347+0000 7f0397400700 1 -- 172.21.15.6:0/2618844534 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0398106580 msgr2=0x7f039819d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:47.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.347+0000 7f0397400700 1 --2- 172.21.15.6:0/2618844534 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0398106580 0x7f039819d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:47.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.347+0000 7f0397400700 1 -- 172.21.15.6:0/2618844534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0398106ed0 msgr2=0x7f039819d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:47.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.347+0000 7f0397400700 1 --2- 172.21.15.6:0/2618844534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0398106ed0 0x7f039819d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:47.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.348+0000 7f0397400700 1 -- 172.21.15.6:0/2618844534 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03981a1ee0 con 0x7f039810c5c0 2024-05-09T04:48:47.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.348+0000 7f0396a00700 1 --2- 172.21.15.6:0/2618844534 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0398106580 0x7f039819d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:48:47.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.348+0000 7f0397400700 1 --2- 172.21.15.6:0/2618844534 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f039810c5c0 0x7f03981a1840 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f039000b070 tx=0x7f0390009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:47.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.348+0000 7f038fe00700 1 -- 172.21.15.6:0/2618844534 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0390021070 con 0x7f039810c5c0 2024-05-09T04:48:47.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.348+0000 7f039da00700 1 -- 172.21.15.6:0/2618844534 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03981a2170 con 0x7f039810c5c0 2024-05-09T04:48:47.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.348+0000 7f038fe00700 1 -- 172.21.15.6:0/2618844534 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0390004e40 con 0x7f039810c5c0 2024-05-09T04:48:47.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.348+0000 7f038fe00700 1 -- 172.21.15.6:0/2618844534 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0390004630 con 0x7f039810c5c0 2024-05-09T04:48:47.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.349+0000 7f039da00700 1 -- 172.21.15.6:0/2618844534 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03981a9af0 con 0x7f039810c5c0 2024-05-09T04:48:47.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.350+0000 7f039da00700 1 -- 172.21.15.6:0/2618844534 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f039810e980 con 0x7f039810c5c0 2024-05-09T04:48:47.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.352+0000 7f038fe00700 1 -- 172.21.15.6:0/2618844534 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f0390007980 con 0x7f039810c5c0 2024-05-09T04:48:47.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.354+0000 7f038fe00700 1 --2- 172.21.15.6:0/2618844534 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f037c0b2900 0x7f037c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:47.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.354+0000 7f038fe00700 1 -- 172.21.15.6:0/2618844534 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f039000f020 con 0x7f039810c5c0 2024-05-09T04:48:47.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.354+0000 7f0396a00700 1 --2- 172.21.15.6:0/2618844534 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f037c0b2900 0x7f037c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:47.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.355+0000 7f0396a00700 1 --2- 172.21.15.6:0/2618844534 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f037c0b2900 0x7f037c0b4e10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f03981079f0 tx=0x7f0388009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:47.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.357+0000 7f038fe00700 1 -- 172.21.15.6:0/2618844534 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f039002a020 con 0x7f039810c5c0 2024-05-09T04:48:47.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.650+0000 7f039da00700 1 -- 172.21.15.6:0/2618844534 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0398107ca0 con 0x7f039810c5c0 2024-05-09T04:48:47.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.651+0000 7f038fe00700 1 -- 172.21.15.6:0/2618844534 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3040 (secure 0 0 0) 0x7f03900ab660 con 0x7f039810c5c0 2024-05-09T04:48:47.652 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:48:47.652 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/162094599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":162094599},{"type":"v1","addr":"172.21.15.6:6835","nonce":162094599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/2977145206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2977145206},{"type":"v1","addr":"172.21.15.186:6833","nonce":2977145206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/133345948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":133345948},{"type":"v1","addr":"172.21.15.169:6833","nonce":133345948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-09T04:48:47.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.654+0000 7f039da00700 1 -- 172.21.15.6:0/2618844534 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f037c0b2900 msgr2=0x7f037c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:47.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.654+0000 7f039da00700 1 --2- 172.21.15.6:0/2618844534 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f037c0b2900 0x7f037c0b4e10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f03981079f0 tx=0x7f0388009260 comp rx=0 tx=0).stop 2024-05-09T04:48:47.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.654+0000 7f039da00700 1 -- 172.21.15.6:0/2618844534 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f039810c5c0 msgr2=0x7f03981a1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:47.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.654+0000 7f039da00700 1 --2- 172.21.15.6:0/2618844534 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f039810c5c0 0x7f03981a1840 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f039000b070 tx=0x7f0390009c90 comp rx=0 tx=0).stop 2024-05-09T04:48:47.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.654+0000 7f039da00700 1 -- 172.21.15.6:0/2618844534 shutdown_connections 2024-05-09T04:48:47.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.654+0000 7f039da00700 1 --2- 172.21.15.6:0/2618844534 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f039810c5c0 0x7f03981a1840 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:47.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.654+0000 7f039da00700 1 --2- 172.21.15.6:0/2618844534 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0398106580 0x7f039819d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:47.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.654+0000 7f039da00700 1 --2- 172.21.15.6:0/2618844534 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f037c0b2900 0x7f037c0b4e10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:47.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.654+0000 7f039da00700 1 --2- 172.21.15.6:0/2618844534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0398106ed0 0x7f039819d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:47.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.654+0000 7f039da00700 1 -- 172.21.15.6:0/2618844534 >> 172.21.15.6:0/2618844534 conn(0x7f0398102080 msgr2=0x7f0398104620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:47.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.654+0000 7f039da00700 1 -- 172.21.15.6:0/2618844534 shutdown_connections 2024-05-09T04:48:47.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:47.655+0000 7f039da00700 1 -- 172.21.15.6:0/2618844534 wait complete. 2024-05-09T04:48:47.657 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 4 2024-05-09T04:48:47.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:47 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:47.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:47 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:47.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:47 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:47.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:47 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:48:47.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:47 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:48:47.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:47 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:47.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:47 smithi006 ceph-mon[29835]: Deploying daemon mds.e on smithi169 2024-05-09T04:48:47.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:47 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/2618844534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:48:47.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:47 smithi006 ceph-mon[29835]: mds.? [v2:172.21.15.6:6836/1281521088,v1:172.21.15.6:6837/1281521088] up:boot 2024-05-09T04:48:47.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:47 smithi006 ceph-mon[29835]: fsmap 4 up:standby 2024-05-09T04:48:47.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:47 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T04:48:48.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:47 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:48.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:47 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:48.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:47 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:48.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:47 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:48:48.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:47 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:48:48.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:47 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:48.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:47 smithi186 ceph-mon[34668]: Deploying daemon mds.e on smithi169 2024-05-09T04:48:48.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:47 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/2618844534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:48:48.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:47 smithi186 ceph-mon[34668]: mds.? [v2:172.21.15.6:6836/1281521088,v1:172.21.15.6:6837/1281521088] up:boot 2024-05-09T04:48:48.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:47 smithi186 ceph-mon[34668]: fsmap 4 up:standby 2024-05-09T04:48:48.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:47 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T04:48:48.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:47 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:48.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:47 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:48.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:47 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:48.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:47 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:48:48.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:47 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:48:48.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:47 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:48.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:47 smithi169 ceph-mon[34075]: Deploying daemon mds.e on smithi169 2024-05-09T04:48:48.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:47 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/2618844534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:48:48.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:47 smithi169 ceph-mon[34075]: mds.? [v2:172.21.15.6:6836/1281521088,v1:172.21.15.6:6837/1281521088] up:boot 2024-05-09T04:48:48.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:47 smithi169 ceph-mon[34075]: fsmap 4 up:standby 2024-05-09T04:48:48.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:47 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T04:48:49.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:48 smithi186 ceph-mon[34668]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:49.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:48 smithi169 ceph-mon[34075]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:49.117 INFO:journalctl@ceph.mds.e.smithi169.stdout:May 09 04:48:48 smithi169 systemd[1]: Starting Ceph mds.e for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T04:48:49.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:48 smithi006 ceph-mon[29835]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:49.424 INFO:journalctl@ceph.mds.e.smithi169.stdout:May 09 04:48:49 smithi169 podman[64292]: 2024-05-09 04:48:49.031298399 +0000 UTC m=+0.018366374 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T04:48:49.425 INFO:journalctl@ceph.mds.e.smithi169.stdout:May 09 04:48:49 smithi169 podman[64292]: 2024-05-09 04:48:49.130405356 +0000 UTC m=+0.117473320 container create 67aa99d0163a6f1b85c5651a87547231a4461e9b79c5103d8f368068c9b16510 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-e, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-09T04:48:49.425 INFO:journalctl@ceph.mds.e.smithi169.stdout:May 09 04:48:49 smithi169 podman[64292]: 2024-05-09 04:48:49.425068705 +0000 UTC m=+0.412136670 container init 67aa99d0163a6f1b85c5651a87547231a4461e9b79c5103d8f368068c9b16510 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-e, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-09T04:48:49.703 INFO:journalctl@ceph.mds.e.smithi169.stdout:May 09 04:48:49 smithi169 podman[64292]: 2024-05-09 04:48:49.427881031 +0000 UTC m=+0.414949003 container start 67aa99d0163a6f1b85c5651a87547231a4461e9b79c5103d8f368068c9b16510 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-09T04:48:49.703 INFO:journalctl@ceph.mds.e.smithi169.stdout:May 09 04:48:49 smithi169 ceph-mds[64312]: -- 172.21.15.169:0/914090646 <== mon.2 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d8081f5a40 con 0x55d80823da80 2024-05-09T04:48:49.704 INFO:journalctl@ceph.mds.e.smithi169.stdout:May 09 04:48:49 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-e[64307]: starting mds.e at 2024-05-09T04:48:49.704 INFO:journalctl@ceph.mds.e.smithi169.stdout:May 09 04:48:49 smithi169 bash[64292]: 67aa99d0163a6f1b85c5651a87547231a4461e9b79c5103d8f368068c9b16510 2024-05-09T04:48:49.704 INFO:journalctl@ceph.mds.e.smithi169.stdout:May 09 04:48:49 smithi169 systemd[1]: Started Ceph mds.e for f74d813c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:48:50.579 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:48:50.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:50 smithi186 ceph-mon[34668]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:50.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:50 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:50.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:50 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:50.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:50 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:50.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:50 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:48:50.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:50 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:48:50.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:50 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:50.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:50 smithi186 ceph-mon[34668]: Deploying daemon mds.f on smithi186 2024-05-09T04:48:50.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:50 smithi186 ceph-mon[34668]: mds.? [v2:172.21.15.169:6834/3811005052,v1:172.21.15.169:6835/3811005052] up:boot 2024-05-09T04:48:50.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:50 smithi186 ceph-mon[34668]: fsmap 5 up:standby 2024-05-09T04:48:50.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:50 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T04:48:50.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:50 smithi169 ceph-mon[34075]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:50.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:50 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:50.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:50 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:50.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:50 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:50.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:50 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:48:50.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:50 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:48:50.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:50 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:50.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:50 smithi169 ceph-mon[34075]: Deploying daemon mds.f on smithi186 2024-05-09T04:48:50.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:50 smithi169 ceph-mon[34075]: mds.? [v2:172.21.15.169:6834/3811005052,v1:172.21.15.169:6835/3811005052] up:boot 2024-05-09T04:48:50.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:50 smithi169 ceph-mon[34075]: fsmap 5 up:standby 2024-05-09T04:48:50.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:50 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T04:48:50.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:50 smithi006 ceph-mon[29835]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:50.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:50 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:50.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:50 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:50.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:50 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:50.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:50 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:48:50.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:50 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:48:50.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:50 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:50.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:50 smithi006 ceph-mon[29835]: Deploying daemon mds.f on smithi186 2024-05-09T04:48:50.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:50 smithi006 ceph-mon[29835]: mds.? [v2:172.21.15.169:6834/3811005052,v1:172.21.15.169:6835/3811005052] up:boot 2024-05-09T04:48:50.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:50 smithi006 ceph-mon[29835]: fsmap 5 up:standby 2024-05-09T04:48:50.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:50 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T04:48:51.034 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:52.099 INFO:journalctl@ceph.mds.f.smithi186.stdout:May 09 04:48:51 smithi186 systemd[1]: Starting Ceph mds.f for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T04:48:52.453 INFO:journalctl@ceph.mds.f.smithi186.stdout:May 09 04:48:52 smithi186 podman[65235]: 2024-05-09 04:48:52.067120757 +0000 UTC m=+0.018483607 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T04:48:52.453 INFO:journalctl@ceph.mds.f.smithi186.stdout:May 09 04:48:52 smithi186 podman[65235]: 2024-05-09 04:48:52.208989696 +0000 UTC m=+0.160352546 container create 6d28b7e38c06ba1773736d8ee72bd76b1139cf9b7ed2f3372a4455edaf67961a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-f, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T04:48:52.453 INFO:journalctl@ceph.mds.f.smithi186.stdout:May 09 04:48:52 smithi186 podman[65235]: 2024-05-09 04:48:52.453600122 +0000 UTC m=+0.404962972 container init 6d28b7e38c06ba1773736d8ee72bd76b1139cf9b7ed2f3372a4455edaf67961a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-09T04:48:52.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:52 smithi186 ceph-mon[34668]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:52.849 INFO:journalctl@ceph.mds.f.smithi186.stdout:May 09 04:48:52 smithi186 podman[65235]: 2024-05-09 04:48:52.456215449 +0000 UTC m=+0.407578302 container start 6d28b7e38c06ba1773736d8ee72bd76b1139cf9b7ed2f3372a4455edaf67961a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True) 2024-05-09T04:48:52.849 INFO:journalctl@ceph.mds.f.smithi186.stdout:May 09 04:48:52 smithi186 ceph-mds[65254]: -- 172.21.15.186:0/337519852 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5645f38fda40 con 0x5645f3945600 2024-05-09T04:48:52.850 INFO:journalctl@ceph.mds.f.smithi186.stdout:May 09 04:48:52 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-f[65250]: starting mds.f at 2024-05-09T04:48:52.850 INFO:journalctl@ceph.mds.f.smithi186.stdout:May 09 04:48:52 smithi186 bash[65235]: 6d28b7e38c06ba1773736d8ee72bd76b1139cf9b7ed2f3372a4455edaf67961a 2024-05-09T04:48:52.850 INFO:journalctl@ceph.mds.f.smithi186.stdout:May 09 04:48:52 smithi186 systemd[1]: Started Ceph mds.f for f74d813c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:48:52.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:52 smithi169 ceph-mon[34075]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:52.904 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:52 smithi006 ceph-mon[29835]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:53.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.515+0000 7faeb4e00700 1 -- 172.21.15.6:0/3739335123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faeb010c5c0 msgr2=0x7faeb010c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:53.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.515+0000 7faeb4e00700 1 --2- 172.21.15.6:0/3739335123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faeb010c5c0 0x7faeb010c9d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7faea0009990 tx=0x7faea0009ca0 comp rx=0 tx=0).stop 2024-05-09T04:48:53.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.515+0000 7faeb4e00700 1 -- 172.21.15.6:0/3739335123 shutdown_connections 2024-05-09T04:48:53.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.515+0000 7faeb4e00700 1 --2- 172.21.15.6:0/3739335123 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faeb0106ed0 0x7faeb0113360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:53.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.515+0000 7faeb4e00700 1 --2- 172.21.15.6:0/3739335123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faeb010c5c0 0x7faeb010c9d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:53.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.515+0000 7faeb4e00700 1 --2- 172.21.15.6:0/3739335123 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faeb0106580 0x7faeb0106990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:53.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.515+0000 7faeb4e00700 1 -- 172.21.15.6:0/3739335123 >> 172.21.15.6:0/3739335123 conn(0x7faeb0102080 msgr2=0x7faeb01044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:53.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.516+0000 7faeb4e00700 1 -- 172.21.15.6:0/3739335123 shutdown_connections 2024-05-09T04:48:53.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.516+0000 7faeb4e00700 1 -- 172.21.15.6:0/3739335123 wait complete. 2024-05-09T04:48:53.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.517+0000 7faeb4e00700 1 Processor -- start 2024-05-09T04:48:53.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.517+0000 7faeb4e00700 1 -- start start 2024-05-09T04:48:53.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.518+0000 7faeb4e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faeb0106580 0x7faeb019d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:53.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.518+0000 7faeb4e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faeb0106ed0 0x7faeb019d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:53.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.518+0000 7faeb4e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faeb010c5c0 0x7faeb01a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:53.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.518+0000 7faeb4e00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7faeb01166d0 con 0x7faeb010c5c0 2024-05-09T04:48:53.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.518+0000 7faeb4e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7faeb0116550 con 0x7faeb0106ed0 2024-05-09T04:48:53.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.518+0000 7faeb4e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7faeb0116850 con 0x7faeb0106580 2024-05-09T04:48:53.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.518+0000 7faeae800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faeb010c5c0 0x7faeb01a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:53.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.518+0000 7faeae800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faeb010c5c0 0x7faeb01a1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37558/0 (socket says 172.21.15.6:37558) 2024-05-09T04:48:53.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.518+0000 7faeade00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faeb0106580 0x7faeb019d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:53.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.518+0000 7faeae800700 1 -- 172.21.15.6:0/258518265 learned_addr learned my addr 172.21.15.6:0/258518265 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:48:53.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.518+0000 7faead400700 1 --2- 172.21.15.6:0/258518265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faeb0106ed0 0x7faeb019d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:53.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.518+0000 7faeade00700 1 -- 172.21.15.6:0/258518265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faeb0106ed0 msgr2=0x7faeb019d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:53.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.518+0000 7faeade00700 1 --2- 172.21.15.6:0/258518265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faeb0106ed0 0x7faeb019d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:53.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.518+0000 7faeade00700 1 -- 172.21.15.6:0/258518265 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faeb010c5c0 msgr2=0x7faeb01a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:53.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.519+0000 7faeade00700 1 --2- 172.21.15.6:0/258518265 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faeb010c5c0 0x7faeb01a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:53.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.519+0000 7faeade00700 1 -- 172.21.15.6:0/258518265 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb01a1f10 con 0x7faeb0106580 2024-05-09T04:48:53.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.519+0000 7faeade00700 1 --2- 172.21.15.6:0/258518265 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faeb0106580 0x7faeb019d040 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faea00098e0 tx=0x7faea000bfc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:53.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.519+0000 7faea6a00700 1 -- 172.21.15.6:0/258518265 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0027070 con 0x7faeb0106580 2024-05-09T04:48:53.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.519+0000 7faea6a00700 1 -- 172.21.15.6:0/258518265 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faea0022470 con 0x7faeb0106580 2024-05-09T04:48:53.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.519+0000 7faeb4e00700 1 -- 172.21.15.6:0/258518265 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeb01a21a0 con 0x7faeb0106580 2024-05-09T04:48:53.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.519+0000 7faea6a00700 1 -- 172.21.15.6:0/258518265 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0017650 con 0x7faeb0106580 2024-05-09T04:48:53.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.519+0000 7faeb4e00700 1 -- 172.21.15.6:0/258518265 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeb01a26a0 con 0x7faeb0106580 2024-05-09T04:48:53.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.521+0000 7faeb4e00700 1 -- 172.21.15.6:0/258518265 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeb00512c0 con 0x7faeb0106580 2024-05-09T04:48:53.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.523+0000 7faea6a00700 1 -- 172.21.15.6:0/258518265 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7faea002a070 con 0x7faeb0106580 2024-05-09T04:48:53.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.525+0000 7faea6a00700 1 --2- 172.21.15.6:0/258518265 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fae940b2900 0x7fae940b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:53.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.525+0000 7faea6a00700 1 -- 172.21.15.6:0/258518265 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7faea00e2af0 con 0x7faeb0106580 2024-05-09T04:48:53.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.525+0000 7faead400700 1 --2- 172.21.15.6:0/258518265 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fae940b2900 0x7fae940b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:53.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.526+0000 7faead400700 1 --2- 172.21.15.6:0/258518265 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fae940b2900 0x7fae940b4e10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7faeb0103c00 tx=0x7fae9c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:53.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.529+0000 7faea6a00700 1 -- 172.21.15.6:0/258518265 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faeb01a26a0 con 0x7faeb0106580 2024-05-09T04:48:53.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.824+0000 7faeb4e00700 1 -- 172.21.15.6:0/258518265 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faeb002e5f0 con 0x7faeb0106580 2024-05-09T04:48:53.825 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:53 smithi006 ceph-mon[29835]: mds.? [v2:172.21.15.186:6834/2268322007,v1:172.21.15.186:6835/2268322007] up:boot 2024-05-09T04:48:53.826 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:53 smithi006 ceph-mon[29835]: fsmap 6 up:standby 2024-05-09T04:48:53.826 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:53 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T04:48:53.826 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:53 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:53.826 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:53 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:53.826 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:53 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:53.826 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:53 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:48:53.826 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:53 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:48:53.826 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:53 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:53.826 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:53 smithi006 ceph-mon[29835]: Deploying daemon mds.c on smithi186 2024-05-09T04:48:53.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.825+0000 7faea6a00700 1 -- 172.21.15.6:0/258518265 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5518 (secure 0 0 0) 0x7faea0013070 con 0x7faeb0106580 2024-05-09T04:48:53.830 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:48:53.830 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/162094599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":162094599},{"type":"v1","addr":"172.21.15.6:6835","nonce":162094599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/2977145206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2977145206},{"type":"v1","addr":"172.21.15.186:6833","nonce":2977145206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/133345948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":133345948},{"type":"v1","addr":"172.21.15.169:6833","nonce":133345948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/1281521088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":1281521088},{"type":"v1","addr":"172.21.15.6:6837","nonce":1281521088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2268322007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2268322007},{"type":"v1","addr":"172.21.15.186:6835","nonce":2268322007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3811005052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3811005052},{"type":"v1","addr":"172.21.15.169:6835","nonce":3811005052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-09T04:48:53.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.828+0000 7faeb4e00700 1 -- 172.21.15.6:0/258518265 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fae940b2900 msgr2=0x7fae940b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:53.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.828+0000 7faeb4e00700 1 --2- 172.21.15.6:0/258518265 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fae940b2900 0x7fae940b4e10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7faeb0103c00 tx=0x7fae9c0073d0 comp rx=0 tx=0).stop 2024-05-09T04:48:53.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.828+0000 7faeb4e00700 1 -- 172.21.15.6:0/258518265 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faeb0106580 msgr2=0x7faeb019d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:53.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.828+0000 7faeb4e00700 1 --2- 172.21.15.6:0/258518265 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faeb0106580 0x7faeb019d040 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faea00098e0 tx=0x7faea000bfc0 comp rx=0 tx=0).stop 2024-05-09T04:48:53.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.828+0000 7faeb4e00700 1 -- 172.21.15.6:0/258518265 shutdown_connections 2024-05-09T04:48:53.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.828+0000 7faeb4e00700 1 --2- 172.21.15.6:0/258518265 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faeb010c5c0 0x7faeb01a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:53.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.828+0000 7faeb4e00700 1 --2- 172.21.15.6:0/258518265 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faeb0106580 0x7faeb019d040 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:53.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.828+0000 7faeb4e00700 1 --2- 172.21.15.6:0/258518265 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fae940b2900 0x7fae940b4e10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:53.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.828+0000 7faeb4e00700 1 --2- 172.21.15.6:0/258518265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faeb0106ed0 0x7faeb019d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:53.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.828+0000 7faeb4e00700 1 -- 172.21.15.6:0/258518265 >> 172.21.15.6:0/258518265 conn(0x7faeb0102080 msgr2=0x7faeb0104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:53.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.829+0000 7faeb4e00700 1 -- 172.21.15.6:0/258518265 shutdown_connections 2024-05-09T04:48:53.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:53.829+0000 7faeb4e00700 1 -- 172.21.15.6:0/258518265 wait complete. 2024-05-09T04:48:53.832 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 7 2024-05-09T04:48:53.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:53 smithi186 ceph-mon[34668]: mds.? [v2:172.21.15.186:6834/2268322007,v1:172.21.15.186:6835/2268322007] up:boot 2024-05-09T04:48:53.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:53 smithi186 ceph-mon[34668]: fsmap 6 up:standby 2024-05-09T04:48:53.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:53 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T04:48:53.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:53 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:53.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:53 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:53.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:53 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:53.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:53 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:48:53.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:53 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:48:53.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:53 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:53.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:53 smithi186 ceph-mon[34668]: Deploying daemon mds.c on smithi186 2024-05-09T04:48:53.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:53 smithi169 ceph-mon[34075]: mds.? [v2:172.21.15.186:6834/2268322007,v1:172.21.15.186:6835/2268322007] up:boot 2024-05-09T04:48:53.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:53 smithi169 ceph-mon[34075]: fsmap 6 up:standby 2024-05-09T04:48:53.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:53 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T04:48:53.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:53 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:53.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:53 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:53.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:53 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:53.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:53 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:48:53.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:53 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:48:53.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:53 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:53.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:53 smithi169 ceph-mon[34075]: Deploying daemon mds.c on smithi186 2024-05-09T04:48:54.737 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:54 smithi186 ceph-mon[34668]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:54.737 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:54 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/258518265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:48:54.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:54 smithi006 ceph-mon[29835]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:54.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:54 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/258518265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:48:55.100 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 09 04:48:54 smithi186 systemd[1]: Starting Ceph mds.c for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T04:48:55.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:54 smithi169 ceph-mon[34075]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:55.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:54 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/258518265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:48:55.434 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 09 04:48:55 smithi186 podman[65543]: 2024-05-09 04:48:55.034680569 +0000 UTC m=+0.018319804 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T04:48:55.434 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 09 04:48:55 smithi186 podman[65543]: 2024-05-09 04:48:55.182012098 +0000 UTC m=+0.165651347 container create c5741dc9744be66f4b72d6096f10095bb938a33e5890f76b3dfc8f2e1d186052 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-c, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-09T04:48:55.434 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 09 04:48:55 smithi186 podman[65543]: 2024-05-09 04:48:55.434959005 +0000 UTC m=+0.418598216 container init c5741dc9744be66f4b72d6096f10095bb938a33e5890f76b3dfc8f2e1d186052 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux ) 2024-05-09T04:48:55.849 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 09 04:48:55 smithi186 podman[65543]: 2024-05-09 04:48:55.437609981 +0000 UTC m=+0.421249189 container start c5741dc9744be66f4b72d6096f10095bb938a33e5890f76b3dfc8f2e1d186052 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-c, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-05-09T04:48:55.850 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 09 04:48:55 smithi186 ceph-mds[65562]: -- 172.21.15.186:0/160024882 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563976e81a40 con 0x563976ec9180 2024-05-09T04:48:55.850 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 09 04:48:55 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-c[65558]: starting mds.c at 2024-05-09T04:48:55.850 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 09 04:48:55 smithi186 bash[65543]: c5741dc9744be66f4b72d6096f10095bb938a33e5890f76b3dfc8f2e1d186052 2024-05-09T04:48:55.850 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 09 04:48:55 smithi186 systemd[1]: Started Ceph mds.c for f74d813c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:48:56.728 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:48:56.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:56 smithi186 ceph-mon[34668]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:56.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:56.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:56.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:56.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:48:56.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:48:56.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:56.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:56 smithi186 ceph-mon[34668]: Deploying daemon mds.h on smithi169 2024-05-09T04:48:56.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:56 smithi186 ceph-mon[34668]: mds.? [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] up:boot 2024-05-09T04:48:56.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:56 smithi186 ceph-mon[34668]: fsmap 7 up:standby 2024-05-09T04:48:56.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T04:48:56.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:56 smithi169 ceph-mon[34075]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:56.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:56 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:56.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:56 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:56.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:56 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:56.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:56 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:48:56.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:56 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:48:56.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:56 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:56.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:56 smithi169 ceph-mon[34075]: Deploying daemon mds.h on smithi169 2024-05-09T04:48:56.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:56 smithi169 ceph-mon[34075]: mds.? [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] up:boot 2024-05-09T04:48:56.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:56 smithi169 ceph-mon[34075]: fsmap 7 up:standby 2024-05-09T04:48:56.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:56 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T04:48:56.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:56 smithi006 ceph-mon[29835]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:56.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:56.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:56.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:56.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:48:56.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:48:56.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:56.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:56 smithi006 ceph-mon[29835]: Deploying daemon mds.h on smithi169 2024-05-09T04:48:56.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:56 smithi006 ceph-mon[29835]: mds.? [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] up:boot 2024-05-09T04:48:56.949 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:56 smithi006 ceph-mon[29835]: fsmap 7 up:standby 2024-05-09T04:48:56.949 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T04:48:57.203 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:57.867 INFO:journalctl@ceph.mds.h.smithi169.stdout:May 09 04:48:57 smithi169 systemd[1]: Starting Ceph mds.h for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T04:48:58.305 INFO:journalctl@ceph.mds.h.smithi169.stdout:May 09 04:48:58 smithi169 podman[64609]: 2024-05-09 04:48:57.918330666 +0000 UTC m=+0.018281913 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T04:48:58.305 INFO:journalctl@ceph.mds.h.smithi169.stdout:May 09 04:48:58 smithi169 podman[64609]: 2024-05-09 04:48:58.044725328 +0000 UTC m=+0.144676564 container create 03ff7e9c27ce244b5fee5d19ff1cd4cf60d110016c546945b8cccfe50fac0a7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-h, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-09T04:48:58.305 INFO:journalctl@ceph.mds.h.smithi169.stdout:May 09 04:48:58 smithi169 podman[64609]: 2024-05-09 04:48:58.305988392 +0000 UTC m=+0.405939628 container init 03ff7e9c27ce244b5fee5d19ff1cd4cf60d110016c546945b8cccfe50fac0a7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-h, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-09T04:48:58.590 INFO:journalctl@ceph.mds.h.smithi169.stdout:May 09 04:48:58 smithi169 podman[64609]: 2024-05-09 04:48:58.308723693 +0000 UTC m=+0.408674936 container start 03ff7e9c27ce244b5fee5d19ff1cd4cf60d110016c546945b8cccfe50fac0a7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-h, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-09T04:48:58.590 INFO:journalctl@ceph.mds.h.smithi169.stdout:May 09 04:48:58 smithi169 ceph-mds[64628]: -- 172.21.15.169:0/860448655 <== mon.2 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5625bbb69a40 con 0x5625bbbb1180 2024-05-09T04:48:58.590 INFO:journalctl@ceph.mds.h.smithi169.stdout:May 09 04:48:58 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-h[64624]: starting mds.h at 2024-05-09T04:48:58.590 INFO:journalctl@ceph.mds.h.smithi169.stdout:May 09 04:48:58 smithi169 bash[64609]: 03ff7e9c27ce244b5fee5d19ff1cd4cf60d110016c546945b8cccfe50fac0a7c 2024-05-09T04:48:58.590 INFO:journalctl@ceph.mds.h.smithi169.stdout:May 09 04:48:58 smithi169 systemd[1]: Started Ceph mds.h for f74d813c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:48:58.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:58 smithi186 ceph-mon[34668]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:58.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:58 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:58.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:58 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:58.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:58 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:58.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:58 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:48:58.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:58 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:48:58.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:58 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:58.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:58 smithi169 ceph-mon[34075]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:58.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:58 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:58.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:58 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:58.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:58 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:58.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:58 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:48:58.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:58 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:48:58.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:58 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:58.891 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:58 smithi006 ceph-mon[29835]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:48:58.891 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:58 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:58.891 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:58 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:58.891 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:58 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:48:58.891 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:58 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:48:58.891 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:58 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:48:58.891 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:58 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:59.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.479+0000 7f68a1000700 1 -- 172.21.15.6:0/637256231 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f689c10b870 msgr2=0x7f689c1131d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:59.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.479+0000 7f68a1000700 1 --2- 172.21.15.6:0/637256231 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f689c10b870 0x7f689c1131d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f689400b0d0 tx=0x7f689400b3e0 comp rx=0 tx=0).stop 2024-05-09T04:48:59.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.479+0000 7f68a1000700 1 -- 172.21.15.6:0/637256231 shutdown_connections 2024-05-09T04:48:59.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.479+0000 7f68a1000700 1 --2- 172.21.15.6:0/637256231 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f689c10b870 0x7f689c1131d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:59.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.479+0000 7f68a1000700 1 --2- 172.21.15.6:0/637256231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f689c106640 0x7f689c106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:59.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.479+0000 7f68a1000700 1 --2- 172.21.15.6:0/637256231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f689c10af20 0x7f689c10b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:59.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.479+0000 7f68a1000700 1 -- 172.21.15.6:0/637256231 >> 172.21.15.6:0/637256231 conn(0x7f689c102030 msgr2=0x7f689c104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:59.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.479+0000 7f68a1000700 1 -- 172.21.15.6:0/637256231 shutdown_connections 2024-05-09T04:48:59.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.479+0000 7f68a1000700 1 -- 172.21.15.6:0/637256231 wait complete. 2024-05-09T04:48:59.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.480+0000 7f68a1000700 1 Processor -- start 2024-05-09T04:48:59.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.480+0000 7f68a1000700 1 -- start start 2024-05-09T04:48:59.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.481+0000 7f68a1000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f689c106640 0x7f689c19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:59.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.481+0000 7f68a1000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f689c10af20 0x7f689c19d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:59.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.481+0000 7f68a1000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f689c10b870 0x7f689c1a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:59.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.481+0000 7f68a1000700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f689c1164f0 con 0x7f689c10b870 2024-05-09T04:48:59.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.481+0000 7f68a1000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f689c116370 con 0x7f689c106640 2024-05-09T04:48:59.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.481+0000 7f68a1000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f689c116670 con 0x7f689c10af20 2024-05-09T04:48:59.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.481+0000 7f689be00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f689c10b870 0x7f689c1a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:59.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.481+0000 7f689aa00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f689c10af20 0x7f689c19d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:59.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.481+0000 7f689b400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f689c106640 0x7f689c19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:59.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.481+0000 7f689be00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f689c10b870 0x7f689c1a1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39648/0 (socket says 172.21.15.6:39648) 2024-05-09T04:48:59.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.481+0000 7f689be00700 1 -- 172.21.15.6:0/3539110353 learned_addr learned my addr 172.21.15.6:0/3539110353 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:48:59.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.482+0000 7f689b400700 1 -- 172.21.15.6:0/3539110353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f689c10af20 msgr2=0x7f689c19d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:59.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.482+0000 7f689b400700 1 --2- 172.21.15.6:0/3539110353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f689c10af20 0x7f689c19d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:59.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.482+0000 7f689b400700 1 -- 172.21.15.6:0/3539110353 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f689c10b870 msgr2=0x7f689c1a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:59.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.482+0000 7f689b400700 1 --2- 172.21.15.6:0/3539110353 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f689c10b870 0x7f689c1a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:59.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.482+0000 7f689b400700 1 -- 172.21.15.6:0/3539110353 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689c1a1cd0 con 0x7f689c106640 2024-05-09T04:48:59.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.482+0000 7f689aa00700 1 --2- 172.21.15.6:0/3539110353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f689c10af20 0x7f689c19d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:48:59.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.482+0000 7f689be00700 1 --2- 172.21.15.6:0/3539110353 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f689c10b870 0x7f689c1a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:48:59.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.482+0000 7f689b400700 1 --2- 172.21.15.6:0/3539110353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f689c106640 0x7f689c19ce00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f688c00e9f0 tx=0x7f688c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:59.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.482+0000 7f6893e00700 1 -- 172.21.15.6:0/3539110353 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c00c9c0 con 0x7f689c106640 2024-05-09T04:48:59.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.482+0000 7f6893e00700 1 -- 172.21.15.6:0/3539110353 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f688c00cb60 con 0x7f689c106640 2024-05-09T04:48:59.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.482+0000 7f68a1000700 1 -- 172.21.15.6:0/3539110353 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f689c1a1fc0 con 0x7f689c106640 2024-05-09T04:48:59.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.482+0000 7f68a1000700 1 -- 172.21.15.6:0/3539110353 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f689c1a98e0 con 0x7f689c106640 2024-05-09T04:48:59.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.483+0000 7f6893e00700 1 -- 172.21.15.6:0/3539110353 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c018680 con 0x7f689c106640 2024-05-09T04:48:59.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.484+0000 7f68a1000700 1 -- 172.21.15.6:0/3539110353 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f687c005330 con 0x7f689c106640 2024-05-09T04:48:59.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.486+0000 7f6893e00700 1 -- 172.21.15.6:0/3539110353 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f688c018820 con 0x7f689c106640 2024-05-09T04:48:59.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.488+0000 7f6893e00700 1 --2- 172.21.15.6:0/3539110353 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f68800b2880 0x7f68800b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:59.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.488+0000 7f6893e00700 1 -- 172.21.15.6:0/3539110353 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f688c0e1ee0 con 0x7f689c106640 2024-05-09T04:48:59.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.488+0000 7f689aa00700 1 --2- 172.21.15.6:0/3539110353 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f68800b2880 0x7f68800b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:59.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.489+0000 7f689aa00700 1 --2- 172.21.15.6:0/3539110353 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f68800b2880 0x7f68800b4d90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f689c105a10 tx=0x7f6888009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:59.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.492+0000 7f6893e00700 1 -- 172.21.15.6:0/3539110353 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f688c0aa000 con 0x7f689c106640 2024-05-09T04:48:59.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.787+0000 7f68a1000700 1 -- 172.21.15.6:0/3539110353 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f687c005620 con 0x7f689c106640 2024-05-09T04:48:59.789 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:59 smithi006 ceph-mon[29835]: Deploying daemon mds.i on smithi186 2024-05-09T04:48:59.789 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:59 smithi006 ceph-mon[29835]: mds.? [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] up:boot 2024-05-09T04:48:59.789 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:59 smithi006 ceph-mon[29835]: fsmap 8 up:standby 2024-05-09T04:48:59.789 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:48:59 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T04:48:59.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.789+0000 7f6893e00700 1 -- 172.21.15.6:0/3539110353 <== mon.2 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7174 (secure 0 0 0) 0x7f688c0a9900 con 0x7f689c106640 2024-05-09T04:48:59.790 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:48:59.790 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/162094599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":162094599},{"type":"v1","addr":"172.21.15.6:6835","nonce":162094599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/2977145206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2977145206},{"type":"v1","addr":"172.21.15.186:6833","nonce":2977145206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/133345948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":133345948},{"type":"v1","addr":"172.21.15.169:6833","nonce":133345948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/1281521088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":1281521088},{"type":"v1","addr":"172.21.15.6:6837","nonce":1281521088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2268322007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2268322007},{"type":"v1","addr":"172.21.15.186:6835","nonce":2268322007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3811005052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3811005052},{"type":"v1","addr":"172.21.15.169:6835","nonce":3811005052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1647607118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1647607118},{"type":"v1","addr":"172.21.15.186:6837","nonce":1647607118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24512,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1845715294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1845715294},{"type":"v1","addr":"172.21.15.169:6837","nonce":1845715294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-09T04:48:59.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.793+0000 7f68a1000700 1 -- 172.21.15.6:0/3539110353 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f68800b2880 msgr2=0x7f68800b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:59.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.793+0000 7f68a1000700 1 --2- 172.21.15.6:0/3539110353 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f68800b2880 0x7f68800b4d90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f689c105a10 tx=0x7f6888009260 comp rx=0 tx=0).stop 2024-05-09T04:48:59.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.793+0000 7f68a1000700 1 -- 172.21.15.6:0/3539110353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f689c106640 msgr2=0x7f689c19ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:59.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.793+0000 7f68a1000700 1 --2- 172.21.15.6:0/3539110353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f689c106640 0x7f689c19ce00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f688c00e9f0 tx=0x7f688c00ed00 comp rx=0 tx=0).stop 2024-05-09T04:48:59.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.793+0000 7f68a1000700 1 -- 172.21.15.6:0/3539110353 shutdown_connections 2024-05-09T04:48:59.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.793+0000 7f68a1000700 1 --2- 172.21.15.6:0/3539110353 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f689c10b870 0x7f689c1a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:59.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.793+0000 7f68a1000700 1 --2- 172.21.15.6:0/3539110353 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f68800b2880 0x7f68800b4d90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:59.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.793+0000 7f68a1000700 1 --2- 172.21.15.6:0/3539110353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f689c106640 0x7f689c19ce00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:59.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.793+0000 7f68a1000700 1 --2- 172.21.15.6:0/3539110353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f689c10af20 0x7f689c19d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:59.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.793+0000 7f68a1000700 1 -- 172.21.15.6:0/3539110353 >> 172.21.15.6:0/3539110353 conn(0x7f689c102030 msgr2=0x7f689c10a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:59.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.794+0000 7f68a1000700 1 -- 172.21.15.6:0/3539110353 shutdown_connections 2024-05-09T04:48:59.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:48:59.794+0000 7f68a1000700 1 -- 172.21.15.6:0/3539110353 wait complete. 2024-05-09T04:48:59.796 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 9 2024-05-09T04:48:59.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:59 smithi186 ceph-mon[34668]: Deploying daemon mds.i on smithi186 2024-05-09T04:48:59.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:59 smithi186 ceph-mon[34668]: mds.? [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] up:boot 2024-05-09T04:48:59.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:59 smithi186 ceph-mon[34668]: fsmap 8 up:standby 2024-05-09T04:48:59.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:48:59 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T04:48:59.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:59 smithi169 ceph-mon[34075]: Deploying daemon mds.i on smithi186 2024-05-09T04:48:59.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:59 smithi169 ceph-mon[34075]: mds.? [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] up:boot 2024-05-09T04:48:59.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:59 smithi169 ceph-mon[34075]: fsmap 8 up:standby 2024-05-09T04:48:59.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:48:59 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T04:49:00.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:00 smithi186 ceph-mon[34668]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:00.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:00 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/3539110353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:49:00.850 INFO:journalctl@ceph.mds.i.smithi186.stdout:May 09 04:49:00 smithi186 systemd[1]: Starting Ceph mds.i for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T04:49:00.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:00 smithi169 ceph-mon[34075]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:00.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:00 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/3539110353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:49:00.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:00 smithi006 ceph-mon[29835]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:00.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:00 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/3539110353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:49:01.240 INFO:journalctl@ceph.mds.i.smithi186.stdout:May 09 04:49:00 smithi186 podman[65847]: 2024-05-09 04:49:00.85139043 +0000 UTC m=+0.018488308 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T04:49:01.240 INFO:journalctl@ceph.mds.i.smithi186.stdout:May 09 04:49:00 smithi186 podman[65847]: 2024-05-09 04:49:00.996128265 +0000 UTC m=+0.163226174 container create 4e2839b809cc7e0eb94f27cff72206490eb070c57e62331819c33b1594867916 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-i, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-05-09T04:49:01.240 INFO:journalctl@ceph.mds.i.smithi186.stdout:May 09 04:49:01 smithi186 podman[65847]: 2024-05-09 04:49:01.240429398 +0000 UTC m=+0.407527261 container init 4e2839b809cc7e0eb94f27cff72206490eb070c57e62331819c33b1594867916 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T04:49:01.599 INFO:journalctl@ceph.mds.i.smithi186.stdout:May 09 04:49:01 smithi186 podman[65847]: 2024-05-09 04:49:01.243330136 +0000 UTC m=+0.410428008 container start 4e2839b809cc7e0eb94f27cff72206490eb070c57e62331819c33b1594867916 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-i, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-09T04:49:01.600 INFO:journalctl@ceph.mds.i.smithi186.stdout:May 09 04:49:01 smithi186 ceph-mds[65866]: -- 172.21.15.186:0/393430845 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56000090da40 con 0x560000955600 2024-05-09T04:49:01.600 INFO:journalctl@ceph.mds.i.smithi186.stdout:May 09 04:49:01 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-i[65862]: starting mds.i at 2024-05-09T04:49:01.600 INFO:journalctl@ceph.mds.i.smithi186.stdout:May 09 04:49:01 smithi186 bash[65847]: 4e2839b809cc7e0eb94f27cff72206490eb070c57e62331819c33b1594867916 2024-05-09T04:49:01.600 INFO:journalctl@ceph.mds.i.smithi186.stdout:May 09 04:49:01 smithi186 systemd[1]: Started Ceph mds.i for f74d813c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:49:02.651 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:02 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:02.651 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:02 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:02.651 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:02 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:02.651 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:02 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:49:02.651 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:02 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:49:02.651 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:02 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:49:02.651 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:02 smithi186 ceph-mon[34668]: Deploying daemon mds.g on smithi006 2024-05-09T04:49:02.651 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:02 smithi186 ceph-mon[34668]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:02.651 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:02 smithi186 ceph-mon[34668]: mds.? [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] up:boot 2024-05-09T04:49:02.651 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:02 smithi186 ceph-mon[34668]: fsmap 9 up:standby 2024-05-09T04:49:02.651 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:02 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T04:49:02.676 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:02 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:02.676 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:02 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:02.676 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:02 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:02.676 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:02 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:49:02.676 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:02 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:49:02.677 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:02 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:49:02.677 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:02 smithi006 ceph-mon[29835]: Deploying daemon mds.g on smithi006 2024-05-09T04:49:02.677 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:02 smithi006 ceph-mon[29835]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:02.677 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:02 smithi006 ceph-mon[29835]: mds.? [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] up:boot 2024-05-09T04:49:02.677 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:02 smithi006 ceph-mon[29835]: fsmap 9 up:standby 2024-05-09T04:49:02.677 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:02 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T04:49:02.703 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:49:02.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:02 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:02.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:02 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:02.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:02 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:02.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:02 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:49:02.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:02 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:49:02.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:02 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:49:02.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:02 smithi169 ceph-mon[34075]: Deploying daemon mds.g on smithi006 2024-05-09T04:49:02.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:02 smithi169 ceph-mon[34075]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:02.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:02 smithi169 ceph-mon[34075]: mds.? [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] up:boot 2024-05-09T04:49:02.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:02 smithi169 ceph-mon[34075]: fsmap 9 up:standby 2024-05-09T04:49:02.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:02 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T04:49:04.026 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:49:04.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:04 smithi186 ceph-mon[34668]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:04.870 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:04 smithi169 ceph-mon[34075]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:04.888 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:04 smithi006 ceph-mon[29835]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:06.271 INFO:journalctl@ceph.mds.g.smithi006.stdout:May 09 04:49:06 smithi006 podman[68719]: 2024-05-09 04:49:06.009448429 +0000 UTC m=+0.144521002 container create 1e1230aa6c00ecb35b0ff366f3ff58c2eef2396131e3fb18993ed145db5d3bcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-g, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-09T04:49:06.826 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:06 smithi006 ceph-mon[29835]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:06.827 INFO:journalctl@ceph.mds.g.smithi006.stdout:May 09 04:49:06 smithi006 podman[68719]: 2024-05-09 04:49:06.54165359 +0000 UTC m=+0.676726177 container init 1e1230aa6c00ecb35b0ff366f3ff58c2eef2396131e3fb18993ed145db5d3bcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-g, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-09T04:49:06.827 INFO:journalctl@ceph.mds.g.smithi006.stdout:May 09 04:49:06 smithi006 podman[68719]: 2024-05-09 04:49:06.553631051 +0000 UTC m=+0.688703607 container start 1e1230aa6c00ecb35b0ff366f3ff58c2eef2396131e3fb18993ed145db5d3bcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-g, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-09T04:49:06.827 INFO:journalctl@ceph.mds.g.smithi006.stdout:May 09 04:49:06 smithi006 bash[68719]: 1e1230aa6c00ecb35b0ff366f3ff58c2eef2396131e3fb18993ed145db5d3bcb 2024-05-09T04:49:06.827 INFO:journalctl@ceph.mds.g.smithi006.stdout:May 09 04:49:06 smithi006 systemd[1]: Started Ceph mds.g for f74d813c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:49:06.827 INFO:journalctl@ceph.mds.g.smithi006.stdout:May 09 04:49:06 smithi006 ceph-mds[68756]: -- 172.21.15.6:0/2169367681 <== mon.2 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558fabc17a40 con 0x558fabc5f180 2024-05-09T04:49:06.827 INFO:journalctl@ceph.mds.g.smithi006.stdout:May 09 04:49:06 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-g[68748]: starting mds.g at 2024-05-09T04:49:06.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:06 smithi186 ceph-mon[34668]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:06.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:06 smithi169 ceph-mon[34075]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:07.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.079+0000 7fb3cf000700 1 -- 172.21.15.6:0/624138509 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb3c8108f20 msgr2=0x7fb3c8109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:07.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.079+0000 7fb3cf000700 1 --2- 172.21.15.6:0/624138509 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb3c8108f20 0x7fb3c8109330 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4009920 tx=0x7fb3b4009c30 comp rx=0 tx=0).stop 2024-05-09T04:49:07.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.079+0000 7fb3cf000700 1 -- 172.21.15.6:0/624138509 shutdown_connections 2024-05-09T04:49:07.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.079+0000 7fb3cf000700 1 --2- 172.21.15.6:0/624138509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3c8109870 0x7fb3c81111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:07.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.079+0000 7fb3cf000700 1 --2- 172.21.15.6:0/624138509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3c81045f0 0x7fb3c8104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:07.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.079+0000 7fb3cf000700 1 --2- 172.21.15.6:0/624138509 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb3c8108f20 0x7fb3c8109330 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:07.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.079+0000 7fb3cf000700 1 -- 172.21.15.6:0/624138509 >> 172.21.15.6:0/624138509 conn(0x7fb3c80fffe0 msgr2=0x7fb3c8102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:07.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.079+0000 7fb3cf000700 1 -- 172.21.15.6:0/624138509 shutdown_connections 2024-05-09T04:49:07.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.079+0000 7fb3cf000700 1 -- 172.21.15.6:0/624138509 wait complete. 2024-05-09T04:49:07.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.081+0000 7fb3cf000700 1 Processor -- start 2024-05-09T04:49:07.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.081+0000 7fb3cf000700 1 -- start start 2024-05-09T04:49:07.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.081+0000 7fb3cf000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb3c81045f0 0x7fb3c819d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:07.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.081+0000 7fb3cf000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3c8108f20 0x7fb3c819d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:07.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.081+0000 7fb3cf000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3c8109870 0x7fb3c81a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:07.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.081+0000 7fb3cf000700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c81144d0 con 0x7fb3c81045f0 2024-05-09T04:49:07.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.081+0000 7fb3cf000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c8114350 con 0x7fb3c8109870 2024-05-09T04:49:07.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.081+0000 7fb3cf000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c8114650 con 0x7fb3c8108f20 2024-05-09T04:49:07.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.082+0000 7fb3c7e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb3c81045f0 0x7fb3c819d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:07.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.082+0000 7fb3c7e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb3c81045f0 0x7fb3c819d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51464/0 (socket says 172.21.15.6:51464) 2024-05-09T04:49:07.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.082+0000 7fb3c7400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3c8108f20 0x7fb3c819d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:07.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.082+0000 7fb3c7e00700 1 -- 172.21.15.6:0/1451832095 learned_addr learned my addr 172.21.15.6:0/1451832095 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:49:07.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.082+0000 7fb3cd000700 1 --2- 172.21.15.6:0/1451832095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3c8109870 0x7fb3c81a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:07.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.082+0000 7fb3c7400700 1 -- 172.21.15.6:0/1451832095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3c8109870 msgr2=0x7fb3c81a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:07.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.082+0000 7fb3c7400700 1 --2- 172.21.15.6:0/1451832095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3c8109870 0x7fb3c81a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:07.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.082+0000 7fb3c7400700 1 -- 172.21.15.6:0/1451832095 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb3c81045f0 msgr2=0x7fb3c819d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:07.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.082+0000 7fb3c7400700 1 --2- 172.21.15.6:0/1451832095 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb3c81045f0 0x7fb3c819d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:07.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.082+0000 7fb3c7400700 1 -- 172.21.15.6:0/1451832095 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3c81a20b0 con 0x7fb3c8108f20 2024-05-09T04:49:07.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.083+0000 7fb3c7400700 1 --2- 172.21.15.6:0/1451832095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3c8108f20 0x7fb3c819d780 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4000c00 tx=0x7fb3b400f950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:07.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.083+0000 7fb3c4c00700 1 -- 172.21.15.6:0/1451832095 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4005040 con 0x7fb3c8108f20 2024-05-09T04:49:07.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.083+0000 7fb3c4c00700 1 -- 172.21.15.6:0/1451832095 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3b4005690 con 0x7fb3c8108f20 2024-05-09T04:49:07.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.083+0000 7fb3c4c00700 1 -- 172.21.15.6:0/1451832095 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4003970 con 0x7fb3c8108f20 2024-05-09T04:49:07.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.083+0000 7fb3cf000700 1 -- 172.21.15.6:0/1451832095 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3c81a2340 con 0x7fb3c8108f20 2024-05-09T04:49:07.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.083+0000 7fb3cf000700 1 -- 172.21.15.6:0/1451832095 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3c81a27d0 con 0x7fb3c8108f20 2024-05-09T04:49:07.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.084+0000 7fb3cf000700 1 -- 172.21.15.6:0/1451832095 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3c80512c0 con 0x7fb3c8108f20 2024-05-09T04:49:07.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.088+0000 7fb3c4c00700 1 -- 172.21.15.6:0/1451832095 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fb3b400b7f0 con 0x7fb3c8108f20 2024-05-09T04:49:07.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.089+0000 7fb3c4c00700 1 --2- 172.21.15.6:0/1451832095 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fb3ac0b2880 0x7fb3ac0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:07.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.090+0000 7fb3c4c00700 1 -- 172.21.15.6:0/1451832095 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fb3b40e2ed0 con 0x7fb3c8108f20 2024-05-09T04:49:07.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.090+0000 7fb3c7e00700 1 --2- 172.21.15.6:0/1451832095 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fb3ac0b2880 0x7fb3ac0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:07.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.090+0000 7fb3c7e00700 1 --2- 172.21.15.6:0/1451832095 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fb3ac0b2880 0x7fb3ac0b4d90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8005950 tx=0x7fb3b8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:07.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.091+0000 7fb3c4c00700 1 -- 172.21.15.6:0/1451832095 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb3b4013030 con 0x7fb3c8108f20 2024-05-09T04:49:07.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.380+0000 7fb3cf000700 1 -- 172.21.15.6:0/1451832095 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3c81a9cc0 con 0x7fb3c8108f20 2024-05-09T04:49:07.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.381+0000 7fb3c4c00700 1 -- 172.21.15.6:0/1451832095 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8004 (secure 0 0 0) 0x7fb3b40ab080 con 0x7fb3c8108f20 2024-05-09T04:49:07.382 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:49:07.382 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/162094599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":162094599},{"type":"v1","addr":"172.21.15.6:6835","nonce":162094599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/2977145206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2977145206},{"type":"v1","addr":"172.21.15.186:6833","nonce":2977145206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/133345948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":133345948},{"type":"v1","addr":"172.21.15.169:6833","nonce":133345948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/1281521088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":1281521088},{"type":"v1","addr":"172.21.15.6:6837","nonce":1281521088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2268322007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2268322007},{"type":"v1","addr":"172.21.15.186:6835","nonce":2268322007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3811005052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3811005052},{"type":"v1","addr":"172.21.15.169:6835","nonce":3811005052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1647607118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1647607118},{"type":"v1","addr":"172.21.15.186:6837","nonce":1647607118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/3954883105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":3954883105},{"type":"v1","addr":"172.21.15.186:6839","nonce":3954883105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24512,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1845715294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1845715294},{"type":"v1","addr":"172.21.15.169:6837","nonce":1845715294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-09T04:49:07.386 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.386+0000 7fb3cf000700 1 -- 172.21.15.6:0/1451832095 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fb3ac0b2880 msgr2=0x7fb3ac0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:07.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.386+0000 7fb3cf000700 1 --2- 172.21.15.6:0/1451832095 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fb3ac0b2880 0x7fb3ac0b4d90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8005950 tx=0x7fb3b8009310 comp rx=0 tx=0).stop 2024-05-09T04:49:07.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.386+0000 7fb3cf000700 1 -- 172.21.15.6:0/1451832095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3c8108f20 msgr2=0x7fb3c819d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:07.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.386+0000 7fb3cf000700 1 --2- 172.21.15.6:0/1451832095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3c8108f20 0x7fb3c819d780 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4000c00 tx=0x7fb3b400f950 comp rx=0 tx=0).stop 2024-05-09T04:49:07.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.386+0000 7fb3cf000700 1 -- 172.21.15.6:0/1451832095 shutdown_connections 2024-05-09T04:49:07.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.386+0000 7fb3cf000700 1 --2- 172.21.15.6:0/1451832095 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fb3ac0b2880 0x7fb3ac0b4d90 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:07.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.386+0000 7fb3cf000700 1 --2- 172.21.15.6:0/1451832095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3c8109870 0x7fb3c81a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:07.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.386+0000 7fb3cf000700 1 --2- 172.21.15.6:0/1451832095 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb3c81045f0 0x7fb3c819d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:07.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.386+0000 7fb3cf000700 1 --2- 172.21.15.6:0/1451832095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3c8108f20 0x7fb3c819d780 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:07.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.386+0000 7fb3cf000700 1 -- 172.21.15.6:0/1451832095 >> 172.21.15.6:0/1451832095 conn(0x7fb3c80fffe0 msgr2=0x7fb3c8107430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:07.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.386+0000 7fb3cf000700 1 -- 172.21.15.6:0/1451832095 shutdown_connections 2024-05-09T04:49:07.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:07.386+0000 7fb3cf000700 1 -- 172.21.15.6:0/1451832095 wait complete. 2024-05-09T04:49:07.389 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 10 2024-05-09T04:49:08.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:07 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:08.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:07 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:08.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:07 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:08.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:07 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:49:08.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:07 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:49:08.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:07 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:49:08.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:07 smithi186 ceph-mon[34668]: Deploying daemon mds.a on smithi006 2024-05-09T04:49:08.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:07 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/1451832095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:49:08.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:07 smithi186 ceph-mon[34668]: mds.? [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] up:boot 2024-05-09T04:49:08.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:07 smithi186 ceph-mon[34668]: fsmap 10 up:standby 2024-05-09T04:49:08.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:07 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T04:49:08.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:07 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:08.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:07 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:08.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:07 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:08.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:07 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:49:08.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:07 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:49:08.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:07 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:49:08.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:07 smithi169 ceph-mon[34075]: Deploying daemon mds.a on smithi006 2024-05-09T04:49:08.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:07 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/1451832095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:49:08.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:07 smithi169 ceph-mon[34075]: mds.? [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] up:boot 2024-05-09T04:49:08.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:07 smithi169 ceph-mon[34075]: fsmap 10 up:standby 2024-05-09T04:49:08.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:07 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T04:49:08.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:07 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:08.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:07 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:08.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:07 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:08.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:07 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:49:08.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:07 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:49:08.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:07 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:49:08.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:07 smithi006 ceph-mon[29835]: Deploying daemon mds.a on smithi006 2024-05-09T04:49:08.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:07 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1451832095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:49:08.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:07 smithi006 ceph-mon[29835]: mds.? [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] up:boot 2024-05-09T04:49:08.199 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:07 smithi006 ceph-mon[29835]: fsmap 10 up:standby 2024-05-09T04:49:08.199 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:07 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T04:49:08.933 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:08 smithi006 ceph-mon[29835]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:09.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:08 smithi186 ceph-mon[34668]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:09.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:08 smithi169 ceph-mon[34075]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:10.293 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:49:10.781 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:49:10.956 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:10 smithi006 ceph-mon[29835]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:11.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:10 smithi186 ceph-mon[34668]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:11.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:10 smithi169 ceph-mon[34075]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:12.380 INFO:journalctl@ceph.mds.a.smithi006.stdout:May 09 04:49:12 smithi006 podman[69164]: 2024-05-09 04:49:12.271274262 +0000 UTC m=+0.038955656 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T04:49:12.380 INFO:journalctl@ceph.mds.a.smithi006.stdout:May 09 04:49:12 smithi006 podman[69164]: 2024-05-09 04:49:12.37876289 +0000 UTC m=+0.146444322 container create 1a921a25b562d644a9a12a0414bc0692449e91e82b86ba061d5889097ddb893a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-09T04:49:13.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:12 smithi186 ceph-mon[34668]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:13.100 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:12 smithi006 ceph-mon[29835]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:13.101 INFO:journalctl@ceph.mds.a.smithi006.stdout:May 09 04:49:12 smithi006 podman[69164]: 2024-05-09 04:49:12.749729667 +0000 UTC m=+0.517411106 container init 1a921a25b562d644a9a12a0414bc0692449e91e82b86ba061d5889097ddb893a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-a, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-09T04:49:13.101 INFO:journalctl@ceph.mds.a.smithi006.stdout:May 09 04:49:12 smithi006 podman[69164]: 2024-05-09 04:49:12.760446936 +0000 UTC m=+0.528128340 container start 1a921a25b562d644a9a12a0414bc0692449e91e82b86ba061d5889097ddb893a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T04:49:13.101 INFO:journalctl@ceph.mds.a.smithi006.stdout:May 09 04:49:12 smithi006 bash[69164]: 1a921a25b562d644a9a12a0414bc0692449e91e82b86ba061d5889097ddb893a 2024-05-09T04:49:13.101 INFO:journalctl@ceph.mds.a.smithi006.stdout:May 09 04:49:12 smithi006 systemd[1]: Started Ceph mds.a for f74d813c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:49:13.101 INFO:journalctl@ceph.mds.a.smithi006.stdout:May 09 04:49:12 smithi006 ceph-mds[69201]: -- 172.21.15.6:0/511258310 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e8e1dffa40 con 0x55e8e1e47600 2024-05-09T04:49:13.101 INFO:journalctl@ceph.mds.a.smithi006.stdout:May 09 04:49:12 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-a[69196]: starting mds.a at 2024-05-09T04:49:13.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:12 smithi169 ceph-mon[34075]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:13.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.507+0000 7f1a84800700 1 -- 172.21.15.6:0/409131625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a7c107df0 msgr2=0x7f1a7c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:13.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.507+0000 7f1a84800700 1 --2- 172.21.15.6:0/409131625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a7c107df0 0x7f1a7c108200 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c009990 tx=0x7f1a6c009ca0 comp rx=0 tx=0).stop 2024-05-09T04:49:13.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.508+0000 7f1a84800700 1 -- 172.21.15.6:0/409131625 shutdown_connections 2024-05-09T04:49:13.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.508+0000 7f1a84800700 1 --2- 172.21.15.6:0/409131625 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a7c108740 0x7f1a7c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:13.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.508+0000 7f1a84800700 1 --2- 172.21.15.6:0/409131625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a7c1034c0 0x7f1a7c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:13.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.508+0000 7f1a84800700 1 --2- 172.21.15.6:0/409131625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a7c107df0 0x7f1a7c108200 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:13.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.508+0000 7f1a84800700 1 -- 172.21.15.6:0/409131625 >> 172.21.15.6:0/409131625 conn(0x7f1a7c0fedf0 msgr2=0x7f1a7c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:13.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.508+0000 7f1a84800700 1 -- 172.21.15.6:0/409131625 shutdown_connections 2024-05-09T04:49:13.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.508+0000 7f1a84800700 1 -- 172.21.15.6:0/409131625 wait complete. 2024-05-09T04:49:13.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.509+0000 7f1a84800700 1 Processor -- start 2024-05-09T04:49:13.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.509+0000 7f1a84800700 1 -- start start 2024-05-09T04:49:13.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.510+0000 7f1a84800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a7c1034c0 0x7f1a7c19ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:13.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.510+0000 7f1a84800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a7c107df0 0x7f1a7c19b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:13.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.510+0000 7f1a84800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a7c108740 0x7f1a7c19f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:13.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.510+0000 7f1a84800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7c1133a0 con 0x7f1a7c107df0 2024-05-09T04:49:13.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.510+0000 7f1a84800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7c113220 con 0x7f1a7c1034c0 2024-05-09T04:49:13.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.510+0000 7f1a84800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7c113520 con 0x7f1a7c108740 2024-05-09T04:49:13.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.510+0000 7f1a81e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a7c1034c0 0x7f1a7c19ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:13.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.510+0000 7f1a81e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a7c1034c0 0x7f1a7c19ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:60334/0 (socket says 172.21.15.6:60334) 2024-05-09T04:49:13.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.510+0000 7f1a81e00700 1 -- 172.21.15.6:0/1424270224 learned_addr learned my addr 172.21.15.6:0/1424270224 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:49:13.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.510+0000 7f1a81400700 1 --2- 172.21.15.6:0/1424270224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a7c107df0 0x7f1a7c19b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:13.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.511+0000 7f1a82800700 1 --2- 172.21.15.6:0/1424270224 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a7c108740 0x7f1a7c19f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:13.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.511+0000 7f1a82800700 1 -- 172.21.15.6:0/1424270224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a7c1034c0 msgr2=0x7f1a7c19ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:13.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.511+0000 7f1a82800700 1 --2- 172.21.15.6:0/1424270224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a7c1034c0 0x7f1a7c19ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:13.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.511+0000 7f1a82800700 1 -- 172.21.15.6:0/1424270224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a7c107df0 msgr2=0x7f1a7c19b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:13.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.511+0000 7f1a82800700 1 --2- 172.21.15.6:0/1424270224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a7c107df0 0x7f1a7c19b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:13.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.511+0000 7f1a82800700 1 -- 172.21.15.6:0/1424270224 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a7c19fd30 con 0x7f1a7c108740 2024-05-09T04:49:13.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.511+0000 7f1a81e00700 1 --2- 172.21.15.6:0/1424270224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a7c1034c0 0x7f1a7c19ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:49:13.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.511+0000 7f1a82800700 1 --2- 172.21.15.6:0/1424270224 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a7c108740 0x7f1a7c19f690 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1a7800efc0 tx=0x7f1a7800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:13.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.512+0000 7f1a76a00700 1 -- 172.21.15.6:0/1424270224 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a780092e0 con 0x7f1a7c108740 2024-05-09T04:49:13.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.512+0000 7f1a76a00700 1 -- 172.21.15.6:0/1424270224 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a78004bb0 con 0x7f1a7c108740 2024-05-09T04:49:13.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.512+0000 7f1a76a00700 1 -- 172.21.15.6:0/1424270224 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a78004020 con 0x7f1a7c108740 2024-05-09T04:49:13.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.512+0000 7f1a84800700 1 -- 172.21.15.6:0/1424270224 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a7c19ffc0 con 0x7f1a7c108740 2024-05-09T04:49:13.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.512+0000 7f1a84800700 1 -- 172.21.15.6:0/1424270224 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a7c1a04b0 con 0x7f1a7c108740 2024-05-09T04:49:13.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.513+0000 7f1a84800700 1 -- 172.21.15.6:0/1424270224 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a7c0512c0 con 0x7f1a7c108740 2024-05-09T04:49:13.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.516+0000 7f1a76a00700 1 -- 172.21.15.6:0/1424270224 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f1a7801b030 con 0x7f1a7c108740 2024-05-09T04:49:13.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.517+0000 7f1a76a00700 1 --2- 172.21.15.6:0/1424270224 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f1a640b2900 0x7f1a640b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:13.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.518+0000 7f1a76a00700 1 -- 172.21.15.6:0/1424270224 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f1a780e21e0 con 0x7f1a7c108740 2024-05-09T04:49:13.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.518+0000 7f1a81e00700 1 --2- 172.21.15.6:0/1424270224 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f1a640b2900 0x7f1a640b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:13.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.518+0000 7f1a81e00700 1 --2- 172.21.15.6:0/1424270224 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f1a640b2900 0x7f1a640b4e10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1a70011550 tx=0x7f1a70009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:13.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.519+0000 7f1a76a00700 1 -- 172.21.15.6:0/1424270224 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1a78022020 con 0x7f1a7c108740 2024-05-09T04:49:13.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.813+0000 7f1a84800700 1 -- 172.21.15.6:0/1424270224 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a7c02e5f0 con 0x7f1a7c108740 2024-05-09T04:49:13.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.814+0000 7f1a76a00700 1 -- 172.21.15.6:0/1424270224 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8827 (secure 0 0 0) 0x7f1a78010040 con 0x7f1a7c108740 2024-05-09T04:49:13.815 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:49:13.815 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/162094599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":162094599},{"type":"v1","addr":"172.21.15.6:6835","nonce":162094599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/2977145206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2977145206},{"type":"v1","addr":"172.21.15.186:6833","nonce":2977145206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/133345948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":133345948},{"type":"v1","addr":"172.21.15.169:6833","nonce":133345948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/1281521088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":1281521088},{"type":"v1","addr":"172.21.15.6:6837","nonce":1281521088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2268322007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2268322007},{"type":"v1","addr":"172.21.15.186:6835","nonce":2268322007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3811005052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3811005052},{"type":"v1","addr":"172.21.15.169:6835","nonce":3811005052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1647607118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1647607118},{"type":"v1","addr":"172.21.15.186:6837","nonce":1647607118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/3954883105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":3954883105},{"type":"v1","addr":"172.21.15.186:6839","nonce":3954883105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24512,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1845715294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1845715294},{"type":"v1","addr":"172.21.15.169:6837","nonce":1845715294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24530,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/2780306492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":2780306492},{"type":"v1","addr":"172.21.15.6:6839","nonce":2780306492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-05-09T04:49:13.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.817+0000 7f1a84800700 1 -- 172.21.15.6:0/1424270224 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f1a640b2900 msgr2=0x7f1a640b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:13.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.817+0000 7f1a84800700 1 --2- 172.21.15.6:0/1424270224 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f1a640b2900 0x7f1a640b4e10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1a70011550 tx=0x7f1a70009260 comp rx=0 tx=0).stop 2024-05-09T04:49:13.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.817+0000 7f1a84800700 1 -- 172.21.15.6:0/1424270224 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a7c108740 msgr2=0x7f1a7c19f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:13.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.817+0000 7f1a84800700 1 --2- 172.21.15.6:0/1424270224 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a7c108740 0x7f1a7c19f690 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1a7800efc0 tx=0x7f1a7800c3a0 comp rx=0 tx=0).stop 2024-05-09T04:49:13.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.817+0000 7f1a84800700 1 -- 172.21.15.6:0/1424270224 shutdown_connections 2024-05-09T04:49:13.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.817+0000 7f1a84800700 1 --2- 172.21.15.6:0/1424270224 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a7c108740 0x7f1a7c19f690 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:13.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.817+0000 7f1a84800700 1 --2- 172.21.15.6:0/1424270224 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f1a640b2900 0x7f1a640b4e10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:13.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.817+0000 7f1a84800700 1 --2- 172.21.15.6:0/1424270224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a7c1034c0 0x7f1a7c19ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:13.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.817+0000 7f1a84800700 1 --2- 172.21.15.6:0/1424270224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a7c107df0 0x7f1a7c19b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:13.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.817+0000 7f1a84800700 1 -- 172.21.15.6:0/1424270224 >> 172.21.15.6:0/1424270224 conn(0x7f1a7c0fedf0 msgr2=0x7f1a7c075dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:13.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.818+0000 7f1a84800700 1 -- 172.21.15.6:0/1424270224 shutdown_connections 2024-05-09T04:49:13.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:13.818+0000 7f1a84800700 1 -- 172.21.15.6:0/1424270224 wait complete. 2024-05-09T04:49:13.820 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 11 2024-05-09T04:49:14.177 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:13 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:14.177 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:13 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:14.177 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:13 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:14.177 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:13 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:49:14.177 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:13 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:49:14.178 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:13 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:49:14.178 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:13 smithi006 ceph-mon[29835]: Deploying daemon mds.b on smithi169 2024-05-09T04:49:14.178 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:13 smithi006 ceph-mon[29835]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:14.178 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:13 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1424270224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:49:14.178 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:13 smithi006 ceph-mon[29835]: mds.? [v2:172.21.15.6:6840/589294363,v1:172.21.15.6:6841/589294363] up:boot 2024-05-09T04:49:14.178 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:13 smithi006 ceph-mon[29835]: fsmap 11 up:standby 2024-05-09T04:49:14.178 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:13 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T04:49:14.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:13 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:14.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:13 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:14.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:13 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:14.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:13 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:49:14.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:13 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:49:14.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:13 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:49:14.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:13 smithi186 ceph-mon[34668]: Deploying daemon mds.b on smithi169 2024-05-09T04:49:14.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:13 smithi186 ceph-mon[34668]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:14.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:13 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/1424270224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:49:14.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:13 smithi186 ceph-mon[34668]: mds.? [v2:172.21.15.6:6840/589294363,v1:172.21.15.6:6841/589294363] up:boot 2024-05-09T04:49:14.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:13 smithi186 ceph-mon[34668]: fsmap 11 up:standby 2024-05-09T04:49:14.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:13 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T04:49:14.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:13 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:14.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:13 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:14.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:13 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:14.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:13 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:49:14.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:13 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:49:14.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:13 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:49:14.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:13 smithi169 ceph-mon[34075]: Deploying daemon mds.b on smithi169 2024-05-09T04:49:14.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:13 smithi169 ceph-mon[34075]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:14.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:13 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/1424270224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:49:14.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:13 smithi169 ceph-mon[34075]: mds.? [v2:172.21.15.6:6840/589294363,v1:172.21.15.6:6841/589294363] up:boot 2024-05-09T04:49:14.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:13 smithi169 ceph-mon[34075]: fsmap 11 up:standby 2024-05-09T04:49:14.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:13 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T04:49:15.528 INFO:journalctl@ceph.mds.b.smithi169.stdout:May 09 04:49:15 smithi169 systemd[1]: Starting Ceph mds.b for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T04:49:15.813 INFO:journalctl@ceph.mds.b.smithi169.stdout:May 09 04:49:15 smithi169 podman[64923]: 2024-05-09 04:49:15.431742419 +0000 UTC m=+0.018188374 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T04:49:15.813 INFO:journalctl@ceph.mds.b.smithi169.stdout:May 09 04:49:15 smithi169 podman[64923]: 2024-05-09 04:49:15.561042752 +0000 UTC m=+0.147488679 container create a0c62ed0cc4211281ba66c5ad0884d34ab3623019c4a485d6ba2e49626f24168 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, org.label-schema.build-date=20240507) 2024-05-09T04:49:15.813 INFO:journalctl@ceph.mds.b.smithi169.stdout:May 09 04:49:15 smithi169 podman[64923]: 2024-05-09 04:49:15.813912193 +0000 UTC m=+0.400358120 container init a0c62ed0cc4211281ba66c5ad0884d34ab3623019c4a485d6ba2e49626f24168 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-09T04:49:16.116 INFO:journalctl@ceph.mds.b.smithi169.stdout:May 09 04:49:15 smithi169 podman[64923]: 2024-05-09 04:49:15.816803384 +0000 UTC m=+0.403249312 container start a0c62ed0cc4211281ba66c5ad0884d34ab3623019c4a485d6ba2e49626f24168 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-b, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-09T04:49:16.117 INFO:journalctl@ceph.mds.b.smithi169.stdout:May 09 04:49:15 smithi169 ceph-mds[64943]: -- 172.21.15.169:0/497334443 <== mon.2 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5646376bba40 con 0x564637703180 2024-05-09T04:49:16.117 INFO:journalctl@ceph.mds.b.smithi169.stdout:May 09 04:49:15 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-b[64939]: starting mds.b at 2024-05-09T04:49:16.117 INFO:journalctl@ceph.mds.b.smithi169.stdout:May 09 04:49:15 smithi169 bash[64923]: a0c62ed0cc4211281ba66c5ad0884d34ab3623019c4a485d6ba2e49626f24168 2024-05-09T04:49:16.117 INFO:journalctl@ceph.mds.b.smithi169.stdout:May 09 04:49:15 smithi169 systemd[1]: Started Ceph mds.b for f74d813c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:49:16.711 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:49:16.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:16 smithi186 ceph-mon[34668]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:16.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:16.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:16.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:16.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:16.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:16.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:16 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:49:16.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:16 smithi169 ceph-mon[34075]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:16.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:16.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:16.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:16.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:16.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:16.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:16 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:49:16.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:16 smithi006 ceph-mon[29835]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:16.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:16.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:16.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:16.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:16.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:16.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:16 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:49:17.167 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:49:17.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:17 smithi186 ceph-mon[34668]: mds.? [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] up:boot 2024-05-09T04:49:17.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:17 smithi186 ceph-mon[34668]: fsmap 12 up:standby 2024-05-09T04:49:17.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:17 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T04:49:17.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:17 smithi169 ceph-mon[34075]: mds.? [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] up:boot 2024-05-09T04:49:17.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:17 smithi169 ceph-mon[34075]: fsmap 12 up:standby 2024-05-09T04:49:17.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:17 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T04:49:17.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:17 smithi006 ceph-mon[29835]: mds.? [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] up:boot 2024-05-09T04:49:17.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:17 smithi006 ceph-mon[29835]: fsmap 12 up:standby 2024-05-09T04:49:17.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:17 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T04:49:18.731 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:18 smithi169 ceph-mon[34075]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:18.732 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:18 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:18.732 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:18 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:18.732 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:18 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:18.732 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:18 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:18.853 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:18 smithi006 ceph-mon[29835]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:18.853 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:18 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:18.853 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:18 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:18.853 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:18 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:18.854 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:18 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:19.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:18 smithi186 ceph-mon[34668]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:19.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:18 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:19.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:18 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:19.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:18 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:19.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:18 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:20.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.030+0000 7ff24e200700 1 -- 172.21.15.6:0/2672802699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2481045d0 msgr2=0x7ff2481049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:20.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.030+0000 7ff24e200700 1 --2- 172.21.15.6:0/2672802699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2481045d0 0x7ff2481049e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff238009990 tx=0x7ff238009ca0 comp rx=0 tx=0).stop 2024-05-09T04:49:20.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.031+0000 7ff24e200700 1 -- 172.21.15.6:0/2672802699 shutdown_connections 2024-05-09T04:49:20.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.031+0000 7ff24e200700 1 --2- 172.21.15.6:0/2672802699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff248109850 0x7ff2481111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:20.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.031+0000 7ff24e200700 1 --2- 172.21.15.6:0/2672802699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2481045d0 0x7ff2481049e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:20.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.031+0000 7ff24e200700 1 --2- 172.21.15.6:0/2672802699 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff248108f00 0x7ff248109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:20.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.031+0000 7ff24e200700 1 -- 172.21.15.6:0/2672802699 >> 172.21.15.6:0/2672802699 conn(0x7ff2480ffea0 msgr2=0x7ff2481022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:20.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.031+0000 7ff24e200700 1 -- 172.21.15.6:0/2672802699 shutdown_connections 2024-05-09T04:49:20.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.031+0000 7ff24e200700 1 -- 172.21.15.6:0/2672802699 wait complete. 2024-05-09T04:49:20.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.032+0000 7ff24e200700 1 Processor -- start 2024-05-09T04:49:20.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.032+0000 7ff24e200700 1 -- start start 2024-05-09T04:49:20.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.033+0000 7ff24e200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2481045d0 0x7ff24819ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:20.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.033+0000 7ff24e200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff248108f00 0x7ff24819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:20.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.033+0000 7ff24e200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff248109850 0x7ff24819f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:20.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.033+0000 7ff24e200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff2481144d0 con 0x7ff248109850 2024-05-09T04:49:20.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.033+0000 7ff24e200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff248114350 con 0x7ff2481045d0 2024-05-09T04:49:20.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.033+0000 7ff24e200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff248114650 con 0x7ff248108f00 2024-05-09T04:49:20.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.033+0000 7ff247400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2481045d0 0x7ff24819ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:20.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.033+0000 7ff247e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff248109850 0x7ff24819f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:20.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.033+0000 7ff247400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2481045d0 0x7ff24819ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:38616/0 (socket says 172.21.15.6:38616) 2024-05-09T04:49:20.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.033+0000 7ff247400700 1 -- 172.21.15.6:0/2593235938 learned_addr learned my addr 172.21.15.6:0/2593235938 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:49:20.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.033+0000 7ff247e00700 1 --2- 172.21.15.6:0/2593235938 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff248109850 0x7ff24819f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:50630/0 (socket says 172.21.15.6:50630) 2024-05-09T04:49:20.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.033+0000 7ff23fe00700 1 --2- 172.21.15.6:0/2593235938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff248108f00 0x7ff24819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:20.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.034+0000 7ff247400700 1 -- 172.21.15.6:0/2593235938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff248108f00 msgr2=0x7ff24819b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:20.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.034+0000 7ff247400700 1 --2- 172.21.15.6:0/2593235938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff248108f00 0x7ff24819b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:20.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.034+0000 7ff247400700 1 -- 172.21.15.6:0/2593235938 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff248109850 msgr2=0x7ff24819f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:20.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.034+0000 7ff247400700 1 --2- 172.21.15.6:0/2593235938 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff248109850 0x7ff24819f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:20.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.034+0000 7ff247400700 1 -- 172.21.15.6:0/2593235938 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff24819fcd0 con 0x7ff2481045d0 2024-05-09T04:49:20.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.035+0000 7ff247400700 1 --2- 172.21.15.6:0/2593235938 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2481045d0 0x7ff24819ae00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff238005e80 tx=0x7ff238005eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:20.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.035+0000 7ff244c00700 1 -- 172.21.15.6:0/2593235938 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23800f930 con 0x7ff2481045d0 2024-05-09T04:49:20.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.035+0000 7ff24e200700 1 -- 172.21.15.6:0/2593235938 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff24819ff60 con 0x7ff2481045d0 2024-05-09T04:49:20.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.035+0000 7ff24e200700 1 -- 172.21.15.6:0/2593235938 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff248076b00 con 0x7ff2481045d0 2024-05-09T04:49:20.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.035+0000 7ff244c00700 1 -- 172.21.15.6:0/2593235938 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff23801f4d0 con 0x7ff2481045d0 2024-05-09T04:49:20.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.035+0000 7ff244c00700 1 -- 172.21.15.6:0/2593235938 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff238017650 con 0x7ff2481045d0 2024-05-09T04:49:20.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.037+0000 7ff24e200700 1 -- 172.21.15.6:0/2593235938 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff248105a80 con 0x7ff2481045d0 2024-05-09T04:49:20.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.041+0000 7ff244c00700 1 -- 172.21.15.6:0/2593235938 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7ff238009d10 con 0x7ff2481045d0 2024-05-09T04:49:20.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.043+0000 7ff244c00700 1 --2- 172.21.15.6:0/2593235938 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff22c0b2900 0x7ff22c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:20.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.044+0000 7ff244c00700 1 -- 172.21.15.6:0/2593235938 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7ff2380e7120 con 0x7ff2481045d0 2024-05-09T04:49:20.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.044+0000 7ff23fe00700 1 --2- 172.21.15.6:0/2593235938 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff22c0b2900 0x7ff22c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:20.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.044+0000 7ff244c00700 1 -- 172.21.15.6:0/2593235938 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff238013070 con 0x7ff2481045d0 2024-05-09T04:49:20.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.045+0000 7ff23fe00700 1 --2- 172.21.15.6:0/2593235938 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff22c0b2900 0x7ff22c0b4e10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff2481076b0 tx=0x7ff230009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:20.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.372+0000 7ff24e200700 1 -- 172.21.15.6:0/2593235938 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff24819c160 con 0x7ff2481045d0 2024-05-09T04:49:20.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.373+0000 7ff244c00700 1 -- 172.21.15.6:0/2593235938 <== mon.2 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10473 (secure 0 0 0) 0x7ff2380af2d0 con 0x7ff2481045d0 2024-05-09T04:49:20.374 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:49:20.374 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/162094599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":162094599},{"type":"v1","addr":"172.21.15.6:6835","nonce":162094599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/589294363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":589294363},{"type":"v1","addr":"172.21.15.6:6841","nonce":589294363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/2977145206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2977145206},{"type":"v1","addr":"172.21.15.186:6833","nonce":2977145206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/133345948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":133345948},{"type":"v1","addr":"172.21.15.169:6833","nonce":133345948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/1281521088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":1281521088},{"type":"v1","addr":"172.21.15.6:6837","nonce":1281521088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2268322007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2268322007},{"type":"v1","addr":"172.21.15.186:6835","nonce":2268322007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3811005052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3811005052},{"type":"v1","addr":"172.21.15.169:6835","nonce":3811005052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1647607118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1647607118},{"type":"v1","addr":"172.21.15.186:6837","nonce":1647607118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/3954883105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":3954883105},{"type":"v1","addr":"172.21.15.186:6839","nonce":3954883105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24512,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1845715294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1845715294},{"type":"v1","addr":"172.21.15.169:6837","nonce":1845715294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24530,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/2780306492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":2780306492},{"type":"v1","addr":"172.21.15.6:6839","nonce":2780306492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24554,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6839/614388771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":614388771},{"type":"v1","addr":"172.21.15.169:6839","nonce":614388771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-05-09T04:49:20.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.377+0000 7ff24e200700 1 -- 172.21.15.6:0/2593235938 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff22c0b2900 msgr2=0x7ff22c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:20.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.377+0000 7ff24e200700 1 --2- 172.21.15.6:0/2593235938 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff22c0b2900 0x7ff22c0b4e10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff2481076b0 tx=0x7ff230009260 comp rx=0 tx=0).stop 2024-05-09T04:49:20.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.377+0000 7ff24e200700 1 -- 172.21.15.6:0/2593235938 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2481045d0 msgr2=0x7ff24819ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:20.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.377+0000 7ff24e200700 1 --2- 172.21.15.6:0/2593235938 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2481045d0 0x7ff24819ae00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff238005e80 tx=0x7ff238005eb0 comp rx=0 tx=0).stop 2024-05-09T04:49:20.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.377+0000 7ff24e200700 1 -- 172.21.15.6:0/2593235938 shutdown_connections 2024-05-09T04:49:20.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.377+0000 7ff24e200700 1 --2- 172.21.15.6:0/2593235938 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff248109850 0x7ff24819f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:20.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.377+0000 7ff24e200700 1 --2- 172.21.15.6:0/2593235938 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff22c0b2900 0x7ff22c0b4e10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:20.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.377+0000 7ff24e200700 1 --2- 172.21.15.6:0/2593235938 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2481045d0 0x7ff24819ae00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:20.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.377+0000 7ff24e200700 1 --2- 172.21.15.6:0/2593235938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff248108f00 0x7ff24819b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:20.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.377+0000 7ff24e200700 1 -- 172.21.15.6:0/2593235938 >> 172.21.15.6:0/2593235938 conn(0x7ff2480ffea0 msgr2=0x7ff248108130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:20.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.378+0000 7ff24e200700 1 -- 172.21.15.6:0/2593235938 shutdown_connections 2024-05-09T04:49:20.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:20.378+0000 7ff24e200700 1 -- 172.21.15.6:0/2593235938 wait complete. 2024-05-09T04:49:20.380 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 13 2024-05-09T04:49:20.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:20 smithi006 ceph-mon[29835]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:20.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:20 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/2593235938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:49:20.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:20 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:21.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:20 smithi186 ceph-mon[34668]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:21.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:20 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/2593235938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:49:21.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:20 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:21.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:20 smithi169 ceph-mon[34075]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:21.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:20 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/2593235938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:49:21.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:20 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:21.359 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-09T04:49:21.797 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:49:22.800 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:22 smithi186 ceph-mon[34668]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:22.800 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:22.800 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:22.800 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:49:22.800 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:49:22.800 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:22.801 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:22 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:49:22.837 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:22 smithi169 ceph-mon[34075]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:22.837 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:22.837 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:22.837 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:49:22.837 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:49:22.837 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:22.837 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:22 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:49:22.873 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:22 smithi006 ceph-mon[29835]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:22.873 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:22.873 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:22.873 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:49:22.873 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:49:22.873 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:22.873 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:22 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:49:24.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.489+0000 7f65c2c00700 1 -- 172.21.15.6:0/3621170032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65bc105e30 msgr2=0x7f65bc10e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:24.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.489+0000 7f65c2c00700 1 --2- 172.21.15.6:0/3621170032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65bc105e30 0x7f65bc10e380 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f65a4009990 tx=0x7f65a4009ca0 comp rx=0 tx=0).stop 2024-05-09T04:49:24.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.489+0000 7f65c2c00700 1 -- 172.21.15.6:0/3621170032 shutdown_connections 2024-05-09T04:49:24.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.489+0000 7f65c2c00700 1 --2- 172.21.15.6:0/3621170032 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f65bc10e910 0x7f65bc110df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:24.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.489+0000 7f65c2c00700 1 --2- 172.21.15.6:0/3621170032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65bc1054e0 0x7f65bc1058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:24.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.489+0000 7f65c2c00700 1 --2- 172.21.15.6:0/3621170032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65bc105e30 0x7f65bc10e380 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:24.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.489+0000 7f65c2c00700 1 -- 172.21.15.6:0/3621170032 >> 172.21.15.6:0/3621170032 conn(0x7f65bc0fedc0 msgr2=0x7f65bc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:24.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.490+0000 7f65c2c00700 1 -- 172.21.15.6:0/3621170032 shutdown_connections 2024-05-09T04:49:24.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.490+0000 7f65c2c00700 1 -- 172.21.15.6:0/3621170032 wait complete. 2024-05-09T04:49:24.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.491+0000 7f65c2c00700 1 Processor -- start 2024-05-09T04:49:24.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.493+0000 7f65c2c00700 1 -- start start 2024-05-09T04:49:24.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.493+0000 7f65c2c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f65bc1054e0 0x7f65bc1031d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:24.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.493+0000 7f65c2c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65bc105e30 0x7f65bc103710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:24.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.493+0000 7f65c2c00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65bc10e910 0x7f65bc103c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:24.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.493+0000 7f65c2c00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc114440 con 0x7f65bc1054e0 2024-05-09T04:49:24.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.493+0000 7f65c2c00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc1142c0 con 0x7f65bc10e910 2024-05-09T04:49:24.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.493+0000 7f65c2c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc1145c0 con 0x7f65bc105e30 2024-05-09T04:49:24.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.493+0000 7f65bbe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f65bc1054e0 0x7f65bc1031d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:24.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.493+0000 7f65b3400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65bc105e30 0x7f65bc103710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:24.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.493+0000 7f65bbe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f65bc1054e0 0x7f65bc1031d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:50644/0 (socket says 172.21.15.6:50644) 2024-05-09T04:49:24.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.493+0000 7f65bbe00700 1 -- 172.21.15.6:0/3618713777 learned_addr learned my addr 172.21.15.6:0/3618713777 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:49:24.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.493+0000 7f65c0c00700 1 --2- 172.21.15.6:0/3618713777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65bc10e910 0x7f65bc103c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:24.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.493+0000 7f65b3400700 1 -- 172.21.15.6:0/3618713777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65bc10e910 msgr2=0x7f65bc103c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:24.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.493+0000 7f65b3400700 1 --2- 172.21.15.6:0/3618713777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65bc10e910 0x7f65bc103c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:24.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.493+0000 7f65b3400700 1 -- 172.21.15.6:0/3618713777 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f65bc1054e0 msgr2=0x7f65bc1031d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:24.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.493+0000 7f65b3400700 1 --2- 172.21.15.6:0/3618713777 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f65bc1054e0 0x7f65bc1031d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:24.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.493+0000 7f65b3400700 1 -- 172.21.15.6:0/3618713777 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65bc1a91a0 con 0x7f65bc105e30 2024-05-09T04:49:24.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.493+0000 7f65bbe00700 1 --2- 172.21.15.6:0/3618713777 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f65bc1054e0 0x7f65bc1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:49:24.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.494+0000 7f65b3400700 1 --2- 172.21.15.6:0/3618713777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65bc105e30 0x7f65bc103710 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f65a4006010 tx=0x7f65a40057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:24.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.494+0000 7f65b9600700 1 -- 172.21.15.6:0/3618713777 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a4027070 con 0x7f65bc105e30 2024-05-09T04:49:24.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.494+0000 7f65b9600700 1 -- 172.21.15.6:0/3618713777 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65a4022470 con 0x7f65bc105e30 2024-05-09T04:49:24.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.494+0000 7f65b9600700 1 -- 172.21.15.6:0/3618713777 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a4017650 con 0x7f65bc105e30 2024-05-09T04:49:24.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.494+0000 7f65c2c00700 1 -- 172.21.15.6:0/3618713777 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65bc1a9430 con 0x7f65bc105e30 2024-05-09T04:49:24.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.494+0000 7f65c2c00700 1 -- 172.21.15.6:0/3618713777 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65bc1a98c0 con 0x7f65bc105e30 2024-05-09T04:49:24.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.495+0000 7f65c2c00700 1 -- 172.21.15.6:0/3618713777 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65bc0512c0 con 0x7f65bc105e30 2024-05-09T04:49:24.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.499+0000 7f65b9600700 1 -- 172.21.15.6:0/3618713777 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f65a402a070 con 0x7f65bc105e30 2024-05-09T04:49:24.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.499+0000 7f65b9600700 1 --2- 172.21.15.6:0/3618713777 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f65a00b2900 0x7f65a00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:24.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.499+0000 7f65b9600700 1 -- 172.21.15.6:0/3618713777 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f65a40e2c40 con 0x7f65bc105e30 2024-05-09T04:49:24.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.500+0000 7f65bbe00700 1 --2- 172.21.15.6:0/3618713777 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f65a00b2900 0x7f65a00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:24.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.500+0000 7f65bbe00700 1 --2- 172.21.15.6:0/3618713777 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f65a00b2900 0x7f65a00b4e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f65ac009c10 tx=0x7f65ac00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:24.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.501+0000 7f65b9600700 1 -- 172.21.15.6:0/3618713777 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f65a4013070 con 0x7f65bc105e30 2024-05-09T04:49:24.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.748+0000 7f65c2c00700 1 -- 172.21.15.6:0/3618713777 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f65bc02e5f0 con 0x7f65bc105e30 2024-05-09T04:49:24.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.748+0000 7f65b9600700 1 -- 172.21.15.6:0/3618713777 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) ==== 84+0+65 (secure 0 0 0) 0x7f65a40aadf0 con 0x7f65bc105e30 2024-05-09T04:49:24.748 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:49:24.748 INFO:teuthology.orchestra.run.smithi006.stdout:[ 2024-05-09T04:49:24.748 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:49:24.748 INFO:teuthology.orchestra.run.smithi006.stdout: "poolnum": 1, 2024-05-09T04:49:24.748 INFO:teuthology.orchestra.run.smithi006.stdout: "poolname": ".mgr" 2024-05-09T04:49:24.749 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:49:24.749 INFO:teuthology.orchestra.run.smithi006.stdout:] 2024-05-09T04:49:24.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.751+0000 7f65c2c00700 1 -- 172.21.15.6:0/3618713777 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f65a00b2900 msgr2=0x7f65a00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:24.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.751+0000 7f65c2c00700 1 --2- 172.21.15.6:0/3618713777 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f65a00b2900 0x7f65a00b4e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f65ac009c10 tx=0x7f65ac00b040 comp rx=0 tx=0).stop 2024-05-09T04:49:24.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.751+0000 7f65c2c00700 1 -- 172.21.15.6:0/3618713777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65bc105e30 msgr2=0x7f65bc103710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:24.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.751+0000 7f65c2c00700 1 --2- 172.21.15.6:0/3618713777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65bc105e30 0x7f65bc103710 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f65a4006010 tx=0x7f65a40057c0 comp rx=0 tx=0).stop 2024-05-09T04:49:24.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.752+0000 7f65c2c00700 1 -- 172.21.15.6:0/3618713777 shutdown_connections 2024-05-09T04:49:24.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.752+0000 7f65c2c00700 1 --2- 172.21.15.6:0/3618713777 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f65a00b2900 0x7f65a00b4e10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:24.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.752+0000 7f65c2c00700 1 --2- 172.21.15.6:0/3618713777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65bc10e910 0x7f65bc103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:24.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.752+0000 7f65c2c00700 1 --2- 172.21.15.6:0/3618713777 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f65bc1054e0 0x7f65bc1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:24.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.752+0000 7f65c2c00700 1 --2- 172.21.15.6:0/3618713777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65bc105e30 0x7f65bc103710 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:24.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.752+0000 7f65c2c00700 1 -- 172.21.15.6:0/3618713777 >> 172.21.15.6:0/3618713777 conn(0x7f65bc0fedc0 msgr2=0x7f65bc0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:24.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.752+0000 7f65c2c00700 1 -- 172.21.15.6:0/3618713777 shutdown_connections 2024-05-09T04:49:24.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:24.752+0000 7f65c2c00700 1 -- 172.21.15.6:0/3618713777 wait complete. 2024-05-09T04:49:24.778 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:24 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:24.778 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:24 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:24.778 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:24 smithi006 ceph-mon[29835]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:24.778 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:24 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:24.779 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:24 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:24.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:24 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:24.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:24 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:24.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:24 smithi186 ceph-mon[34668]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:24.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:24 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:24.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:24 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:24.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:24 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:24.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:24 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:24.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:24 smithi169 ceph-mon[34075]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:24.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:24 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:24.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:24 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:25.696 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-09T04:49:25.696 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-09T04:49:25.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:25 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/3618713777' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-09T04:49:25.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:25 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/3618713777' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-09T04:49:25.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:25 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/3618713777' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-09T04:49:26.144 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:49:26.636 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:26 smithi006 ceph-mon[29835]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:26.636 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:26 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:26.636 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:26 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:26.636 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:26 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:49:26.637 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:26 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:49:26.637 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:26 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:26.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:26 smithi186 ceph-mon[34668]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:26.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:26 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:26.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:26 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:26.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:26 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:49:26.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:26 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:49:26.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:26 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:26.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:26 smithi169 ceph-mon[34075]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:26.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:26 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:26.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:26 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:26.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:26 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:49:26.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:26 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:49:26.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:26 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:49:28.282 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:27 smithi006 ceph-mon[29835]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:28.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:27 smithi186 ceph-mon[34668]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:28.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:27 smithi169 ceph-mon[34075]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:28.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.540+0000 7f44c4800700 1 -- 172.21.15.6:0/3737809279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44bc110c60 msgr2=0x7f44bc113080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:28.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.540+0000 7f44c4800700 1 --2- 172.21.15.6:0/3737809279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44bc110c60 0x7f44bc113080 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f44b800b0a0 tx=0x7f44b800b3b0 comp rx=0 tx=0).stop 2024-05-09T04:49:28.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.540+0000 7f44c4800700 1 -- 172.21.15.6:0/3737809279 shutdown_connections 2024-05-09T04:49:28.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.540+0000 7f44c4800700 1 --2- 172.21.15.6:0/3737809279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44bc110c60 0x7f44bc113080 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:28.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.540+0000 7f44c4800700 1 --2- 172.21.15.6:0/3737809279 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44bc075b00 0x7f44bc075f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:28.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.540+0000 7f44c4800700 1 --2- 172.21.15.6:0/3737809279 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44bc076450 0x7f44bc110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:28.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.540+0000 7f44c4800700 1 -- 172.21.15.6:0/3737809279 >> 172.21.15.6:0/3737809279 conn(0x7f44bc079f00 msgr2=0x7f44bc07a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:28.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.540+0000 7f44c4800700 1 -- 172.21.15.6:0/3737809279 shutdown_connections 2024-05-09T04:49:28.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.540+0000 7f44c4800700 1 -- 172.21.15.6:0/3737809279 wait complete. 2024-05-09T04:49:28.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.541+0000 7f44c4800700 1 Processor -- start 2024-05-09T04:49:28.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.541+0000 7f44c4800700 1 -- start start 2024-05-09T04:49:28.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.542+0000 7f44c4800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44bc075b00 0x7f44bc19d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:28.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.542+0000 7f44c4800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44bc076450 0x7f44bc19d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:28.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.542+0000 7f44c4800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44bc110c60 0x7f44bc1a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:28.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.542+0000 7f44c4800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc1166d0 con 0x7f44bc075b00 2024-05-09T04:49:28.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.542+0000 7f44c4800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc116550 con 0x7f44bc076450 2024-05-09T04:49:28.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.542+0000 7f44c4800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc116850 con 0x7f44bc110c60 2024-05-09T04:49:28.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.542+0000 7f44c1e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44bc075b00 0x7f44bc19d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:28.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.542+0000 7f44c1e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44bc075b00 0x7f44bc19d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35114/0 (socket says 172.21.15.6:35114) 2024-05-09T04:49:28.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.542+0000 7f44c1e00700 1 -- 172.21.15.6:0/4260404269 learned_addr learned my addr 172.21.15.6:0/4260404269 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:49:28.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.542+0000 7f44c1400700 1 --2- 172.21.15.6:0/4260404269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44bc076450 0x7f44bc19d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:28.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.542+0000 7f44c2800700 1 --2- 172.21.15.6:0/4260404269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44bc110c60 0x7f44bc1a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:28.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.543+0000 7f44c1e00700 1 -- 172.21.15.6:0/4260404269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44bc110c60 msgr2=0x7f44bc1a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:28.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.543+0000 7f44c1e00700 1 --2- 172.21.15.6:0/4260404269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44bc110c60 0x7f44bc1a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:28.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.543+0000 7f44c1e00700 1 -- 172.21.15.6:0/4260404269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44bc076450 msgr2=0x7f44bc19d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:28.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.543+0000 7f44c1e00700 1 --2- 172.21.15.6:0/4260404269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44bc076450 0x7f44bc19d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:28.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.543+0000 7f44c1e00700 1 -- 172.21.15.6:0/4260404269 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44bc1a1f10 con 0x7f44bc075b00 2024-05-09T04:49:28.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.543+0000 7f44c2800700 1 --2- 172.21.15.6:0/4260404269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44bc110c60 0x7f44bc1a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:49:28.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.543+0000 7f44c1400700 1 --2- 172.21.15.6:0/4260404269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44bc076450 0x7f44bc19d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:49:28.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.543+0000 7f44c1e00700 1 --2- 172.21.15.6:0/4260404269 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44bc075b00 0x7f44bc19d040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f44b000e9a0 tx=0x7f44b000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:28.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.543+0000 7f44b6a00700 1 -- 172.21.15.6:0/4260404269 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b000ca10 con 0x7f44bc075b00 2024-05-09T04:49:28.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.543+0000 7f44b6a00700 1 -- 172.21.15.6:0/4260404269 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44b000cbb0 con 0x7f44bc075b00 2024-05-09T04:49:28.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.543+0000 7f44c4800700 1 -- 172.21.15.6:0/4260404269 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44bc1a2200 con 0x7f44bc075b00 2024-05-09T04:49:28.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.543+0000 7f44b6a00700 1 -- 172.21.15.6:0/4260404269 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b0018680 con 0x7f44bc075b00 2024-05-09T04:49:28.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.544+0000 7f44c4800700 1 -- 172.21.15.6:0/4260404269 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44bc1a9b20 con 0x7f44bc075b00 2024-05-09T04:49:28.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.545+0000 7f44c4800700 1 -- 172.21.15.6:0/4260404269 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44bc0512c0 con 0x7f44bc075b00 2024-05-09T04:49:28.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.548+0000 7f44b6a00700 1 -- 172.21.15.6:0/4260404269 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f44b0010430 con 0x7f44bc075b00 2024-05-09T04:49:28.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.549+0000 7f44b6a00700 1 --2- 172.21.15.6:0/4260404269 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f44a40b2880 0x7f44a40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:28.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.549+0000 7f44b6a00700 1 -- 172.21.15.6:0/4260404269 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f44b00e22f0 con 0x7f44bc075b00 2024-05-09T04:49:28.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.550+0000 7f44c1400700 1 --2- 172.21.15.6:0/4260404269 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f44a40b2880 0x7f44a40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:28.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.550+0000 7f44c1400700 1 --2- 172.21.15.6:0/4260404269 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f44a40b2880 0x7f44a40b4d90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f44bc112b70 tx=0x7f44ac00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:28.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.551+0000 7f44b6a00700 1 -- 172.21.15.6:0/4260404269 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f44b00aa410 con 0x7f44bc075b00 2024-05-09T04:49:28.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:28.788+0000 7f44c4800700 1 -- 172.21.15.6:0/4260404269 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f44bc02e5f0 con 0x7f44bc075b00 2024-05-09T04:49:29.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:29.024+0000 7f44b6a00700 1 -- 172.21.15.6:0/4260404269 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v73) ==== 138+0+0 (secure 0 0 0) 0x7f44b00a9d10 con 0x7f44bc075b00 2024-05-09T04:49:29.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:29.030+0000 7f44c4800700 1 -- 172.21.15.6:0/4260404269 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f44a40b2880 msgr2=0x7f44a40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:29.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:29.030+0000 7f44c4800700 1 --2- 172.21.15.6:0/4260404269 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f44a40b2880 0x7f44a40b4d90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f44bc112b70 tx=0x7f44ac00b410 comp rx=0 tx=0).stop 2024-05-09T04:49:29.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:29.030+0000 7f44c4800700 1 -- 172.21.15.6:0/4260404269 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44bc075b00 msgr2=0x7f44bc19d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:29.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:29.030+0000 7f44c4800700 1 --2- 172.21.15.6:0/4260404269 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44bc075b00 0x7f44bc19d040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f44b000e9a0 tx=0x7f44b000ecb0 comp rx=0 tx=0).stop 2024-05-09T04:49:29.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:29.034+0000 7f44c4800700 1 -- 172.21.15.6:0/4260404269 shutdown_connections 2024-05-09T04:49:29.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:29.034+0000 7f44c4800700 1 --2- 172.21.15.6:0/4260404269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44bc110c60 0x7f44bc1a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:29.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:29.034+0000 7f44c4800700 1 --2- 172.21.15.6:0/4260404269 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44bc075b00 0x7f44bc19d040 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:29.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:29.034+0000 7f44c4800700 1 --2- 172.21.15.6:0/4260404269 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f44a40b2880 0x7f44a40b4d90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:29.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:29.034+0000 7f44c4800700 1 --2- 172.21.15.6:0/4260404269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44bc076450 0x7f44bc19d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:29.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:29.034+0000 7f44c4800700 1 -- 172.21.15.6:0/4260404269 >> 172.21.15.6:0/4260404269 conn(0x7f44bc079f00 msgr2=0x7f44bc101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:29.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:29.036+0000 7f44c4800700 1 -- 172.21.15.6:0/4260404269 shutdown_connections 2024-05-09T04:49:29.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:29.036+0000 7f44c4800700 1 -- 172.21.15.6:0/4260404269 wait complete. 2024-05-09T04:49:29.039 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_metadata' created 2024-05-09T04:49:29.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:29 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/4260404269' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T04:49:29.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:29 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/4260404269' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T04:49:29.429 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:29 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/4260404269' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T04:49:29.885 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-09T04:49:30.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:30 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/4260404269' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-09T04:49:30.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:30 smithi006 ceph-mon[29835]: osdmap e73: 12 total, 12 up, 12 in 2024-05-09T04:49:30.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:30 smithi006 ceph-mon[29835]: pgmap v320: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:30.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:30 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/4260404269' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-09T04:49:30.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:30 smithi186 ceph-mon[34668]: osdmap e73: 12 total, 12 up, 12 in 2024-05-09T04:49:30.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:30 smithi186 ceph-mon[34668]: pgmap v320: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:30.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:30 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/4260404269' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-09T04:49:30.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:30 smithi169 ceph-mon[34075]: osdmap e73: 12 total, 12 up, 12 in 2024-05-09T04:49:30.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:30 smithi169 ceph-mon[34075]: pgmap v320: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:30.367 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:49:31.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:31 smithi186 ceph-mon[34668]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T04:49:31.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:31 smithi186 ceph-mon[34668]: osdmap e74: 12 total, 12 up, 12 in 2024-05-09T04:49:31.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:31 smithi169 ceph-mon[34075]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T04:49:31.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:31 smithi169 ceph-mon[34075]: osdmap e74: 12 total, 12 up, 12 in 2024-05-09T04:49:31.409 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:31 smithi006 ceph-mon[29835]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T04:49:31.409 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:31 smithi006 ceph-mon[29835]: osdmap e74: 12 total, 12 up, 12 in 2024-05-09T04:49:32.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:32 smithi186 ceph-mon[34668]: osdmap e75: 12 total, 12 up, 12 in 2024-05-09T04:49:32.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:32 smithi186 ceph-mon[34668]: pgmap v323: 33 pgs: 3 creating+peering, 12 unknown, 18 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:32.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:32 smithi169 ceph-mon[34075]: osdmap e75: 12 total, 12 up, 12 in 2024-05-09T04:49:32.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:32 smithi169 ceph-mon[34075]: pgmap v323: 33 pgs: 3 creating+peering, 12 unknown, 18 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:32.438 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:32 smithi006 ceph-mon[29835]: osdmap e75: 12 total, 12 up, 12 in 2024-05-09T04:49:32.438 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:32 smithi006 ceph-mon[29835]: pgmap v323: 33 pgs: 3 creating+peering, 12 unknown, 18 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:32.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.700+0000 7f4495e00700 1 -- 172.21.15.6:0/2698180256 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44901045d0 msgr2=0x7f44901049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:32.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.700+0000 7f4495e00700 1 --2- 172.21.15.6:0/2698180256 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44901045d0 0x7f44901049e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f44880099c0 tx=0x7f4488009cd0 comp rx=0 tx=0).stop 2024-05-09T04:49:32.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.701+0000 7f4495e00700 1 -- 172.21.15.6:0/2698180256 shutdown_connections 2024-05-09T04:49:32.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.701+0000 7f4495e00700 1 --2- 172.21.15.6:0/2698180256 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4490109850 0x7f44901111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:32.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.701+0000 7f4495e00700 1 --2- 172.21.15.6:0/2698180256 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44901045d0 0x7f44901049e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:32.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.701+0000 7f4495e00700 1 --2- 172.21.15.6:0/2698180256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4490108f00 0x7f4490109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:32.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.701+0000 7f4495e00700 1 -- 172.21.15.6:0/2698180256 >> 172.21.15.6:0/2698180256 conn(0x7f44900ffea0 msgr2=0x7f44901022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:32.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.701+0000 7f4495e00700 1 -- 172.21.15.6:0/2698180256 shutdown_connections 2024-05-09T04:49:32.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.701+0000 7f4495e00700 1 -- 172.21.15.6:0/2698180256 wait complete. 2024-05-09T04:49:32.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.702+0000 7f4495e00700 1 Processor -- start 2024-05-09T04:49:32.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.702+0000 7f4495e00700 1 -- start start 2024-05-09T04:49:32.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.702+0000 7f4495e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44901045d0 0x7f449019ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:32.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.702+0000 7f4495e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4490108f00 0x7f449019b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:32.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.703+0000 7f4495e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4490109850 0x7f449019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:32.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.703+0000 7f4495e00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4490114480 con 0x7f4490108f00 2024-05-09T04:49:32.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.703+0000 7f4495e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4490114300 con 0x7f44901045d0 2024-05-09T04:49:32.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.703+0000 7f4495e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4490114600 con 0x7f4490109850 2024-05-09T04:49:32.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.703+0000 7f448ea00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44901045d0 0x7f449019ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:32.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.703+0000 7f448ea00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44901045d0 0x7f449019ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:53892/0 (socket says 172.21.15.6:53892) 2024-05-09T04:49:32.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.703+0000 7f448ea00700 1 -- 172.21.15.6:0/2403257583 learned_addr learned my addr 172.21.15.6:0/2403257583 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:49:32.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.703+0000 7f448e000700 1 --2- 172.21.15.6:0/2403257583 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4490108f00 0x7f449019b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:32.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.703+0000 7f448f400700 1 --2- 172.21.15.6:0/2403257583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4490109850 0x7f449019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:32.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.703+0000 7f448ea00700 1 -- 172.21.15.6:0/2403257583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4490109850 msgr2=0x7f449019f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:32.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.703+0000 7f448ea00700 1 --2- 172.21.15.6:0/2403257583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4490109850 0x7f449019f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:32.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.703+0000 7f448ea00700 1 -- 172.21.15.6:0/2403257583 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4490108f00 msgr2=0x7f449019b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:32.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.703+0000 7f448ea00700 1 --2- 172.21.15.6:0/2403257583 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4490108f00 0x7f449019b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:32.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.703+0000 7f448ea00700 1 -- 172.21.15.6:0/2403257583 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f449019fd60 con 0x7f44901045d0 2024-05-09T04:49:32.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.704+0000 7f448f400700 1 --2- 172.21.15.6:0/2403257583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4490109850 0x7f449019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:49:32.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.704+0000 7f448e000700 1 --2- 172.21.15.6:0/2403257583 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4490108f00 0x7f449019b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:49:32.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.704+0000 7f448ea00700 1 --2- 172.21.15.6:0/2403257583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44901045d0 0x7f449019ae90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4488006010 tx=0x7f4488005280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:32.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.704+0000 7f4494a00700 1 -- 172.21.15.6:0/2403257583 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4488026070 con 0x7f44901045d0 2024-05-09T04:49:32.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.704+0000 7f4494a00700 1 -- 172.21.15.6:0/2403257583 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4488005680 con 0x7f44901045d0 2024-05-09T04:49:32.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.704+0000 7f4494a00700 1 -- 172.21.15.6:0/2403257583 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448800fb20 con 0x7f44901045d0 2024-05-09T04:49:32.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.704+0000 7f4495e00700 1 -- 172.21.15.6:0/2403257583 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f449019fff0 con 0x7f44901045d0 2024-05-09T04:49:32.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.705+0000 7f4495e00700 1 -- 172.21.15.6:0/2403257583 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44901a04d0 con 0x7f44901045d0 2024-05-09T04:49:32.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.706+0000 7f4495e00700 1 -- 172.21.15.6:0/2403257583 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44900512c0 con 0x7f44901045d0 2024-05-09T04:49:32.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.708+0000 7f4494a00700 1 -- 172.21.15.6:0/2403257583 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f4488029070 con 0x7f44901045d0 2024-05-09T04:49:32.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.710+0000 7f4494a00700 1 --2- 172.21.15.6:0/2403257583 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f44700b2880 0x7f44700b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:32.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.710+0000 7f4494a00700 1 -- 172.21.15.6:0/2403257583 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7497+0+0 (secure 0 0 0) 0x7f44880e2e70 con 0x7f44901045d0 2024-05-09T04:49:32.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.710+0000 7f448e000700 1 --2- 172.21.15.6:0/2403257583 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f44700b2880 0x7f44700b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:32.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.711+0000 7f448e000700 1 --2- 172.21.15.6:0/2403257583 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f44700b2880 0x7f44700b4d90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f44901034b0 tx=0x7f448000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:32.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.712+0000 7f4494a00700 1 -- 172.21.15.6:0/2403257583 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4488030080 con 0x7f44901045d0 2024-05-09T04:49:32.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:32.951+0000 7f4495e00700 1 -- 172.21.15.6:0/2403257583 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f449002e5f0 con 0x7f44901045d0 2024-05-09T04:49:33.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:33.099+0000 7f4494a00700 1 -- 172.21.15.6:0/2403257583 <== mon.2 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v76) ==== 171+0+0 (secure 0 0 0) 0x7f4488013070 con 0x7f44901045d0 2024-05-09T04:49:33.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:33.108+0000 7f4495e00700 1 -- 172.21.15.6:0/2403257583 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f44700b2880 msgr2=0x7f44700b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:33.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:33.108+0000 7f4495e00700 1 --2- 172.21.15.6:0/2403257583 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f44700b2880 0x7f44700b4d90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f44901034b0 tx=0x7f448000b040 comp rx=0 tx=0).stop 2024-05-09T04:49:33.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:33.108+0000 7f4495e00700 1 -- 172.21.15.6:0/2403257583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44901045d0 msgr2=0x7f449019ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:33.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:33.108+0000 7f4495e00700 1 --2- 172.21.15.6:0/2403257583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44901045d0 0x7f449019ae90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4488006010 tx=0x7f4488005280 comp rx=0 tx=0).stop 2024-05-09T04:49:33.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:33.109+0000 7f4495e00700 1 -- 172.21.15.6:0/2403257583 shutdown_connections 2024-05-09T04:49:33.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:33.109+0000 7f4495e00700 1 --2- 172.21.15.6:0/2403257583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4490109850 0x7f449019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:33.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:33.109+0000 7f4495e00700 1 --2- 172.21.15.6:0/2403257583 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f44700b2880 0x7f44700b4d90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:33.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:33.109+0000 7f4495e00700 1 --2- 172.21.15.6:0/2403257583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44901045d0 0x7f449019ae90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:33.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:33.109+0000 7f4495e00700 1 --2- 172.21.15.6:0/2403257583 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4490108f00 0x7f449019b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:33.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:33.109+0000 7f4495e00700 1 -- 172.21.15.6:0/2403257583 >> 172.21.15.6:0/2403257583 conn(0x7f44900ffea0 msgr2=0x7f449010a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:33.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:33.110+0000 7f4495e00700 1 -- 172.21.15.6:0/2403257583 shutdown_connections 2024-05-09T04:49:33.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:33.110+0000 7f4495e00700 1 -- 172.21.15.6:0/2403257583 wait complete. 2024-05-09T04:49:33.114 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_data' created 2024-05-09T04:49:33.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:33 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/2403257583' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-09T04:49:33.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:33 smithi186 ceph-mon[34668]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-09T04:49:33.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:33 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/2403257583' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-09T04:49:33.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:33 smithi169 ceph-mon[34075]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-09T04:49:33.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:33 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/2403257583' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-09T04:49:33.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:33 smithi006 ceph-mon[29835]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-09T04:49:33.886 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-09T04:49:34.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:34 smithi186 ceph-mon[34668]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-09T04:49:34.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:34 smithi186 ceph-mon[34668]: osdmap e76: 12 total, 12 up, 12 in 2024-05-09T04:49:34.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:34 smithi186 ceph-mon[34668]: pgmap v325: 97 pgs: 3 creating+peering, 64 unknown, 30 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:34.356 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:49:34.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:34 smithi169 ceph-mon[34075]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-09T04:49:34.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:34 smithi169 ceph-mon[34075]: osdmap e76: 12 total, 12 up, 12 in 2024-05-09T04:49:34.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:34 smithi169 ceph-mon[34075]: pgmap v325: 97 pgs: 3 creating+peering, 64 unknown, 30 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:34.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:34 smithi006 ceph-mon[29835]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-09T04:49:34.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:34 smithi006 ceph-mon[29835]: osdmap e76: 12 total, 12 up, 12 in 2024-05-09T04:49:34.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:34 smithi006 ceph-mon[29835]: pgmap v325: 97 pgs: 3 creating+peering, 64 unknown, 30 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:35.375 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:35 smithi006 ceph-mon[29835]: osdmap e77: 12 total, 12 up, 12 in 2024-05-09T04:49:35.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:35 smithi186 ceph-mon[34668]: osdmap e77: 12 total, 12 up, 12 in 2024-05-09T04:49:35.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:35 smithi169 ceph-mon[34075]: osdmap e77: 12 total, 12 up, 12 in 2024-05-09T04:49:36.360 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:36 smithi006 ceph-mon[29835]: osdmap e78: 12 total, 12 up, 12 in 2024-05-09T04:49:36.360 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:36 smithi006 ceph-mon[29835]: pgmap v328: 97 pgs: 52 unknown, 45 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:36.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:36 smithi186 ceph-mon[34668]: osdmap e78: 12 total, 12 up, 12 in 2024-05-09T04:49:36.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:36 smithi186 ceph-mon[34668]: pgmap v328: 97 pgs: 52 unknown, 45 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:36.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:36 smithi169 ceph-mon[34075]: osdmap e78: 12 total, 12 up, 12 in 2024-05-09T04:49:36.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:36 smithi169 ceph-mon[34075]: pgmap v328: 97 pgs: 52 unknown, 45 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:36.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.616+0000 7f67f4c00700 1 -- 172.21.15.6:0/2834811002 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67f010b2e0 msgr2=0x7f67f0117810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:36.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.616+0000 7f67f4c00700 1 --2- 172.21.15.6:0/2834811002 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67f010b2e0 0x7f67f0117810 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f67e800b0a0 tx=0x7f67e800b3b0 comp rx=0 tx=0).stop 2024-05-09T04:49:36.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.616+0000 7f67f4c00700 1 -- 172.21.15.6:0/2834811002 shutdown_connections 2024-05-09T04:49:36.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.616+0000 7f67f4c00700 1 --2- 172.21.15.6:0/2834811002 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67f010b2e0 0x7f67f0117810 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:36.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.616+0000 7f67f4c00700 1 --2- 172.21.15.6:0/2834811002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67f0110970 0x7f67f0110d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:36.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.616+0000 7f67f4c00700 1 --2- 172.21.15.6:0/2834811002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67f010a990 0x7f67f010ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:36.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.616+0000 7f67f4c00700 1 -- 172.21.15.6:0/2834811002 >> 172.21.15.6:0/2834811002 conn(0x7f67f0077c80 msgr2=0x7f67f00780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:36.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.616+0000 7f67f4c00700 1 -- 172.21.15.6:0/2834811002 shutdown_connections 2024-05-09T04:49:36.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.616+0000 7f67f4c00700 1 -- 172.21.15.6:0/2834811002 wait complete. 2024-05-09T04:49:36.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.618+0000 7f67f4c00700 1 Processor -- start 2024-05-09T04:49:36.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.618+0000 7f67f4c00700 1 -- start start 2024-05-09T04:49:36.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.618+0000 7f67f4c00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67f010a990 0x7f67f01a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:36.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.619+0000 7f67f4c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67f010b2e0 0x7f67f01a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:36.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.619+0000 7f67f4c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67f0110970 0x7f67f01a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:36.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.619+0000 7f67f4c00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f67f011aae0 con 0x7f67f0110970 2024-05-09T04:49:36.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.619+0000 7f67f4c00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f67f011a960 con 0x7f67f010a990 2024-05-09T04:49:36.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.619+0000 7f67f4c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f67f011ac60 con 0x7f67f010b2e0 2024-05-09T04:49:36.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.619+0000 7f67ede00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67f010a990 0x7f67f01a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:36.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.619+0000 7f67ede00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67f010a990 0x7f67f01a1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:53910/0 (socket says 172.21.15.6:53910) 2024-05-09T04:49:36.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.619+0000 7f67ede00700 1 -- 172.21.15.6:0/2135893798 learned_addr learned my addr 172.21.15.6:0/2135893798 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:49:36.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.619+0000 7f67ee800700 1 --2- 172.21.15.6:0/2135893798 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67f0110970 0x7f67f01a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:36.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.619+0000 7f67ed400700 1 --2- 172.21.15.6:0/2135893798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67f010b2e0 0x7f67f01a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:36.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.619+0000 7f67ede00700 1 -- 172.21.15.6:0/2135893798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67f010b2e0 msgr2=0x7f67f01a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:36.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.619+0000 7f67ede00700 1 --2- 172.21.15.6:0/2135893798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67f010b2e0 0x7f67f01a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:36.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.619+0000 7f67ede00700 1 -- 172.21.15.6:0/2135893798 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67f0110970 msgr2=0x7f67f01a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:36.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.619+0000 7f67ede00700 1 --2- 172.21.15.6:0/2135893798 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67f0110970 0x7f67f01a5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:36.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.619+0000 7f67ede00700 1 -- 172.21.15.6:0/2135893798 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f01a62e0 con 0x7f67f010a990 2024-05-09T04:49:36.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.620+0000 7f67ede00700 1 --2- 172.21.15.6:0/2135893798 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67f010a990 0x7f67f01a1410 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f67e000ea30 tx=0x7f67e000ed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:36.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.620+0000 7f67e6a00700 1 -- 172.21.15.6:0/2135893798 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e000cbd0 con 0x7f67f010a990 2024-05-09T04:49:36.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.620+0000 7f67f4c00700 1 -- 172.21.15.6:0/2135893798 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67f01a65d0 con 0x7f67f010a990 2024-05-09T04:49:36.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.620+0000 7f67e6a00700 1 -- 172.21.15.6:0/2135893798 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67e00044d0 con 0x7f67f010a990 2024-05-09T04:49:36.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.620+0000 7f67e6a00700 1 -- 172.21.15.6:0/2135893798 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e0003dc0 con 0x7f67f010a990 2024-05-09T04:49:36.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.620+0000 7f67f4c00700 1 -- 172.21.15.6:0/2135893798 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67f01adef0 con 0x7f67f010a990 2024-05-09T04:49:36.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.622+0000 7f67f4c00700 1 -- 172.21.15.6:0/2135893798 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67f00512c0 con 0x7f67f010a990 2024-05-09T04:49:36.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.624+0000 7f67e6a00700 1 -- 172.21.15.6:0/2135893798 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f67e0004000 con 0x7f67f010a990 2024-05-09T04:49:36.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.625+0000 7f67e6a00700 1 --2- 172.21.15.6:0/2135893798 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f67d40b2900 0x7f67d40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:36.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.626+0000 7f67e6a00700 1 -- 172.21.15.6:0/2135893798 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7879+0+0 (secure 0 0 0) 0x7f67e00e20e0 con 0x7f67f010a990 2024-05-09T04:49:36.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.626+0000 7f67ed400700 1 --2- 172.21.15.6:0/2135893798 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f67d40b2900 0x7f67d40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:36.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.627+0000 7f67ed400700 1 --2- 172.21.15.6:0/2135893798 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f67d40b2900 0x7f67d40b4e10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f67f01097a0 tx=0x7f67dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:36.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.630+0000 7f67e6a00700 1 -- 172.21.15.6:0/2135893798 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f67e00042f0 con 0x7f67f010a990 2024-05-09T04:49:36.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:36.918+0000 7f67f4c00700 1 -- 172.21.15.6:0/2135893798 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f67f002e5f0 con 0x7f67f010a990 2024-05-09T04:49:37.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:37.159+0000 7f67e6a00700 1 -- 172.21.15.6:0/2135893798 <== mon.2 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f67e00a9f10 con 0x7f67f010a990 2024-05-09T04:49:37.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:37.166+0000 7f67f4c00700 1 -- 172.21.15.6:0/2135893798 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f67d40b2900 msgr2=0x7f67d40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:37.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:37.166+0000 7f67f4c00700 1 --2- 172.21.15.6:0/2135893798 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f67d40b2900 0x7f67d40b4e10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f67f01097a0 tx=0x7f67dc009260 comp rx=0 tx=0).stop 2024-05-09T04:49:37.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:37.166+0000 7f67f4c00700 1 -- 172.21.15.6:0/2135893798 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67f010a990 msgr2=0x7f67f01a1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:37.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:37.166+0000 7f67f4c00700 1 --2- 172.21.15.6:0/2135893798 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67f010a990 0x7f67f01a1410 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f67e000ea30 tx=0x7f67e000ed40 comp rx=0 tx=0).stop 2024-05-09T04:49:37.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:37.166+0000 7f67f4c00700 1 -- 172.21.15.6:0/2135893798 shutdown_connections 2024-05-09T04:49:37.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:37.166+0000 7f67f4c00700 1 --2- 172.21.15.6:0/2135893798 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67f0110970 0x7f67f01a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:37.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:37.166+0000 7f67f4c00700 1 --2- 172.21.15.6:0/2135893798 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f67d40b2900 0x7f67d40b4e10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:37.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:37.166+0000 7f67f4c00700 1 --2- 172.21.15.6:0/2135893798 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67f010a990 0x7f67f01a1410 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:37.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:37.166+0000 7f67f4c00700 1 --2- 172.21.15.6:0/2135893798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67f010b2e0 0x7f67f01a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:37.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:37.166+0000 7f67f4c00700 1 -- 172.21.15.6:0/2135893798 >> 172.21.15.6:0/2135893798 conn(0x7f67f0077c80 msgr2=0x7f67f007b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:37.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:37.167+0000 7f67f4c00700 1 -- 172.21.15.6:0/2135893798 shutdown_connections 2024-05-09T04:49:37.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:37.169+0000 7f67f4c00700 1 -- 172.21.15.6:0/2135893798 wait complete. 2024-05-09T04:49:37.170 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:37 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/2135893798' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T04:49:37.170 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:37 smithi006 ceph-mon[29835]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T04:49:37.170 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:37 smithi006 ceph-mon[29835]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-09T04:49:37.170 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:37 smithi006 ceph-mon[29835]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T04:49:37.173 INFO:teuthology.orchestra.run.smithi006.stderr:new fs with metadata pool 2 and data pool 3 2024-05-09T04:49:37.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:37 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/2135893798' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T04:49:37.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:37 smithi186 ceph-mon[34668]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T04:49:37.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:37 smithi186 ceph-mon[34668]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-09T04:49:37.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:37 smithi186 ceph-mon[34668]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T04:49:37.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:37 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/2135893798' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T04:49:37.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:37 smithi169 ceph-mon[34075]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T04:49:37.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:37 smithi169 ceph-mon[34075]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-09T04:49:37.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:37 smithi169 ceph-mon[34075]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T04:49:37.972 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-09T04:49:37.973 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-09T04:49:38.443 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:49:38.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:38 smithi006 ceph-mon[29835]: osdmap e79: 12 total, 12 up, 12 in 2024-05-09T04:49:38.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:38 smithi006 ceph-mon[29835]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-09T04:49:38.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:38 smithi006 ceph-mon[29835]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-09T04:49:38.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:38 smithi006 ceph-mon[29835]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-09T04:49:38.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:38 smithi006 ceph-mon[29835]: pgmap v330: 97 pgs: 22 unknown, 75 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:38.501 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:38 smithi186 ceph-mon[34668]: osdmap e79: 12 total, 12 up, 12 in 2024-05-09T04:49:38.501 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:38 smithi186 ceph-mon[34668]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-09T04:49:38.501 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:38 smithi186 ceph-mon[34668]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-09T04:49:38.501 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:38 smithi186 ceph-mon[34668]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-09T04:49:38.501 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:38 smithi186 ceph-mon[34668]: pgmap v330: 97 pgs: 22 unknown, 75 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:38.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:38 smithi169 ceph-mon[34075]: osdmap e79: 12 total, 12 up, 12 in 2024-05-09T04:49:38.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:38 smithi169 ceph-mon[34075]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-09T04:49:38.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:38 smithi169 ceph-mon[34075]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-09T04:49:38.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:38 smithi169 ceph-mon[34075]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-09T04:49:38.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:38 smithi169 ceph-mon[34075]: pgmap v330: 97 pgs: 22 unknown, 75 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:39.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:39 smithi006 ceph-mon[29835]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-09T04:49:39.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:39 smithi006 ceph-mon[29835]: Cluster is now healthy 2024-05-09T04:49:39.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:39 smithi006 ceph-mon[29835]: mds.? [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] up:active 2024-05-09T04:49:39.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:39 smithi006 ceph-mon[29835]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T04:49:39.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:39 smithi006 ceph-mon[29835]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T04:49:39.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:39 smithi186 ceph-mon[34668]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-09T04:49:39.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:39 smithi186 ceph-mon[34668]: Cluster is now healthy 2024-05-09T04:49:39.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:39 smithi186 ceph-mon[34668]: mds.? [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] up:active 2024-05-09T04:49:39.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:39 smithi186 ceph-mon[34668]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T04:49:39.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:39 smithi186 ceph-mon[34668]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T04:49:39.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:39 smithi169 ceph-mon[34075]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-09T04:49:39.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:39 smithi169 ceph-mon[34075]: Cluster is now healthy 2024-05-09T04:49:39.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:39 smithi169 ceph-mon[34075]: mds.? [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] up:active 2024-05-09T04:49:39.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:39 smithi169 ceph-mon[34075]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T04:49:39.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:39 smithi169 ceph-mon[34075]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T04:49:40.331 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:40 smithi006 ceph-mon[29835]: pgmap v331: 97 pgs: 97 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 1 op/s 2024-05-09T04:49:40.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:40 smithi186 ceph-mon[34668]: pgmap v331: 97 pgs: 97 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 1 op/s 2024-05-09T04:49:40.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:40 smithi169 ceph-mon[34075]: pgmap v331: 97 pgs: 97 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 1 op/s 2024-05-09T04:49:41.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.017+0000 7f5c63600700 1 -- 172.21.15.6:0/2901547275 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c5c110970 msgr2=0x7f5c5c110d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:41.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.017+0000 7f5c63600700 1 --2- 172.21.15.6:0/2901547275 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c5c110970 0x7f5c5c110d80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009990 tx=0x7f5c50009ca0 comp rx=0 tx=0).stop 2024-05-09T04:49:41.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.017+0000 7f5c63600700 1 -- 172.21.15.6:0/2901547275 shutdown_connections 2024-05-09T04:49:41.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.017+0000 7f5c63600700 1 --2- 172.21.15.6:0/2901547275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c5c10b2e0 0x7f5c5c117810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:41.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.017+0000 7f5c63600700 1 --2- 172.21.15.6:0/2901547275 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c5c110970 0x7f5c5c110d80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:41.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.017+0000 7f5c63600700 1 --2- 172.21.15.6:0/2901547275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c5c10a990 0x7f5c5c10ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:41.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.017+0000 7f5c63600700 1 -- 172.21.15.6:0/2901547275 >> 172.21.15.6:0/2901547275 conn(0x7f5c5c077c80 msgr2=0x7f5c5c0780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:41.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.018+0000 7f5c63600700 1 -- 172.21.15.6:0/2901547275 shutdown_connections 2024-05-09T04:49:41.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.018+0000 7f5c63600700 1 -- 172.21.15.6:0/2901547275 wait complete. 2024-05-09T04:49:41.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.019+0000 7f5c63600700 1 Processor -- start 2024-05-09T04:49:41.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.019+0000 7f5c63600700 1 -- start start 2024-05-09T04:49:41.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.020+0000 7f5c63600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c5c10a990 0x7f5c5c1a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:41.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.020+0000 7f5c63600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c5c10b2e0 0x7f5c5c1a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:41.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.020+0000 7f5c63600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c5c110970 0x7f5c5c1a5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:41.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.020+0000 7f5c63600700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c11aad0 con 0x7f5c5c10a990 2024-05-09T04:49:41.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.020+0000 7f5c63600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c11a950 con 0x7f5c5c10b2e0 2024-05-09T04:49:41.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.020+0000 7f5c63600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c11ac50 con 0x7f5c5c110970 2024-05-09T04:49:41.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.020+0000 7f5c60c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c5c10a990 0x7f5c5c1a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:41.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.020+0000 7f5c60c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c5c10a990 0x7f5c5c1a14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58162/0 (socket says 172.21.15.6:58162) 2024-05-09T04:49:41.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.020+0000 7f5c60c00700 1 -- 172.21.15.6:0/2698132487 learned_addr learned my addr 172.21.15.6:0/2698132487 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:49:41.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.020+0000 7f5c5be00700 1 --2- 172.21.15.6:0/2698132487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c5c10b2e0 0x7f5c5c1a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:41.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.020+0000 7f5c61600700 1 --2- 172.21.15.6:0/2698132487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c5c110970 0x7f5c5c1a5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:41.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.020+0000 7f5c60c00700 1 -- 172.21.15.6:0/2698132487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c5c110970 msgr2=0x7f5c5c1a5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:41.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.020+0000 7f5c60c00700 1 --2- 172.21.15.6:0/2698132487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c5c110970 0x7f5c5c1a5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:41.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.021+0000 7f5c60c00700 1 -- 172.21.15.6:0/2698132487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c5c10b2e0 msgr2=0x7f5c5c1a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:41.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.021+0000 7f5c60c00700 1 --2- 172.21.15.6:0/2698132487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c5c10b2e0 0x7f5c5c1a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:41.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.021+0000 7f5c60c00700 1 -- 172.21.15.6:0/2698132487 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c5c1a6370 con 0x7f5c5c10a990 2024-05-09T04:49:41.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.021+0000 7f5c60c00700 1 --2- 172.21.15.6:0/2698132487 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c5c10a990 0x7f5c5c1a14a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5c50000c00 tx=0x7f5c500047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:41.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.021+0000 7f5c59600700 1 -- 172.21.15.6:0/2698132487 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50026070 con 0x7f5c5c10a990 2024-05-09T04:49:41.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.021+0000 7f5c59600700 1 -- 172.21.15.6:0/2698132487 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c50021410 con 0x7f5c5c10a990 2024-05-09T04:49:41.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.021+0000 7f5c63600700 1 -- 172.21.15.6:0/2698132487 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c5c1a6600 con 0x7f5c5c10a990 2024-05-09T04:49:41.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.021+0000 7f5c59600700 1 -- 172.21.15.6:0/2698132487 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50017650 con 0x7f5c5c10a990 2024-05-09T04:49:41.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.022+0000 7f5c63600700 1 -- 172.21.15.6:0/2698132487 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c5c1adf80 con 0x7f5c5c10a990 2024-05-09T04:49:41.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.023+0000 7f5c63600700 1 -- 172.21.15.6:0/2698132487 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c5c10c080 con 0x7f5c5c10a990 2024-05-09T04:49:41.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.027+0000 7f5c59600700 1 -- 172.21.15.6:0/2698132487 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f5c50029070 con 0x7f5c5c10a990 2024-05-09T04:49:41.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.029+0000 7f5c59600700 1 --2- 172.21.15.6:0/2698132487 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f5c440b2900 0x7f5c440b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:41.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.029+0000 7f5c59600700 1 -- 172.21.15.6:0/2698132487 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f5c500e3100 con 0x7f5c5c10a990 2024-05-09T04:49:41.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.029+0000 7f5c5be00700 1 --2- 172.21.15.6:0/2698132487 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f5c440b2900 0x7f5c440b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:41.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.030+0000 7f5c5be00700 1 --2- 172.21.15.6:0/2698132487 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f5c440b2900 0x7f5c440b4e10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c06b930 tx=0x7f5c48009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:41.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.032+0000 7f5c59600700 1 -- 172.21.15.6:0/2698132487 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5c50013070 con 0x7f5c5c10a990 2024-05-09T04:49:41.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.283+0000 7f5c63600700 1 -- 172.21.15.6:0/2698132487 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f5c5c1a2800 con 0x7f5c5c10a990 2024-05-09T04:49:41.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.338+0000 7f5c59600700 1 -- 172.21.15.6:0/2698132487 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v80) ==== 157+0+0 (secure 0 0 0) 0x7f5c500aaf60 con 0x7f5c5c10a990 2024-05-09T04:49:41.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.342+0000 7f5c63600700 1 -- 172.21.15.6:0/2698132487 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f5c440b2900 msgr2=0x7f5c440b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:41.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.342+0000 7f5c63600700 1 --2- 172.21.15.6:0/2698132487 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f5c440b2900 0x7f5c440b4e10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c06b930 tx=0x7f5c48009260 comp rx=0 tx=0).stop 2024-05-09T04:49:41.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.342+0000 7f5c63600700 1 -- 172.21.15.6:0/2698132487 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c5c10a990 msgr2=0x7f5c5c1a14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:41.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.342+0000 7f5c63600700 1 --2- 172.21.15.6:0/2698132487 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c5c10a990 0x7f5c5c1a14a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5c50000c00 tx=0x7f5c500047b0 comp rx=0 tx=0).stop 2024-05-09T04:49:41.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.343+0000 7f5c63600700 1 -- 172.21.15.6:0/2698132487 shutdown_connections 2024-05-09T04:49:41.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.343+0000 7f5c63600700 1 --2- 172.21.15.6:0/2698132487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c5c110970 0x7f5c5c1a5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:41.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.343+0000 7f5c63600700 1 --2- 172.21.15.6:0/2698132487 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c5c10a990 0x7f5c5c1a14a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:41.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.343+0000 7f5c63600700 1 --2- 172.21.15.6:0/2698132487 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f5c440b2900 0x7f5c440b4e10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:41.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.343+0000 7f5c63600700 1 --2- 172.21.15.6:0/2698132487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c5c10b2e0 0x7f5c5c1a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:41.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.343+0000 7f5c63600700 1 -- 172.21.15.6:0/2698132487 >> 172.21.15.6:0/2698132487 conn(0x7f5c5c077c80 msgr2=0x7f5c5c07ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:41.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.344+0000 7f5c63600700 1 -- 172.21.15.6:0/2698132487 shutdown_connections 2024-05-09T04:49:41.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:41.345+0000 7f5c63600700 1 -- 172.21.15.6:0/2698132487 wait complete. 2024-05-09T04:49:41.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:41 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/2698132487' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-09T04:49:41.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:41 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/2698132487' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-09T04:49:41.617 INFO:journalctl@ceph.mds.b.smithi169.stdout:May 09 04:49:41 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-b[64939]: 2024-05-09T04:49:41.596+0000 7f0e0841d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T04:49:41.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:41 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/2698132487' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-09T04:49:42.199 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-09T04:49:42.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:42 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/2698132487' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-09T04:49:42.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:42 smithi006 ceph-mon[29835]: osdmap e80: 12 total, 12 up, 12 in 2024-05-09T04:49:42.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:42 smithi006 ceph-mon[29835]: pgmap v333: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 6 op/s 2024-05-09T04:49:42.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:42 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/2698132487' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-09T04:49:42.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:42 smithi169 ceph-mon[34075]: osdmap e80: 12 total, 12 up, 12 in 2024-05-09T04:49:42.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:42 smithi169 ceph-mon[34075]: pgmap v333: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 6 op/s 2024-05-09T04:49:42.679 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:49:42.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:42 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/2698132487' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-09T04:49:42.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:42 smithi186 ceph-mon[34668]: osdmap e80: 12 total, 12 up, 12 in 2024-05-09T04:49:42.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:42 smithi186 ceph-mon[34668]: pgmap v333: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 6 op/s 2024-05-09T04:49:44.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:44 smithi006 ceph-mon[29835]: pgmap v334: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-09T04:49:44.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:44 smithi186 ceph-mon[34668]: pgmap v334: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-09T04:49:44.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:44 smithi169 ceph-mon[34075]: pgmap v334: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-09T04:49:45.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.084+0000 7f2e5e800700 1 -- 172.21.15.6:0/3224028182 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e58108d30 msgr2=0x7f2e58109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:45.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.084+0000 7f2e5e800700 1 --2- 172.21.15.6:0/3224028182 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e58108d30 0x7f2e58109140 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2e440099c0 tx=0x7f2e44009cd0 comp rx=0 tx=0).stop 2024-05-09T04:49:45.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.084+0000 7f2e5e800700 1 -- 172.21.15.6:0/3224028182 shutdown_connections 2024-05-09T04:49:45.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.084+0000 7f2e5e800700 1 --2- 172.21.15.6:0/3224028182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e58109680 0x7f2e58110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:45.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.084+0000 7f2e5e800700 1 --2- 172.21.15.6:0/3224028182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e58104400 0x7f2e58104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:45.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.084+0000 7f2e5e800700 1 --2- 172.21.15.6:0/3224028182 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e58108d30 0x7f2e58109140 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:45.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.084+0000 7f2e5e800700 1 -- 172.21.15.6:0/3224028182 >> 172.21.15.6:0/3224028182 conn(0x7f2e58079d10 msgr2=0x7f2e5807a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:45.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.084+0000 7f2e5e800700 1 -- 172.21.15.6:0/3224028182 shutdown_connections 2024-05-09T04:49:45.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.084+0000 7f2e5e800700 1 -- 172.21.15.6:0/3224028182 wait complete. 2024-05-09T04:49:45.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.086+0000 7f2e5e800700 1 Processor -- start 2024-05-09T04:49:45.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.086+0000 7f2e5e800700 1 -- start start 2024-05-09T04:49:45.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.086+0000 7f2e5e800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e58104400 0x7f2e5819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:45.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.086+0000 7f2e5e800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e58108d30 0x7f2e5819d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:45.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.086+0000 7f2e5e800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e58109680 0x7f2e581a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:45.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.086+0000 7f2e5e800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2e58114290 con 0x7f2e58108d30 2024-05-09T04:49:45.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.086+0000 7f2e5e800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2e58114110 con 0x7f2e58109680 2024-05-09T04:49:45.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.086+0000 7f2e5e800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2e58114410 con 0x7f2e58104400 2024-05-09T04:49:45.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.087+0000 7f2e5ca00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e58108d30 0x7f2e5819d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:45.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.087+0000 7f2e5ca00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e58108d30 0x7f2e5819d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58176/0 (socket says 172.21.15.6:58176) 2024-05-09T04:49:45.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.087+0000 7f2e5ca00700 1 -- 172.21.15.6:0/2536467351 learned_addr learned my addr 172.21.15.6:0/2536467351 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:49:45.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.087+0000 7f2e5d400700 1 --2- 172.21.15.6:0/2536467351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e58104400 0x7f2e5819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:45.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.087+0000 7f2e5de00700 1 --2- 172.21.15.6:0/2536467351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e58109680 0x7f2e581a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:45.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.087+0000 7f2e5ca00700 1 -- 172.21.15.6:0/2536467351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e58104400 msgr2=0x7f2e5819ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:45.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.087+0000 7f2e5ca00700 1 --2- 172.21.15.6:0/2536467351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e58104400 0x7f2e5819ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:45.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.087+0000 7f2e5ca00700 1 -- 172.21.15.6:0/2536467351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e58109680 msgr2=0x7f2e581a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:45.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.087+0000 7f2e5ca00700 1 --2- 172.21.15.6:0/2536467351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e58109680 0x7f2e581a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:45.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.087+0000 7f2e5ca00700 1 -- 172.21.15.6:0/2536467351 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e581a1cf0 con 0x7f2e58108d30 2024-05-09T04:49:45.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.088+0000 7f2e5ca00700 1 --2- 172.21.15.6:0/2536467351 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e58108d30 0x7f2e5819d3c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2e44006010 tx=0x7f2e44005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:45.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.088+0000 7f2e56000700 1 -- 172.21.15.6:0/2536467351 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e44026070 con 0x7f2e58108d30 2024-05-09T04:49:45.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.088+0000 7f2e5e800700 1 -- 172.21.15.6:0/2536467351 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e581a1f80 con 0x7f2e58108d30 2024-05-09T04:49:45.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.088+0000 7f2e5e800700 1 -- 172.21.15.6:0/2536467351 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e581a9900 con 0x7f2e58108d30 2024-05-09T04:49:45.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.088+0000 7f2e56000700 1 -- 172.21.15.6:0/2536467351 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e4400b790 con 0x7f2e58108d30 2024-05-09T04:49:45.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.088+0000 7f2e56000700 1 -- 172.21.15.6:0/2536467351 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e44030060 con 0x7f2e58108d30 2024-05-09T04:49:45.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.089+0000 7f2e5e800700 1 -- 172.21.15.6:0/2536467351 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e38005330 con 0x7f2e58108d30 2024-05-09T04:49:45.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.093+0000 7f2e56000700 1 -- 172.21.15.6:0/2536467351 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f2e44029070 con 0x7f2e58108d30 2024-05-09T04:49:45.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.094+0000 7f2e56000700 1 --2- 172.21.15.6:0/2536467351 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f2e400b2880 0x7f2e400b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:45.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.095+0000 7f2e56000700 1 -- 172.21.15.6:0/2536467351 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8277+0+0 (secure 0 0 0) 0x7f2e440e2a30 con 0x7f2e58108d30 2024-05-09T04:49:45.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.095+0000 7f2e5d400700 1 --2- 172.21.15.6:0/2536467351 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f2e400b2880 0x7f2e400b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:45.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.096+0000 7f2e5d400700 1 --2- 172.21.15.6:0/2536467351 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f2e400b2880 0x7f2e400b4d90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c009c60 tx=0x7f2e4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:45.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.102+0000 7f2e56000700 1 -- 172.21.15.6:0/2536467351 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2e44013070 con 0x7f2e58108d30 2024-05-09T04:49:45.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:45.341+0000 7f2e5e800700 1 -- 172.21.15.6:0/2536467351 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2e38005620 con 0x7f2e58108d30 2024-05-09T04:49:45.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:45 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/2536467351' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-09T04:49:45.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:45 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/2536467351' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-09T04:49:45.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:45 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/2536467351' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-09T04:49:46.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:46.621+0000 7f2e56000700 1 -- 172.21.15.6:0/2536467351 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v82) ==== 228+0+0 (secure 0 0 0) 0x7f2e440aa750 con 0x7f2e58108d30 2024-05-09T04:49:46.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:46.626+0000 7f2e5e800700 1 -- 172.21.15.6:0/2536467351 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f2e400b2880 msgr2=0x7f2e400b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:46.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:46.627+0000 7f2e5e800700 1 --2- 172.21.15.6:0/2536467351 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f2e400b2880 0x7f2e400b4d90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c009c60 tx=0x7f2e4c009260 comp rx=0 tx=0).stop 2024-05-09T04:49:46.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:46.627+0000 7f2e5e800700 1 -- 172.21.15.6:0/2536467351 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e58108d30 msgr2=0x7f2e5819d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:46.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:46.627+0000 7f2e5e800700 1 --2- 172.21.15.6:0/2536467351 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e58108d30 0x7f2e5819d3c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2e44006010 tx=0x7f2e44005280 comp rx=0 tx=0).stop 2024-05-09T04:49:46.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:46.627+0000 7f2e5e800700 1 -- 172.21.15.6:0/2536467351 shutdown_connections 2024-05-09T04:49:46.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:46.627+0000 7f2e5e800700 1 --2- 172.21.15.6:0/2536467351 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f2e400b2880 0x7f2e400b4d90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:46.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:46.627+0000 7f2e5e800700 1 --2- 172.21.15.6:0/2536467351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e58109680 0x7f2e581a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:46.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:46.627+0000 7f2e5e800700 1 --2- 172.21.15.6:0/2536467351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e58104400 0x7f2e5819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:46.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:46.627+0000 7f2e5e800700 1 --2- 172.21.15.6:0/2536467351 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e58108d30 0x7f2e5819d3c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:46.633 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:46.627+0000 7f2e5e800700 1 -- 172.21.15.6:0/2536467351 >> 172.21.15.6:0/2536467351 conn(0x7f2e58079d10 msgr2=0x7f2e58101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:46.633 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:46.628+0000 7f2e5e800700 1 -- 172.21.15.6:0/2536467351 shutdown_connections 2024-05-09T04:49:46.633 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:46.629+0000 7f2e5e800700 1 -- 172.21.15.6:0/2536467351 wait complete. 2024-05-09T04:49:46.633 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_data_ec' created 2024-05-09T04:49:46.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:46 smithi169 ceph-mon[34075]: pgmap v335: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-09T04:49:46.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:46 smithi169 ceph-mon[34075]: osdmap e81: 12 total, 12 up, 12 in 2024-05-09T04:49:46.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:46 smithi006 ceph-mon[29835]: pgmap v335: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-09T04:49:46.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:46 smithi006 ceph-mon[29835]: osdmap e81: 12 total, 12 up, 12 in 2024-05-09T04:49:47.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:46 smithi186 ceph-mon[34668]: pgmap v335: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-09T04:49:47.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:46 smithi186 ceph-mon[34668]: osdmap e81: 12 total, 12 up, 12 in 2024-05-09T04:49:47.705 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-09T04:49:47.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:47 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/2536467351' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-09T04:49:47.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:47 smithi006 ceph-mon[29835]: osdmap e82: 12 total, 12 up, 12 in 2024-05-09T04:49:48.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:47 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/2536467351' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-09T04:49:48.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:47 smithi186 ceph-mon[34668]: osdmap e82: 12 total, 12 up, 12 in 2024-05-09T04:49:48.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:47 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/2536467351' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-09T04:49:48.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:47 smithi169 ceph-mon[34075]: osdmap e82: 12 total, 12 up, 12 in 2024-05-09T04:49:48.135 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:49:48.944 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:48 smithi006 ceph-mon[29835]: pgmap v338: 129 pgs: 9 creating+peering, 23 unknown, 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 0 op/s 2024-05-09T04:49:48.944 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:48 smithi006 ceph-mon[29835]: osdmap e83: 12 total, 12 up, 12 in 2024-05-09T04:49:48.944 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:48 smithi006 ceph-mon[29835]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T04:49:48.945 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:48 smithi006 ceph-mon[29835]: osdmap e84: 12 total, 12 up, 12 in 2024-05-09T04:49:49.018 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:48 smithi169 ceph-mon[34075]: pgmap v338: 129 pgs: 9 creating+peering, 23 unknown, 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 0 op/s 2024-05-09T04:49:49.018 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:48 smithi169 ceph-mon[34075]: osdmap e83: 12 total, 12 up, 12 in 2024-05-09T04:49:49.018 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:48 smithi169 ceph-mon[34075]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T04:49:49.018 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:48 smithi169 ceph-mon[34075]: osdmap e84: 12 total, 12 up, 12 in 2024-05-09T04:49:49.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:48 smithi186 ceph-mon[34668]: pgmap v338: 129 pgs: 9 creating+peering, 23 unknown, 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 0 op/s 2024-05-09T04:49:49.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:48 smithi186 ceph-mon[34668]: osdmap e83: 12 total, 12 up, 12 in 2024-05-09T04:49:49.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:48 smithi186 ceph-mon[34668]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T04:49:49.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:48 smithi186 ceph-mon[34668]: osdmap e84: 12 total, 12 up, 12 in 2024-05-09T04:49:50.785 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:50 smithi006 ceph-mon[29835]: pgmap v341: 129 pgs: 1 creating+activating, 10 creating+peering, 13 unknown, 105 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:50.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.930+0000 7fa73c600700 1 -- 172.21.15.6:0/2512771848 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa734110970 msgr2=0x7fa734110d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:50.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.930+0000 7fa73c600700 1 --2- 172.21.15.6:0/2512771848 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa734110970 0x7fa734110d80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa728009990 tx=0x7fa728009ca0 comp rx=0 tx=0).stop 2024-05-09T04:49:50.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.930+0000 7fa73c600700 1 -- 172.21.15.6:0/2512771848 shutdown_connections 2024-05-09T04:49:50.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.930+0000 7fa73c600700 1 --2- 172.21.15.6:0/2512771848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa73410b2e0 0x7fa734117810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:50.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.930+0000 7fa73c600700 1 --2- 172.21.15.6:0/2512771848 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa734110970 0x7fa734110d80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:50.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.930+0000 7fa73c600700 1 --2- 172.21.15.6:0/2512771848 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa73410a990 0x7fa73410ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:50.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.930+0000 7fa73c600700 1 -- 172.21.15.6:0/2512771848 >> 172.21.15.6:0/2512771848 conn(0x7fa734077c80 msgr2=0x7fa7340780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:50.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.930+0000 7fa73c600700 1 -- 172.21.15.6:0/2512771848 shutdown_connections 2024-05-09T04:49:50.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.930+0000 7fa73c600700 1 -- 172.21.15.6:0/2512771848 wait complete. 2024-05-09T04:49:50.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.932+0000 7fa73c600700 1 Processor -- start 2024-05-09T04:49:50.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.932+0000 7fa73c600700 1 -- start start 2024-05-09T04:49:50.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.933+0000 7fa73c600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa73410a990 0x7fa7341a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:50.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.933+0000 7fa73c600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa73410b2e0 0x7fa7341a1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:50.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.933+0000 7fa73c600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7341a5de0 0x7fa7341a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:50.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.933+0000 7fa73c600700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa73411aac0 con 0x7fa73410b2e0 2024-05-09T04:49:50.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.933+0000 7fa73c600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa73411a940 con 0x7fa7341a5de0 2024-05-09T04:49:50.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.933+0000 7fa73c600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa73411ac40 con 0x7fa73410a990 2024-05-09T04:49:50.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.933+0000 7fa739c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa73410a990 0x7fa7341a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:50.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.933+0000 7fa739200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa73410b2e0 0x7fa7341a1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:50.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.933+0000 7fa739c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa73410a990 0x7fa7341a1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:56044/0 (socket says 172.21.15.6:56044) 2024-05-09T04:49:50.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.933+0000 7fa739c00700 1 -- 172.21.15.6:0/546740406 learned_addr learned my addr 172.21.15.6:0/546740406 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:49:50.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.933+0000 7fa73a600700 1 --2- 172.21.15.6:0/546740406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7341a5de0 0x7fa7341a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:50.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.933+0000 7fa739200700 1 -- 172.21.15.6:0/546740406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa73410a990 msgr2=0x7fa7341a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:50.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.933+0000 7fa739200700 1 --2- 172.21.15.6:0/546740406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa73410a990 0x7fa7341a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:50.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.933+0000 7fa739200700 1 -- 172.21.15.6:0/546740406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7341a5de0 msgr2=0x7fa7341a61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:50.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.933+0000 7fa739200700 1 --2- 172.21.15.6:0/546740406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7341a5de0 0x7fa7341a61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:50.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.933+0000 7fa739200700 1 -- 172.21.15.6:0/546740406 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7341a6930 con 0x7fa73410b2e0 2024-05-09T04:49:50.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.934+0000 7fa739c00700 1 --2- 172.21.15.6:0/546740406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa73410a990 0x7fa7341a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:49:50.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.934+0000 7fa73a600700 1 --2- 172.21.15.6:0/546740406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7341a5de0 0x7fa7341a61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:49:50.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.934+0000 7fa739200700 1 --2- 172.21.15.6:0/546740406 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa73410b2e0 0x7fa7341a1c10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa73406d120 tx=0x7fa72000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:50.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.934+0000 7fa732000700 1 -- 172.21.15.6:0/546740406 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa720016070 con 0x7fa73410b2e0 2024-05-09T04:49:50.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.934+0000 7fa732000700 1 -- 172.21.15.6:0/546740406 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa72000fd40 con 0x7fa73410b2e0 2024-05-09T04:49:50.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.934+0000 7fa73c600700 1 -- 172.21.15.6:0/546740406 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7341a6c20 con 0x7fa73410b2e0 2024-05-09T04:49:50.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.934+0000 7fa732000700 1 -- 172.21.15.6:0/546740406 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72000e3b0 con 0x7fa73410b2e0 2024-05-09T04:49:50.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.934+0000 7fa73c600700 1 -- 172.21.15.6:0/546740406 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7341ae450 con 0x7fa73410b2e0 2024-05-09T04:49:50.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.935+0000 7fa73c600700 1 -- 172.21.15.6:0/546740406 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7340512c0 con 0x7fa73410b2e0 2024-05-09T04:49:50.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.938+0000 7fa732000700 1 -- 172.21.15.6:0/546740406 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fa720019070 con 0x7fa73410b2e0 2024-05-09T04:49:50.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.940+0000 7fa732000700 1 --2- 172.21.15.6:0/546740406 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa71c0b2900 0x7fa71c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:50.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.940+0000 7fa732000700 1 -- 172.21.15.6:0/546740406 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8770+0+0 (secure 0 0 0) 0x7fa7200e22a0 con 0x7fa73410b2e0 2024-05-09T04:49:50.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.940+0000 7fa739c00700 1 --2- 172.21.15.6:0/546740406 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa71c0b2900 0x7fa71c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:50.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.941+0000 7fa739c00700 1 --2- 172.21.15.6:0/546740406 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa71c0b2900 0x7fa71c0b4e10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa728009750 tx=0x7fa72800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:50.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:50.942+0000 7fa732000700 1 -- 172.21.15.6:0/546740406 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa7200a9d40 con 0x7fa73410b2e0 2024-05-09T04:49:51.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:50 smithi186 ceph-mon[34668]: pgmap v341: 129 pgs: 1 creating+activating, 10 creating+peering, 13 unknown, 105 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:51.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:50 smithi169 ceph-mon[34075]: pgmap v341: 129 pgs: 1 creating+activating, 10 creating+peering, 13 unknown, 105 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:51.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:51.174+0000 7fa73c600700 1 -- 172.21.15.6:0/546740406 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fa73402e5f0 con 0x7fa73410b2e0 2024-05-09T04:49:51.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:51.674+0000 7fa732000700 1 -- 172.21.15.6:0/546740406 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v85) ==== 169+0+0 (secure 0 0 0) 0x7fa7200a9640 con 0x7fa73410b2e0 2024-05-09T04:49:51.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:51.679+0000 7fa73c600700 1 -- 172.21.15.6:0/546740406 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa71c0b2900 msgr2=0x7fa71c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:51.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:51.680+0000 7fa73c600700 1 --2- 172.21.15.6:0/546740406 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa71c0b2900 0x7fa71c0b4e10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa728009750 tx=0x7fa72800b260 comp rx=0 tx=0).stop 2024-05-09T04:49:51.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:51.680+0000 7fa73c600700 1 -- 172.21.15.6:0/546740406 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa73410b2e0 msgr2=0x7fa7341a1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:51.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:51.680+0000 7fa73c600700 1 --2- 172.21.15.6:0/546740406 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa73410b2e0 0x7fa7341a1c10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa73406d120 tx=0x7fa72000dc30 comp rx=0 tx=0).stop 2024-05-09T04:49:51.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:51.681+0000 7fa73c600700 1 -- 172.21.15.6:0/546740406 shutdown_connections 2024-05-09T04:49:51.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:51.681+0000 7fa73c600700 1 --2- 172.21.15.6:0/546740406 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa71c0b2900 0x7fa71c0b4e10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:51.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:51.681+0000 7fa73c600700 1 --2- 172.21.15.6:0/546740406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7341a5de0 0x7fa7341a61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:51.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:51.681+0000 7fa73c600700 1 --2- 172.21.15.6:0/546740406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa73410a990 0x7fa7341a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:51.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:51.681+0000 7fa73c600700 1 --2- 172.21.15.6:0/546740406 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa73410b2e0 0x7fa7341a1c10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:51.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:51.681+0000 7fa73c600700 1 -- 172.21.15.6:0/546740406 >> 172.21.15.6:0/546740406 conn(0x7fa734077c80 msgr2=0x7fa73407be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:51.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:51.682+0000 7fa73c600700 1 -- 172.21.15.6:0/546740406 shutdown_connections 2024-05-09T04:49:51.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:51.682+0000 7fa73c600700 1 -- 172.21.15.6:0/546740406 wait complete. 2024-05-09T04:49:51.686 INFO:teuthology.orchestra.run.smithi006.stderr:set pool 4 allow_ec_overwrites to true 2024-05-09T04:49:51.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:51 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/546740406' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-09T04:49:52.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:51 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/546740406' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-09T04:49:52.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:51 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/546740406' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-09T04:49:52.518 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-09T04:49:52.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:52 smithi006 ceph-mon[29835]: pgmap v342: 129 pgs: 1 creating+activating, 10 creating+peering, 118 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:52.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:52 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/546740406' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-09T04:49:52.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:52 smithi006 ceph-mon[29835]: osdmap e85: 12 total, 12 up, 12 in 2024-05-09T04:49:53.000 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:49:53.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:52 smithi186 ceph-mon[34668]: pgmap v342: 129 pgs: 1 creating+activating, 10 creating+peering, 118 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:53.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:52 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/546740406' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-09T04:49:53.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:52 smithi186 ceph-mon[34668]: osdmap e85: 12 total, 12 up, 12 in 2024-05-09T04:49:53.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:52 smithi169 ceph-mon[34075]: pgmap v342: 129 pgs: 1 creating+activating, 10 creating+peering, 118 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:53.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:52 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/546740406' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-09T04:49:53.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:52 smithi169 ceph-mon[34075]: osdmap e85: 12 total, 12 up, 12 in 2024-05-09T04:49:54.937 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:54 smithi006 ceph-mon[29835]: pgmap v344: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:55.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:54 smithi186 ceph-mon[34668]: pgmap v344: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:55.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:54 smithi169 ceph-mon[34075]: pgmap v344: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:55.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.193+0000 7fad69800700 1 -- 172.21.15.6:0/2796168269 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad64106640 msgr2=0x7fad64106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:55.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.193+0000 7fad69800700 1 --2- 172.21.15.6:0/2796168269 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad64106640 0x7fad64106a50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fad540099c0 tx=0x7fad54009cd0 comp rx=0 tx=0).stop 2024-05-09T04:49:55.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.193+0000 7fad69800700 1 -- 172.21.15.6:0/2796168269 shutdown_connections 2024-05-09T04:49:55.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.193+0000 7fad69800700 1 --2- 172.21.15.6:0/2796168269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad6410b870 0x7fad641131d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:55.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.193+0000 7fad69800700 1 --2- 172.21.15.6:0/2796168269 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad64106640 0x7fad64106a50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:55.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.193+0000 7fad69800700 1 --2- 172.21.15.6:0/2796168269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad6410af20 0x7fad6410b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:55.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.193+0000 7fad69800700 1 -- 172.21.15.6:0/2796168269 >> 172.21.15.6:0/2796168269 conn(0x7fad64102030 msgr2=0x7fad64104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:55.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.193+0000 7fad69800700 1 -- 172.21.15.6:0/2796168269 shutdown_connections 2024-05-09T04:49:55.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.193+0000 7fad69800700 1 -- 172.21.15.6:0/2796168269 wait complete. 2024-05-09T04:49:55.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.195+0000 7fad69800700 1 Processor -- start 2024-05-09T04:49:55.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.195+0000 7fad69800700 1 -- start start 2024-05-09T04:49:55.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.195+0000 7fad69800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad64106640 0x7fad6419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:55.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.195+0000 7fad69800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad6410af20 0x7fad6419d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:55.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.195+0000 7fad69800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad6410b870 0x7fad641a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:55.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.195+0000 7fad69800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fad641164a0 con 0x7fad6410b870 2024-05-09T04:49:55.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.195+0000 7fad69800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fad64116320 con 0x7fad6410af20 2024-05-09T04:49:55.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.195+0000 7fad69800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fad64116620 con 0x7fad64106640 2024-05-09T04:49:55.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.196+0000 7fad63400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad6410af20 0x7fad6419d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:55.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.196+0000 7fad63e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad64106640 0x7fad6419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:55.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.196+0000 7fad63400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad6410af20 0x7fad6419d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:60064/0 (socket says 172.21.15.6:60064) 2024-05-09T04:49:55.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.196+0000 7fad63400700 1 -- 172.21.15.6:0/3041686268 learned_addr learned my addr 172.21.15.6:0/3041686268 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:49:55.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.196+0000 7fad68e00700 1 --2- 172.21.15.6:0/3041686268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad6410b870 0x7fad641a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:55.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.196+0000 7fad63400700 1 -- 172.21.15.6:0/3041686268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad64106640 msgr2=0x7fad6419cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:55.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.196+0000 7fad63400700 1 --2- 172.21.15.6:0/3041686268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad64106640 0x7fad6419cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:55.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.196+0000 7fad63400700 1 -- 172.21.15.6:0/3041686268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad6410b870 msgr2=0x7fad641a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:55.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.196+0000 7fad63400700 1 --2- 172.21.15.6:0/3041686268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad6410b870 0x7fad641a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:55.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.196+0000 7fad63400700 1 -- 172.21.15.6:0/3041686268 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad641a1cc0 con 0x7fad6410af20 2024-05-09T04:49:55.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.197+0000 7fad63400700 1 --2- 172.21.15.6:0/3041686268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad6410af20 0x7fad6419d390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fad5000e970 tx=0x7fad5000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:55.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.197+0000 7fad60c00700 1 -- 172.21.15.6:0/3041686268 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad50004a30 con 0x7fad6410af20 2024-05-09T04:49:55.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.197+0000 7fad69800700 1 -- 172.21.15.6:0/3041686268 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad641a1fb0 con 0x7fad6410af20 2024-05-09T04:49:55.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.197+0000 7fad60c00700 1 -- 172.21.15.6:0/3041686268 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad5000cc90 con 0x7fad6410af20 2024-05-09T04:49:55.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.197+0000 7fad60c00700 1 -- 172.21.15.6:0/3041686268 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad50005610 con 0x7fad6410af20 2024-05-09T04:49:55.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.197+0000 7fad69800700 1 -- 172.21.15.6:0/3041686268 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad641a98d0 con 0x7fad6410af20 2024-05-09T04:49:55.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.198+0000 7fad69800700 1 -- 172.21.15.6:0/3041686268 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad640512c0 con 0x7fad6410af20 2024-05-09T04:49:55.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.202+0000 7fad60c00700 1 -- 172.21.15.6:0/3041686268 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fad5000c7e0 con 0x7fad6410af20 2024-05-09T04:49:55.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.203+0000 7fad60c00700 1 --2- 172.21.15.6:0/3041686268 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fad480b2900 0x7fad480b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:55.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.204+0000 7fad60c00700 1 -- 172.21.15.6:0/3041686268 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8770+0+0 (secure 0 0 0) 0x7fad500e27d0 con 0x7fad6410af20 2024-05-09T04:49:55.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.204+0000 7fad63e00700 1 --2- 172.21.15.6:0/3041686268 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fad480b2900 0x7fad480b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:55.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.205+0000 7fad63e00700 1 --2- 172.21.15.6:0/3041686268 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fad480b2900 0x7fad480b4e10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fad54003760 tx=0x7fad5401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:55.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.205+0000 7fad60c00700 1 -- 172.21.15.6:0/3041686268 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fad50054df0 con 0x7fad6410af20 2024-05-09T04:49:55.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.498+0000 7fad69800700 1 -- 172.21.15.6:0/3041686268 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fad6402e5f0 con 0x7fad6410af20 2024-05-09T04:49:55.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.710+0000 7fad60c00700 1 -- 172.21.15.6:0/3041686268 <== mon.2 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7fad500cae20 con 0x7fad6410af20 2024-05-09T04:49:55.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.717+0000 7fad69800700 1 -- 172.21.15.6:0/3041686268 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fad480b2900 msgr2=0x7fad480b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:55.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.717+0000 7fad69800700 1 --2- 172.21.15.6:0/3041686268 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fad480b2900 0x7fad480b4e10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fad54003760 tx=0x7fad5401a040 comp rx=0 tx=0).stop 2024-05-09T04:49:55.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.717+0000 7fad69800700 1 -- 172.21.15.6:0/3041686268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad6410af20 msgr2=0x7fad6419d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:55.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.717+0000 7fad69800700 1 --2- 172.21.15.6:0/3041686268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad6410af20 0x7fad6419d390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fad5000e970 tx=0x7fad5000ec80 comp rx=0 tx=0).stop 2024-05-09T04:49:55.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.717+0000 7fad69800700 1 -- 172.21.15.6:0/3041686268 shutdown_connections 2024-05-09T04:49:55.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.717+0000 7fad69800700 1 --2- 172.21.15.6:0/3041686268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad6410b870 0x7fad641a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:55.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.717+0000 7fad69800700 1 --2- 172.21.15.6:0/3041686268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad64106640 0x7fad6419cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:55.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.717+0000 7fad69800700 1 --2- 172.21.15.6:0/3041686268 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fad480b2900 0x7fad480b4e10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:55.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.717+0000 7fad69800700 1 --2- 172.21.15.6:0/3041686268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad6410af20 0x7fad6419d390 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:55.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.717+0000 7fad69800700 1 -- 172.21.15.6:0/3041686268 >> 172.21.15.6:0/3041686268 conn(0x7fad64102030 msgr2=0x7fad6410a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:55.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.717+0000 7fad69800700 1 -- 172.21.15.6:0/3041686268 shutdown_connections 2024-05-09T04:49:55.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:55.718+0000 7fad69800700 1 -- 172.21.15.6:0/3041686268 wait complete. 2024-05-09T04:49:55.720 INFO:teuthology.orchestra.run.smithi006.stderr:added data pool 4 to fsmap 2024-05-09T04:49:55.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:55 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 1]}]: dispatch 2024-05-09T04:49:55.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:55 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-05-09T04:49:55.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:55 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-05-09T04:49:55.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:55 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-05-09T04:49:55.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:55 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-09T04:49:55.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:55 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]: dispatch 2024-05-09T04:49:55.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:55 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-05-09T04:49:55.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:55 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 10]}]: dispatch 2024-05-09T04:49:55.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:55 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-05-09T04:49:55.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:55 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/3041686268' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-09T04:49:55.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:55 smithi006 ceph-mon[29835]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-09T04:49:56.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:55 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 1]}]: dispatch 2024-05-09T04:49:56.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:55 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-05-09T04:49:56.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:55 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-05-09T04:49:56.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:55 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-05-09T04:49:56.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:55 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-09T04:49:56.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:55 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]: dispatch 2024-05-09T04:49:56.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:55 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-05-09T04:49:56.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:55 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 10]}]: dispatch 2024-05-09T04:49:56.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:55 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-05-09T04:49:56.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:55 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/3041686268' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-09T04:49:56.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:55 smithi186 ceph-mon[34668]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-09T04:49:56.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:55 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 1]}]: dispatch 2024-05-09T04:49:56.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:55 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-05-09T04:49:56.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:55 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-05-09T04:49:56.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:55 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-05-09T04:49:56.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:55 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-09T04:49:56.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:55 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]: dispatch 2024-05-09T04:49:56.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:55 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-05-09T04:49:56.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:55 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 10]}]: dispatch 2024-05-09T04:49:56.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:55 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-05-09T04:49:56.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:55 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/3041686268' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-09T04:49:56.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:55 smithi169 ceph-mon[34075]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-09T04:49:56.557 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:49:56.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:56 smithi006 ceph-mon[29835]: pgmap v345: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:56.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 1]}]': finished 2024-05-09T04:49:56.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-05-09T04:49:56.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-05-09T04:49:56.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-05-09T04:49:56.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-09T04:49:56.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]': finished 2024-05-09T04:49:56.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-05-09T04:49:56.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 10]}]': finished 2024-05-09T04:49:56.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-05-09T04:49:56.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:56 smithi006 ceph-mon[29835]: osdmap e86: 12 total, 12 up, 12 in 2024-05-09T04:49:56.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:56 smithi006 ceph-mon[29835]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-09T04:49:56.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:56 smithi006 ceph-mon[29835]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T04:49:56.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:56 smithi006 ceph-mon[29835]: osdmap e87: 12 total, 12 up, 12 in 2024-05-09T04:49:57.055 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:49:57.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:56 smithi186 ceph-mon[34668]: pgmap v345: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:57.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 1]}]': finished 2024-05-09T04:49:57.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-05-09T04:49:57.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-05-09T04:49:57.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-05-09T04:49:57.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-09T04:49:57.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]': finished 2024-05-09T04:49:57.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-05-09T04:49:57.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 10]}]': finished 2024-05-09T04:49:57.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-05-09T04:49:57.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:56 smithi186 ceph-mon[34668]: osdmap e86: 12 total, 12 up, 12 in 2024-05-09T04:49:57.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:56 smithi186 ceph-mon[34668]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-09T04:49:57.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:56 smithi186 ceph-mon[34668]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T04:49:57.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:56 smithi186 ceph-mon[34668]: osdmap e87: 12 total, 12 up, 12 in 2024-05-09T04:49:57.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:56 smithi169 ceph-mon[34075]: pgmap v345: 129 pgs: 129 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:49:57.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:56 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 1]}]': finished 2024-05-09T04:49:57.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:56 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-05-09T04:49:57.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:56 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-05-09T04:49:57.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:56 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-05-09T04:49:57.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:56 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-09T04:49:57.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:56 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]': finished 2024-05-09T04:49:57.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:56 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-05-09T04:49:57.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:56 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 10]}]': finished 2024-05-09T04:49:57.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:56 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-05-09T04:49:57.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:56 smithi169 ceph-mon[34075]: osdmap e86: 12 total, 12 up, 12 in 2024-05-09T04:49:57.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:56 smithi169 ceph-mon[34075]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-09T04:49:57.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:56 smithi169 ceph-mon[34075]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T04:49:57.118 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:56 smithi169 ceph-mon[34075]: osdmap e87: 12 total, 12 up, 12 in 2024-05-09T04:49:58.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:57 smithi186 ceph-mon[34668]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-09T04:49:58.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:57 smithi186 ceph-mon[34668]: Cluster is now healthy 2024-05-09T04:49:58.118 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:57 smithi169 ceph-mon[34075]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-09T04:49:58.118 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:57 smithi169 ceph-mon[34075]: Cluster is now healthy 2024-05-09T04:49:58.125 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:57 smithi006 ceph-mon[29835]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-09T04:49:58.125 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:57 smithi006 ceph-mon[29835]: Cluster is now healthy 2024-05-09T04:49:59.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:58 smithi186 ceph-mon[34668]: pgmap v348: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-09T04:49:59.100 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:58 smithi006 ceph-mon[29835]: pgmap v348: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-09T04:49:59.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:58 smithi169 ceph-mon[34075]: pgmap v348: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-09T04:49:59.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.349+0000 7f8ee7200700 1 -- 172.21.15.6:0/1549080415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ee01045f0 msgr2=0x7f8ee0104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:59.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.349+0000 7f8ee7200700 1 --2- 172.21.15.6:0/1549080415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ee01045f0 0x7f8ee0104a00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0009920 tx=0x7f8ed0009c30 comp rx=0 tx=0).stop 2024-05-09T04:49:59.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.349+0000 7f8ee7200700 1 -- 172.21.15.6:0/1549080415 shutdown_connections 2024-05-09T04:49:59.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.349+0000 7f8ee7200700 1 --2- 172.21.15.6:0/1549080415 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ee0109870 0x7f8ee01111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:59.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.349+0000 7f8ee7200700 1 --2- 172.21.15.6:0/1549080415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ee01045f0 0x7f8ee0104a00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:59.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.349+0000 7f8ee7200700 1 --2- 172.21.15.6:0/1549080415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ee0108f20 0x7f8ee0109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:59.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.349+0000 7f8ee7200700 1 -- 172.21.15.6:0/1549080415 >> 172.21.15.6:0/1549080415 conn(0x7f8ee0100040 msgr2=0x7f8ee0102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:59.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.350+0000 7f8ee7200700 1 -- 172.21.15.6:0/1549080415 shutdown_connections 2024-05-09T04:49:59.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.350+0000 7f8ee7200700 1 -- 172.21.15.6:0/1549080415 wait complete. 2024-05-09T04:49:59.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.351+0000 7f8ee7200700 1 Processor -- start 2024-05-09T04:49:59.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.351+0000 7f8ee7200700 1 -- start start 2024-05-09T04:49:59.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.352+0000 7f8ee7200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ee01045f0 0x7f8ee019d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:59.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.352+0000 7f8ee7200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ee0108f20 0x7f8ee019d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:59.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.352+0000 7f8ee7200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ee0109870 0x7f8ee01a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:59.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.352+0000 7f8ee7200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee01144f0 con 0x7f8ee01045f0 2024-05-09T04:49:59.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.352+0000 7f8ee7200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee0114370 con 0x7f8ee0109870 2024-05-09T04:49:59.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.352+0000 7f8ee7200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee0114670 con 0x7f8ee0108f20 2024-05-09T04:49:59.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.352+0000 7f8edfe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ee01045f0 0x7f8ee019d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:59.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.352+0000 7f8ee5200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ee0109870 0x7f8ee01a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:59.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.352+0000 7f8edfe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ee01045f0 0x7f8ee019d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39120/0 (socket says 172.21.15.6:39120) 2024-05-09T04:49:59.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.352+0000 7f8edfe00700 1 -- 172.21.15.6:0/1081507710 learned_addr learned my addr 172.21.15.6:0/1081507710 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:49:59.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.352+0000 7f8edf400700 1 --2- 172.21.15.6:0/1081507710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ee0108f20 0x7f8ee019d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:59.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.352+0000 7f8ee5200700 1 -- 172.21.15.6:0/1081507710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ee0108f20 msgr2=0x7f8ee019d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:59.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.352+0000 7f8ee5200700 1 --2- 172.21.15.6:0/1081507710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ee0108f20 0x7f8ee019d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:59.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.352+0000 7f8ee5200700 1 -- 172.21.15.6:0/1081507710 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ee01045f0 msgr2=0x7f8ee019d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:59.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.352+0000 7f8ee5200700 1 --2- 172.21.15.6:0/1081507710 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ee01045f0 0x7f8ee019d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:59.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.353+0000 7f8ee5200700 1 -- 172.21.15.6:0/1081507710 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee01a2090 con 0x7f8ee0109870 2024-05-09T04:49:59.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.353+0000 7f8edfe00700 1 --2- 172.21.15.6:0/1081507710 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ee01045f0 0x7f8ee019d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:49:59.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.353+0000 7f8ee5200700 1 --2- 172.21.15.6:0/1081507710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ee0109870 0x7f8ee01a19f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8ed800efb0 tx=0x7f8ed800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:59.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.353+0000 7f8edcc00700 1 -- 172.21.15.6:0/1081507710 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8003a00 con 0x7f8ee0109870 2024-05-09T04:49:59.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.353+0000 7f8ee7200700 1 -- 172.21.15.6:0/1081507710 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ee01a2380 con 0x7f8ee0109870 2024-05-09T04:49:59.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.353+0000 7f8edcc00700 1 -- 172.21.15.6:0/1081507710 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ed800ce50 con 0x7f8ee0109870 2024-05-09T04:49:59.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.353+0000 7f8edcc00700 1 -- 172.21.15.6:0/1081507710 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed80047c0 con 0x7f8ee0109870 2024-05-09T04:49:59.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.353+0000 7f8ee7200700 1 -- 172.21.15.6:0/1081507710 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ee01a9ca0 con 0x7f8ee0109870 2024-05-09T04:49:59.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.354+0000 7f8ee7200700 1 -- 172.21.15.6:0/1081507710 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ee00512c0 con 0x7f8ee0109870 2024-05-09T04:49:59.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.357+0000 7f8edcc00700 1 -- 172.21.15.6:0/1081507710 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f8ed8012050 con 0x7f8ee0109870 2024-05-09T04:49:59.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.358+0000 7f8edcc00700 1 --2- 172.21.15.6:0/1081507710 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f8ec40b2880 0x7f8ec40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:59.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.359+0000 7f8edcc00700 1 -- 172.21.15.6:0/1081507710 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f8ed80e1ff0 con 0x7f8ee0109870 2024-05-09T04:49:59.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.359+0000 7f8edfe00700 1 --2- 172.21.15.6:0/1081507710 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f8ec40b2880 0x7f8ec40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:59.360 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.359+0000 7f8edfe00700 1 --2- 172.21.15.6:0/1081507710 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f8ec40b2880 0x7f8ec40b4d90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0005f40 tx=0x7f8ed0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:59.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.361+0000 7f8edcc00700 1 -- 172.21.15.6:0/1081507710 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8ed8022060 con 0x7f8ee0109870 2024-05-09T04:49:59.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.648+0000 7f8ee7200700 1 -- 172.21.15.6:0/1081507710 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ee01a9e70 con 0x7f8ee0109870 2024-05-09T04:49:59.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.650+0000 7f8edcc00700 1 -- 172.21.15.6:0/1081507710 <== mon.2 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11791 (secure 0 0 0) 0x7f8ed8010040 con 0x7f8ee0109870 2024-05-09T04:49:59.651 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:49:59.651 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/162094599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":162094599},{"type":"v1","addr":"172.21.15.6:6835","nonce":162094599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/589294363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":589294363},{"type":"v1","addr":"172.21.15.6:6841","nonce":589294363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/2977145206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2977145206},{"type":"v1","addr":"172.21.15.186:6833","nonce":2977145206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/133345948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":133345948},{"type":"v1","addr":"172.21.15.169:6833","nonce":133345948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/1281521088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":1281521088},{"type":"v1","addr":"172.21.15.6:6837","nonce":1281521088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2268322007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2268322007},{"type":"v1","addr":"172.21.15.186:6835","nonce":2268322007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3811005052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3811005052},{"type":"v1","addr":"172.21.15.169:6835","nonce":3811005052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1647607118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1647607118},{"type":"v1","addr":"172.21.15.186:6837","nonce":1647607118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/3954883105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":3954883105},{"type":"v1","addr":"172.21.15.186:6839","nonce":3954883105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24512,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1845715294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1845715294},{"type":"v1","addr":"172.21.15.169:6837","nonce":1845715294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24530,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/2780306492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":2780306492},{"type":"v1","addr":"172.21.15.6:6839","nonce":2780306492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T04:49:36.920376+0000","modified":"2024-05-09T04:49:55.501674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24554},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.169:6839/614388771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":614388771},{"type":"v1","addr":"172.21.15.169:6839","nonce":614388771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24554,"qdb_cluster":[24554]},"id":1}]} 2024-05-09T04:49:59.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.654+0000 7f8ee7200700 1 -- 172.21.15.6:0/1081507710 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f8ec40b2880 msgr2=0x7f8ec40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:59.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.654+0000 7f8ee7200700 1 --2- 172.21.15.6:0/1081507710 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f8ec40b2880 0x7f8ec40b4d90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0005f40 tx=0x7f8ed0019040 comp rx=0 tx=0).stop 2024-05-09T04:49:59.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.654+0000 7f8ee7200700 1 -- 172.21.15.6:0/1081507710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ee0109870 msgr2=0x7f8ee01a19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:59.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.654+0000 7f8ee7200700 1 --2- 172.21.15.6:0/1081507710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ee0109870 0x7f8ee01a19f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8ed800efb0 tx=0x7f8ed800c3a0 comp rx=0 tx=0).stop 2024-05-09T04:49:59.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.654+0000 7f8ee7200700 1 -- 172.21.15.6:0/1081507710 shutdown_connections 2024-05-09T04:49:59.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.654+0000 7f8ee7200700 1 --2- 172.21.15.6:0/1081507710 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f8ec40b2880 0x7f8ec40b4d90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:59.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.654+0000 7f8ee7200700 1 --2- 172.21.15.6:0/1081507710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ee0109870 0x7f8ee01a19f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:59.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.654+0000 7f8ee7200700 1 --2- 172.21.15.6:0/1081507710 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ee01045f0 0x7f8ee019d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:59.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.654+0000 7f8ee7200700 1 --2- 172.21.15.6:0/1081507710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ee0108f20 0x7f8ee019d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:59.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.654+0000 7f8ee7200700 1 -- 172.21.15.6:0/1081507710 >> 172.21.15.6:0/1081507710 conn(0x7f8ee0100040 msgr2=0x7f8ee010a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:59.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.654+0000 7f8ee7200700 1 -- 172.21.15.6:0/1081507710 shutdown_connections 2024-05-09T04:49:59.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:49:59.654+0000 7f8ee7200700 1 -- 172.21.15.6:0/1081507710 wait complete. 2024-05-09T04:49:59.656 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 17 2024-05-09T04:49:59.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:49:59 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1081507710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:50:00.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:49:59 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/1081507710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:50:00.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:49:59 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/1081507710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:50:00.423 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T04:50:00.926 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:50:01.084 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:00 smithi006 ceph-mon[29835]: pgmap v349: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-09T04:50:01.084 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:00 smithi006 ceph-mon[29835]: overall HEALTH_OK 2024-05-09T04:50:01.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:00 smithi186 ceph-mon[34668]: pgmap v349: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-09T04:50:01.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:00 smithi186 ceph-mon[34668]: overall HEALTH_OK 2024-05-09T04:50:01.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:00 smithi169 ceph-mon[34075]: pgmap v349: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-09T04:50:01.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:00 smithi169 ceph-mon[34075]: overall HEALTH_OK 2024-05-09T04:50:02.929 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:02 smithi006 ceph-mon[29835]: pgmap v350: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-09T04:50:03.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:02 smithi186 ceph-mon[34668]: pgmap v350: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-09T04:50:03.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:02 smithi169 ceph-mon[34075]: pgmap v350: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-09T04:50:03.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.181+0000 7f4d3b600700 1 -- 172.21.15.6:0/2224642224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d341034c0 msgr2=0x7f4d341038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:03.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.181+0000 7f4d3b600700 1 --2- 172.21.15.6:0/2224642224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d341034c0 0x7f4d341038d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4d240098f0 tx=0x7f4d24009c00 comp rx=0 tx=0).stop 2024-05-09T04:50:03.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.182+0000 7f4d3b600700 1 -- 172.21.15.6:0/2224642224 shutdown_connections 2024-05-09T04:50:03.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.182+0000 7f4d3b600700 1 --2- 172.21.15.6:0/2224642224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d34108740 0x7f4d341100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:03.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.182+0000 7f4d3b600700 1 --2- 172.21.15.6:0/2224642224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d341034c0 0x7f4d341038d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:03.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.182+0000 7f4d3b600700 1 --2- 172.21.15.6:0/2224642224 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d34107df0 0x7f4d34108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:03.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.182+0000 7f4d3b600700 1 -- 172.21.15.6:0/2224642224 >> 172.21.15.6:0/2224642224 conn(0x7f4d340fedc0 msgr2=0x7f4d341011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:03.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.182+0000 7f4d3b600700 1 -- 172.21.15.6:0/2224642224 shutdown_connections 2024-05-09T04:50:03.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.182+0000 7f4d3b600700 1 -- 172.21.15.6:0/2224642224 wait complete. 2024-05-09T04:50:03.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.183+0000 7f4d3b600700 1 Processor -- start 2024-05-09T04:50:03.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.183+0000 7f4d3b600700 1 -- start start 2024-05-09T04:50:03.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.184+0000 7f4d3b600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d341034c0 0x7f4d3419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:03.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.184+0000 7f4d3b600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d34107df0 0x7f4d3419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:03.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.184+0000 7f4d3b600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d34108740 0x7f4d3419f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:03.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.184+0000 7f4d3b600700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4d34113370 con 0x7f4d341034c0 2024-05-09T04:50:03.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.184+0000 7f4d3b600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4d341131f0 con 0x7f4d34108740 2024-05-09T04:50:03.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.184+0000 7f4d3b600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4d341134f0 con 0x7f4d34107df0 2024-05-09T04:50:03.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.184+0000 7f4d38c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d341034c0 0x7f4d3419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:03.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.184+0000 7f4d38c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d341034c0 0x7f4d3419ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39128/0 (socket says 172.21.15.6:39128) 2024-05-09T04:50:03.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.184+0000 7f4d39600700 1 --2- 172.21.15.6:0/2537409736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d34108740 0x7f4d3419f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:03.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.184+0000 7f4d38c00700 1 -- 172.21.15.6:0/2537409736 learned_addr learned my addr 172.21.15.6:0/2537409736 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:50:03.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.184+0000 7f4d2be00700 1 --2- 172.21.15.6:0/2537409736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d34107df0 0x7f4d3419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:03.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.185+0000 7f4d39600700 1 -- 172.21.15.6:0/2537409736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d34107df0 msgr2=0x7f4d3419b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:03.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.185+0000 7f4d39600700 1 --2- 172.21.15.6:0/2537409736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d34107df0 0x7f4d3419b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.185+0000 7f4d39600700 1 -- 172.21.15.6:0/2537409736 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d341034c0 msgr2=0x7f4d3419ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.185+0000 7f4d39600700 1 --2- 172.21.15.6:0/2537409736 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d341034c0 0x7f4d3419ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.185+0000 7f4d39600700 1 -- 172.21.15.6:0/2537409736 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d3419fce0 con 0x7f4d34108740 2024-05-09T04:50:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.185+0000 7f4d39600700 1 --2- 172.21.15.6:0/2537409736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d34108740 0x7f4d3419f640 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c00efc0 tx=0x7f4d2c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.185+0000 7f4d32000700 1 -- 172.21.15.6:0/2537409736 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c0092e0 con 0x7f4d34108740 2024-05-09T04:50:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.185+0000 7f4d32000700 1 -- 172.21.15.6:0/2537409736 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d2c004bb0 con 0x7f4d34108740 2024-05-09T04:50:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.185+0000 7f4d3b600700 1 -- 172.21.15.6:0/2537409736 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d3419ff70 con 0x7f4d34108740 2024-05-09T04:50:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.185+0000 7f4d32000700 1 -- 172.21.15.6:0/2537409736 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c004020 con 0x7f4d34108740 2024-05-09T04:50:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.186+0000 7f4d3b600700 1 -- 172.21.15.6:0/2537409736 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d341a0460 con 0x7f4d34108740 2024-05-09T04:50:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.187+0000 7f4d3b600700 1 -- 172.21.15.6:0/2537409736 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d340512c0 con 0x7f4d34108740 2024-05-09T04:50:03.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.189+0000 7f4d32000700 1 -- 172.21.15.6:0/2537409736 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f4d2c01b030 con 0x7f4d34108740 2024-05-09T04:50:03.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.191+0000 7f4d32000700 1 --2- 172.21.15.6:0/2537409736 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4d180b2880 0x7f4d180b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:03.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.191+0000 7f4d32000700 1 -- 172.21.15.6:0/2537409736 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f4d2c0e2960 con 0x7f4d34108740 2024-05-09T04:50:03.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.191+0000 7f4d38c00700 1 --2- 172.21.15.6:0/2537409736 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4d180b2880 0x7f4d180b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:03.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.192+0000 7f4d38c00700 1 --2- 172.21.15.6:0/2537409736 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4d180b2880 0x7f4d180b4d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4d24009680 tx=0x7f4d24019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:03.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.193+0000 7f4d32000700 1 -- 172.21.15.6:0/2537409736 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4d2c022020 con 0x7f4d34108740 2024-05-09T04:50:03.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.435+0000 7f4d3b600700 1 -- 172.21.15.6:0/2537409736 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4d3402e5f0 con 0x7f4d34108740 2024-05-09T04:50:03.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.437+0000 7f4d32000700 1 -- 172.21.15.6:0/2537409736 <== mon.2 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24566 (secure 0 0 0) 0x7f4d2c0b0020 con 0x7f4d34108740 2024-05-09T04:50:03.441 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:50:03.441 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":87,"fsid":"f74d813c-0dbd-11ef-bc97-c7b262605968","created":"2024-05-09T04:39:35.287470+0000","modified":"2024-05-09T04:49:56.686042+0000","last_up_change":"2024-05-09T04:48:16.919842+0000","last_in_change":"2024-05-09T04:47:56.452645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:43:47.538306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:49:28.805447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T04:49:32.970287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T04:49:45.637000+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1b732fd3-e648-4b0d-aece-083bdaaa5d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6803","nonce":1738750914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6805","nonce":1738750914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6809","nonce":1738750914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6807","nonce":1738750914}]},"public_addr":"172.21.15.6:6803/1738750914","cluster_addr":"172.21.15.6:6805/1738750914","heartbeat_back_addr":"172.21.15.6:6809/1738750914","heartbeat_front_addr":"172.21.15.6:6807/1738750914","state":["exists","up"]},{"osd":1,"uuid":"9770c07d-4243-43bf-a0cd-9d2ac558556c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6801","nonce":2552207737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6803","nonce":2552207737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6807","nonce":2552207737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6805","nonce":2552207737}]},"public_addr":"172.21.15.169:6801/2552207737","cluster_addr":"172.21.15.169:6803/2552207737","heartbeat_back_addr":"172.21.15.169:6807/2552207737","heartbeat_front_addr":"172.21.15.169:6805/2552207737","state":["exists","up"]},{"osd":2,"uuid":"3f50ed89-b2a8-415e-b78d-8510d98d9368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6801","nonce":231435105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6803","nonce":231435105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6807","nonce":231435105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6805","nonce":231435105}]},"public_addr":"172.21.15.186:6801/231435105","cluster_addr":"172.21.15.186:6803/231435105","heartbeat_back_addr":"172.21.15.186:6807/231435105","heartbeat_front_addr":"172.21.15.186:6805/231435105","state":["exists","up"]},{"osd":3,"uuid":"afcbc36b-5669-4044-8dc5-2ebfe7970774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6811","nonce":2846613703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6813","nonce":2846613703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6817","nonce":2846613703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6815","nonce":2846613703}]},"public_addr":"172.21.15.6:6811/2846613703","cluster_addr":"172.21.15.6:6813/2846613703","heartbeat_back_addr":"172.21.15.6:6817/2846613703","heartbeat_front_addr":"172.21.15.6:6815/2846613703","state":["exists","up"]},{"osd":4,"uuid":"bb84cc5e-0297-4e23-be50-7ee2f7e59665","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6809","nonce":1928722453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6811","nonce":1928722453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6815","nonce":1928722453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6813","nonce":1928722453}]},"public_addr":"172.21.15.169:6809/1928722453","cluster_addr":"172.21.15.169:6811/1928722453","heartbeat_back_addr":"172.21.15.169:6815/1928722453","heartbeat_front_addr":"172.21.15.169:6813/1928722453","state":["exists","up"]},{"osd":5,"uuid":"830dc7f3-352e-4b01-8d4c-a9b68dcb40d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6809","nonce":3644445439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6811","nonce":3644445439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6815","nonce":3644445439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6813","nonce":3644445439}]},"public_addr":"172.21.15.186:6809/3644445439","cluster_addr":"172.21.15.186:6811/3644445439","heartbeat_back_addr":"172.21.15.186:6815/3644445439","heartbeat_front_addr":"172.21.15.186:6813/3644445439","state":["exists","up"]},{"osd":6,"uuid":"2e8478cb-c8c9-47ce-a3a5-48ffc959ab24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6819","nonce":3068252891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6821","nonce":3068252891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6825","nonce":3068252891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6823","nonce":3068252891}]},"public_addr":"172.21.15.6:6819/3068252891","cluster_addr":"172.21.15.6:6821/3068252891","heartbeat_back_addr":"172.21.15.6:6825/3068252891","heartbeat_front_addr":"172.21.15.6:6823/3068252891","state":["exists","up"]},{"osd":7,"uuid":"0f432ee2-42a4-4efa-9233-8e5a2fd65d10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6817","nonce":1303584711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6819","nonce":1303584711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6823","nonce":1303584711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6821","nonce":1303584711}]},"public_addr":"172.21.15.169:6817/1303584711","cluster_addr":"172.21.15.169:6819/1303584711","heartbeat_back_addr":"172.21.15.169:6823/1303584711","heartbeat_front_addr":"172.21.15.169:6821/1303584711","state":["exists","up"]},{"osd":8,"uuid":"cf5dca81-131b-48e7-9417-d01d4be47316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6817","nonce":3475163771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6819","nonce":3475163771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6823","nonce":3475163771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6821","nonce":3475163771}]},"public_addr":"172.21.15.186:6817/3475163771","cluster_addr":"172.21.15.186:6819/3475163771","heartbeat_back_addr":"172.21.15.186:6823/3475163771","heartbeat_front_addr":"172.21.15.186:6821/3475163771","state":["exists","up"]},{"osd":9,"uuid":"bca7bedd-1a57-4465-9033-064ca468add2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6827","nonce":3590990973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6829","nonce":3590990973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6833","nonce":3590990973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6831","nonce":3590990973}]},"public_addr":"172.21.15.6:6827/3590990973","cluster_addr":"172.21.15.6:6829/3590990973","heartbeat_back_addr":"172.21.15.6:6833/3590990973","heartbeat_front_addr":"172.21.15.6:6831/3590990973","state":["exists","up"]},{"osd":10,"uuid":"97b0e058-64f8-42ee-9b99-3ef8e39c8a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6825","nonce":61007341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6827","nonce":61007341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6831","nonce":61007341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6829","nonce":61007341}]},"public_addr":"172.21.15.169:6825/61007341","cluster_addr":"172.21.15.169:6827/61007341","heartbeat_back_addr":"172.21.15.169:6831/61007341","heartbeat_front_addr":"172.21.15.169:6829/61007341","state":["exists","up"]},{"osd":11,"uuid":"7f2220b0-24cf-48b2-95ae-1d66e00e57fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6825","nonce":1138940441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6827","nonce":1138940441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6831","nonce":1138940441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6829","nonce":1138940441}]},"public_addr":"172.21.15.186:6825/1138940441","cluster_addr":"172.21.15.186:6827/1138940441","heartbeat_back_addr":"172.21.15.186:6831/1138940441","heartbeat_front_addr":"172.21.15.186:6829/1138940441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:53.766295+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:20.195243+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:44.153812+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:20.472086+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:48.415371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:13.795415+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:50.236398+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:18.066623+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:43.781009+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:47:20.084719+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:47:48.219901+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:48:14.965536+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/3742411767":"2024-05-10T04:40:55.357057+0000","172.21.15.6:0/2410771702":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6801/937713898":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6800/937713898":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6801/3021503778":"2024-05-10T04:40:19.639151+0000","172.21.15.6:6800/3021503778":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/2559510850":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/3810474743":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/4133970555":"2024-05-10T04:40:55.357057+0000","172.21.15.6:0/664817743":"2024-05-10T04:40:19.639151+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:50:03.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.441+0000 7f4d3b600700 1 -- 172.21.15.6:0/2537409736 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4d180b2880 msgr2=0x7f4d180b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:03.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.441+0000 7f4d3b600700 1 --2- 172.21.15.6:0/2537409736 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4d180b2880 0x7f4d180b4d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4d24009680 tx=0x7f4d24019040 comp rx=0 tx=0).stop 2024-05-09T04:50:03.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.441+0000 7f4d3b600700 1 -- 172.21.15.6:0/2537409736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d34108740 msgr2=0x7f4d3419f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:03.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.441+0000 7f4d3b600700 1 --2- 172.21.15.6:0/2537409736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d34108740 0x7f4d3419f640 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c00efc0 tx=0x7f4d2c00c3a0 comp rx=0 tx=0).stop 2024-05-09T04:50:03.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.441+0000 7f4d3b600700 1 -- 172.21.15.6:0/2537409736 shutdown_connections 2024-05-09T04:50:03.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.441+0000 7f4d3b600700 1 --2- 172.21.15.6:0/2537409736 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4d180b2880 0x7f4d180b4d90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:03.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.441+0000 7f4d3b600700 1 --2- 172.21.15.6:0/2537409736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d34108740 0x7f4d3419f640 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:03.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.441+0000 7f4d3b600700 1 --2- 172.21.15.6:0/2537409736 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d341034c0 0x7f4d3419ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:03.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.441+0000 7f4d3b600700 1 --2- 172.21.15.6:0/2537409736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d34107df0 0x7f4d3419b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:03.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.441+0000 7f4d3b600700 1 -- 172.21.15.6:0/2537409736 >> 172.21.15.6:0/2537409736 conn(0x7f4d340fedc0 msgr2=0x7f4d3406b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:03.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.441+0000 7f4d3b600700 1 -- 172.21.15.6:0/2537409736 shutdown_connections 2024-05-09T04:50:03.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:03.441+0000 7f4d3b600700 1 -- 172.21.15.6:0/2537409736 wait complete. 2024-05-09T04:50:04.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:03 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/2537409736' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:50:04.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:03 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/2537409736' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:50:04.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:03 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/2537409736' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:50:04.312 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T04:50:04.799 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:50:05.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:04 smithi186 ceph-mon[34668]: pgmap v351: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-09T04:50:05.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:04 smithi169 ceph-mon[34075]: pgmap v351: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-09T04:50:05.137 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:04 smithi006 ceph-mon[29835]: pgmap v351: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-09T04:50:07.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.028+0000 7f3ad7800700 1 -- 172.21.15.6:0/725848188 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ad010e8d0 msgr2=0x7f3ad0110de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:07.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.028+0000 7f3ad7800700 1 --2- 172.21.15.6:0/725848188 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ad010e8d0 0x7f3ad0110de0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3ac800b0a0 tx=0x7f3ac800b3b0 comp rx=0 tx=0).stop 2024-05-09T04:50:07.030 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:06 smithi006 ceph-mon[29835]: pgmap v352: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-09T04:50:07.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.028+0000 7f3ad7800700 1 -- 172.21.15.6:0/725848188 shutdown_connections 2024-05-09T04:50:07.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.028+0000 7f3ad7800700 1 --2- 172.21.15.6:0/725848188 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ad010e8d0 0x7f3ad0110de0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:07.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.028+0000 7f3ad7800700 1 --2- 172.21.15.6:0/725848188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ad01054a0 0x7f3ad01058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:07.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.028+0000 7f3ad7800700 1 --2- 172.21.15.6:0/725848188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ad0105df0 0x7f3ad010e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:07.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.028+0000 7f3ad7800700 1 -- 172.21.15.6:0/725848188 >> 172.21.15.6:0/725848188 conn(0x7f3ad00fedf0 msgr2=0x7f3ad0101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:07.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.028+0000 7f3ad7800700 1 -- 172.21.15.6:0/725848188 shutdown_connections 2024-05-09T04:50:07.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.028+0000 7f3ad7800700 1 -- 172.21.15.6:0/725848188 wait complete. 2024-05-09T04:50:07.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.029+0000 7f3ad7800700 1 Processor -- start 2024-05-09T04:50:07.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.030+0000 7f3ad7800700 1 -- start start 2024-05-09T04:50:07.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.030+0000 7f3ad7800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ad01054a0 0x7f3ad019ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:07.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.030+0000 7f3ad7800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ad0105df0 0x7f3ad019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:07.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.030+0000 7f3ad7800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ad010e8d0 0x7f3ad019f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:07.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.030+0000 7f3ad7800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad01144b0 con 0x7f3ad0105df0 2024-05-09T04:50:07.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.030+0000 7f3ad7800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad0114330 con 0x7f3ad01054a0 2024-05-09T04:50:07.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.030+0000 7f3ad7800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad0114630 con 0x7f3ad010e8d0 2024-05-09T04:50:07.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.030+0000 7f3ad4e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ad01054a0 0x7f3ad019ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:07.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.030+0000 7f3ad4e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ad01054a0 0x7f3ad019ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:38038/0 (socket says 172.21.15.6:38038) 2024-05-09T04:50:07.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.030+0000 7f3ad4e00700 1 -- 172.21.15.6:0/2486728415 learned_addr learned my addr 172.21.15.6:0/2486728415 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:50:07.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.030+0000 7f3ad5800700 1 --2- 172.21.15.6:0/2486728415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ad010e8d0 0x7f3ad019f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:07.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.030+0000 7f3acfe00700 1 --2- 172.21.15.6:0/2486728415 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ad0105df0 0x7f3ad019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:07.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.031+0000 7f3ad5800700 1 -- 172.21.15.6:0/2486728415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ad01054a0 msgr2=0x7f3ad019ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:07.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.031+0000 7f3ad5800700 1 --2- 172.21.15.6:0/2486728415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ad01054a0 0x7f3ad019ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:07.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.031+0000 7f3ad5800700 1 -- 172.21.15.6:0/2486728415 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ad0105df0 msgr2=0x7f3ad019b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:07.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.031+0000 7f3ad5800700 1 --2- 172.21.15.6:0/2486728415 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ad0105df0 0x7f3ad019b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:07.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.031+0000 7f3ad5800700 1 -- 172.21.15.6:0/2486728415 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ad019fcb0 con 0x7f3ad010e8d0 2024-05-09T04:50:07.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.031+0000 7f3ad5800700 1 --2- 172.21.15.6:0/2486728415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ad010e8d0 0x7f3ad019f610 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3ac800b070 tx=0x7f3ac8004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:07.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.032+0000 7f3acd600700 1 -- 172.21.15.6:0/2486728415 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8021070 con 0x7f3ad010e8d0 2024-05-09T04:50:07.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.032+0000 7f3acd600700 1 -- 172.21.15.6:0/2486728415 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ac8007e30 con 0x7f3ad010e8d0 2024-05-09T04:50:07.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.032+0000 7f3acd600700 1 -- 172.21.15.6:0/2486728415 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac80045f0 con 0x7f3ad010e8d0 2024-05-09T04:50:07.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.032+0000 7f3ad7800700 1 -- 172.21.15.6:0/2486728415 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ad019ff40 con 0x7f3ad010e8d0 2024-05-09T04:50:07.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.032+0000 7f3ad7800700 1 -- 172.21.15.6:0/2486728415 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ad01a78c0 con 0x7f3ad010e8d0 2024-05-09T04:50:07.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.033+0000 7f3ad7800700 1 -- 172.21.15.6:0/2486728415 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ad0076290 con 0x7f3ad010e8d0 2024-05-09T04:50:07.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.036+0000 7f3acd600700 1 -- 172.21.15.6:0/2486728415 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f3ac8007980 con 0x7f3ad010e8d0 2024-05-09T04:50:07.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.037+0000 7f3acd600700 1 --2- 172.21.15.6:0/2486728415 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f3ab80b2900 0x7f3ab80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:07.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.037+0000 7f3acd600700 1 -- 172.21.15.6:0/2486728415 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f3ac800f020 con 0x7f3ad010e8d0 2024-05-09T04:50:07.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.037+0000 7f3ad4e00700 1 --2- 172.21.15.6:0/2486728415 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f3ab80b2900 0x7f3ab80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:07.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.038+0000 7f3ad4e00700 1 --2- 172.21.15.6:0/2486728415 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f3ab80b2900 0x7f3ab80b4e10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4005950 tx=0x7f3ac4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:07.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.039+0000 7f3acd600700 1 -- 172.21.15.6:0/2486728415 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3ac802a020 con 0x7f3ad010e8d0 2024-05-09T04:50:07.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:06 smithi186 ceph-mon[34668]: pgmap v352: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-09T04:50:07.118 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:06 smithi169 ceph-mon[34075]: pgmap v352: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-09T04:50:07.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.281+0000 7f3ad7800700 1 -- 172.21.15.6:0/2486728415 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3ad002e5f0 con 0x7f3ad010e8d0 2024-05-09T04:50:07.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.284+0000 7f3acd600700 1 -- 172.21.15.6:0/2486728415 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24566 (secure 0 0 0) 0x7f3ac80ab590 con 0x7f3ad010e8d0 2024-05-09T04:50:07.285 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:50:07.285 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":87,"fsid":"f74d813c-0dbd-11ef-bc97-c7b262605968","created":"2024-05-09T04:39:35.287470+0000","modified":"2024-05-09T04:49:56.686042+0000","last_up_change":"2024-05-09T04:48:16.919842+0000","last_in_change":"2024-05-09T04:47:56.452645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:43:47.538306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:49:28.805447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T04:49:32.970287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T04:49:45.637000+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1b732fd3-e648-4b0d-aece-083bdaaa5d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6803","nonce":1738750914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6805","nonce":1738750914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6809","nonce":1738750914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6807","nonce":1738750914}]},"public_addr":"172.21.15.6:6803/1738750914","cluster_addr":"172.21.15.6:6805/1738750914","heartbeat_back_addr":"172.21.15.6:6809/1738750914","heartbeat_front_addr":"172.21.15.6:6807/1738750914","state":["exists","up"]},{"osd":1,"uuid":"9770c07d-4243-43bf-a0cd-9d2ac558556c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6801","nonce":2552207737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6803","nonce":2552207737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6807","nonce":2552207737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6805","nonce":2552207737}]},"public_addr":"172.21.15.169:6801/2552207737","cluster_addr":"172.21.15.169:6803/2552207737","heartbeat_back_addr":"172.21.15.169:6807/2552207737","heartbeat_front_addr":"172.21.15.169:6805/2552207737","state":["exists","up"]},{"osd":2,"uuid":"3f50ed89-b2a8-415e-b78d-8510d98d9368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6801","nonce":231435105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6803","nonce":231435105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6807","nonce":231435105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6805","nonce":231435105}]},"public_addr":"172.21.15.186:6801/231435105","cluster_addr":"172.21.15.186:6803/231435105","heartbeat_back_addr":"172.21.15.186:6807/231435105","heartbeat_front_addr":"172.21.15.186:6805/231435105","state":["exists","up"]},{"osd":3,"uuid":"afcbc36b-5669-4044-8dc5-2ebfe7970774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6811","nonce":2846613703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6813","nonce":2846613703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6817","nonce":2846613703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6815","nonce":2846613703}]},"public_addr":"172.21.15.6:6811/2846613703","cluster_addr":"172.21.15.6:6813/2846613703","heartbeat_back_addr":"172.21.15.6:6817/2846613703","heartbeat_front_addr":"172.21.15.6:6815/2846613703","state":["exists","up"]},{"osd":4,"uuid":"bb84cc5e-0297-4e23-be50-7ee2f7e59665","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6809","nonce":1928722453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6811","nonce":1928722453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6815","nonce":1928722453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6813","nonce":1928722453}]},"public_addr":"172.21.15.169:6809/1928722453","cluster_addr":"172.21.15.169:6811/1928722453","heartbeat_back_addr":"172.21.15.169:6815/1928722453","heartbeat_front_addr":"172.21.15.169:6813/1928722453","state":["exists","up"]},{"osd":5,"uuid":"830dc7f3-352e-4b01-8d4c-a9b68dcb40d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6809","nonce":3644445439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6811","nonce":3644445439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6815","nonce":3644445439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6813","nonce":3644445439}]},"public_addr":"172.21.15.186:6809/3644445439","cluster_addr":"172.21.15.186:6811/3644445439","heartbeat_back_addr":"172.21.15.186:6815/3644445439","heartbeat_front_addr":"172.21.15.186:6813/3644445439","state":["exists","up"]},{"osd":6,"uuid":"2e8478cb-c8c9-47ce-a3a5-48ffc959ab24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6819","nonce":3068252891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6821","nonce":3068252891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6825","nonce":3068252891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6823","nonce":3068252891}]},"public_addr":"172.21.15.6:6819/3068252891","cluster_addr":"172.21.15.6:6821/3068252891","heartbeat_back_addr":"172.21.15.6:6825/3068252891","heartbeat_front_addr":"172.21.15.6:6823/3068252891","state":["exists","up"]},{"osd":7,"uuid":"0f432ee2-42a4-4efa-9233-8e5a2fd65d10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6817","nonce":1303584711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6819","nonce":1303584711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6823","nonce":1303584711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6821","nonce":1303584711}]},"public_addr":"172.21.15.169:6817/1303584711","cluster_addr":"172.21.15.169:6819/1303584711","heartbeat_back_addr":"172.21.15.169:6823/1303584711","heartbeat_front_addr":"172.21.15.169:6821/1303584711","state":["exists","up"]},{"osd":8,"uuid":"cf5dca81-131b-48e7-9417-d01d4be47316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6817","nonce":3475163771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6819","nonce":3475163771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6823","nonce":3475163771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6821","nonce":3475163771}]},"public_addr":"172.21.15.186:6817/3475163771","cluster_addr":"172.21.15.186:6819/3475163771","heartbeat_back_addr":"172.21.15.186:6823/3475163771","heartbeat_front_addr":"172.21.15.186:6821/3475163771","state":["exists","up"]},{"osd":9,"uuid":"bca7bedd-1a57-4465-9033-064ca468add2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6827","nonce":3590990973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6829","nonce":3590990973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6833","nonce":3590990973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6831","nonce":3590990973}]},"public_addr":"172.21.15.6:6827/3590990973","cluster_addr":"172.21.15.6:6829/3590990973","heartbeat_back_addr":"172.21.15.6:6833/3590990973","heartbeat_front_addr":"172.21.15.6:6831/3590990973","state":["exists","up"]},{"osd":10,"uuid":"97b0e058-64f8-42ee-9b99-3ef8e39c8a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6825","nonce":61007341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6827","nonce":61007341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6831","nonce":61007341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6829","nonce":61007341}]},"public_addr":"172.21.15.169:6825/61007341","cluster_addr":"172.21.15.169:6827/61007341","heartbeat_back_addr":"172.21.15.169:6831/61007341","heartbeat_front_addr":"172.21.15.169:6829/61007341","state":["exists","up"]},{"osd":11,"uuid":"7f2220b0-24cf-48b2-95ae-1d66e00e57fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6825","nonce":1138940441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6827","nonce":1138940441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6831","nonce":1138940441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6829","nonce":1138940441}]},"public_addr":"172.21.15.186:6825/1138940441","cluster_addr":"172.21.15.186:6827/1138940441","heartbeat_back_addr":"172.21.15.186:6831/1138940441","heartbeat_front_addr":"172.21.15.186:6829/1138940441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:53.766295+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:20.195243+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:44.153812+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:20.472086+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:48.415371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:13.795415+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:50.236398+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:18.066623+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:43.781009+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:47:20.084719+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:47:48.219901+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:48:14.965536+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/3742411767":"2024-05-10T04:40:55.357057+0000","172.21.15.6:0/2410771702":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6801/937713898":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6800/937713898":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6801/3021503778":"2024-05-10T04:40:19.639151+0000","172.21.15.6:6800/3021503778":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/2559510850":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/3810474743":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/4133970555":"2024-05-10T04:40:55.357057+0000","172.21.15.6:0/664817743":"2024-05-10T04:40:19.639151+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:50:07.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.287+0000 7f3ad7800700 1 -- 172.21.15.6:0/2486728415 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f3ab80b2900 msgr2=0x7f3ab80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:07.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.287+0000 7f3ad7800700 1 --2- 172.21.15.6:0/2486728415 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f3ab80b2900 0x7f3ab80b4e10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4005950 tx=0x7f3ac4009260 comp rx=0 tx=0).stop 2024-05-09T04:50:07.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.287+0000 7f3ad7800700 1 -- 172.21.15.6:0/2486728415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ad010e8d0 msgr2=0x7f3ad019f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:07.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.287+0000 7f3ad7800700 1 --2- 172.21.15.6:0/2486728415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ad010e8d0 0x7f3ad019f610 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3ac800b070 tx=0x7f3ac8004c80 comp rx=0 tx=0).stop 2024-05-09T04:50:07.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.287+0000 7f3ad7800700 1 -- 172.21.15.6:0/2486728415 shutdown_connections 2024-05-09T04:50:07.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.287+0000 7f3ad7800700 1 --2- 172.21.15.6:0/2486728415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ad010e8d0 0x7f3ad019f610 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:07.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.287+0000 7f3ad7800700 1 --2- 172.21.15.6:0/2486728415 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f3ab80b2900 0x7f3ab80b4e10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:07.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.287+0000 7f3ad7800700 1 --2- 172.21.15.6:0/2486728415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ad01054a0 0x7f3ad019ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:07.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.287+0000 7f3ad7800700 1 --2- 172.21.15.6:0/2486728415 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ad0105df0 0x7f3ad019b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:07.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.287+0000 7f3ad7800700 1 -- 172.21.15.6:0/2486728415 >> 172.21.15.6:0/2486728415 conn(0x7f3ad00fedf0 msgr2=0x7f3ad00ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:07.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.287+0000 7f3ad7800700 1 -- 172.21.15.6:0/2486728415 shutdown_connections 2024-05-09T04:50:07.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:07.288+0000 7f3ad7800700 1 -- 172.21.15.6:0/2486728415 wait complete. 2024-05-09T04:50:07.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:07 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/2486728415' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:50:08.101 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:50:08.101 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:07 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/2486728415' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:50:08.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:07 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/2486728415' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:50:08.569 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:50:09.034 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:08 smithi006 ceph-mon[29835]: pgmap v353: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-09T04:50:09.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:08 smithi186 ceph-mon[34668]: pgmap v353: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-09T04:50:09.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:08 smithi169 ceph-mon[34075]: pgmap v353: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-09T04:50:10.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.903+0000 7f6461200700 1 -- 172.21.15.6:0/1963796774 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f645c104400 msgr2=0x7f645c104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:10.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.903+0000 7f6461200700 1 --2- 172.21.15.6:0/1963796774 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f645c104400 0x7f645c104810 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f644c0099c0 tx=0x7f644c009cd0 comp rx=0 tx=0).stop 2024-05-09T04:50:10.905 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:10 smithi006 ceph-mon[29835]: pgmap v354: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-09T04:50:10.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.903+0000 7f6461200700 1 -- 172.21.15.6:0/1963796774 shutdown_connections 2024-05-09T04:50:10.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.903+0000 7f6461200700 1 --2- 172.21.15.6:0/1963796774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f645c109680 0x7f645c110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:10.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.903+0000 7f6461200700 1 --2- 172.21.15.6:0/1963796774 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f645c104400 0x7f645c104810 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:10.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.903+0000 7f6461200700 1 --2- 172.21.15.6:0/1963796774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f645c108d30 0x7f645c109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:10.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.903+0000 7f6461200700 1 -- 172.21.15.6:0/1963796774 >> 172.21.15.6:0/1963796774 conn(0x7f645c079d10 msgr2=0x7f645c07a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:10.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.903+0000 7f6461200700 1 -- 172.21.15.6:0/1963796774 shutdown_connections 2024-05-09T04:50:10.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.903+0000 7f6461200700 1 -- 172.21.15.6:0/1963796774 wait complete. 2024-05-09T04:50:10.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.905+0000 7f6461200700 1 Processor -- start 2024-05-09T04:50:10.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.905+0000 7f6461200700 1 -- start start 2024-05-09T04:50:10.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.905+0000 7f6461200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f645c104400 0x7f645c19ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:10.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.905+0000 7f6461200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f645c108d30 0x7f645c19d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:10.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.906+0000 7f6461200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f645c109680 0x7f645c1a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:10.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.906+0000 7f6461200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f645c114300 con 0x7f645c108d30 2024-05-09T04:50:10.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.906+0000 7f6461200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f645c114180 con 0x7f645c104400 2024-05-09T04:50:10.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.906+0000 7f6461200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f645c114480 con 0x7f645c109680 2024-05-09T04:50:10.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.906+0000 7f6452a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f645c108d30 0x7f645c19d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:10.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.906+0000 7f645b400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f645c104400 0x7f645c19ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:10.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.906+0000 7f6452a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f645c108d30 0x7f645c19d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41384/0 (socket says 172.21.15.6:41384) 2024-05-09T04:50:10.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.906+0000 7f6452a00700 1 -- 172.21.15.6:0/3506038091 learned_addr learned my addr 172.21.15.6:0/3506038091 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:50:10.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.906+0000 7f645be00700 1 --2- 172.21.15.6:0/3506038091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f645c109680 0x7f645c1a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:10.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.906+0000 7f6452a00700 1 -- 172.21.15.6:0/3506038091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f645c109680 msgr2=0x7f645c1a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:10.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.906+0000 7f6452a00700 1 --2- 172.21.15.6:0/3506038091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f645c109680 0x7f645c1a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:10.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.906+0000 7f6452a00700 1 -- 172.21.15.6:0/3506038091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f645c104400 msgr2=0x7f645c19ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:10.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.906+0000 7f6452a00700 1 --2- 172.21.15.6:0/3506038091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f645c104400 0x7f645c19ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:10.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.906+0000 7f6452a00700 1 -- 172.21.15.6:0/3506038091 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645c1a1d40 con 0x7f645c108d30 2024-05-09T04:50:10.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.906+0000 7f645b400700 1 --2- 172.21.15.6:0/3506038091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f645c104400 0x7f645c19ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:50:10.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.906+0000 7f645be00700 1 --2- 172.21.15.6:0/3506038091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f645c109680 0x7f645c1a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:50:10.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.907+0000 7f6452a00700 1 --2- 172.21.15.6:0/3506038091 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f645c108d30 0x7f645c19d410 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f644400e970 tx=0x7f644400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:10.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.907+0000 7f6458c00700 1 -- 172.21.15.6:0/3506038091 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6444004a30 con 0x7f645c108d30 2024-05-09T04:50:10.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.907+0000 7f6458c00700 1 -- 172.21.15.6:0/3506038091 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f644400cc90 con 0x7f645c108d30 2024-05-09T04:50:10.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.907+0000 7f6461200700 1 -- 172.21.15.6:0/3506038091 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f645c1a2030 con 0x7f645c108d30 2024-05-09T04:50:10.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.907+0000 7f6458c00700 1 -- 172.21.15.6:0/3506038091 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6444005610 con 0x7f645c108d30 2024-05-09T04:50:10.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.907+0000 7f6461200700 1 -- 172.21.15.6:0/3506038091 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f645c1a9950 con 0x7f645c108d30 2024-05-09T04:50:10.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.908+0000 7f6461200700 1 -- 172.21.15.6:0/3506038091 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f645c1058b0 con 0x7f645c108d30 2024-05-09T04:50:10.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.911+0000 7f6458c00700 1 -- 172.21.15.6:0/3506038091 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f644400c7e0 con 0x7f645c108d30 2024-05-09T04:50:10.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.913+0000 7f6458c00700 1 --2- 172.21.15.6:0/3506038091 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f64400b2880 0x7f64400b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:10.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.913+0000 7f6458c00700 1 -- 172.21.15.6:0/3506038091 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f64440e2950 con 0x7f645c108d30 2024-05-09T04:50:10.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.913+0000 7f645b400700 1 --2- 172.21.15.6:0/3506038091 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f64400b2880 0x7f64400b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:10.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.914+0000 7f645b400700 1 --2- 172.21.15.6:0/3506038091 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f64400b2880 0x7f64400b4d90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f644c000c00 tx=0x7f644c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:10.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:10.916+0000 7f6458c00700 1 -- 172.21.15.6:0/3506038091 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6444054df0 con 0x7f645c108d30 2024-05-09T04:50:11.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:10 smithi186 ceph-mon[34668]: pgmap v354: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-09T04:50:11.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:10 smithi169 ceph-mon[34075]: pgmap v354: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-09T04:50:11.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:11.206+0000 7f6461200700 1 -- 172.21.15.6:0/3506038091 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f645c0512c0 con 0x7f645c108d30 2024-05-09T04:50:11.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:11.207+0000 7f6458c00700 1 -- 172.21.15.6:0/3506038091 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11791 (secure 0 0 0) 0x7f64440aa2d0 con 0x7f645c108d30 2024-05-09T04:50:11.208 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:50:11.209 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/162094599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":162094599},{"type":"v1","addr":"172.21.15.6:6835","nonce":162094599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/589294363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":589294363},{"type":"v1","addr":"172.21.15.6:6841","nonce":589294363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/2977145206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2977145206},{"type":"v1","addr":"172.21.15.186:6833","nonce":2977145206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/133345948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":133345948},{"type":"v1","addr":"172.21.15.169:6833","nonce":133345948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/1281521088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":1281521088},{"type":"v1","addr":"172.21.15.6:6837","nonce":1281521088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2268322007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2268322007},{"type":"v1","addr":"172.21.15.186:6835","nonce":2268322007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3811005052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3811005052},{"type":"v1","addr":"172.21.15.169:6835","nonce":3811005052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1647607118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1647607118},{"type":"v1","addr":"172.21.15.186:6837","nonce":1647607118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/3954883105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":3954883105},{"type":"v1","addr":"172.21.15.186:6839","nonce":3954883105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24512,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1845715294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1845715294},{"type":"v1","addr":"172.21.15.169:6837","nonce":1845715294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24530,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/2780306492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":2780306492},{"type":"v1","addr":"172.21.15.6:6839","nonce":2780306492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T04:49:36.920376+0000","modified":"2024-05-09T04:49:55.501674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24554},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.169:6839/614388771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":614388771},{"type":"v1","addr":"172.21.15.169:6839","nonce":614388771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24554,"qdb_cluster":[24554]},"id":1}]} 2024-05-09T04:50:11.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:11.210+0000 7f6461200700 1 -- 172.21.15.6:0/3506038091 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f64400b2880 msgr2=0x7f64400b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:11.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:11.210+0000 7f6461200700 1 --2- 172.21.15.6:0/3506038091 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f64400b2880 0x7f64400b4d90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f644c000c00 tx=0x7f644c01a040 comp rx=0 tx=0).stop 2024-05-09T04:50:11.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:11.210+0000 7f6461200700 1 -- 172.21.15.6:0/3506038091 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f645c108d30 msgr2=0x7f645c19d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:11.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:11.210+0000 7f6461200700 1 --2- 172.21.15.6:0/3506038091 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f645c108d30 0x7f645c19d410 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f644400e970 tx=0x7f644400ec80 comp rx=0 tx=0).stop 2024-05-09T04:50:11.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:11.211+0000 7f6461200700 1 -- 172.21.15.6:0/3506038091 shutdown_connections 2024-05-09T04:50:11.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:11.211+0000 7f6461200700 1 --2- 172.21.15.6:0/3506038091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f645c109680 0x7f645c1a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:11.211+0000 7f6461200700 1 --2- 172.21.15.6:0/3506038091 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f64400b2880 0x7f64400b4d90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:11.211+0000 7f6461200700 1 --2- 172.21.15.6:0/3506038091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f645c104400 0x7f645c19ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:11.211+0000 7f6461200700 1 --2- 172.21.15.6:0/3506038091 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f645c108d30 0x7f645c19d410 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:11.211+0000 7f6461200700 1 -- 172.21.15.6:0/3506038091 >> 172.21.15.6:0/3506038091 conn(0x7f645c079d10 msgr2=0x7f645c101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:11.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:11.211+0000 7f6461200700 1 -- 172.21.15.6:0/3506038091 shutdown_connections 2024-05-09T04:50:11.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:11.211+0000 7f6461200700 1 -- 172.21.15.6:0/3506038091 wait complete. 2024-05-09T04:50:11.213 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 17 2024-05-09T04:50:11.938 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:11 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/3506038091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:50:11.994 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T04:50:12.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:11 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/3506038091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:50:12.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:11 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/3506038091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:50:12.462 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:50:12.866 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:12 smithi006 ceph-mon[29835]: pgmap v355: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:13.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:12 smithi186 ceph-mon[34668]: pgmap v355: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:13.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:12 smithi169 ceph-mon[34075]: pgmap v355: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:14.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.694+0000 7ff097200700 1 -- 172.21.15.6:0/2126084334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff09010e8d0 msgr2=0x7ff090110de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:14.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.694+0000 7ff097200700 1 --2- 172.21.15.6:0/2126084334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff09010e8d0 0x7ff090110de0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff088009f90 tx=0x7ff08800b330 comp rx=0 tx=0).stop 2024-05-09T04:50:14.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.694+0000 7ff097200700 1 -- 172.21.15.6:0/2126084334 shutdown_connections 2024-05-09T04:50:14.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.694+0000 7ff097200700 1 --2- 172.21.15.6:0/2126084334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff09010e8d0 0x7ff090110de0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.694+0000 7ff097200700 1 --2- 172.21.15.6:0/2126084334 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff090105460 0x7ff090105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.694+0000 7ff097200700 1 --2- 172.21.15.6:0/2126084334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff090105db0 0x7ff09010e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.694+0000 7ff097200700 1 -- 172.21.15.6:0/2126084334 >> 172.21.15.6:0/2126084334 conn(0x7ff0900fedf0 msgr2=0x7ff090101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:14.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.694+0000 7ff097200700 1 -- 172.21.15.6:0/2126084334 shutdown_connections 2024-05-09T04:50:14.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.695+0000 7ff097200700 1 -- 172.21.15.6:0/2126084334 wait complete. 2024-05-09T04:50:14.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.696+0000 7ff097200700 1 Processor -- start 2024-05-09T04:50:14.697 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.696+0000 7ff097200700 1 -- start start 2024-05-09T04:50:14.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.697+0000 7ff097200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff090105460 0x7ff09019add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:14.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.697+0000 7ff097200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff090105db0 0x7ff09019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:14.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.697+0000 7ff097200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff09010e8d0 0x7ff09019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:14.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.697+0000 7ff097200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff090114480 con 0x7ff09010e8d0 2024-05-09T04:50:14.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.697+0000 7ff097200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff090114300 con 0x7ff090105460 2024-05-09T04:50:14.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.697+0000 7ff097200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff090114600 con 0x7ff090105db0 2024-05-09T04:50:14.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.697+0000 7ff08fe00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff090105460 0x7ff09019add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:14.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.697+0000 7ff08fe00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff090105460 0x7ff09019add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:38078/0 (socket says 172.21.15.6:38078) 2024-05-09T04:50:14.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.697+0000 7ff08fe00700 1 -- 172.21.15.6:0/1443647399 learned_addr learned my addr 172.21.15.6:0/1443647399 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:50:14.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.697+0000 7ff095200700 1 --2- 172.21.15.6:0/1443647399 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff09010e8d0 0x7ff09019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:14.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.697+0000 7ff08f400700 1 --2- 172.21.15.6:0/1443647399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff090105db0 0x7ff09019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:14.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.698+0000 7ff095200700 1 -- 172.21.15.6:0/1443647399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff090105db0 msgr2=0x7ff09019b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:14.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.698+0000 7ff095200700 1 --2- 172.21.15.6:0/1443647399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff090105db0 0x7ff09019b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.698+0000 7ff095200700 1 -- 172.21.15.6:0/1443647399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff090105460 msgr2=0x7ff09019add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:14.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.698+0000 7ff095200700 1 --2- 172.21.15.6:0/1443647399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff090105460 0x7ff09019add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.698+0000 7ff095200700 1 -- 172.21.15.6:0/1443647399 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff09019fca0 con 0x7ff09010e8d0 2024-05-09T04:50:14.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.698+0000 7ff08fe00700 1 --2- 172.21.15.6:0/1443647399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff090105460 0x7ff09019add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:50:14.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.698+0000 7ff08f400700 1 --2- 172.21.15.6:0/1443647399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff090105db0 0x7ff09019b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:50:14.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.698+0000 7ff095200700 1 --2- 172.21.15.6:0/1443647399 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff09010e8d0 0x7ff09019f600 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff088009790 tx=0x7ff08800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:14.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.699+0000 7ff08cc00700 1 -- 172.21.15.6:0/1443647399 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff088021070 con 0x7ff09010e8d0 2024-05-09T04:50:14.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.699+0000 7ff097200700 1 -- 172.21.15.6:0/1443647399 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff09019ff30 con 0x7ff09010e8d0 2024-05-09T04:50:14.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.699+0000 7ff08cc00700 1 -- 172.21.15.6:0/1443647399 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff088007ac0 con 0x7ff09010e8d0 2024-05-09T04:50:14.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.699+0000 7ff08cc00700 1 -- 172.21.15.6:0/1443647399 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff088005040 con 0x7ff09010e8d0 2024-05-09T04:50:14.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.699+0000 7ff097200700 1 -- 172.21.15.6:0/1443647399 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff090103080 con 0x7ff09010e8d0 2024-05-09T04:50:14.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.700+0000 7ff097200700 1 -- 172.21.15.6:0/1443647399 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0900762d0 con 0x7ff09010e8d0 2024-05-09T04:50:14.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.703+0000 7ff08cc00700 1 -- 172.21.15.6:0/1443647399 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7ff088024070 con 0x7ff09010e8d0 2024-05-09T04:50:14.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.705+0000 7ff08cc00700 1 --2- 172.21.15.6:0/1443647399 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff0740b2900 0x7ff0740b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:14.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.706+0000 7ff08cc00700 1 -- 172.21.15.6:0/1443647399 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7ff08800f020 con 0x7ff09010e8d0 2024-05-09T04:50:14.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.706+0000 7ff08fe00700 1 --2- 172.21.15.6:0/1443647399 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff0740b2900 0x7ff0740b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:14.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.707+0000 7ff08fe00700 1 --2- 172.21.15.6:0/1443647399 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff0740b2900 0x7ff0740b4e10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff080009fd0 tx=0x7ff080009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:14.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.709+0000 7ff08cc00700 1 -- 172.21.15.6:0/1443647399 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff08802b020 con 0x7ff09010e8d0 2024-05-09T04:50:14.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.945+0000 7ff097200700 1 -- 172.21.15.6:0/1443647399 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff0900512c0 con 0x7ff09010e8d0 2024-05-09T04:50:14.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:14 smithi006 ceph-mon[29835]: pgmap v356: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:14.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.948+0000 7ff08cc00700 1 -- 172.21.15.6:0/1443647399 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24566 (secure 0 0 0) 0x7ff0900512c0 con 0x7ff09010e8d0 2024-05-09T04:50:14.950 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:50:14.950 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":87,"fsid":"f74d813c-0dbd-11ef-bc97-c7b262605968","created":"2024-05-09T04:39:35.287470+0000","modified":"2024-05-09T04:49:56.686042+0000","last_up_change":"2024-05-09T04:48:16.919842+0000","last_in_change":"2024-05-09T04:47:56.452645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:43:47.538306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:49:28.805447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T04:49:32.970287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T04:49:45.637000+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1b732fd3-e648-4b0d-aece-083bdaaa5d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6803","nonce":1738750914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6805","nonce":1738750914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6809","nonce":1738750914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6807","nonce":1738750914}]},"public_addr":"172.21.15.6:6803/1738750914","cluster_addr":"172.21.15.6:6805/1738750914","heartbeat_back_addr":"172.21.15.6:6809/1738750914","heartbeat_front_addr":"172.21.15.6:6807/1738750914","state":["exists","up"]},{"osd":1,"uuid":"9770c07d-4243-43bf-a0cd-9d2ac558556c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6801","nonce":2552207737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6803","nonce":2552207737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6807","nonce":2552207737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6805","nonce":2552207737}]},"public_addr":"172.21.15.169:6801/2552207737","cluster_addr":"172.21.15.169:6803/2552207737","heartbeat_back_addr":"172.21.15.169:6807/2552207737","heartbeat_front_addr":"172.21.15.169:6805/2552207737","state":["exists","up"]},{"osd":2,"uuid":"3f50ed89-b2a8-415e-b78d-8510d98d9368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6801","nonce":231435105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6803","nonce":231435105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6807","nonce":231435105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6805","nonce":231435105}]},"public_addr":"172.21.15.186:6801/231435105","cluster_addr":"172.21.15.186:6803/231435105","heartbeat_back_addr":"172.21.15.186:6807/231435105","heartbeat_front_addr":"172.21.15.186:6805/231435105","state":["exists","up"]},{"osd":3,"uuid":"afcbc36b-5669-4044-8dc5-2ebfe7970774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6811","nonce":2846613703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6813","nonce":2846613703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6817","nonce":2846613703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6815","nonce":2846613703}]},"public_addr":"172.21.15.6:6811/2846613703","cluster_addr":"172.21.15.6:6813/2846613703","heartbeat_back_addr":"172.21.15.6:6817/2846613703","heartbeat_front_addr":"172.21.15.6:6815/2846613703","state":["exists","up"]},{"osd":4,"uuid":"bb84cc5e-0297-4e23-be50-7ee2f7e59665","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6809","nonce":1928722453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6811","nonce":1928722453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6815","nonce":1928722453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6813","nonce":1928722453}]},"public_addr":"172.21.15.169:6809/1928722453","cluster_addr":"172.21.15.169:6811/1928722453","heartbeat_back_addr":"172.21.15.169:6815/1928722453","heartbeat_front_addr":"172.21.15.169:6813/1928722453","state":["exists","up"]},{"osd":5,"uuid":"830dc7f3-352e-4b01-8d4c-a9b68dcb40d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6809","nonce":3644445439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6811","nonce":3644445439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6815","nonce":3644445439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6813","nonce":3644445439}]},"public_addr":"172.21.15.186:6809/3644445439","cluster_addr":"172.21.15.186:6811/3644445439","heartbeat_back_addr":"172.21.15.186:6815/3644445439","heartbeat_front_addr":"172.21.15.186:6813/3644445439","state":["exists","up"]},{"osd":6,"uuid":"2e8478cb-c8c9-47ce-a3a5-48ffc959ab24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6819","nonce":3068252891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6821","nonce":3068252891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6825","nonce":3068252891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6823","nonce":3068252891}]},"public_addr":"172.21.15.6:6819/3068252891","cluster_addr":"172.21.15.6:6821/3068252891","heartbeat_back_addr":"172.21.15.6:6825/3068252891","heartbeat_front_addr":"172.21.15.6:6823/3068252891","state":["exists","up"]},{"osd":7,"uuid":"0f432ee2-42a4-4efa-9233-8e5a2fd65d10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6817","nonce":1303584711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6819","nonce":1303584711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6823","nonce":1303584711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6821","nonce":1303584711}]},"public_addr":"172.21.15.169:6817/1303584711","cluster_addr":"172.21.15.169:6819/1303584711","heartbeat_back_addr":"172.21.15.169:6823/1303584711","heartbeat_front_addr":"172.21.15.169:6821/1303584711","state":["exists","up"]},{"osd":8,"uuid":"cf5dca81-131b-48e7-9417-d01d4be47316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6817","nonce":3475163771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6819","nonce":3475163771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6823","nonce":3475163771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6821","nonce":3475163771}]},"public_addr":"172.21.15.186:6817/3475163771","cluster_addr":"172.21.15.186:6819/3475163771","heartbeat_back_addr":"172.21.15.186:6823/3475163771","heartbeat_front_addr":"172.21.15.186:6821/3475163771","state":["exists","up"]},{"osd":9,"uuid":"bca7bedd-1a57-4465-9033-064ca468add2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6827","nonce":3590990973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6829","nonce":3590990973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6833","nonce":3590990973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6831","nonce":3590990973}]},"public_addr":"172.21.15.6:6827/3590990973","cluster_addr":"172.21.15.6:6829/3590990973","heartbeat_back_addr":"172.21.15.6:6833/3590990973","heartbeat_front_addr":"172.21.15.6:6831/3590990973","state":["exists","up"]},{"osd":10,"uuid":"97b0e058-64f8-42ee-9b99-3ef8e39c8a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6825","nonce":61007341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6827","nonce":61007341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6831","nonce":61007341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6829","nonce":61007341}]},"public_addr":"172.21.15.169:6825/61007341","cluster_addr":"172.21.15.169:6827/61007341","heartbeat_back_addr":"172.21.15.169:6831/61007341","heartbeat_front_addr":"172.21.15.169:6829/61007341","state":["exists","up"]},{"osd":11,"uuid":"7f2220b0-24cf-48b2-95ae-1d66e00e57fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6825","nonce":1138940441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6827","nonce":1138940441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6831","nonce":1138940441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6829","nonce":1138940441}]},"public_addr":"172.21.15.186:6825/1138940441","cluster_addr":"172.21.15.186:6827/1138940441","heartbeat_back_addr":"172.21.15.186:6831/1138940441","heartbeat_front_addr":"172.21.15.186:6829/1138940441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:53.766295+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:20.195243+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:44.153812+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:20.472086+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:48.415371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:13.795415+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:50.236398+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:18.066623+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:43.781009+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:47:20.084719+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:47:48.219901+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:48:14.965536+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/3742411767":"2024-05-10T04:40:55.357057+0000","172.21.15.6:0/2410771702":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6801/937713898":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6800/937713898":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6801/3021503778":"2024-05-10T04:40:19.639151+0000","172.21.15.6:6800/3021503778":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/2559510850":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/3810474743":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/4133970555":"2024-05-10T04:40:55.357057+0000","172.21.15.6:0/664817743":"2024-05-10T04:40:19.639151+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:50:14.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.951+0000 7ff097200700 1 -- 172.21.15.6:0/1443647399 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff0740b2900 msgr2=0x7ff0740b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:14.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.951+0000 7ff097200700 1 --2- 172.21.15.6:0/1443647399 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff0740b2900 0x7ff0740b4e10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff080009fd0 tx=0x7ff080009310 comp rx=0 tx=0).stop 2024-05-09T04:50:14.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.951+0000 7ff097200700 1 -- 172.21.15.6:0/1443647399 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff09010e8d0 msgr2=0x7ff09019f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:14.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.951+0000 7ff097200700 1 --2- 172.21.15.6:0/1443647399 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff09010e8d0 0x7ff09019f600 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff088009790 tx=0x7ff08800ccb0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.952+0000 7ff097200700 1 -- 172.21.15.6:0/1443647399 shutdown_connections 2024-05-09T04:50:14.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.952+0000 7ff097200700 1 --2- 172.21.15.6:0/1443647399 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff09010e8d0 0x7ff09019f600 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.952+0000 7ff097200700 1 --2- 172.21.15.6:0/1443647399 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff0740b2900 0x7ff0740b4e10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.952+0000 7ff097200700 1 --2- 172.21.15.6:0/1443647399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff090105460 0x7ff09019add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.952+0000 7ff097200700 1 --2- 172.21.15.6:0/1443647399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff090105db0 0x7ff09019b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.952+0000 7ff097200700 1 -- 172.21.15.6:0/1443647399 >> 172.21.15.6:0/1443647399 conn(0x7ff0900fedf0 msgr2=0x7ff0901083e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:14.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.952+0000 7ff097200700 1 -- 172.21.15.6:0/1443647399 shutdown_connections 2024-05-09T04:50:14.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:14.952+0000 7ff097200700 1 -- 172.21.15.6:0/1443647399 wait complete. 2024-05-09T04:50:15.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:14 smithi186 ceph-mon[34668]: pgmap v356: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:15.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:14 smithi169 ceph-mon[34075]: pgmap v356: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:15.843 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T04:49:36.920376+0000', 'modified': '2024-05-09T04:49:55.501674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24554}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24554': {'gid': 24554, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.169:6839/614388771', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6838', 'nonce': 614388771}, {'type': 'v1', 'addr': '172.21.15.169:6839', 'nonce': 614388771}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24554, 'qdb_cluster': [24554]} 2024-05-09T04:50:15.844 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-09T04:50:15.844 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-09T04:50:16.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:15 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/1443647399' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:50:16.196 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:15 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1443647399' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:50:16.317 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:50:16.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:15 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/1443647399' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:50:17.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:16 smithi006 ceph-mon[29835]: pgmap v357: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:17.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:16 smithi186 ceph-mon[34668]: pgmap v357: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:17.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:16 smithi169 ceph-mon[34075]: pgmap v357: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:18.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.645+0000 7fa513200700 1 -- 172.21.15.6:0/1431939921 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa50c107df0 msgr2=0x7fa50c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:18.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.645+0000 7fa513200700 1 --2- 172.21.15.6:0/1431939921 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa50c107df0 0x7fa50c108200 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8009920 tx=0x7fa4f8009c30 comp rx=0 tx=0).stop 2024-05-09T04:50:18.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.645+0000 7fa513200700 1 -- 172.21.15.6:0/1431939921 shutdown_connections 2024-05-09T04:50:18.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.645+0000 7fa513200700 1 --2- 172.21.15.6:0/1431939921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa50c108740 0x7fa50c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:18.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.645+0000 7fa513200700 1 --2- 172.21.15.6:0/1431939921 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa50c1034c0 0x7fa50c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:18.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.645+0000 7fa513200700 1 --2- 172.21.15.6:0/1431939921 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa50c107df0 0x7fa50c108200 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:18.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.645+0000 7fa513200700 1 -- 172.21.15.6:0/1431939921 >> 172.21.15.6:0/1431939921 conn(0x7fa50c0fedc0 msgr2=0x7fa50c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:18.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.645+0000 7fa513200700 1 -- 172.21.15.6:0/1431939921 shutdown_connections 2024-05-09T04:50:18.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.645+0000 7fa513200700 1 -- 172.21.15.6:0/1431939921 wait complete. 2024-05-09T04:50:18.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.647+0000 7fa513200700 1 Processor -- start 2024-05-09T04:50:18.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.647+0000 7fa513200700 1 -- start start 2024-05-09T04:50:18.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.648+0000 7fa513200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa50c1034c0 0x7fa50c0ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:18.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.648+0000 7fa513200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa50c107df0 0x7fa50c0ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:18.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.648+0000 7fa513200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa50c108740 0x7fa50c0ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:18.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.648+0000 7fa513200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c1133c0 con 0x7fa50c108740 2024-05-09T04:50:18.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.648+0000 7fa513200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c113240 con 0x7fa50c1034c0 2024-05-09T04:50:18.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.648+0000 7fa513200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c113540 con 0x7fa50c107df0 2024-05-09T04:50:18.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.648+0000 7fa50be00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa50c1034c0 0x7fa50c0ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:18.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.648+0000 7fa50b400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa50c107df0 0x7fa50c0ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:18.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.648+0000 7fa511200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa50c108740 0x7fa50c0ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:18.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.648+0000 7fa50be00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa50c1034c0 0x7fa50c0ff2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:37774/0 (socket says 172.21.15.6:37774) 2024-05-09T04:50:18.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.648+0000 7fa50be00700 1 -- 172.21.15.6:0/1225542141 learned_addr learned my addr 172.21.15.6:0/1225542141 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:50:18.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.649+0000 7fa50be00700 1 -- 172.21.15.6:0/1225542141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa50c107df0 msgr2=0x7fa50c0ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:18.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.649+0000 7fa50be00700 1 --2- 172.21.15.6:0/1225542141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa50c107df0 0x7fa50c0ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:18.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.649+0000 7fa50be00700 1 -- 172.21.15.6:0/1225542141 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa50c108740 msgr2=0x7fa50c0ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:18.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.649+0000 7fa50be00700 1 --2- 172.21.15.6:0/1225542141 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa50c108740 0x7fa50c0ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:18.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.649+0000 7fa50be00700 1 -- 172.21.15.6:0/1225542141 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa50c1a9260 con 0x7fa50c1034c0 2024-05-09T04:50:18.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.649+0000 7fa50b400700 1 --2- 172.21.15.6:0/1225542141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa50c107df0 0x7fa50c0ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:50:18.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.649+0000 7fa50be00700 1 --2- 172.21.15.6:0/1225542141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa50c1034c0 0x7fa50c0ff2a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00ec10 tx=0x7fa4fc00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:18.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.649+0000 7fa508c00700 1 -- 172.21.15.6:0/1225542141 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc00cc90 con 0x7fa50c1034c0 2024-05-09T04:50:18.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.649+0000 7fa508c00700 1 -- 172.21.15.6:0/1225542141 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4fc00ce30 con 0x7fa50c1034c0 2024-05-09T04:50:18.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.649+0000 7fa513200700 1 -- 172.21.15.6:0/1225542141 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa50c1a9550 con 0x7fa50c1034c0 2024-05-09T04:50:18.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.650+0000 7fa508c00700 1 -- 172.21.15.6:0/1225542141 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc010430 con 0x7fa50c1034c0 2024-05-09T04:50:18.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.650+0000 7fa513200700 1 -- 172.21.15.6:0/1225542141 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa50c1a9ab0 con 0x7fa50c1034c0 2024-05-09T04:50:18.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.655+0000 7fa521d5ab80 1 Processor -- start 2024-05-09T04:50:18.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.655+0000 7fa521d5ab80 1 -- start start 2024-05-09T04:50:18.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.656+0000 7fa508c00700 1 -- 172.21.15.6:0/1225542141 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fa4fc0105d0 con 0x7fa50c1034c0 2024-05-09T04:50:18.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.656+0000 7fa521d5ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a690af4450 0x55a690c76ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:18.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.656+0000 7fa521d5ab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a690c77410 0x55a690c714c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:18.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.656+0000 7fa521d5ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a690c71a00 0x55a690c71e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:18.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.656+0000 7fa521d5ab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a690b94260 con 0x55a690c77410 2024-05-09T04:50:18.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.656+0000 7fa521d5ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a690b32090 con 0x55a690af4450 2024-05-09T04:50:18.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.656+0000 7fa521d5ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a690b90220 con 0x55a690c71a00 2024-05-09T04:50:18.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.656+0000 7fa50be00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a690c77410 0x55a690c714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:18.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.656+0000 7fa511200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a690af4450 0x55a690c76ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:18.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.656+0000 7fa511200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a690af4450 0x55a690c76ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:37780/0 (socket says 172.21.15.6:37780) 2024-05-09T04:50:18.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.656+0000 7fa511200700 1 -- 172.21.15.6:0/126059462 learned_addr learned my addr 172.21.15.6:0/126059462 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:50:18.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.656+0000 7fa50b400700 1 --2- 172.21.15.6:0/126059462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a690c71a00 0x55a690c71e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:18.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.657+0000 7fa50b400700 1 -- 172.21.15.6:0/126059462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a690af4450 msgr2=0x55a690c76ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:18.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.657+0000 7fa50b400700 1 --2- 172.21.15.6:0/126059462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a690af4450 0x55a690c76ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:18.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.657+0000 7fa50b400700 1 -- 172.21.15.6:0/126059462 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a690c77410 msgr2=0x55a690c714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:18.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.657+0000 7fa50b400700 1 --2- 172.21.15.6:0/126059462 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a690c77410 0x55a690c714c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:18.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.657+0000 7fa50b400700 1 -- 172.21.15.6:0/126059462 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a690a0f440 con 0x55a690c71a00 2024-05-09T04:50:18.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.657+0000 7fa50be00700 1 --2- 172.21.15.6:0/126059462 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a690c77410 0x55a690c714c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:50:18.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.658+0000 7fa50b400700 1 --2- 172.21.15.6:0/126059462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a690c71a00 0x55a690c71e10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8000c00 tx=0x7fa4f800bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6a9b9f6473bff60 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:18.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.658+0000 7fa501600700 1 -- 172.21.15.6:0/126059462 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f800f660 con 0x55a690c71a00 2024-05-09T04:50:18.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.658+0000 7fa501600700 1 -- 172.21.15.6:0/126059462 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4f8028070 con 0x55a690c71a00 2024-05-09T04:50:18.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.658+0000 7fa501600700 1 -- 172.21.15.6:0/126059462 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8004910 con 0x55a690c71a00 2024-05-09T04:50:18.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.658+0000 7fa508c00700 1 --2- 172.21.15.6:0/1225542141 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa4f00b2900 0x7fa4f00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:18.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.658+0000 7fa508c00700 1 -- 172.21.15.6:0/1225542141 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7fa4fc0e27b0 con 0x7fa50c1034c0 2024-05-09T04:50:18.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.659+0000 7fa511200700 1 --2- 172.21.15.6:0/1225542141 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa4f00b2900 0x7fa4f00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:18.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.659+0000 7fa521d5ab80 1 -- 172.21.15.6:0/126059462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a690c71a00 msgr2=0x55a690c71e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:18.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.659+0000 7fa521d5ab80 1 --2- 172.21.15.6:0/126059462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a690c71a00 0x55a690c71e10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8000c00 tx=0x7fa4f800bf60 comp rx=0 tx=0).stop 2024-05-09T04:50:18.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.660+0000 7fa521d5ab80 1 -- 172.21.15.6:0/126059462 shutdown_connections 2024-05-09T04:50:18.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.660+0000 7fa521d5ab80 1 --2- 172.21.15.6:0/126059462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a690c71a00 0x55a690c71e10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:18.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.660+0000 7fa521d5ab80 1 --2- 172.21.15.6:0/126059462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a690af4450 0x55a690c76ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:18.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.660+0000 7fa521d5ab80 1 --2- 172.21.15.6:0/126059462 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a690c77410 0x55a690c714c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:18.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.660+0000 7fa521d5ab80 1 -- 172.21.15.6:0/126059462 >> 172.21.15.6:0/126059462 conn(0x55a690b38e20 msgr2=0x55a690c6be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:18.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.660+0000 7fa521d5ab80 1 -- 172.21.15.6:0/126059462 shutdown_connections 2024-05-09T04:50:18.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.660+0000 7fa511200700 1 --2- 172.21.15.6:0/1225542141 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa4f00b2900 0x7fa4f00b4e10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa5040075c0 tx=0x7fa50400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:18.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.660+0000 7fa521d5ab80 1 -- 172.21.15.6:0/126059462 wait complete. 2024-05-09T04:50:18.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.661+0000 7fa521d5ab80 1 -- start start 2024-05-09T04:50:18.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.661+0000 7fa521d5ab80 1 Processor -- start 2024-05-09T04:50:18.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.661+0000 7fa521d5ab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a690af4450 0x55a690c7aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:18.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.661+0000 7fa521d5ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a690c71a00 0x55a690c7aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:18.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.661+0000 7fa521d5ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a690c77410 0x55a690c7b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:18.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.662+0000 7fa521d5ab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a690c7dea0 con 0x55a690af4450 2024-05-09T04:50:18.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.662+0000 7fa521d5ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a690af9670 con 0x55a690c71a00 2024-05-09T04:50:18.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.662+0000 7fa521d5ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a690c7e020 con 0x55a690c77410 2024-05-09T04:50:18.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.662+0000 7fa521d5ab80 10 client.0 _ll_register_callbacks cb 0x55a690b2b540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:50:18.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.662+0000 7fa50be00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a690af4450 0x55a690c7aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:18.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.662+0000 7fa50be00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a690af4450 0x55a690c7aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56678/0 (socket says 172.21.15.6:56678) 2024-05-09T04:50:18.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.662+0000 7fa50be00700 1 -- 172.21.15.6:0/1384347903 learned_addr learned my addr 172.21.15.6:0/1384347903 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:50:18.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.662+0000 7fa50b400700 1 --2- 172.21.15.6:0/1384347903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a690c71a00 0x55a690c7aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:18.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.662+0000 7fa511200700 1 --2- 172.21.15.6:0/1384347903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a690c77410 0x55a690c7b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:18.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.662+0000 7fa50b400700 1 -- 172.21.15.6:0/1384347903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a690c77410 msgr2=0x55a690c7b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:18.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.662+0000 7fa50b400700 1 --2- 172.21.15.6:0/1384347903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a690c77410 0x55a690c7b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:18.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.662+0000 7fa50b400700 1 -- 172.21.15.6:0/1384347903 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a690af4450 msgr2=0x55a690c7aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:18.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.662+0000 7fa50b400700 1 --2- 172.21.15.6:0/1384347903 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a690af4450 0x55a690c7aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:18.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.662+0000 7fa50b400700 1 -- 172.21.15.6:0/1384347903 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a690c87350 con 0x55a690c71a00 2024-05-09T04:50:18.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.663+0000 7fa50be00700 1 --2- 172.21.15.6:0/1384347903 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a690af4450 0x55a690c7aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:50:18.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.663+0000 7fa50b400700 1 --2- 172.21.15.6:0/1384347903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a690c71a00 0x55a690c7aff0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8009c70 tx=0x7fa4f8005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a54de25ea36fbd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:18.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.663+0000 7fa4f6000700 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-05-09T04:50:18.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.663+0000 7fa4f6000700 1 -- 172.21.15.6:0/1384347903 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8030070 con 0x55a690c71a00 2024-05-09T04:50:18.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.663+0000 7fa521d5ab80 1 -- 172.21.15.6:0/1384347903 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4f8028070 con 0x55a690c71a00 2024-05-09T04:50:18.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.663+0000 7fa4f6000700 1 -- 172.21.15.6:0/1384347903 <== mon.2 v2:172.21.15.169:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7fa4f8005680 con 0x55a690c71a00 2024-05-09T04:50:18.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.664+0000 7fa4f6000700 1 -- 172.21.15.6:0/1384347903 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fa4f803e050 con 0x55a690c71a00 2024-05-09T04:50:18.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.664+0000 7fa521d5ab80 10 client.24650 fetch_fsmap learned FSMap version 17 2024-05-09T04:50:18.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.664+0000 7fa521d5ab80 1 -- 172.21.15.6:0/1384347903 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55a690c88b80 con 0x55a690c71a00 2024-05-09T04:50:18.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.664+0000 7fa4f6000700 1 -- 172.21.15.6:0/1384347903 <== mon.2 v2:172.21.15.169:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4f8028070 con 0x55a690c71a00 2024-05-09T04:50:18.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.664+0000 7fa4f6000700 1 -- 172.21.15.6:0/1384347903 <== mon.2 v2:172.21.15.169:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7fa4f800b9b0 con 0x55a690c71a00 2024-05-09T04:50:18.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.664+0000 7fa521d5ab80 10 client.24650 fetch_fsmap finished waiting for FSMap version 17 2024-05-09T04:50:18.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.665+0000 7fa521d5ab80 10 client.24650 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T04:50:18.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.665+0000 7fa521d5ab80 20 client.24650 populate_metadata read hostname 'smithi006' 2024-05-09T04:50:18.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.665+0000 7fa521d5ab80 1 --2- 172.21.15.6:0/1384347903 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x55a690c89ed0 0x55a690c8c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:18.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.665+0000 7fa521d5ab80 4 client.24650 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:50:18.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.665+0000 7fa521d5ab80 1 -- 172.21.15.6:0/1384347903 --> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a690c3d820 con 0x55a690c89ed0 2024-05-09T04:50:18.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.665+0000 7fa50be00700 1 --2- 172.21.15.6:0/1384347903 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x55a690c89ed0 0x55a690c8c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:18.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.666+0000 7fa50be00700 1 --2- 172.21.15.6:0/1384347903 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x55a690c89ed0 0x55a690c8c040 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edc7685a460dfe0d server_cookie=4199d24b2d147491 in_seq=0 out_seq=0 2024-05-09T04:50:18.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.666+0000 7fa4f6000700 10 client.24650 ms_handle_connect on v2:172.21.15.169:6838/614388771 2024-05-09T04:50:18.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.667+0000 7fa4f6000700 1 -- 172.21.15.6:0/1384347903 <== mds.0 v2:172.21.15.169:6838/614388771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a690c3d820 con 0x55a690c89ed0 2024-05-09T04:50:18.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.667+0000 7fa4f6000700 10 client.24650 handle_command_reply: tid=0 2024-05-09T04:50:18.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.667+0000 7fa521d5ab80 1 client.24650 shutdown 2024-05-09T04:50:18.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.667+0000 7fa521d5ab80 1 -- 172.21.15.6:0/1384347903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a690c71a00 msgr2=0x55a690c7aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:18.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.667+0000 7fa521d5ab80 1 --2- 172.21.15.6:0/1384347903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a690c71a00 0x55a690c7aff0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8009c70 tx=0x7fa4f8005040 comp rx=0 tx=0).stop 2024-05-09T04:50:18.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.667+0000 7fa521d5ab80 1 -- 172.21.15.6:0/1384347903 shutdown_connections 2024-05-09T04:50:18.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.667+0000 7fa521d5ab80 1 --2- 172.21.15.6:0/1384347903 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x55a690c89ed0 0x55a690c8c040 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:18.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.667+0000 7fa521d5ab80 1 --2- 172.21.15.6:0/1384347903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a690c77410 0x55a690c7b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:18.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.667+0000 7fa521d5ab80 1 --2- 172.21.15.6:0/1384347903 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a690af4450 0x55a690c7aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:18.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.667+0000 7fa521d5ab80 1 --2- 172.21.15.6:0/1384347903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a690c71a00 0x55a690c7aff0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:18.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.667+0000 7fa521d5ab80 1 -- 172.21.15.6:0/1384347903 >> 172.21.15.6:0/1384347903 conn(0x55a690b38e20 msgr2=0x55a690c6be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:18.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.667+0000 7fa4f6000700 0 client.24650 ms_handle_reset on v2:172.21.15.169:6838/614388771 2024-05-09T04:50:18.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.667+0000 7fa521d5ab80 1 -- 172.21.15.6:0/1384347903 shutdown_connections 2024-05-09T04:50:18.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.667+0000 7fa521d5ab80 1 -- 172.21.15.6:0/1384347903 wait complete. 2024-05-09T04:50:18.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.668+0000 7fa521d5ab80 20 client.24650 trim_cache size 0 max 16384 2024-05-09T04:50:18.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.713+0000 7fa521d5ab80 1 Processor -- start 2024-05-09T04:50:18.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.713+0000 7fa521d5ab80 1 -- start start 2024-05-09T04:50:18.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.714+0000 7fa521d5ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a690af4450 0x55a690c714c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:18.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.714+0000 7fa521d5ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a690c71a00 0x55a690c752c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:18.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.714+0000 7fa521d5ab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a690c77410 0x55a690c86270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:18.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.714+0000 7fa521d5ab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a690c8a380 con 0x55a690c77410 2024-05-09T04:50:18.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.714+0000 7fa521d5ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a690c8aa50 con 0x55a690af4450 2024-05-09T04:50:18.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.714+0000 7fa521d5ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a690c8b120 con 0x55a690c71a00 2024-05-09T04:50:18.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.714+0000 7fa511200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a690c77410 0x55a690c86270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:18.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.714+0000 7fa50be00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a690af4450 0x55a690c714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:18.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.714+0000 7fa50b400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a690c71a00 0x55a690c752c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:18.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.714+0000 7fa50be00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a690af4450 0x55a690c714c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:37784/0 (socket says 172.21.15.6:37784) 2024-05-09T04:50:18.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.714+0000 7fa50b400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a690c71a00 0x55a690c752c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:43094/0 (socket says 172.21.15.6:43094) 2024-05-09T04:50:18.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.714+0000 7fa50be00700 1 -- 172.21.15.6:0/2167922634 learned_addr learned my addr 172.21.15.6:0/2167922634 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:50:18.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.715+0000 7fa50be00700 1 -- 172.21.15.6:0/2167922634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a690c71a00 msgr2=0x55a690c752c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:18.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.715+0000 7fa50be00700 1 --2- 172.21.15.6:0/2167922634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a690c71a00 0x55a690c752c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:18.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.715+0000 7fa50be00700 1 -- 172.21.15.6:0/2167922634 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a690c77410 msgr2=0x55a690c86270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:18.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.715+0000 7fa50be00700 1 --2- 172.21.15.6:0/2167922634 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a690c77410 0x55a690c86270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:18.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.715+0000 7fa50be00700 1 -- 172.21.15.6:0/2167922634 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a690c6ddc0 con 0x55a690af4450 2024-05-09T04:50:18.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.715+0000 7fa50be00700 1 --2- 172.21.15.6:0/2167922634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a690af4450 0x55a690c714c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc0e02e0 tx=0x7fa4fc0e8550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaf8382ff0838a70 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:18.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.716+0000 7fa500c00700 1 -- 172.21.15.6:0/2167922634 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc0a93e0 con 0x55a690af4450 2024-05-09T04:50:18.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.716+0000 7fa500c00700 1 -- 172.21.15.6:0/2167922634 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4fc0b2240 con 0x55a690af4450 2024-05-09T04:50:18.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.716+0000 7fa500c00700 1 -- 172.21.15.6:0/2167922634 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc0a5220 con 0x55a690af4450 2024-05-09T04:50:18.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.717+0000 7fa521d5ab80 1 -- 172.21.15.6:0/2167922634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a690af4450 msgr2=0x55a690c714c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:18.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.717+0000 7fa521d5ab80 1 --2- 172.21.15.6:0/2167922634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a690af4450 0x55a690c714c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc0e02e0 tx=0x7fa4fc0e8550 comp rx=0 tx=0).stop 2024-05-09T04:50:18.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.717+0000 7fa521d5ab80 1 -- 172.21.15.6:0/2167922634 shutdown_connections 2024-05-09T04:50:18.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.717+0000 7fa521d5ab80 1 --2- 172.21.15.6:0/2167922634 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a690c77410 0x55a690c86270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:18.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.717+0000 7fa521d5ab80 1 --2- 172.21.15.6:0/2167922634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a690af4450 0x55a690c714c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:18.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.717+0000 7fa521d5ab80 1 --2- 172.21.15.6:0/2167922634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a690c71a00 0x55a690c752c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:18.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.717+0000 7fa521d5ab80 1 -- 172.21.15.6:0/2167922634 >> 172.21.15.6:0/2167922634 conn(0x55a690b38e20 msgr2=0x55a690c6c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:18.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.717+0000 7fa521d5ab80 1 -- 172.21.15.6:0/2167922634 shutdown_connections 2024-05-09T04:50:18.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.717+0000 7fa521d5ab80 1 -- 172.21.15.6:0/2167922634 wait complete. 2024-05-09T04:50:18.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.717+0000 7fa521d5ab80 1 -- start start 2024-05-09T04:50:18.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.718+0000 7fa521d5ab80 1 Processor -- start 2024-05-09T04:50:18.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.718+0000 7fa521d5ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a690af4450 0x55a690c8fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:18.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.718+0000 7fa521d5ab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a690c71a00 0x55a690c8ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:18.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.718+0000 7fa521d5ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a690c77410 0x55a690c904c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:18.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.718+0000 7fa521d5ab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a690c8bec0 con 0x55a690c71a00 2024-05-09T04:50:18.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.718+0000 7fa521d5ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a690c8b7f0 con 0x55a690c77410 2024-05-09T04:50:18.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.718+0000 7fa521d5ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a690c8c590 con 0x55a690af4450 2024-05-09T04:50:18.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.718+0000 7fa521d5ab80 10 client.0 _ll_register_callbacks cb 0x55a690b2b540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:50:18.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.719+0000 7fa50b400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a690c71a00 0x55a690c8ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:18.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.719+0000 7fa50be00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a690af4450 0x55a690c8fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:18.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.719+0000 7fa50b400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a690c71a00 0x55a690c8ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56696/0 (socket says 172.21.15.6:56696) 2024-05-09T04:50:18.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.719+0000 7fa50b400700 1 -- 172.21.15.6:0/1672891019 learned_addr learned my addr 172.21.15.6:0/1672891019 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:50:18.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.719+0000 7fa511200700 1 --2- 172.21.15.6:0/1672891019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a690c77410 0x55a690c904c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:18.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.719+0000 7fa50be00700 1 -- 172.21.15.6:0/1672891019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a690c77410 msgr2=0x55a690c904c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:18.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.719+0000 7fa50be00700 1 --2- 172.21.15.6:0/1672891019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a690c77410 0x55a690c904c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:18.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.719+0000 7fa50be00700 1 -- 172.21.15.6:0/1672891019 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a690c71a00 msgr2=0x55a690c8ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:18.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.719+0000 7fa50be00700 1 --2- 172.21.15.6:0/1672891019 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a690c71a00 0x55a690c8ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:18.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.719+0000 7fa50be00700 1 -- 172.21.15.6:0/1672891019 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a690c94a90 con 0x55a690af4450 2024-05-09T04:50:18.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.719+0000 7fa50b400700 1 --2- 172.21.15.6:0/1672891019 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a690c71a00 0x55a690c8ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:50:18.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.719+0000 7fa511200700 1 --2- 172.21.15.6:0/1672891019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a690c77410 0x55a690c904c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:50:18.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.720+0000 7fa50be00700 1 --2- 172.21.15.6:0/1672891019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a690af4450 0x55a690c8fa40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc0e02e0 tx=0x7fa4fc003a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d86fa4c1319812ef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:18.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.720+0000 7fa4f6000700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-09T04:50:18.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.720+0000 7fa4f6000700 1 -- 172.21.15.6:0/1672891019 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc08e020 con 0x55a690af4450 2024-05-09T04:50:18.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.720+0000 7fa4f6000700 1 -- 172.21.15.6:0/1672891019 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7fa4fc0a9cf0 con 0x55a690af4450 2024-05-09T04:50:18.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.720+0000 7fa521d5ab80 1 -- 172.21.15.6:0/1672891019 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4fc0b2240 con 0x55a690af4450 2024-05-09T04:50:18.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.720+0000 7fa4f6000700 1 -- 172.21.15.6:0/1672891019 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fa4fc0a9200 con 0x55a690af4450 2024-05-09T04:50:18.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.720+0000 7fa521d5ab80 10 client.24637 fetch_fsmap learned FSMap version 17 2024-05-09T04:50:18.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.720+0000 7fa521d5ab80 1 -- 172.21.15.6:0/1672891019 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55a690cf6220 con 0x55a690af4450 2024-05-09T04:50:18.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.720+0000 7fa4f6000700 1 -- 172.21.15.6:0/1672891019 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4fc0b2240 con 0x55a690af4450 2024-05-09T04:50:18.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.721+0000 7fa4f6000700 1 -- 172.21.15.6:0/1672891019 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7fa4fc0b2460 con 0x55a690af4450 2024-05-09T04:50:18.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.721+0000 7fa521d5ab80 10 client.24637 fetch_fsmap finished waiting for FSMap version 17 2024-05-09T04:50:18.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.721+0000 7fa521d5ab80 10 client.24637 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T04:50:18.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.721+0000 7fa521d5ab80 20 client.24637 populate_metadata read hostname 'smithi006' 2024-05-09T04:50:18.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.721+0000 7fa521d5ab80 1 --2- 172.21.15.6:0/1672891019 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x55a690cf8180 0x55a690cf95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:18.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.721+0000 7fa521d5ab80 4 client.24637 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T04:50:18.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.721+0000 7fa521d5ab80 1 -- 172.21.15.6:0/1672891019 --> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] -- command(tid 0: {"prefix": "status"}) -- 0x55a690ba1c40 con 0x55a690cf8180 2024-05-09T04:50:18.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.721+0000 7fa50b400700 1 --2- 172.21.15.6:0/1672891019 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x55a690cf8180 0x55a690cf95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:18.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.722+0000 7fa4f6000700 10 client.24637 ms_handle_connect on v2:172.21.15.169:6838/614388771 2024-05-09T04:50:18.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.722+0000 7fa50b400700 1 --2- 172.21.15.6:0/1672891019 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x55a690cf8180 0x55a690cf95f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a23d43f9bf81063a server_cookie=1b4b7f7f0611355c in_seq=0 out_seq=0 2024-05-09T04:50:18.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.722+0000 7fa4f6000700 1 -- 172.21.15.6:0/1672891019 <== mds.0 v2:172.21.15.169:6838/614388771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55a690ba1c40 con 0x55a690cf8180 2024-05-09T04:50:18.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.722+0000 7fa4f6000700 10 client.24637 handle_command_reply: tid=0 2024-05-09T04:50:18.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.722+0000 7fa521d5ab80 1 client.24637 shutdown 2024-05-09T04:50:18.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.723+0000 7fa521d5ab80 1 -- 172.21.15.6:0/1672891019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a690af4450 msgr2=0x55a690c8fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:18.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.723+0000 7fa521d5ab80 1 --2- 172.21.15.6:0/1672891019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a690af4450 0x55a690c8fa40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc0e02e0 tx=0x7fa4fc003a50 comp rx=0 tx=0).stop 2024-05-09T04:50:18.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.723+0000 7fa521d5ab80 1 -- 172.21.15.6:0/1672891019 shutdown_connections 2024-05-09T04:50:18.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.723+0000 7fa521d5ab80 1 --2- 172.21.15.6:0/1672891019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a690c77410 0x55a690c904c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:18.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.723+0000 7fa521d5ab80 1 --2- 172.21.15.6:0/1672891019 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x55a690cf8180 0x55a690cf95f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:18.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.723+0000 7fa521d5ab80 1 --2- 172.21.15.6:0/1672891019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a690af4450 0x55a690c8fa40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:18.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.723+0000 7fa521d5ab80 1 --2- 172.21.15.6:0/1672891019 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a690c71a00 0x55a690c8ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:18.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.723+0000 7fa4f6000700 0 client.24637 ms_handle_reset on v2:172.21.15.169:6838/614388771 2024-05-09T04:50:18.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.723+0000 7fa521d5ab80 1 -- 172.21.15.6:0/1672891019 >> 172.21.15.6:0/1672891019 conn(0x55a690b38e20 msgr2=0x55a690c764a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:18.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.723+0000 7fa521d5ab80 1 -- 172.21.15.6:0/1672891019 shutdown_connections 2024-05-09T04:50:18.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.723+0000 7fa521d5ab80 1 -- 172.21.15.6:0/1672891019 wait complete. 2024-05-09T04:50:18.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.723+0000 7fa521d5ab80 20 client.24637 trim_cache size 0 max 16384 2024-05-09T04:50:18.726 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-09T04:50:18.726 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "f74d813c-0dbd-11ef-bc97-c7b262605968", 2024-05-09T04:50:18.726 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 0, 2024-05-09T04:50:18.726 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24554, 2024-05-09T04:50:18.727 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-05-09T04:50:18.727 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-05-09T04:50:18.727 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-05-09T04:50:18.727 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 41.567488245, 2024-05-09T04:50:18.727 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 17, 2024-05-09T04:50:18.727 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 87, 2024-05-09T04:50:18.727 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 86, 2024-05-09T04:50:18.727 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 62.870891538999999 2024-05-09T04:50:18.727 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-09T04:50:18.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.724+0000 7fa513200700 1 -- 172.21.15.6:0/1225542141 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa4f00b2900 msgr2=0x7fa4f00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:18.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.724+0000 7fa513200700 1 --2- 172.21.15.6:0/1225542141 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa4f00b2900 0x7fa4f00b4e10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa5040075c0 tx=0x7fa50400b040 comp rx=0 tx=0).stop 2024-05-09T04:50:18.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.724+0000 7fa513200700 1 -- 172.21.15.6:0/1225542141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa50c1034c0 msgr2=0x7fa50c0ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:18.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.724+0000 7fa513200700 1 --2- 172.21.15.6:0/1225542141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa50c1034c0 0x7fa50c0ff2a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00ec10 tx=0x7fa4fc00ef20 comp rx=0 tx=0).stop 2024-05-09T04:50:18.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.724+0000 7fa513200700 1 -- 172.21.15.6:0/1225542141 shutdown_connections 2024-05-09T04:50:18.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.724+0000 7fa513200700 1 --2- 172.21.15.6:0/1225542141 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa50c108740 0x7fa50c0ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:18.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.724+0000 7fa513200700 1 --2- 172.21.15.6:0/1225542141 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa4f00b2900 0x7fa4f00b4e10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:18.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.724+0000 7fa513200700 1 --2- 172.21.15.6:0/1225542141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa50c1034c0 0x7fa50c0ff2a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:18.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.724+0000 7fa513200700 1 --2- 172.21.15.6:0/1225542141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa50c107df0 0x7fa50c0ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:18.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.724+0000 7fa513200700 1 -- 172.21.15.6:0/1225542141 >> 172.21.15.6:0/1225542141 conn(0x7fa50c0fedc0 msgr2=0x7fa50c06b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:18.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.724+0000 7fa513200700 1 -- 172.21.15.6:0/1225542141 shutdown_connections 2024-05-09T04:50:18.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:18.724+0000 7fa513200700 1 -- 172.21.15.6:0/1225542141 wait complete. 2024-05-09T04:50:18.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:18 smithi006 ceph-mon[29835]: pgmap v358: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:19.169 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:18 smithi169 ceph-mon[34075]: pgmap v358: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:19.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:18 smithi186 ceph-mon[34668]: pgmap v358: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:19.654 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T04:50:19.654 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-09T04:50:19.654 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-09T04:50:20.101 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:50:21.129 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:20 smithi006 ceph-mon[29835]: pgmap v359: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:21.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:20 smithi186 ceph-mon[34668]: pgmap v359: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:21.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:20 smithi169 ceph-mon[34075]: pgmap v359: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:22.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.362+0000 7f258c600700 1 -- 172.21.15.6:0/156811632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2584075d30 msgr2=0x7f2584076140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:22.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.362+0000 7f258c600700 1 --2- 172.21.15.6:0/156811632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2584075d30 0x7f2584076140 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f25780098f0 tx=0x7f2578009c00 comp rx=0 tx=0).stop 2024-05-09T04:50:22.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.362+0000 7f258c600700 1 -- 172.21.15.6:0/156811632 shutdown_connections 2024-05-09T04:50:22.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.362+0000 7f258c600700 1 --2- 172.21.15.6:0/156811632 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2584110ba0 0x7f2584113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:22.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.362+0000 7f258c600700 1 --2- 172.21.15.6:0/156811632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2584075d30 0x7f2584076140 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:22.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.362+0000 7f258c600700 1 --2- 172.21.15.6:0/156811632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2584076680 0x7f2584110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:22.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.362+0000 7f258c600700 1 -- 172.21.15.6:0/156811632 >> 172.21.15.6:0/156811632 conn(0x7f2584100040 msgr2=0x7f2584102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:22.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.362+0000 7f258c600700 1 -- 172.21.15.6:0/156811632 shutdown_connections 2024-05-09T04:50:22.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.363+0000 7f258c600700 1 -- 172.21.15.6:0/156811632 wait complete. 2024-05-09T04:50:22.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.364+0000 7f258c600700 1 Processor -- start 2024-05-09T04:50:22.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.364+0000 7f258c600700 1 -- start start 2024-05-09T04:50:22.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.365+0000 7f258c600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2584075d30 0x7f258419d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:22.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.365+0000 7f258c600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2584076680 0x7f258419d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:22.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.365+0000 7f258c600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2584110ba0 0x7f25841a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:22.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.365+0000 7f258c600700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f25841166e0 con 0x7f2584075d30 2024-05-09T04:50:22.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.365+0000 7f258c600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2584116560 con 0x7f2584076680 2024-05-09T04:50:22.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.365+0000 7f258c600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2584116860 con 0x7f2584110ba0 2024-05-09T04:50:22.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.365+0000 7f2589200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2584076680 0x7f258419d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:22.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.365+0000 7f2589c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2584075d30 0x7f258419d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:22.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.365+0000 7f2589200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2584076680 0x7f258419d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:37812/0 (socket says 172.21.15.6:37812) 2024-05-09T04:50:22.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.365+0000 7f2589200700 1 -- 172.21.15.6:0/354841103 learned_addr learned my addr 172.21.15.6:0/354841103 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:50:22.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.365+0000 7f258a600700 1 --2- 172.21.15.6:0/354841103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2584110ba0 0x7f25841a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:22.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.366+0000 7f2589200700 1 -- 172.21.15.6:0/354841103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2584110ba0 msgr2=0x7f25841a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:22.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.366+0000 7f2589200700 1 --2- 172.21.15.6:0/354841103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2584110ba0 0x7f25841a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:22.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.366+0000 7f2589200700 1 -- 172.21.15.6:0/354841103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2584075d30 msgr2=0x7f258419d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:22.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.366+0000 7f2589200700 1 --2- 172.21.15.6:0/354841103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2584075d30 0x7f258419d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:22.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.366+0000 7f2589200700 1 -- 172.21.15.6:0/354841103 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25841a1f40 con 0x7f2584076680 2024-05-09T04:50:22.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.366+0000 7f2589200700 1 --2- 172.21.15.6:0/354841103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2584076680 0x7f258419d610 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f257000e9a0 tx=0x7f257000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:22.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.366+0000 7f2582000700 1 -- 172.21.15.6:0/354841103 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257000ca10 con 0x7f2584076680 2024-05-09T04:50:22.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.366+0000 7f258c600700 1 -- 172.21.15.6:0/354841103 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25841a2230 con 0x7f2584076680 2024-05-09T04:50:22.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.366+0000 7f2582000700 1 -- 172.21.15.6:0/354841103 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f257000cbb0 con 0x7f2584076680 2024-05-09T04:50:22.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.366+0000 7f2582000700 1 -- 172.21.15.6:0/354841103 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2570018680 con 0x7f2584076680 2024-05-09T04:50:22.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.367+0000 7f258c600700 1 -- 172.21.15.6:0/354841103 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25841a9b50 con 0x7f2584076680 2024-05-09T04:50:22.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.368+0000 7f258c600700 1 -- 172.21.15.6:0/354841103 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25840512c0 con 0x7f2584076680 2024-05-09T04:50:22.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.370+0000 7f2582000700 1 -- 172.21.15.6:0/354841103 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f2570010430 con 0x7f2584076680 2024-05-09T04:50:22.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.372+0000 7f2582000700 1 --2- 172.21.15.6:0/354841103 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f256c0b2880 0x7f256c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:22.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.372+0000 7f2582000700 1 -- 172.21.15.6:0/354841103 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f25700e2a60 con 0x7f2584076680 2024-05-09T04:50:22.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.372+0000 7f2589c00700 1 --2- 172.21.15.6:0/354841103 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f256c0b2880 0x7f256c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:22.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.373+0000 7f2589c00700 1 --2- 172.21.15.6:0/354841103 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f256c0b2880 0x7f256c0b4d90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f25780098c0 tx=0x7f2578019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:22.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.374+0000 7f2582000700 1 -- 172.21.15.6:0/354841103 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f25700aa3e0 con 0x7f2584076680 2024-05-09T04:50:22.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.663+0000 7f258c600700 1 -- 172.21.15.6:0/354841103 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f258419e3d0 con 0x7f2584076680 2024-05-09T04:50:22.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.664+0000 7f2582000700 1 -- 172.21.15.6:0/354841103 <== mon.2 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f25700a9ce0 con 0x7f2584076680 2024-05-09T04:50:22.665 INFO:teuthology.orchestra.run.smithi006.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-09T04:50:22.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.668+0000 7f258c600700 1 -- 172.21.15.6:0/354841103 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f256c0b2880 msgr2=0x7f256c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:22.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.668+0000 7f258c600700 1 --2- 172.21.15.6:0/354841103 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f256c0b2880 0x7f256c0b4d90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f25780098c0 tx=0x7f2578019040 comp rx=0 tx=0).stop 2024-05-09T04:50:22.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.668+0000 7f258c600700 1 -- 172.21.15.6:0/354841103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2584076680 msgr2=0x7f258419d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:22.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.668+0000 7f258c600700 1 --2- 172.21.15.6:0/354841103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2584076680 0x7f258419d610 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f257000e9a0 tx=0x7f257000ecb0 comp rx=0 tx=0).stop 2024-05-09T04:50:22.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.668+0000 7f258c600700 1 -- 172.21.15.6:0/354841103 shutdown_connections 2024-05-09T04:50:22.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.668+0000 7f258c600700 1 --2- 172.21.15.6:0/354841103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2584110ba0 0x7f25841a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:22.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.668+0000 7f258c600700 1 --2- 172.21.15.6:0/354841103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2584075d30 0x7f258419d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:22.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.668+0000 7f258c600700 1 --2- 172.21.15.6:0/354841103 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f256c0b2880 0x7f256c0b4d90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:22.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.668+0000 7f258c600700 1 --2- 172.21.15.6:0/354841103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2584076680 0x7f258419d610 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:22.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.668+0000 7f258c600700 1 -- 172.21.15.6:0/354841103 >> 172.21.15.6:0/354841103 conn(0x7f2584100040 msgr2=0x7f2584100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:22.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.668+0000 7f258c600700 1 -- 172.21.15.6:0/354841103 shutdown_connections 2024-05-09T04:50:22.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:22.668+0000 7f258c600700 1 -- 172.21.15.6:0/354841103 wait complete. 2024-05-09T04:50:22.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:22 smithi006 ceph-mon[29835]: pgmap v360: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:22.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:22 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/354841103' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T04:50:23.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:22 smithi186 ceph-mon[34668]: pgmap v360: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:23.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:22 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/354841103' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T04:50:23.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:22 smithi169 ceph-mon[34075]: pgmap v360: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:23.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:22 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/354841103' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T04:50:23.578 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-09T04:50:23.578 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T04:50:23.578 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi006.front.sepia.ceph.com 2024-05-09T04:50:23.578 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-09T04:50:23.578 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-09T04:50:23.578 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-09T04:50:23.578 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T04:50:23.579 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T04:50:23.579 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-09T04:50:23.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:50:23.579 DEBUG:teuthology.orchestra.run.smithi006:> ip addr 2024-05-09T04:50:23.640 INFO:teuthology.orchestra.run.smithi006.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T04:50:23.640 INFO:teuthology.orchestra.run.smithi006.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T04:50:23.640 INFO:teuthology.orchestra.run.smithi006.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T04:50:23.640 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:50:23.640 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 ::1/128 scope host 2024-05-09T04:50:23.640 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:50:23.640 INFO:teuthology.orchestra.run.smithi006.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T04:50:23.640 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e6 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:50:23.640 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp7s0f0 2024-05-09T04:50:23.640 INFO:teuthology.orchestra.run.smithi006.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T04:50:23.640 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e7 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:50:23.640 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp7s0f1 2024-05-09T04:50:23.641 INFO:teuthology.orchestra.run.smithi006.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T04:50:23.641 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e4 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:50:23.641 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp1s0f0 2024-05-09T04:50:23.641 INFO:teuthology.orchestra.run.smithi006.stdout: inet 172.21.15.6/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-09T04:50:23.641 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft 41942sec preferred_lft 41942sec 2024-05-09T04:50:23.641 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::ec4:7aff:febd:15e4/64 scope link noprefixroute 2024-05-09T04:50:23.641 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:50:23.641 INFO:teuthology.orchestra.run.smithi006.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T04:50:23.641 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e5 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:50:23.641 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp1s0f1 2024-05-09T04:50:23.642 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-09T04:50:23.642 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:50:23.642 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2024-05-09T04:50:23.642 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link add name ceph-brx type bridge 2024-05-09T04:50:23.642 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip addr flush dev ceph-brx 2024-05-09T04:50:23.642 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set ceph-brx up 2024-05-09T04:50:23.642 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-09T04:50:23.642 DEBUG:teuthology.orchestra.run.smithi006:> ') 2024-05-09T04:50:23.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:50:24.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:50:24.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:50:24.166 DEBUG:teuthology.orchestra.run.smithi006:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T04:50:24.239 INFO:teuthology.orchestra.run.smithi006.stdout:1 2024-05-09T04:50:24.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:50:24.243 DEBUG:teuthology.orchestra.run.smithi006:> route 2024-05-09T04:50:24.307 INFO:teuthology.orchestra.run.smithi006.stdout:Kernel IP routing table 2024-05-09T04:50:24.308 INFO:teuthology.orchestra.run.smithi006.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T04:50:24.308 INFO:teuthology.orchestra.run.smithi006.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T04:50:24.308 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T04:50:24.308 INFO:teuthology.orchestra.run.smithi006.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T04:50:24.308 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:50:24.308 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2024-05-09T04:50:24.309 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T04:50:24.309 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T04:50:24.309 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T04:50:24.309 DEBUG:teuthology.orchestra.run.smithi006:> ') 2024-05-09T04:50:24.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:50:25.085 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:24 smithi006 ceph-mon[29835]: pgmap v361: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:25.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:50:25.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:50:25.100 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2024-05-09T04:50:25.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:50:25.154 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list-id 2024-05-09T04:50:25.211 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:50:25.211 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2024-05-09T04:50:25.211 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T04:50:25.211 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-09T04:50:25.211 DEBUG:teuthology.orchestra.run.smithi006:> ') 2024-05-09T04:50:25.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:50:25.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:24 smithi186 ceph-mon[34668]: pgmap v361: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:25.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:24 smithi169 ceph-mon[34075]: pgmap v361: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:25.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:50:25.411 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-09T04:50:25.412 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:50:25.412 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2024-05-09T04:50:25.412 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-09T04:50:25.412 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-09T04:50:25.412 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-09T04:50:25.412 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-09T04:50:25.412 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-09T04:50:25.412 DEBUG:teuthology.orchestra.run.smithi006:> ') 2024-05-09T04:50:25.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:50:25.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:50:25.732 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:50:25.732 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2024-05-09T04:50:25.732 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set brx.0 up 2024-05-09T04:50:25.732 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set dev brx.0 master ceph-brx 2024-05-09T04:50:25.732 DEBUG:teuthology.orchestra.run.smithi006:> ') 2024-05-09T04:50:25.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:50:25.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:50:25.955 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-09T04:50:25.955 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T04:50:25.955 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-09T04:50:26.002 INFO:teuthology.orchestra.run.smithi006.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-09T04:50:26.003 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T04:50:26.003 DEBUG:teuthology.orchestra.run.smithi006:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-09T04:50:26.099 DEBUG:teuthology.orchestra.run.smithi006:> sudo modprobe fuse 2024-05-09T04:50:26.184 DEBUG:teuthology.orchestra.run.smithi006:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T04:50:26.241 INFO:teuthology.orchestra.run.smithi006.stdout:/proc 2024-05-09T04:50:26.241 INFO:teuthology.orchestra.run.smithi006.stdout:/sys 2024-05-09T04:50:26.241 INFO:teuthology.orchestra.run.smithi006.stdout:/dev 2024-05-09T04:50:26.241 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/security 2024-05-09T04:50:26.241 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/shm 2024-05-09T04:50:26.241 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/pts 2024-05-09T04:50:26.241 INFO:teuthology.orchestra.run.smithi006.stdout:/run 2024-05-09T04:50:26.241 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/cgroup 2024-05-09T04:50:26.241 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/pstore 2024-05-09T04:50:26.241 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/bpf 2024-05-09T04:50:26.241 INFO:teuthology.orchestra.run.smithi006.stdout:/ 2024-05-09T04:50:26.241 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/selinux 2024-05-09T04:50:26.242 INFO:teuthology.orchestra.run.smithi006.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T04:50:26.242 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/mqueue 2024-05-09T04:50:26.242 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/hugepages 2024-05-09T04:50:26.242 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/debug 2024-05-09T04:50:26.242 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/tracing 2024-05-09T04:50:26.242 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/fuse/connections 2024-05-09T04:50:26.242 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T04:50:26.242 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/config 2024-05-09T04:50:26.242 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T04:50:26.242 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T04:50:26.242 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T04:50:26.242 INFO:teuthology.orchestra.run.smithi006.stdout:/run/user/1000 2024-05-09T04:50:26.242 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph 2024-05-09T04:50:26.242 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/containers/storage/overlay 2024-05-09T04:50:26.243 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/containers/storage/overlay/6348e3a836580359ded965ccca9c5ed8f431b9fb889efea942c3c25368e48089/merged 2024-05-09T04:50:26.243 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/containers/storage/overlay/7682fa9a25a21a009a11dcd956dff23541b57dc43a16dd45cdf2c3fc32a924a9/merged 2024-05-09T04:50:26.243 INFO:teuthology.orchestra.run.smithi006.stdout:/run/user/0 2024-05-09T04:50:26.243 INFO:teuthology.orchestra.run.smithi006.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T04:50:26.243 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/containers/storage/overlay/d8cc6174a1bff49d41794a2957f9da81d345fd1193f201e5098f10c6b1944d6d/merged 2024-05-09T04:50:26.243 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/containers/storage/overlay/71ef540917fd7849848b8d5a3106baecf1537bcfcf8f91aeece86cdbabfa6006/merged 2024-05-09T04:50:26.243 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/containers/storage/overlay/d5bd3a81624b5d2029f395d4ea08cf58a48ac78335629ba8b15d427be31df430/merged 2024-05-09T04:50:26.243 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/containers/storage/overlay/c9e5db8715bbbbd5752c6903193b81fc7fe47982d435bd33bf8747c71b78eb00/merged 2024-05-09T04:50:26.243 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/containers/storage/overlay/c0e4a559e6a893f0a43da0996b739677775e7d3de1e86a9afdacd71c101f8d8c/merged 2024-05-09T04:50:26.243 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/containers/storage/overlay/192f823c8d2d603f9adbc4403f42856732e5d9d9ed62e28f0c0ccde44bd95469/merged 2024-05-09T04:50:26.243 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/containers/storage/overlay/66e7f3e6a903bb665418c10b577236bd2e13a7c0a801152b0262c658553f0560/merged 2024-05-09T04:50:26.243 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/containers/storage/overlay/dd808f3ff55e23849fb0a58df1c323d71c716751ffc408dfd83921997aa2c98d/merged 2024-05-09T04:50:26.243 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns 2024-05-09T04:50:26.243 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T04:50:26.243 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T04:50:26.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:50:26.244 DEBUG:teuthology.orchestra.run.smithi006:> ls /sys/fs/fuse/connections 2024-05-09T04:50:26.297 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-09T04:50:26.297 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-09T04:50:26.304 DEBUG:teuthology.orchestra.run.smithi006:> sudo modprobe fuse 2024-05-09T04:50:26.380 DEBUG:teuthology.orchestra.run.smithi006:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T04:50:26.427 INFO:teuthology.orchestra.run.smithi006.stdout:/proc 2024-05-09T04:50:26.427 INFO:teuthology.orchestra.run.smithi006.stdout:/sys 2024-05-09T04:50:26.428 INFO:teuthology.orchestra.run.smithi006.stdout:/dev 2024-05-09T04:50:26.428 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/security 2024-05-09T04:50:26.428 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/shm 2024-05-09T04:50:26.428 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/pts 2024-05-09T04:50:26.428 INFO:teuthology.orchestra.run.smithi006.stdout:/run 2024-05-09T04:50:26.428 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/cgroup 2024-05-09T04:50:26.428 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/pstore 2024-05-09T04:50:26.428 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/bpf 2024-05-09T04:50:26.428 INFO:teuthology.orchestra.run.smithi006.stdout:/ 2024-05-09T04:50:26.428 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/selinux 2024-05-09T04:50:26.428 INFO:teuthology.orchestra.run.smithi006.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T04:50:26.428 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/mqueue 2024-05-09T04:50:26.428 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/hugepages 2024-05-09T04:50:26.428 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/debug 2024-05-09T04:50:26.428 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/tracing 2024-05-09T04:50:26.429 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/fuse/connections 2024-05-09T04:50:26.429 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T04:50:26.429 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/config 2024-05-09T04:50:26.429 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T04:50:26.429 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T04:50:26.429 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T04:50:26.429 INFO:teuthology.orchestra.run.smithi006.stdout:/run/user/1000 2024-05-09T04:50:26.429 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph 2024-05-09T04:50:26.429 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/containers/storage/overlay 2024-05-09T04:50:26.429 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/containers/storage/overlay/6348e3a836580359ded965ccca9c5ed8f431b9fb889efea942c3c25368e48089/merged 2024-05-09T04:50:26.429 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/containers/storage/overlay/7682fa9a25a21a009a11dcd956dff23541b57dc43a16dd45cdf2c3fc32a924a9/merged 2024-05-09T04:50:26.429 INFO:teuthology.orchestra.run.smithi006.stdout:/run/user/0 2024-05-09T04:50:26.429 INFO:teuthology.orchestra.run.smithi006.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T04:50:26.429 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/containers/storage/overlay/d8cc6174a1bff49d41794a2957f9da81d345fd1193f201e5098f10c6b1944d6d/merged 2024-05-09T04:50:26.430 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/containers/storage/overlay/71ef540917fd7849848b8d5a3106baecf1537bcfcf8f91aeece86cdbabfa6006/merged 2024-05-09T04:50:26.430 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/containers/storage/overlay/d5bd3a81624b5d2029f395d4ea08cf58a48ac78335629ba8b15d427be31df430/merged 2024-05-09T04:50:26.430 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/containers/storage/overlay/c9e5db8715bbbbd5752c6903193b81fc7fe47982d435bd33bf8747c71b78eb00/merged 2024-05-09T04:50:26.430 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/containers/storage/overlay/c0e4a559e6a893f0a43da0996b739677775e7d3de1e86a9afdacd71c101f8d8c/merged 2024-05-09T04:50:26.430 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/containers/storage/overlay/192f823c8d2d603f9adbc4403f42856732e5d9d9ed62e28f0c0ccde44bd95469/merged 2024-05-09T04:50:26.430 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/containers/storage/overlay/66e7f3e6a903bb665418c10b577236bd2e13a7c0a801152b0262c658553f0560/merged 2024-05-09T04:50:26.430 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/containers/storage/overlay/dd808f3ff55e23849fb0a58df1c323d71c716751ffc408dfd83921997aa2c98d/merged 2024-05-09T04:50:26.430 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns 2024-05-09T04:50:26.430 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T04:50:26.430 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T04:50:26.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:50:26.431 DEBUG:teuthology.orchestra.run.smithi006:> ls /sys/fs/fuse/connections 2024-05-09T04:50:26.592 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi006.stderr:2024-05-09T04:50:26.591+0000 7fa936893480 -1 init, newargv = 0x7fa918055f10 newargc=15 2024-05-09T04:50:26.592 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi006.stderr:ceph-fuse[73003]: starting ceph client 2024-05-09T04:50:26.603 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi006.stderr:ceph-fuse[73003]: starting fuse 2024-05-09T04:50:27.189 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:26 smithi186 ceph-mon[34668]: pgmap v362: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:27.189 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:26 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:50:27.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:26 smithi006 ceph-mon[29835]: pgmap v362: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:27.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:26 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:50:27.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:26 smithi169 ceph-mon[34075]: pgmap v362: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:27.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:26 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:50:27.579 DEBUG:teuthology.orchestra.run.smithi006:> sudo modprobe fuse 2024-05-09T04:50:27.670 DEBUG:teuthology.orchestra.run.smithi006:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T04:50:27.715 INFO:teuthology.orchestra.run.smithi006.stdout:/proc 2024-05-09T04:50:27.716 INFO:teuthology.orchestra.run.smithi006.stdout:/sys 2024-05-09T04:50:27.716 INFO:teuthology.orchestra.run.smithi006.stdout:/dev 2024-05-09T04:50:27.716 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/security 2024-05-09T04:50:27.716 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/shm 2024-05-09T04:50:27.716 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/pts 2024-05-09T04:50:27.716 INFO:teuthology.orchestra.run.smithi006.stdout:/run 2024-05-09T04:50:27.716 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/cgroup 2024-05-09T04:50:27.716 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/pstore 2024-05-09T04:50:27.716 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/bpf 2024-05-09T04:50:27.716 INFO:teuthology.orchestra.run.smithi006.stdout:/ 2024-05-09T04:50:27.716 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/selinux 2024-05-09T04:50:27.716 INFO:teuthology.orchestra.run.smithi006.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T04:50:27.716 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/mqueue 2024-05-09T04:50:27.717 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/hugepages 2024-05-09T04:50:27.717 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/debug 2024-05-09T04:50:27.717 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/tracing 2024-05-09T04:50:27.717 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/fuse/connections 2024-05-09T04:50:27.717 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T04:50:27.717 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/config 2024-05-09T04:50:27.717 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T04:50:27.717 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T04:50:27.717 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T04:50:27.717 INFO:teuthology.orchestra.run.smithi006.stdout:/run/user/1000 2024-05-09T04:50:27.717 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph 2024-05-09T04:50:27.717 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/containers/storage/overlay 2024-05-09T04:50:27.717 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/containers/storage/overlay/6348e3a836580359ded965ccca9c5ed8f431b9fb889efea942c3c25368e48089/merged 2024-05-09T04:50:27.717 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/containers/storage/overlay/7682fa9a25a21a009a11dcd956dff23541b57dc43a16dd45cdf2c3fc32a924a9/merged 2024-05-09T04:50:27.718 INFO:teuthology.orchestra.run.smithi006.stdout:/run/user/0 2024-05-09T04:50:27.718 INFO:teuthology.orchestra.run.smithi006.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T04:50:27.718 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/containers/storage/overlay/d8cc6174a1bff49d41794a2957f9da81d345fd1193f201e5098f10c6b1944d6d/merged 2024-05-09T04:50:27.718 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/containers/storage/overlay/71ef540917fd7849848b8d5a3106baecf1537bcfcf8f91aeece86cdbabfa6006/merged 2024-05-09T04:50:27.718 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/containers/storage/overlay/d5bd3a81624b5d2029f395d4ea08cf58a48ac78335629ba8b15d427be31df430/merged 2024-05-09T04:50:27.718 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/containers/storage/overlay/c9e5db8715bbbbd5752c6903193b81fc7fe47982d435bd33bf8747c71b78eb00/merged 2024-05-09T04:50:27.718 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/containers/storage/overlay/c0e4a559e6a893f0a43da0996b739677775e7d3de1e86a9afdacd71c101f8d8c/merged 2024-05-09T04:50:27.718 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/containers/storage/overlay/192f823c8d2d603f9adbc4403f42856732e5d9d9ed62e28f0c0ccde44bd95469/merged 2024-05-09T04:50:27.718 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/containers/storage/overlay/66e7f3e6a903bb665418c10b577236bd2e13a7c0a801152b0262c658553f0560/merged 2024-05-09T04:50:27.718 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/containers/storage/overlay/dd808f3ff55e23849fb0a58df1c323d71c716751ffc408dfd83921997aa2c98d/merged 2024-05-09T04:50:27.718 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns 2024-05-09T04:50:27.718 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T04:50:27.718 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T04:50:27.718 INFO:teuthology.orchestra.run.smithi006.stdout:/home/ubuntu/cephtest/mnt.admin 2024-05-09T04:50:27.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:50:27.719 DEBUG:teuthology.orchestra.run.smithi006:> ls /sys/fs/fuse/connections 2024-05-09T04:50:27.812 INFO:teuthology.orchestra.run.smithi006.stdout:114 2024-05-09T04:50:27.812 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [114] 2024-05-09T04:50:27.813 DEBUG:teuthology.orchestra.run.smithi006:> sudo stdin-killer -- python3 -c ' 2024-05-09T04:50:27.813 DEBUG:teuthology.orchestra.run.smithi006:> import glob 2024-05-09T04:50:27.813 DEBUG:teuthology.orchestra.run.smithi006:> import re 2024-05-09T04:50:27.813 DEBUG:teuthology.orchestra.run.smithi006:> import os 2024-05-09T04:50:27.813 DEBUG:teuthology.orchestra.run.smithi006:> import subprocess 2024-05-09T04:50:27.813 DEBUG:teuthology.orchestra.run.smithi006:> 2024-05-09T04:50:27.813 DEBUG:teuthology.orchestra.run.smithi006:> def _find_admin_socket(client_name): 2024-05-09T04:50:27.813 DEBUG:teuthology.orchestra.run.smithi006:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-09T04:50:27.813 DEBUG:teuthology.orchestra.run.smithi006:> files = glob.glob(asok_path) 2024-05-09T04:50:27.813 DEBUG:teuthology.orchestra.run.smithi006:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-09T04:50:27.813 DEBUG:teuthology.orchestra.run.smithi006:> 2024-05-09T04:50:27.813 DEBUG:teuthology.orchestra.run.smithi006:> # Given a non-glob path, it better be there 2024-05-09T04:50:27.814 DEBUG:teuthology.orchestra.run.smithi006:> if "*" not in asok_path: 2024-05-09T04:50:27.814 DEBUG:teuthology.orchestra.run.smithi006:> assert(len(files) == 1) 2024-05-09T04:50:27.814 DEBUG:teuthology.orchestra.run.smithi006:> return files[0] 2024-05-09T04:50:27.814 DEBUG:teuthology.orchestra.run.smithi006:> 2024-05-09T04:50:27.814 DEBUG:teuthology.orchestra.run.smithi006:> for f in files: 2024-05-09T04:50:27.814 DEBUG:teuthology.orchestra.run.smithi006:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T04:50:27.814 DEBUG:teuthology.orchestra.run.smithi006:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T04:50:27.814 DEBUG:teuthology.orchestra.run.smithi006:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T04:50:27.814 DEBUG:teuthology.orchestra.run.smithi006:> contents = proc_f.read() 2024-05-09T04:50:27.814 DEBUG:teuthology.orchestra.run.smithi006:> if mountpoint in contents: 2024-05-09T04:50:27.814 DEBUG:teuthology.orchestra.run.smithi006:> return f 2024-05-09T04:50:27.814 DEBUG:teuthology.orchestra.run.smithi006:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T04:50:27.814 DEBUG:teuthology.orchestra.run.smithi006:> 2024-05-09T04:50:27.814 DEBUG:teuthology.orchestra.run.smithi006:> print(_find_admin_socket("client.admin")) 2024-05-09T04:50:27.814 DEBUG:teuthology.orchestra.run.smithi006:> ' 2024-05-09T04:50:27.982 INFO:teuthology.orchestra.run.smithi006.stdout:/var/run/ceph/ceph-client.admin.73003.asok 2024-05-09T04:50:27.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:50:27.997 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.73003.asok 2024-05-09T04:50:27.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:50:27.997 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.73003.asok status 2024-05-09T04:50:28.300 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-09T04:50:28.300 INFO:teuthology.orchestra.run.smithi006.stdout: "metadata": { 2024-05-09T04:50:28.300 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_sha1": "c9da4049f638c0f84bdd9b4e33c42e86046ac554", 2024-05-09T04:50:28.300 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_version": "ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev)", 2024-05-09T04:50:28.300 INFO:teuthology.orchestra.run.smithi006.stdout: "entity_id": "admin", 2024-05-09T04:50:28.300 INFO:teuthology.orchestra.run.smithi006.stdout: "hostname": "smithi006", 2024-05-09T04:50:28.300 INFO:teuthology.orchestra.run.smithi006.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-09T04:50:28.300 INFO:teuthology.orchestra.run.smithi006.stdout: "pid": "73003", 2024-05-09T04:50:28.300 INFO:teuthology.orchestra.run.smithi006.stdout: "root": "/" 2024-05-09T04:50:28.300 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:50:28.300 INFO:teuthology.orchestra.run.smithi006.stdout: "dentry_count": 0, 2024-05-09T04:50:28.300 INFO:teuthology.orchestra.run.smithi006.stdout: "dentry_pinned_count": 0, 2024-05-09T04:50:28.301 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 14520, 2024-05-09T04:50:28.301 INFO:teuthology.orchestra.run.smithi006.stdout: "inst": { 2024-05-09T04:50:28.301 INFO:teuthology.orchestra.run.smithi006.stdout: "name": { 2024-05-09T04:50:28.301 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "client", 2024-05-09T04:50:28.301 INFO:teuthology.orchestra.run.smithi006.stdout: "num": 14520 2024-05-09T04:50:28.301 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:50:28.301 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": { 2024-05-09T04:50:28.301 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v1", 2024-05-09T04:50:28.301 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "192.168.0.1:0", 2024-05-09T04:50:28.301 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 1165410545 2024-05-09T04:50:28.301 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:50:28.301 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:50:28.301 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": { 2024-05-09T04:50:28.301 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v1", 2024-05-09T04:50:28.301 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "192.168.0.1:0", 2024-05-09T04:50:28.302 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 1165410545 2024-05-09T04:50:28.302 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:50:28.302 INFO:teuthology.orchestra.run.smithi006.stdout: "inst_str": "client.14520 192.168.0.1:0/1165410545", 2024-05-09T04:50:28.302 INFO:teuthology.orchestra.run.smithi006.stdout: "addr_str": "192.168.0.1:0/1165410545", 2024-05-09T04:50:28.302 INFO:teuthology.orchestra.run.smithi006.stdout: "inode_count": 1, 2024-05-09T04:50:28.302 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_epoch": 17, 2024-05-09T04:50:28.302 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_epoch": 87, 2024-05-09T04:50:28.302 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_epoch_barrier": 0, 2024-05-09T04:50:28.302 INFO:teuthology.orchestra.run.smithi006.stdout: "blocklisted": false, 2024-05-09T04:50:28.302 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs" 2024-05-09T04:50:28.302 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-09T04:50:28.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:50:28.325 DEBUG:teuthology.orchestra.run.smithi006:> stat --file-system '--printf=%T 2024-05-09T04:50:28.325 DEBUG:teuthology.orchestra.run.smithi006:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-09T04:50:28.375 INFO:teuthology.orchestra.run.smithi006.stdout:fuseblk 2024-05-09T04:50:28.375 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-09T04:50:28.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:50:28.376 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-09T04:50:28.461 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-09T04:50:28.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T04:50:28.599 INFO:teuthology.orchestra.run.smithi006.stdout:# file: . 2024-05-09T04:50:28.599 INFO:teuthology.orchestra.run.smithi006.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-09T04:50:28.599 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:50:28.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:50:28.613 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T04:50:28.613 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:50:28.658 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-09T04:50:29.139 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:50:29.158 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:28 smithi006 ceph-mon[29835]: pgmap v363: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:29.158 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:28 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:50:29.158 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:28 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:50:29.158 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:28 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:50:29.158 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:28 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:50:29.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:28 smithi186 ceph-mon[34668]: pgmap v363: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:29.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:28 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:50:29.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:28 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:50:29.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:28 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:50:29.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:28 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:50:29.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:28 smithi169 ceph-mon[34075]: pgmap v363: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:29.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:28 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:50:29.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:28 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:50:29.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:28 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:50:29.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:28 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:50:31.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:30 smithi006 ceph-mon[29835]: pgmap v364: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:31.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:30 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:50:31.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:30 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:50:31.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:30 smithi186 ceph-mon[34668]: pgmap v364: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:31.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:30 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:50:31.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:30 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:50:31.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:30 smithi169 ceph-mon[34075]: pgmap v364: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:31.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:30 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:50:31.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:30 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:50:31.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.486+0000 7f1178200700 1 -- 172.21.15.6:0/252216710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1170109870 msgr2=0x7f11701111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:31.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.486+0000 7f1178200700 1 --2- 172.21.15.6:0/252216710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1170109870 0x7f11701111d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f116800b0d0 tx=0x7f116800b3e0 comp rx=0 tx=0).stop 2024-05-09T04:50:31.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.486+0000 7f1178200700 1 -- 172.21.15.6:0/252216710 shutdown_connections 2024-05-09T04:50:31.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.486+0000 7f1178200700 1 --2- 172.21.15.6:0/252216710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1170109870 0x7f11701111d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:31.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.486+0000 7f1178200700 1 --2- 172.21.15.6:0/252216710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11701045f0 0x7f1170104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:31.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.486+0000 7f1178200700 1 --2- 172.21.15.6:0/252216710 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1170108f20 0x7f1170109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:31.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.486+0000 7f1178200700 1 -- 172.21.15.6:0/252216710 >> 172.21.15.6:0/252216710 conn(0x7f1170100040 msgr2=0x7f1170102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:31.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.486+0000 7f1178200700 1 -- 172.21.15.6:0/252216710 shutdown_connections 2024-05-09T04:50:31.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.486+0000 7f1178200700 1 -- 172.21.15.6:0/252216710 wait complete. 2024-05-09T04:50:31.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.488+0000 7f1178200700 1 Processor -- start 2024-05-09T04:50:31.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.488+0000 7f1178200700 1 -- start start 2024-05-09T04:50:31.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.489+0000 7f1178200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f11701045f0 0x7f117019d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:31.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.489+0000 7f1178200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1170108f20 0x7f117019d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:31.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.489+0000 7f1178200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1170109870 0x7f11701a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:31.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.489+0000 7f1178200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f11701144f0 con 0x7f11701045f0 2024-05-09T04:50:31.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.489+0000 7f1178200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1170114370 con 0x7f1170109870 2024-05-09T04:50:31.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.489+0000 7f1178200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1170114670 con 0x7f1170108f20 2024-05-09T04:50:31.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.489+0000 7f1175800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f11701045f0 0x7f117019d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:31.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.489+0000 7f1175800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f11701045f0 0x7f117019d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58282/0 (socket says 172.21.15.6:58282) 2024-05-09T04:50:31.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.489+0000 7f1175800700 1 -- 172.21.15.6:0/1860725951 learned_addr learned my addr 172.21.15.6:0/1860725951 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:50:31.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.489+0000 7f1176200700 1 --2- 172.21.15.6:0/1860725951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1170109870 0x7f11701a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:31.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.489+0000 7f1174e00700 1 --2- 172.21.15.6:0/1860725951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1170108f20 0x7f117019d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:31.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.490+0000 7f1176200700 1 -- 172.21.15.6:0/1860725951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1170108f20 msgr2=0x7f117019d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:31.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.490+0000 7f1176200700 1 --2- 172.21.15.6:0/1860725951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1170108f20 0x7f117019d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:31.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.490+0000 7f1176200700 1 -- 172.21.15.6:0/1860725951 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f11701045f0 msgr2=0x7f117019d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:31.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.490+0000 7f1176200700 1 --2- 172.21.15.6:0/1860725951 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f11701045f0 0x7f117019d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:31.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.490+0000 7f1176200700 1 -- 172.21.15.6:0/1860725951 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11701a1f30 con 0x7f1170109870 2024-05-09T04:50:31.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.490+0000 7f1174e00700 1 --2- 172.21.15.6:0/1860725951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1170108f20 0x7f117019d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:50:31.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.490+0000 7f1176200700 1 --2- 172.21.15.6:0/1860725951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1170109870 0x7f11701a1890 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f116800b770 tx=0x7f1168007980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:31.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.490+0000 7f116e000700 1 -- 172.21.15.6:0/1860725951 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116801f070 con 0x7f1170109870 2024-05-09T04:50:31.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.490+0000 7f1178200700 1 -- 172.21.15.6:0/1860725951 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11701a21c0 con 0x7f1170109870 2024-05-09T04:50:31.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.490+0000 7f116e000700 1 -- 172.21.15.6:0/1860725951 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1168007dc0 con 0x7f1170109870 2024-05-09T04:50:31.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.490+0000 7f116e000700 1 -- 172.21.15.6:0/1860725951 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116801b430 con 0x7f1170109870 2024-05-09T04:50:31.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.491+0000 7f1178200700 1 -- 172.21.15.6:0/1860725951 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11701a9b40 con 0x7f1170109870 2024-05-09T04:50:31.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.492+0000 7f1178200700 1 -- 172.21.15.6:0/1860725951 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1170105aa0 con 0x7f1170109870 2024-05-09T04:50:31.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.494+0000 7f116e000700 1 -- 172.21.15.6:0/1860725951 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f1168022070 con 0x7f1170109870 2024-05-09T04:50:31.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.497+0000 7f116e000700 1 --2- 172.21.15.6:0/1860725951 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f11580b2900 0x7f11580b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:31.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.497+0000 7f116e000700 1 -- 172.21.15.6:0/1860725951 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f116800f020 con 0x7f1170109870 2024-05-09T04:50:31.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.497+0000 7f1175800700 1 --2- 172.21.15.6:0/1860725951 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f11580b2900 0x7f11580b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:31.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.498+0000 7f116e000700 1 -- 172.21.15.6:0/1860725951 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1168029020 con 0x7f1170109870 2024-05-09T04:50:31.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.498+0000 7f1175800700 1 --2- 172.21.15.6:0/1860725951 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f11580b2900 0x7f11580b4e10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1164009d20 tx=0x7f1164009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:31.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.756+0000 7f1178200700 1 -- 172.21.15.6:0/1860725951 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f11701a9d10 con 0x7f1170109870 2024-05-09T04:50:31.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.757+0000 7f116e000700 1 -- 172.21.15.6:0/1860725951 <== mon.2 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) ==== 81+0+579 (secure 0 0 0) 0x7f11680ab3f0 con 0x7f1170109870 2024-05-09T04:50:31.758 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/3742411767 2024-05-10T04:40:55.357057+0000 2024-05-09T04:50:31.758 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/2410771702 2024-05-10T04:40:55.357057+0000 2024-05-09T04:50:31.758 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6801/937713898 2024-05-10T04:40:55.357057+0000 2024-05-09T04:50:31.758 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6800/937713898 2024-05-10T04:40:55.357057+0000 2024-05-09T04:50:31.758 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6801/3021503778 2024-05-10T04:40:19.639151+0000 2024-05-09T04:50:31.759 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6800/3021503778 2024-05-10T04:40:19.639151+0000 2024-05-09T04:50:31.759 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/2559510850 2024-05-10T04:40:19.639151+0000 2024-05-09T04:50:31.759 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/3810474743 2024-05-10T04:40:19.639151+0000 2024-05-09T04:50:31.759 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/4133970555 2024-05-10T04:40:55.357057+0000 2024-05-09T04:50:31.759 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/664817743 2024-05-10T04:40:19.639151+0000 2024-05-09T04:50:31.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.760+0000 7f1178200700 1 -- 172.21.15.6:0/1860725951 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f11580b2900 msgr2=0x7f11580b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:31.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.760+0000 7f1178200700 1 --2- 172.21.15.6:0/1860725951 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f11580b2900 0x7f11580b4e10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1164009d20 tx=0x7f1164009260 comp rx=0 tx=0).stop 2024-05-09T04:50:31.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.760+0000 7f1178200700 1 -- 172.21.15.6:0/1860725951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1170109870 msgr2=0x7f11701a1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:31.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.760+0000 7f1178200700 1 --2- 172.21.15.6:0/1860725951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1170109870 0x7f11701a1890 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f116800b770 tx=0x7f1168007980 comp rx=0 tx=0).stop 2024-05-09T04:50:31.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.761+0000 7f1178200700 1 -- 172.21.15.6:0/1860725951 shutdown_connections 2024-05-09T04:50:31.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.761+0000 7f1178200700 1 --2- 172.21.15.6:0/1860725951 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f11580b2900 0x7f11580b4e10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:31.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.761+0000 7f1178200700 1 --2- 172.21.15.6:0/1860725951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1170109870 0x7f11701a1890 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:31.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.761+0000 7f1178200700 1 --2- 172.21.15.6:0/1860725951 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f11701045f0 0x7f117019d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:31.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.761+0000 7f1178200700 1 --2- 172.21.15.6:0/1860725951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1170108f20 0x7f117019d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:31.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.761+0000 7f1178200700 1 -- 172.21.15.6:0/1860725951 >> 172.21.15.6:0/1860725951 conn(0x7f1170100040 msgr2=0x7f1170110e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:31.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.761+0000 7f1178200700 1 -- 172.21.15.6:0/1860725951 shutdown_connections 2024-05-09T04:50:31.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:31.761+0000 7f1178200700 1 -- 172.21.15.6:0/1860725951 wait complete. 2024-05-09T04:50:31.763 INFO:teuthology.orchestra.run.smithi006.stderr:listed 10 entries 2024-05-09T04:50:32.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:31 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:50:32.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:31 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:50:32.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:31 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:50:32.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:31 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1860725951' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T04:50:32.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:31 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:50:32.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:31 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:50:32.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:31 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:50:32.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:31 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/1860725951' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T04:50:32.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:31 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:50:32.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:31 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:50:32.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:31 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:50:32.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:31 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/1860725951' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T04:50:32.756 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T04:50:32.757 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:50:32.814 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-09T04:50:33.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:32 smithi006 ceph-mon[29835]: pgmap v365: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-09T04:50:33.272 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:50:33.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:32 smithi186 ceph-mon[34668]: pgmap v365: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-09T04:50:33.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:32 smithi169 ceph-mon[34075]: pgmap v365: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-09T04:50:35.083 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:34 smithi006 ceph-mon[29835]: pgmap v366: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-09T04:50:35.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:34 smithi186 ceph-mon[34668]: pgmap v366: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-09T04:50:35.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:34 smithi169 ceph-mon[34075]: pgmap v366: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-09T04:50:35.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.705+0000 7febc4e00700 1 -- 172.21.15.6:0/1072773935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febc0108f20 msgr2=0x7febc0109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:35.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.705+0000 7febc4e00700 1 --2- 172.21.15.6:0/1072773935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febc0108f20 0x7febc0109330 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feba80099c0 tx=0x7feba8009cd0 comp rx=0 tx=0).stop 2024-05-09T04:50:35.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.705+0000 7febc4e00700 1 -- 172.21.15.6:0/1072773935 shutdown_connections 2024-05-09T04:50:35.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.705+0000 7febc4e00700 1 --2- 172.21.15.6:0/1072773935 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7febc0109870 0x7febc01111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:35.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.705+0000 7febc4e00700 1 --2- 172.21.15.6:0/1072773935 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febc01045f0 0x7febc0104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:35.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.705+0000 7febc4e00700 1 --2- 172.21.15.6:0/1072773935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febc0108f20 0x7febc0109330 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:35.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.705+0000 7febc4e00700 1 -- 172.21.15.6:0/1072773935 >> 172.21.15.6:0/1072773935 conn(0x7febc00fffe0 msgr2=0x7febc0102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:35.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.705+0000 7febc4e00700 1 -- 172.21.15.6:0/1072773935 shutdown_connections 2024-05-09T04:50:35.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.706+0000 7febc4e00700 1 -- 172.21.15.6:0/1072773935 wait complete. 2024-05-09T04:50:35.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.707+0000 7febc4e00700 1 Processor -- start 2024-05-09T04:50:35.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.707+0000 7febc4e00700 1 -- start start 2024-05-09T04:50:35.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.708+0000 7febc4e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febc01045f0 0x7febc019d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:35.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.708+0000 7febc4e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febc0108f20 0x7febc019d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:35.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.708+0000 7febc4e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7febc0109870 0x7febc01a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:35.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.708+0000 7febc4e00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7febc01144d0 con 0x7febc0109870 2024-05-09T04:50:35.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.708+0000 7febc4e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7febc0114350 con 0x7febc01045f0 2024-05-09T04:50:35.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.708+0000 7febc4e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7febc0114650 con 0x7febc0108f20 2024-05-09T04:50:35.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.708+0000 7febb7e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febc0108f20 0x7febc019d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:35.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.708+0000 7febbe800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7febc0109870 0x7febc01a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:35.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.708+0000 7febb7e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febc0108f20 0x7febc019d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:53028/0 (socket says 172.21.15.6:53028) 2024-05-09T04:50:35.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.708+0000 7febbe800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7febc0109870 0x7febc01a1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58298/0 (socket says 172.21.15.6:58298) 2024-05-09T04:50:35.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.708+0000 7febb7e00700 1 -- 172.21.15.6:0/471660688 learned_addr learned my addr 172.21.15.6:0/471660688 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:50:35.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.708+0000 7febbde00700 1 --2- 172.21.15.6:0/471660688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febc01045f0 0x7febc019d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:35.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.709+0000 7febb7e00700 1 -- 172.21.15.6:0/471660688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febc01045f0 msgr2=0x7febc019d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:35.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.709+0000 7febb7e00700 1 --2- 172.21.15.6:0/471660688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febc01045f0 0x7febc019d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:35.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.709+0000 7febb7e00700 1 -- 172.21.15.6:0/471660688 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7febc0109870 msgr2=0x7febc01a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:35.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.709+0000 7febb7e00700 1 --2- 172.21.15.6:0/471660688 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7febc0109870 0x7febc01a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:35.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.709+0000 7febb7e00700 1 -- 172.21.15.6:0/471660688 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc01a2010 con 0x7febc0108f20 2024-05-09T04:50:35.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.709+0000 7febbde00700 1 --2- 172.21.15.6:0/471660688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febc01045f0 0x7febc019d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:50:35.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.709+0000 7febb7e00700 1 --2- 172.21.15.6:0/471660688 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febc0108f20 0x7febc019d6e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7feba8005f50 tx=0x7feba8005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:35.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.709+0000 7febb6a00700 1 -- 172.21.15.6:0/471660688 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba8026070 con 0x7febc0108f20 2024-05-09T04:50:35.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.710+0000 7febb6a00700 1 -- 172.21.15.6:0/471660688 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feba8003c70 con 0x7febc0108f20 2024-05-09T04:50:35.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.710+0000 7febc4e00700 1 -- 172.21.15.6:0/471660688 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febc01a22a0 con 0x7febc0108f20 2024-05-09T04:50:35.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.710+0000 7febb6a00700 1 -- 172.21.15.6:0/471660688 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba800f960 con 0x7febc0108f20 2024-05-09T04:50:35.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.710+0000 7febc4e00700 1 -- 172.21.15.6:0/471660688 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febc01a9c20 con 0x7febc0108f20 2024-05-09T04:50:35.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.711+0000 7febc4e00700 1 -- 172.21.15.6:0/471660688 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febc0105aa0 con 0x7febc0108f20 2024-05-09T04:50:35.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.714+0000 7febb6a00700 1 -- 172.21.15.6:0/471660688 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7feba8029070 con 0x7febc0108f20 2024-05-09T04:50:35.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.715+0000 7febb6a00700 1 --2- 172.21.15.6:0/471660688 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7feba40b2900 0x7feba40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:35.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.715+0000 7febb6a00700 1 -- 172.21.15.6:0/471660688 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7feba80e3110 con 0x7febc0108f20 2024-05-09T04:50:35.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.716+0000 7febbde00700 1 --2- 172.21.15.6:0/471660688 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7feba40b2900 0x7feba40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:35.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.716+0000 7febbde00700 1 --2- 172.21.15.6:0/471660688 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7feba40b2900 0x7feba40b4e10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7febb0011490 tx=0x7febb0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:35.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.717+0000 7febb6a00700 1 -- 172.21.15.6:0/471660688 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7feba8030080 con 0x7febc0108f20 2024-05-09T04:50:35.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.975+0000 7febc4e00700 1 -- 172.21.15.6:0/471660688 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7febc019e4a0 con 0x7febc0108f20 2024-05-09T04:50:35.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.976+0000 7febb6a00700 1 -- 172.21.15.6:0/471660688 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) ==== 81+0+579 (secure 0 0 0) 0x7feba8013070 con 0x7febc0108f20 2024-05-09T04:50:35.977 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/3742411767 2024-05-10T04:40:55.357057+0000 2024-05-09T04:50:35.977 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/2410771702 2024-05-10T04:40:55.357057+0000 2024-05-09T04:50:35.977 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6801/937713898 2024-05-10T04:40:55.357057+0000 2024-05-09T04:50:35.977 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6800/937713898 2024-05-10T04:40:55.357057+0000 2024-05-09T04:50:35.977 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6801/3021503778 2024-05-10T04:40:19.639151+0000 2024-05-09T04:50:35.977 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6800/3021503778 2024-05-10T04:40:19.639151+0000 2024-05-09T04:50:35.977 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/2559510850 2024-05-10T04:40:19.639151+0000 2024-05-09T04:50:35.977 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/3810474743 2024-05-10T04:40:19.639151+0000 2024-05-09T04:50:35.977 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/4133970555 2024-05-10T04:40:55.357057+0000 2024-05-09T04:50:35.977 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/664817743 2024-05-10T04:40:19.639151+0000 2024-05-09T04:50:35.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.979+0000 7febc4e00700 1 -- 172.21.15.6:0/471660688 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7feba40b2900 msgr2=0x7feba40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:35.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.979+0000 7febc4e00700 1 --2- 172.21.15.6:0/471660688 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7feba40b2900 0x7feba40b4e10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7febb0011490 tx=0x7febb0009260 comp rx=0 tx=0).stop 2024-05-09T04:50:35.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.979+0000 7febc4e00700 1 -- 172.21.15.6:0/471660688 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febc0108f20 msgr2=0x7febc019d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:35.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.979+0000 7febc4e00700 1 --2- 172.21.15.6:0/471660688 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febc0108f20 0x7febc019d6e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7feba8005f50 tx=0x7feba8005280 comp rx=0 tx=0).stop 2024-05-09T04:50:35.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.979+0000 7febc4e00700 1 -- 172.21.15.6:0/471660688 shutdown_connections 2024-05-09T04:50:35.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.979+0000 7febc4e00700 1 --2- 172.21.15.6:0/471660688 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7febc0109870 0x7febc01a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:35.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.979+0000 7febc4e00700 1 --2- 172.21.15.6:0/471660688 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7feba40b2900 0x7feba40b4e10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:35.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.979+0000 7febc4e00700 1 --2- 172.21.15.6:0/471660688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febc01045f0 0x7febc019d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:35.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.979+0000 7febc4e00700 1 --2- 172.21.15.6:0/471660688 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febc0108f20 0x7febc019d6e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:35.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.979+0000 7febc4e00700 1 -- 172.21.15.6:0/471660688 >> 172.21.15.6:0/471660688 conn(0x7febc00fffe0 msgr2=0x7febc0110ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:35.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.979+0000 7febc4e00700 1 -- 172.21.15.6:0/471660688 shutdown_connections 2024-05-09T04:50:35.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:35.980+0000 7febc4e00700 1 -- 172.21.15.6:0/471660688 wait complete. 2024-05-09T04:50:35.982 INFO:teuthology.orchestra.run.smithi006.stderr:listed 10 entries 2024-05-09T04:50:36.844 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi006.front.sepia.ceph.com... 2024-05-09T04:50:36.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:50:36.845 DEBUG:teuthology.orchestra.run.smithi006:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-09T04:50:36.938 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi006.stderr:ceph-fuse[73003]: fuse finished with error 0 and tester_r 0 2024-05-09T04:50:36.944 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T04:50:37.114 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:36 smithi006 ceph-mon[29835]: pgmap v367: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-09T04:50:37.114 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:36 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/471660688' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T04:50:37.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:36 smithi186 ceph-mon[34668]: pgmap v367: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-09T04:50:37.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:36 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/471660688' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T04:50:37.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:36 smithi169 ceph-mon[34075]: pgmap v367: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-09T04:50:37.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:36 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/471660688' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T04:50:39.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:38 smithi006 ceph-mon[29835]: pgmap v368: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-09T04:50:39.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:38 smithi186 ceph-mon[34668]: pgmap v368: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-09T04:50:39.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:38 smithi169 ceph-mon[34075]: pgmap v368: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-09T04:50:41.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:40 smithi006 ceph-mon[29835]: pgmap v369: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-09T04:50:41.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:40 smithi186 ceph-mon[34668]: pgmap v369: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-09T04:50:41.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:40 smithi169 ceph-mon[34075]: pgmap v369: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-09T04:50:42.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:41 smithi186 ceph-mon[34668]: pgmap v370: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-09T04:50:42.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:41 smithi169 ceph-mon[34075]: pgmap v370: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-09T04:50:42.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:41 smithi006 ceph-mon[29835]: pgmap v370: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-09T04:50:43.046 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2024-05-09T04:50:43.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:50:43.046 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-09T04:50:43.129 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T04:50:43.624 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:50:44.814 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:44 smithi006 ceph-mon[29835]: pgmap v371: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-09T04:50:45.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:44 smithi186 ceph-mon[34668]: pgmap v371: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-09T04:50:45.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:44 smithi169 ceph-mon[34075]: pgmap v371: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-09T04:50:45.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.955+0000 7fafa7a00700 1 -- 172.21.15.6:0/4020569777 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafa010c5a0 msgr2=0x7fafa010c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:45.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.955+0000 7fafa7a00700 1 --2- 172.21.15.6:0/4020569777 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafa010c5a0 0x7fafa010c9b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faf94009990 tx=0x7faf94009ca0 comp rx=0 tx=0).stop 2024-05-09T04:50:45.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.955+0000 7fafa7a00700 1 -- 172.21.15.6:0/4020569777 shutdown_connections 2024-05-09T04:50:45.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.955+0000 7fafa7a00700 1 --2- 172.21.15.6:0/4020569777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafa0106eb0 0x7fafa0113340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:45.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.955+0000 7fafa7a00700 1 --2- 172.21.15.6:0/4020569777 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafa010c5a0 0x7fafa010c9b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:45.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.955+0000 7fafa7a00700 1 --2- 172.21.15.6:0/4020569777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fafa0106560 0x7fafa0106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:45.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.955+0000 7fafa7a00700 1 -- 172.21.15.6:0/4020569777 >> 172.21.15.6:0/4020569777 conn(0x7fafa0102060 msgr2=0x7fafa01044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:45.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.955+0000 7fafa7a00700 1 -- 172.21.15.6:0/4020569777 shutdown_connections 2024-05-09T04:50:45.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.955+0000 7fafa7a00700 1 -- 172.21.15.6:0/4020569777 wait complete. 2024-05-09T04:50:45.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.956+0000 7fafa7a00700 1 Processor -- start 2024-05-09T04:50:45.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.957+0000 7fafa7a00700 1 -- start start 2024-05-09T04:50:45.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.957+0000 7fafa7a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafa0106560 0x7fafa019d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:45.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.957+0000 7fafa7a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafa0106eb0 0x7fafa019d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:45.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.957+0000 7fafa7a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fafa010c5a0 0x7fafa01a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:45.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.957+0000 7fafa7a00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fafa01166b0 con 0x7fafa0106eb0 2024-05-09T04:50:45.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.957+0000 7fafa7a00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fafa0116530 con 0x7fafa0106560 2024-05-09T04:50:45.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.957+0000 7fafa7a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fafa0116830 con 0x7fafa010c5a0 2024-05-09T04:50:45.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.957+0000 7fafa5a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fafa010c5a0 0x7fafa01a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:45.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.957+0000 7fafa5000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafa0106560 0x7fafa019d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:45.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.957+0000 7faf9fe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafa0106eb0 0x7fafa019d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:45.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.957+0000 7fafa5a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fafa010c5a0 0x7fafa01a1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:51876/0 (socket says 172.21.15.6:51876) 2024-05-09T04:50:45.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.957+0000 7fafa5a00700 1 -- 172.21.15.6:0/1434546445 learned_addr learned my addr 172.21.15.6:0/1434546445 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:50:45.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.958+0000 7faf9fe00700 1 -- 172.21.15.6:0/1434546445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fafa010c5a0 msgr2=0x7fafa01a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:45.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.958+0000 7faf9fe00700 1 --2- 172.21.15.6:0/1434546445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fafa010c5a0 0x7fafa01a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:45.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.958+0000 7faf9fe00700 1 -- 172.21.15.6:0/1434546445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafa0106560 msgr2=0x7fafa019d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:45.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.958+0000 7faf9fe00700 1 --2- 172.21.15.6:0/1434546445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafa0106560 0x7fafa019d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:45.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.958+0000 7faf9fe00700 1 -- 172.21.15.6:0/1434546445 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa01a1f10 con 0x7fafa0106eb0 2024-05-09T04:50:45.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.958+0000 7faf9fe00700 1 --2- 172.21.15.6:0/1434546445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafa0106eb0 0x7fafa019d5e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fafa0075b80 tx=0x7faf8c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:45.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.959+0000 7faf9d600700 1 -- 172.21.15.6:0/1434546445 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c010900 con 0x7fafa0106eb0 2024-05-09T04:50:45.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.959+0000 7faf9d600700 1 -- 172.21.15.6:0/1434546445 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf8c00c430 con 0x7fafa0106eb0 2024-05-09T04:50:45.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.959+0000 7fafa7a00700 1 -- 172.21.15.6:0/1434546445 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafa01a2200 con 0x7fafa0106eb0 2024-05-09T04:50:45.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.959+0000 7faf9d600700 1 -- 172.21.15.6:0/1434546445 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c018680 con 0x7fafa0106eb0 2024-05-09T04:50:45.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.959+0000 7fafa7a00700 1 -- 172.21.15.6:0/1434546445 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafa01a9b20 con 0x7fafa0106eb0 2024-05-09T04:50:45.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.960+0000 7fafa7a00700 1 -- 172.21.15.6:0/1434546445 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafa010e960 con 0x7fafa0106eb0 2024-05-09T04:50:45.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.963+0000 7faf9d600700 1 -- 172.21.15.6:0/1434546445 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7faf8c010aa0 con 0x7fafa0106eb0 2024-05-09T04:50:45.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.965+0000 7faf9d600700 1 --2- 172.21.15.6:0/1434546445 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7faf880b2880 0x7faf880b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:45.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.965+0000 7faf9d600700 1 -- 172.21.15.6:0/1434546445 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7faf8c0e28d0 con 0x7fafa0106eb0 2024-05-09T04:50:45.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.965+0000 7fafa5000700 1 --2- 172.21.15.6:0/1434546445 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7faf880b2880 0x7faf880b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:45.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.966+0000 7fafa5000700 1 --2- 172.21.15.6:0/1434546445 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7faf880b2880 0x7faf880b4d90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7faf9400b2e0 tx=0x7faf94005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:45.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:45.968+0000 7faf9d600700 1 -- 172.21.15.6:0/1434546445 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faf8c0aa250 con 0x7fafa0106eb0 2024-05-09T04:50:46.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:46.215+0000 7fafa7a00700 1 -- 172.21.15.6:0/1434546445 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fafa019e3a0 con 0x7fafa0106eb0 2024-05-09T04:50:46.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:46.218+0000 7faf9d600700 1 -- 172.21.15.6:0/1434546445 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24566 (secure 0 0 0) 0x7faf8c0a9b50 con 0x7fafa0106eb0 2024-05-09T04:50:46.220 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:50:46.220 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":87,"fsid":"f74d813c-0dbd-11ef-bc97-c7b262605968","created":"2024-05-09T04:39:35.287470+0000","modified":"2024-05-09T04:49:56.686042+0000","last_up_change":"2024-05-09T04:48:16.919842+0000","last_in_change":"2024-05-09T04:47:56.452645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:43:47.538306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:49:28.805447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T04:49:32.970287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T04:49:45.637000+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1b732fd3-e648-4b0d-aece-083bdaaa5d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6803","nonce":1738750914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6805","nonce":1738750914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6809","nonce":1738750914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6807","nonce":1738750914}]},"public_addr":"172.21.15.6:6803/1738750914","cluster_addr":"172.21.15.6:6805/1738750914","heartbeat_back_addr":"172.21.15.6:6809/1738750914","heartbeat_front_addr":"172.21.15.6:6807/1738750914","state":["exists","up"]},{"osd":1,"uuid":"9770c07d-4243-43bf-a0cd-9d2ac558556c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6801","nonce":2552207737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6803","nonce":2552207737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6807","nonce":2552207737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6805","nonce":2552207737}]},"public_addr":"172.21.15.169:6801/2552207737","cluster_addr":"172.21.15.169:6803/2552207737","heartbeat_back_addr":"172.21.15.169:6807/2552207737","heartbeat_front_addr":"172.21.15.169:6805/2552207737","state":["exists","up"]},{"osd":2,"uuid":"3f50ed89-b2a8-415e-b78d-8510d98d9368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6801","nonce":231435105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6803","nonce":231435105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6807","nonce":231435105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6805","nonce":231435105}]},"public_addr":"172.21.15.186:6801/231435105","cluster_addr":"172.21.15.186:6803/231435105","heartbeat_back_addr":"172.21.15.186:6807/231435105","heartbeat_front_addr":"172.21.15.186:6805/231435105","state":["exists","up"]},{"osd":3,"uuid":"afcbc36b-5669-4044-8dc5-2ebfe7970774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6811","nonce":2846613703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6813","nonce":2846613703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6817","nonce":2846613703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6815","nonce":2846613703}]},"public_addr":"172.21.15.6:6811/2846613703","cluster_addr":"172.21.15.6:6813/2846613703","heartbeat_back_addr":"172.21.15.6:6817/2846613703","heartbeat_front_addr":"172.21.15.6:6815/2846613703","state":["exists","up"]},{"osd":4,"uuid":"bb84cc5e-0297-4e23-be50-7ee2f7e59665","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6809","nonce":1928722453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6811","nonce":1928722453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6815","nonce":1928722453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6813","nonce":1928722453}]},"public_addr":"172.21.15.169:6809/1928722453","cluster_addr":"172.21.15.169:6811/1928722453","heartbeat_back_addr":"172.21.15.169:6815/1928722453","heartbeat_front_addr":"172.21.15.169:6813/1928722453","state":["exists","up"]},{"osd":5,"uuid":"830dc7f3-352e-4b01-8d4c-a9b68dcb40d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6809","nonce":3644445439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6811","nonce":3644445439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6815","nonce":3644445439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6813","nonce":3644445439}]},"public_addr":"172.21.15.186:6809/3644445439","cluster_addr":"172.21.15.186:6811/3644445439","heartbeat_back_addr":"172.21.15.186:6815/3644445439","heartbeat_front_addr":"172.21.15.186:6813/3644445439","state":["exists","up"]},{"osd":6,"uuid":"2e8478cb-c8c9-47ce-a3a5-48ffc959ab24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6819","nonce":3068252891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6821","nonce":3068252891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6825","nonce":3068252891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6823","nonce":3068252891}]},"public_addr":"172.21.15.6:6819/3068252891","cluster_addr":"172.21.15.6:6821/3068252891","heartbeat_back_addr":"172.21.15.6:6825/3068252891","heartbeat_front_addr":"172.21.15.6:6823/3068252891","state":["exists","up"]},{"osd":7,"uuid":"0f432ee2-42a4-4efa-9233-8e5a2fd65d10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6817","nonce":1303584711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6819","nonce":1303584711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6823","nonce":1303584711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6821","nonce":1303584711}]},"public_addr":"172.21.15.169:6817/1303584711","cluster_addr":"172.21.15.169:6819/1303584711","heartbeat_back_addr":"172.21.15.169:6823/1303584711","heartbeat_front_addr":"172.21.15.169:6821/1303584711","state":["exists","up"]},{"osd":8,"uuid":"cf5dca81-131b-48e7-9417-d01d4be47316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6817","nonce":3475163771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6819","nonce":3475163771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6823","nonce":3475163771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6821","nonce":3475163771}]},"public_addr":"172.21.15.186:6817/3475163771","cluster_addr":"172.21.15.186:6819/3475163771","heartbeat_back_addr":"172.21.15.186:6823/3475163771","heartbeat_front_addr":"172.21.15.186:6821/3475163771","state":["exists","up"]},{"osd":9,"uuid":"bca7bedd-1a57-4465-9033-064ca468add2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6827","nonce":3590990973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6829","nonce":3590990973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6833","nonce":3590990973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6831","nonce":3590990973}]},"public_addr":"172.21.15.6:6827/3590990973","cluster_addr":"172.21.15.6:6829/3590990973","heartbeat_back_addr":"172.21.15.6:6833/3590990973","heartbeat_front_addr":"172.21.15.6:6831/3590990973","state":["exists","up"]},{"osd":10,"uuid":"97b0e058-64f8-42ee-9b99-3ef8e39c8a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6825","nonce":61007341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6827","nonce":61007341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6831","nonce":61007341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6829","nonce":61007341}]},"public_addr":"172.21.15.169:6825/61007341","cluster_addr":"172.21.15.169:6827/61007341","heartbeat_back_addr":"172.21.15.169:6831/61007341","heartbeat_front_addr":"172.21.15.169:6829/61007341","state":["exists","up"]},{"osd":11,"uuid":"7f2220b0-24cf-48b2-95ae-1d66e00e57fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6825","nonce":1138940441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6827","nonce":1138940441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6831","nonce":1138940441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6829","nonce":1138940441}]},"public_addr":"172.21.15.186:6825/1138940441","cluster_addr":"172.21.15.186:6827/1138940441","heartbeat_back_addr":"172.21.15.186:6831/1138940441","heartbeat_front_addr":"172.21.15.186:6829/1138940441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:53.766295+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:20.195243+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:44.153812+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:20.472086+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:48.415371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:13.795415+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:50.236398+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:18.066623+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:43.781009+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:47:20.084719+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:47:48.219901+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:48:14.965536+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/3742411767":"2024-05-10T04:40:55.357057+0000","172.21.15.6:0/2410771702":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6801/937713898":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6800/937713898":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6801/3021503778":"2024-05-10T04:40:19.639151+0000","172.21.15.6:6800/3021503778":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/2559510850":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/3810474743":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/4133970555":"2024-05-10T04:40:55.357057+0000","172.21.15.6:0/664817743":"2024-05-10T04:40:19.639151+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:50:46.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:46.221+0000 7fafa7a00700 1 -- 172.21.15.6:0/1434546445 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7faf880b2880 msgr2=0x7faf880b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:46.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:46.221+0000 7fafa7a00700 1 --2- 172.21.15.6:0/1434546445 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7faf880b2880 0x7faf880b4d90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7faf9400b2e0 tx=0x7faf94005960 comp rx=0 tx=0).stop 2024-05-09T04:50:46.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:46.221+0000 7fafa7a00700 1 -- 172.21.15.6:0/1434546445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafa0106eb0 msgr2=0x7fafa019d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:46.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:46.221+0000 7fafa7a00700 1 --2- 172.21.15.6:0/1434546445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafa0106eb0 0x7fafa019d5e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fafa0075b80 tx=0x7faf8c00eca0 comp rx=0 tx=0).stop 2024-05-09T04:50:46.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:46.222+0000 7fafa7a00700 1 -- 172.21.15.6:0/1434546445 shutdown_connections 2024-05-09T04:50:46.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:46.222+0000 7fafa7a00700 1 --2- 172.21.15.6:0/1434546445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fafa010c5a0 0x7fafa01a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:46.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:46.222+0000 7fafa7a00700 1 --2- 172.21.15.6:0/1434546445 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7faf880b2880 0x7faf880b4d90 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:46.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:46.222+0000 7fafa7a00700 1 --2- 172.21.15.6:0/1434546445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafa0106560 0x7fafa019d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:46.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:46.222+0000 7fafa7a00700 1 --2- 172.21.15.6:0/1434546445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafa0106eb0 0x7fafa019d5e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:46.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:46.222+0000 7fafa7a00700 1 -- 172.21.15.6:0/1434546445 >> 172.21.15.6:0/1434546445 conn(0x7fafa0102060 msgr2=0x7fafa0104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:46.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:46.222+0000 7fafa7a00700 1 -- 172.21.15.6:0/1434546445 shutdown_connections 2024-05-09T04:50:46.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:46.222+0000 7fafa7a00700 1 -- 172.21.15.6:0/1434546445 wait complete. 2024-05-09T04:50:46.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:46 smithi006 ceph-mon[29835]: pgmap v372: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-09T04:50:46.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:46 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1434546445' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:50:47.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:46 smithi186 ceph-mon[34668]: pgmap v372: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-09T04:50:47.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:46 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/1434546445' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:50:47.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:46 smithi169 ceph-mon[34075]: pgmap v372: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-09T04:50:47.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:46 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/1434546445' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:50:47.163 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T04:50:47.625 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:50:48.803 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:48 smithi006 ceph-mon[29835]: pgmap v373: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-09T04:50:49.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:48 smithi186 ceph-mon[34668]: pgmap v373: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-09T04:50:49.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:48 smithi169 ceph-mon[34075]: pgmap v373: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-09T04:50:49.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.756+0000 7f7c26e00700 1 -- 172.21.15.6:0/2614346199 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c20076450 msgr2=0x7f7c201106c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:49.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.756+0000 7f7c26e00700 1 --2- 172.21.15.6:0/2614346199 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c20076450 0x7f7c201106c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c0098f0 tx=0x7f7c0c009c00 comp rx=0 tx=0).stop 2024-05-09T04:50:49.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.756+0000 7f7c26e00700 1 -- 172.21.15.6:0/2614346199 shutdown_connections 2024-05-09T04:50:49.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.756+0000 7f7c26e00700 1 --2- 172.21.15.6:0/2614346199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c20110c00 0x7f7c20113020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:49.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.756+0000 7f7c26e00700 1 --2- 172.21.15.6:0/2614346199 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c20075b00 0x7f7c20075f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:49.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.756+0000 7f7c26e00700 1 --2- 172.21.15.6:0/2614346199 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c20076450 0x7f7c201106c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:49.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.756+0000 7f7c26e00700 1 -- 172.21.15.6:0/2614346199 >> 172.21.15.6:0/2614346199 conn(0x7f7c20100040 msgr2=0x7f7c20102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:49.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.756+0000 7f7c26e00700 1 -- 172.21.15.6:0/2614346199 shutdown_connections 2024-05-09T04:50:49.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.756+0000 7f7c26e00700 1 -- 172.21.15.6:0/2614346199 wait complete. 2024-05-09T04:50:49.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.758+0000 7f7c26e00700 1 Processor -- start 2024-05-09T04:50:49.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.758+0000 7f7c26e00700 1 -- start start 2024-05-09T04:50:49.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.758+0000 7f7c26e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c20075b00 0x7f7c2019d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:49.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.758+0000 7f7c26e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c20076450 0x7f7c2019d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:49.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.758+0000 7f7c26e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c20110c00 0x7f7c201a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:49.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.758+0000 7f7c26e00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7c201166e0 con 0x7f7c20075b00 2024-05-09T04:50:49.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.758+0000 7f7c26e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7c20116560 con 0x7f7c20076450 2024-05-09T04:50:49.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.758+0000 7f7c26e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7c20116860 con 0x7f7c20110c00 2024-05-09T04:50:49.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.759+0000 7f7c1fe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c20075b00 0x7f7c2019d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:49.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.759+0000 7f7c1fe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c20075b00 0x7f7c2019d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49240/0 (socket says 172.21.15.6:49240) 2024-05-09T04:50:49.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.759+0000 7f7c1fe00700 1 -- 172.21.15.6:0/190539004 learned_addr learned my addr 172.21.15.6:0/190539004 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:50:49.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.759+0000 7f7c24e00700 1 --2- 172.21.15.6:0/190539004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c20110c00 0x7f7c201a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:49.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.759+0000 7f7c1f400700 1 --2- 172.21.15.6:0/190539004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c20076450 0x7f7c2019d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:49.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.759+0000 7f7c1fe00700 1 -- 172.21.15.6:0/190539004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c20110c00 msgr2=0x7f7c201a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:49.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.759+0000 7f7c1fe00700 1 --2- 172.21.15.6:0/190539004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c20110c00 0x7f7c201a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:49.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.759+0000 7f7c1fe00700 1 -- 172.21.15.6:0/190539004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c20076450 msgr2=0x7f7c2019d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:49.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.759+0000 7f7c1fe00700 1 --2- 172.21.15.6:0/190539004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c20076450 0x7f7c2019d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:49.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.759+0000 7f7c1fe00700 1 -- 172.21.15.6:0/190539004 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c201a2000 con 0x7f7c20075b00 2024-05-09T04:50:49.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.760+0000 7f7c24e00700 1 --2- 172.21.15.6:0/190539004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c20110c00 0x7f7c201a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:50:49.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.760+0000 7f7c1fe00700 1 --2- 172.21.15.6:0/190539004 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c20075b00 0x7f7c2019d130 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000e9f0 tx=0x7f7c1000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:49.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.760+0000 7f7c1cc00700 1 -- 172.21.15.6:0/190539004 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c10004a30 con 0x7f7c20075b00 2024-05-09T04:50:49.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.760+0000 7f7c1cc00700 1 -- 172.21.15.6:0/190539004 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c1000cc70 con 0x7f7c20075b00 2024-05-09T04:50:49.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.760+0000 7f7c26e00700 1 -- 172.21.15.6:0/190539004 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c201a22f0 con 0x7f7c20075b00 2024-05-09T04:50:49.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.760+0000 7f7c1cc00700 1 -- 172.21.15.6:0/190539004 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c10005600 con 0x7f7c20075b00 2024-05-09T04:50:49.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.760+0000 7f7c26e00700 1 -- 172.21.15.6:0/190539004 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c201a9c10 con 0x7f7c20075b00 2024-05-09T04:50:49.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.762+0000 7f7c26e00700 1 -- 172.21.15.6:0/190539004 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c200512c0 con 0x7f7c20075b00 2024-05-09T04:50:49.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.765+0000 7f7c1cc00700 1 -- 172.21.15.6:0/190539004 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f7c10004c60 con 0x7f7c20075b00 2024-05-09T04:50:49.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.767+0000 7f7c1cc00700 1 --2- 172.21.15.6:0/190539004 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f7c040b2900 0x7f7c040b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:49.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.768+0000 7f7c1cc00700 1 -- 172.21.15.6:0/190539004 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f7c100e2b40 con 0x7f7c20075b00 2024-05-09T04:50:49.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.768+0000 7f7c1f400700 1 --2- 172.21.15.6:0/190539004 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f7c040b2900 0x7f7c040b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:49.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.769+0000 7f7c1f400700 1 --2- 172.21.15.6:0/190539004 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f7c040b2900 0x7f7c040b4e10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009230 tx=0x7f7c0c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:49.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:49.771+0000 7f7c1cc00700 1 -- 172.21.15.6:0/190539004 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7c10054df0 con 0x7f7c20075b00 2024-05-09T04:50:50.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:50.009+0000 7f7c26e00700 1 -- 172.21.15.6:0/190539004 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c2019e490 con 0x7f7c20075b00 2024-05-09T04:50:50.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:50.011+0000 7f7c1cc00700 1 -- 172.21.15.6:0/190539004 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24566 (secure 0 0 0) 0x7f7c1000ce20 con 0x7f7c20075b00 2024-05-09T04:50:50.012 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:50:50.012 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":87,"fsid":"f74d813c-0dbd-11ef-bc97-c7b262605968","created":"2024-05-09T04:39:35.287470+0000","modified":"2024-05-09T04:49:56.686042+0000","last_up_change":"2024-05-09T04:48:16.919842+0000","last_in_change":"2024-05-09T04:47:56.452645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:43:47.538306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:49:28.805447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T04:49:32.970287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T04:49:45.637000+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1b732fd3-e648-4b0d-aece-083bdaaa5d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6803","nonce":1738750914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6805","nonce":1738750914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6809","nonce":1738750914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6807","nonce":1738750914}]},"public_addr":"172.21.15.6:6803/1738750914","cluster_addr":"172.21.15.6:6805/1738750914","heartbeat_back_addr":"172.21.15.6:6809/1738750914","heartbeat_front_addr":"172.21.15.6:6807/1738750914","state":["exists","up"]},{"osd":1,"uuid":"9770c07d-4243-43bf-a0cd-9d2ac558556c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6801","nonce":2552207737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6803","nonce":2552207737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6807","nonce":2552207737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6805","nonce":2552207737}]},"public_addr":"172.21.15.169:6801/2552207737","cluster_addr":"172.21.15.169:6803/2552207737","heartbeat_back_addr":"172.21.15.169:6807/2552207737","heartbeat_front_addr":"172.21.15.169:6805/2552207737","state":["exists","up"]},{"osd":2,"uuid":"3f50ed89-b2a8-415e-b78d-8510d98d9368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6801","nonce":231435105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6803","nonce":231435105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6807","nonce":231435105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6805","nonce":231435105}]},"public_addr":"172.21.15.186:6801/231435105","cluster_addr":"172.21.15.186:6803/231435105","heartbeat_back_addr":"172.21.15.186:6807/231435105","heartbeat_front_addr":"172.21.15.186:6805/231435105","state":["exists","up"]},{"osd":3,"uuid":"afcbc36b-5669-4044-8dc5-2ebfe7970774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6811","nonce":2846613703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6813","nonce":2846613703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6817","nonce":2846613703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6815","nonce":2846613703}]},"public_addr":"172.21.15.6:6811/2846613703","cluster_addr":"172.21.15.6:6813/2846613703","heartbeat_back_addr":"172.21.15.6:6817/2846613703","heartbeat_front_addr":"172.21.15.6:6815/2846613703","state":["exists","up"]},{"osd":4,"uuid":"bb84cc5e-0297-4e23-be50-7ee2f7e59665","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6809","nonce":1928722453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6811","nonce":1928722453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6815","nonce":1928722453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6813","nonce":1928722453}]},"public_addr":"172.21.15.169:6809/1928722453","cluster_addr":"172.21.15.169:6811/1928722453","heartbeat_back_addr":"172.21.15.169:6815/1928722453","heartbeat_front_addr":"172.21.15.169:6813/1928722453","state":["exists","up"]},{"osd":5,"uuid":"830dc7f3-352e-4b01-8d4c-a9b68dcb40d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6809","nonce":3644445439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6811","nonce":3644445439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6815","nonce":3644445439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6813","nonce":3644445439}]},"public_addr":"172.21.15.186:6809/3644445439","cluster_addr":"172.21.15.186:6811/3644445439","heartbeat_back_addr":"172.21.15.186:6815/3644445439","heartbeat_front_addr":"172.21.15.186:6813/3644445439","state":["exists","up"]},{"osd":6,"uuid":"2e8478cb-c8c9-47ce-a3a5-48ffc959ab24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6819","nonce":3068252891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6821","nonce":3068252891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6825","nonce":3068252891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6823","nonce":3068252891}]},"public_addr":"172.21.15.6:6819/3068252891","cluster_addr":"172.21.15.6:6821/3068252891","heartbeat_back_addr":"172.21.15.6:6825/3068252891","heartbeat_front_addr":"172.21.15.6:6823/3068252891","state":["exists","up"]},{"osd":7,"uuid":"0f432ee2-42a4-4efa-9233-8e5a2fd65d10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6817","nonce":1303584711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6819","nonce":1303584711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6823","nonce":1303584711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6821","nonce":1303584711}]},"public_addr":"172.21.15.169:6817/1303584711","cluster_addr":"172.21.15.169:6819/1303584711","heartbeat_back_addr":"172.21.15.169:6823/1303584711","heartbeat_front_addr":"172.21.15.169:6821/1303584711","state":["exists","up"]},{"osd":8,"uuid":"cf5dca81-131b-48e7-9417-d01d4be47316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6817","nonce":3475163771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6819","nonce":3475163771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6823","nonce":3475163771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6821","nonce":3475163771}]},"public_addr":"172.21.15.186:6817/3475163771","cluster_addr":"172.21.15.186:6819/3475163771","heartbeat_back_addr":"172.21.15.186:6823/3475163771","heartbeat_front_addr":"172.21.15.186:6821/3475163771","state":["exists","up"]},{"osd":9,"uuid":"bca7bedd-1a57-4465-9033-064ca468add2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6827","nonce":3590990973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6829","nonce":3590990973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6833","nonce":3590990973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6831","nonce":3590990973}]},"public_addr":"172.21.15.6:6827/3590990973","cluster_addr":"172.21.15.6:6829/3590990973","heartbeat_back_addr":"172.21.15.6:6833/3590990973","heartbeat_front_addr":"172.21.15.6:6831/3590990973","state":["exists","up"]},{"osd":10,"uuid":"97b0e058-64f8-42ee-9b99-3ef8e39c8a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6825","nonce":61007341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6827","nonce":61007341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6831","nonce":61007341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6829","nonce":61007341}]},"public_addr":"172.21.15.169:6825/61007341","cluster_addr":"172.21.15.169:6827/61007341","heartbeat_back_addr":"172.21.15.169:6831/61007341","heartbeat_front_addr":"172.21.15.169:6829/61007341","state":["exists","up"]},{"osd":11,"uuid":"7f2220b0-24cf-48b2-95ae-1d66e00e57fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6825","nonce":1138940441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6827","nonce":1138940441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6831","nonce":1138940441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6829","nonce":1138940441}]},"public_addr":"172.21.15.186:6825/1138940441","cluster_addr":"172.21.15.186:6827/1138940441","heartbeat_back_addr":"172.21.15.186:6831/1138940441","heartbeat_front_addr":"172.21.15.186:6829/1138940441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:53.766295+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:20.195243+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:44.153812+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:20.472086+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:48.415371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:13.795415+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:50.236398+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:18.066623+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:43.781009+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:47:20.084719+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:47:48.219901+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:48:14.965536+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/3742411767":"2024-05-10T04:40:55.357057+0000","172.21.15.6:0/2410771702":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6801/937713898":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6800/937713898":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6801/3021503778":"2024-05-10T04:40:19.639151+0000","172.21.15.6:6800/3021503778":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/2559510850":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/3810474743":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/4133970555":"2024-05-10T04:40:55.357057+0000","172.21.15.6:0/664817743":"2024-05-10T04:40:19.639151+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:50:50.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:50.014+0000 7f7c26e00700 1 -- 172.21.15.6:0/190539004 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f7c040b2900 msgr2=0x7f7c040b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:50.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:50.014+0000 7f7c26e00700 1 --2- 172.21.15.6:0/190539004 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f7c040b2900 0x7f7c040b4e10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009230 tx=0x7f7c0c0058e0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:50.014+0000 7f7c26e00700 1 -- 172.21.15.6:0/190539004 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c20075b00 msgr2=0x7f7c2019d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:50.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:50.014+0000 7f7c26e00700 1 --2- 172.21.15.6:0/190539004 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c20075b00 0x7f7c2019d130 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000e9f0 tx=0x7f7c1000ed00 comp rx=0 tx=0).stop 2024-05-09T04:50:50.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:50.015+0000 7f7c26e00700 1 -- 172.21.15.6:0/190539004 shutdown_connections 2024-05-09T04:50:50.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:50.015+0000 7f7c26e00700 1 --2- 172.21.15.6:0/190539004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c20110c00 0x7f7c201a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:50.015+0000 7f7c26e00700 1 --2- 172.21.15.6:0/190539004 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c20075b00 0x7f7c2019d130 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:50.015+0000 7f7c26e00700 1 --2- 172.21.15.6:0/190539004 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f7c040b2900 0x7f7c040b4e10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:50.015+0000 7f7c26e00700 1 --2- 172.21.15.6:0/190539004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c20076450 0x7f7c2019d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:50.015+0000 7f7c26e00700 1 -- 172.21.15.6:0/190539004 >> 172.21.15.6:0/190539004 conn(0x7f7c20100040 msgr2=0x7f7c20100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:50.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:50.015+0000 7f7c26e00700 1 -- 172.21.15.6:0/190539004 shutdown_connections 2024-05-09T04:50:50.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:50.015+0000 7f7c26e00700 1 -- 172.21.15.6:0/190539004 wait complete. 2024-05-09T04:50:50.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:50 smithi006 ceph-mon[29835]: pgmap v374: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-09T04:50:50.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:50 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/190539004' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:50:50.919 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:50 smithi169 ceph-mon[34075]: pgmap v374: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-09T04:50:50.919 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:50 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/190539004' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:50:51.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:50 smithi186 ceph-mon[34668]: pgmap v374: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-09T04:50:51.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:50 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/190539004' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:50:51.105 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-09T04:50:51.576 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:50:52.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:52 smithi006 ceph-mon[29835]: pgmap v375: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-09T04:50:53.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:52 smithi186 ceph-mon[34668]: pgmap v375: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-09T04:50:53.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:52 smithi169 ceph-mon[34075]: pgmap v375: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-09T04:50:53.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.865+0000 7f330ca00700 1 -- 172.21.15.6:0/3012469230 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f330810b8a0 msgr2=0x7f3308113200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:53.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.865+0000 7f330ca00700 1 --2- 172.21.15.6:0/3012469230 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f330810b8a0 0x7f3308113200 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f330000b0d0 tx=0x7f330000b3e0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.866+0000 7f330ca00700 1 -- 172.21.15.6:0/3012469230 shutdown_connections 2024-05-09T04:50:53.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.866+0000 7f330ca00700 1 --2- 172.21.15.6:0/3012469230 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f330810b8a0 0x7f3308113200 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.866+0000 7f330ca00700 1 --2- 172.21.15.6:0/3012469230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3308106620 0x7f3308106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.866+0000 7f330ca00700 1 --2- 172.21.15.6:0/3012469230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f330810af50 0x7f330810b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.866+0000 7f330ca00700 1 -- 172.21.15.6:0/3012469230 >> 172.21.15.6:0/3012469230 conn(0x7f3308102030 msgr2=0x7f3308104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:53.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.866+0000 7f330ca00700 1 -- 172.21.15.6:0/3012469230 shutdown_connections 2024-05-09T04:50:53.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.866+0000 7f330ca00700 1 -- 172.21.15.6:0/3012469230 wait complete. 2024-05-09T04:50:53.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.867+0000 7f330ca00700 1 Processor -- start 2024-05-09T04:50:53.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.867+0000 7f330ca00700 1 -- start start 2024-05-09T04:50:53.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.868+0000 7f330ca00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3308106620 0x7f330819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:53.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.868+0000 7f330ca00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f330810af50 0x7f330819d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:53.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.868+0000 7f330ca00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f330810b8a0 0x7f33081a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:53.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.868+0000 7f330ca00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3308116520 con 0x7f330810af50 2024-05-09T04:50:53.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.868+0000 7f330ca00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f33081163a0 con 0x7f3308106620 2024-05-09T04:50:53.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.868+0000 7f330ca00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f33081166a0 con 0x7f330810b8a0 2024-05-09T04:50:53.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.868+0000 7f3307400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3308106620 0x7f330819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:53.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.868+0000 7f3306a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f330810af50 0x7f330819d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:53.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.868+0000 7f3307400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3308106620 0x7f330819ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:36270/0 (socket says 172.21.15.6:36270) 2024-05-09T04:50:53.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.868+0000 7f3307400700 1 -- 172.21.15.6:0/149271149 learned_addr learned my addr 172.21.15.6:0/149271149 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:50:53.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.868+0000 7f3307e00700 1 --2- 172.21.15.6:0/149271149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f330810b8a0 0x7f33081a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:53.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.869+0000 7f3307e00700 1 -- 172.21.15.6:0/149271149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3308106620 msgr2=0x7f330819ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:53.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.869+0000 7f3307e00700 1 --2- 172.21.15.6:0/149271149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3308106620 0x7f330819ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.869+0000 7f3307e00700 1 -- 172.21.15.6:0/149271149 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f330810af50 msgr2=0x7f330819d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:53.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.869+0000 7f3307e00700 1 --2- 172.21.15.6:0/149271149 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f330810af50 0x7f330819d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.869+0000 7f3307e00700 1 -- 172.21.15.6:0/149271149 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33081a1d00 con 0x7f330810b8a0 2024-05-09T04:50:53.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.869+0000 7f3307400700 1 --2- 172.21.15.6:0/149271149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3308106620 0x7f330819ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:50:53.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.869+0000 7f3307e00700 1 --2- 172.21.15.6:0/149271149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f330810b8a0 0x7f33081a1660 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f330000b770 tx=0x7f33000079a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:53.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.869+0000 7f32ffe00700 1 -- 172.21.15.6:0/149271149 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330001f070 con 0x7f330810b8a0 2024-05-09T04:50:53.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.869+0000 7f32ffe00700 1 -- 172.21.15.6:0/149271149 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3300004c50 con 0x7f330810b8a0 2024-05-09T04:50:53.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.869+0000 7f32ffe00700 1 -- 172.21.15.6:0/149271149 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330001b690 con 0x7f330810b8a0 2024-05-09T04:50:53.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.869+0000 7f330ca00700 1 -- 172.21.15.6:0/149271149 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33081a1f90 con 0x7f330810b8a0 2024-05-09T04:50:53.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.870+0000 7f330ca00700 1 -- 172.21.15.6:0/149271149 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33081a2470 con 0x7f330810b8a0 2024-05-09T04:50:53.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.871+0000 7f330ca00700 1 -- 172.21.15.6:0/149271149 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33080512c0 con 0x7f330810b8a0 2024-05-09T04:50:53.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.873+0000 7f32ffe00700 1 -- 172.21.15.6:0/149271149 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f3300022070 con 0x7f330810b8a0 2024-05-09T04:50:53.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.875+0000 7f32ffe00700 1 --2- 172.21.15.6:0/149271149 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f32ec0b2880 0x7f32ec0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:53.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.875+0000 7f32ffe00700 1 -- 172.21.15.6:0/149271149 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f330000f020 con 0x7f330810b8a0 2024-05-09T04:50:53.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.875+0000 7f3307400700 1 --2- 172.21.15.6:0/149271149 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f32ec0b2880 0x7f32ec0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:53.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.876+0000 7f3307400700 1 --2- 172.21.15.6:0/149271149 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f32ec0b2880 0x7f32ec0b4d90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f32f8005950 tx=0x7f32f800b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:53.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:53.877+0000 7f32ffe00700 1 -- 172.21.15.6:0/149271149 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3300029020 con 0x7f330810b8a0 2024-05-09T04:50:54.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:54.173+0000 7f330ca00700 1 -- 172.21.15.6:0/149271149 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f330802e5f0 con 0x7f330810b8a0 2024-05-09T04:50:54.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:54.665+0000 7f32ffe00700 1 -- 172.21.15.6:0/149271149 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f33000ab4d0 con 0x7f330810b8a0 2024-05-09T04:50:54.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:54.670+0000 7f330ca00700 1 -- 172.21.15.6:0/149271149 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f32ec0b2880 msgr2=0x7f32ec0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:54.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:54.670+0000 7f330ca00700 1 --2- 172.21.15.6:0/149271149 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f32ec0b2880 0x7f32ec0b4d90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f32f8005950 tx=0x7f32f800b500 comp rx=0 tx=0).stop 2024-05-09T04:50:54.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:54.670+0000 7f330ca00700 1 -- 172.21.15.6:0/149271149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f330810b8a0 msgr2=0x7f33081a1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:54.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:54.670+0000 7f330ca00700 1 --2- 172.21.15.6:0/149271149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f330810b8a0 0x7f33081a1660 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f330000b770 tx=0x7f33000079a0 comp rx=0 tx=0).stop 2024-05-09T04:50:54.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:54.670+0000 7f330ca00700 1 -- 172.21.15.6:0/149271149 shutdown_connections 2024-05-09T04:50:54.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:54.670+0000 7f330ca00700 1 --2- 172.21.15.6:0/149271149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f330810b8a0 0x7f33081a1660 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:54.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:54.670+0000 7f330ca00700 1 --2- 172.21.15.6:0/149271149 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f32ec0b2880 0x7f32ec0b4d90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:54.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:54.670+0000 7f330ca00700 1 --2- 172.21.15.6:0/149271149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3308106620 0x7f330819ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:54.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:54.670+0000 7f330ca00700 1 --2- 172.21.15.6:0/149271149 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f330810af50 0x7f330819d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:54.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:54.671+0000 7f330ca00700 1 -- 172.21.15.6:0/149271149 >> 172.21.15.6:0/149271149 conn(0x7f3308102030 msgr2=0x7f330810a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:54.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:54.671+0000 7f330ca00700 1 -- 172.21.15.6:0/149271149 shutdown_connections 2024-05-09T04:50:54.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:54.671+0000 7f330ca00700 1 -- 172.21.15.6:0/149271149 wait complete. 2024-05-09T04:50:54.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:54 smithi006 ceph-mon[29835]: pgmap v376: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:54.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:54 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/149271149' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-09T04:50:54.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:54 smithi006 ceph-mon[29835]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-09T04:50:55.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:54 smithi186 ceph-mon[34668]: pgmap v376: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:55.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:54 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/149271149' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-09T04:50:55.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:54 smithi186 ceph-mon[34668]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-09T04:50:55.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:54 smithi169 ceph-mon[34075]: pgmap v376: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:55.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:54 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/149271149' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-09T04:50:55.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:54 smithi169 ceph-mon[34075]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-09T04:50:55.581 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-05-09T04:50:55.581 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-05-09T04:50:55.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:55 smithi006 ceph-mon[29835]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-09T04:50:55.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:55 smithi006 ceph-mon[29835]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T04:50:56.061 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:50:56.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:55 smithi186 ceph-mon[34668]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-09T04:50:56.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:55 smithi186 ceph-mon[34668]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T04:50:56.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:55 smithi169 ceph-mon[34075]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-09T04:50:56.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:55 smithi169 ceph-mon[34075]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T04:50:56.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:56 smithi006 ceph-mon[29835]: pgmap v377: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:56.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-05-09T04:50:56.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:56 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-05-09T04:50:57.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:56 smithi186 ceph-mon[34668]: pgmap v377: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:57.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-05-09T04:50:57.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:56 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-05-09T04:50:57.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:56 smithi169 ceph-mon[34075]: pgmap v377: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:57.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:56 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-05-09T04:50:57.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:56 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-05-09T04:50:57.838 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:57 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-05-09T04:50:57.838 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:57 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-05-09T04:50:57.839 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:57 smithi006 ceph-mon[29835]: osdmap e88: 12 total, 12 up, 12 in 2024-05-09T04:50:58.101 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:57 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-05-09T04:50:58.102 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:57 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-05-09T04:50:58.102 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:57 smithi186 ceph-mon[34668]: osdmap e88: 12 total, 12 up, 12 in 2024-05-09T04:50:58.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:57 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-05-09T04:50:58.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:57 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-05-09T04:50:58.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:57 smithi169 ceph-mon[34075]: osdmap e88: 12 total, 12 up, 12 in 2024-05-09T04:50:58.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.485+0000 7f71c2800700 1 -- 172.21.15.6:0/2591767192 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f71bc106ed0 msgr2=0x7f71bc113360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:58.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.485+0000 7f71c2800700 1 --2- 172.21.15.6:0/2591767192 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f71bc106ed0 0x7f71bc113360 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f71b400b0a0 tx=0x7f71b400b3b0 comp rx=0 tx=0).stop 2024-05-09T04:50:58.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.485+0000 7f71c2800700 1 -- 172.21.15.6:0/2591767192 shutdown_connections 2024-05-09T04:50:58.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.485+0000 7f71c2800700 1 --2- 172.21.15.6:0/2591767192 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f71bc106ed0 0x7f71bc113360 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:58.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.485+0000 7f71c2800700 1 --2- 172.21.15.6:0/2591767192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71bc10c5c0 0x7f71bc10c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:58.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.485+0000 7f71c2800700 1 --2- 172.21.15.6:0/2591767192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71bc106580 0x7f71bc106990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:58.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.485+0000 7f71c2800700 1 -- 172.21.15.6:0/2591767192 >> 172.21.15.6:0/2591767192 conn(0x7f71bc102080 msgr2=0x7f71bc1044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:58.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.485+0000 7f71c2800700 1 -- 172.21.15.6:0/2591767192 shutdown_connections 2024-05-09T04:50:58.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.485+0000 7f71c2800700 1 -- 172.21.15.6:0/2591767192 wait complete. 2024-05-09T04:50:58.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.487+0000 7f71c2800700 1 Processor -- start 2024-05-09T04:50:58.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.487+0000 7f71c2800700 1 -- start start 2024-05-09T04:50:58.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.487+0000 7f71c2800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71bc106580 0x7f71bc19d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:58.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.487+0000 7f71c2800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f71bc106ed0 0x7f71bc19d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:58.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.488+0000 7f71c2800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71bc10c5c0 0x7f71bc1a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:58.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.488+0000 7f71c2800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc116740 con 0x7f71bc106ed0 2024-05-09T04:50:58.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.488+0000 7f71c2800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc1165c0 con 0x7f71bc106580 2024-05-09T04:50:58.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.488+0000 7f71c2800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc1168c0 con 0x7f71bc10c5c0 2024-05-09T04:50:58.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.488+0000 7f71bb400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71bc106580 0x7f71bc19d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:58.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.488+0000 7f71bb400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71bc106580 0x7f71bc19d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:35284/0 (socket says 172.21.15.6:35284) 2024-05-09T04:50:58.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.488+0000 7f71baa00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f71bc106ed0 0x7f71bc19d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:58.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.488+0000 7f71bb400700 1 -- 172.21.15.6:0/1261006765 learned_addr learned my addr 172.21.15.6:0/1261006765 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:50:58.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.488+0000 7f71bbe00700 1 --2- 172.21.15.6:0/1261006765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71bc10c5c0 0x7f71bc1a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:58.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.488+0000 7f71bb400700 1 -- 172.21.15.6:0/1261006765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71bc10c5c0 msgr2=0x7f71bc1a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:58.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.488+0000 7f71bb400700 1 --2- 172.21.15.6:0/1261006765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71bc10c5c0 0x7f71bc1a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:58.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.488+0000 7f71bb400700 1 -- 172.21.15.6:0/1261006765 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f71bc106ed0 msgr2=0x7f71bc19d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:58.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.488+0000 7f71bb400700 1 --2- 172.21.15.6:0/1261006765 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f71bc106ed0 0x7f71bc19d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:58.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.488+0000 7f71bb400700 1 -- 172.21.15.6:0/1261006765 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71bc1a1ee0 con 0x7f71bc106580 2024-05-09T04:50:58.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.489+0000 7f71bb400700 1 --2- 172.21.15.6:0/1261006765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71bc106580 0x7f71bc19d010 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00e970 tx=0x7f71ac00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:58.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.489+0000 7f71b3e00700 1 -- 172.21.15.6:0/1261006765 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac004a30 con 0x7f71bc106580 2024-05-09T04:50:58.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.489+0000 7f71c2800700 1 -- 172.21.15.6:0/1261006765 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71bc1a21d0 con 0x7f71bc106580 2024-05-09T04:50:58.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.489+0000 7f71b3e00700 1 -- 172.21.15.6:0/1261006765 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71ac00cc90 con 0x7f71bc106580 2024-05-09T04:50:58.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.489+0000 7f71b3e00700 1 -- 172.21.15.6:0/1261006765 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac005610 con 0x7f71bc106580 2024-05-09T04:50:58.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.489+0000 7f71c2800700 1 -- 172.21.15.6:0/1261006765 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71bc1a9af0 con 0x7f71bc106580 2024-05-09T04:50:58.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.490+0000 7f71c2800700 1 -- 172.21.15.6:0/1261006765 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71bc0512c0 con 0x7f71bc106580 2024-05-09T04:50:58.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.493+0000 7f71b3e00700 1 -- 172.21.15.6:0/1261006765 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f71ac00c7e0 con 0x7f71bc106580 2024-05-09T04:50:58.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.495+0000 7f71b3e00700 1 --2- 172.21.15.6:0/1261006765 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f71a00b2880 0x7f71a00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:58.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.495+0000 7f71b3e00700 1 -- 172.21.15.6:0/1261006765 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f71ac0e29d0 con 0x7f71bc106580 2024-05-09T04:50:58.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.495+0000 7f71baa00700 1 --2- 172.21.15.6:0/1261006765 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f71a00b2880 0x7f71a00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:58.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.496+0000 7f71baa00700 1 --2- 172.21.15.6:0/1261006765 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f71a00b2880 0x7f71a00b4d90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f71bc103c00 tx=0x7f71a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:58.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.497+0000 7f71b3e00700 1 -- 172.21.15.6:0/1261006765 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f71ac054df0 con 0x7f71bc106580 2024-05-09T04:50:58.791 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:58 smithi006 ceph-mon[29835]: pgmap v379: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:58.791 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:58 smithi006 ceph-mon[29835]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-05-09T04:50:58.791 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:58 smithi006 ceph-mon[29835]: osdmap e89: 12 total, 12 up, 12 in 2024-05-09T04:50:58.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:58.790+0000 7f71c2800700 1 -- 172.21.15.6:0/1261006765 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f71bc19e370 con 0x7f71bc106580 2024-05-09T04:50:59.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:58 smithi186 ceph-mon[34668]: pgmap v379: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:59.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:58 smithi186 ceph-mon[34668]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-05-09T04:50:59.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:58 smithi186 ceph-mon[34668]: osdmap e89: 12 total, 12 up, 12 in 2024-05-09T04:50:59.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:58 smithi169 ceph-mon[34075]: pgmap v379: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:50:59.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:58 smithi169 ceph-mon[34075]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-05-09T04:50:59.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:58 smithi169 ceph-mon[34075]: osdmap e89: 12 total, 12 up, 12 in 2024-05-09T04:50:59.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:59.716+0000 7f71b3e00700 1 -- 172.21.15.6:0/1261006765 <== mon.2 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f71ac0aa310 con 0x7f71bc106580 2024-05-09T04:50:59.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:59.720+0000 7f71c2800700 1 -- 172.21.15.6:0/1261006765 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f71a00b2880 msgr2=0x7f71a00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:59.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:59.720+0000 7f71c2800700 1 --2- 172.21.15.6:0/1261006765 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f71a00b2880 0x7f71a00b4d90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f71bc103c00 tx=0x7f71a8009260 comp rx=0 tx=0).stop 2024-05-09T04:50:59.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:59.720+0000 7f71c2800700 1 -- 172.21.15.6:0/1261006765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71bc106580 msgr2=0x7f71bc19d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:59.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:59.720+0000 7f71c2800700 1 --2- 172.21.15.6:0/1261006765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71bc106580 0x7f71bc19d010 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00e970 tx=0x7f71ac00ec80 comp rx=0 tx=0).stop 2024-05-09T04:50:59.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:59.720+0000 7f71c2800700 1 -- 172.21.15.6:0/1261006765 shutdown_connections 2024-05-09T04:50:59.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:59.720+0000 7f71c2800700 1 --2- 172.21.15.6:0/1261006765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71bc10c5c0 0x7f71bc1a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:59.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:59.720+0000 7f71c2800700 1 --2- 172.21.15.6:0/1261006765 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f71a00b2880 0x7f71a00b4d90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:59.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:59.720+0000 7f71c2800700 1 --2- 172.21.15.6:0/1261006765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71bc106580 0x7f71bc19d010 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:59.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:59.720+0000 7f71c2800700 1 --2- 172.21.15.6:0/1261006765 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f71bc106ed0 0x7f71bc19d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:59.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:59.720+0000 7f71c2800700 1 -- 172.21.15.6:0/1261006765 >> 172.21.15.6:0/1261006765 conn(0x7f71bc102080 msgr2=0x7f71bc104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:59.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:59.721+0000 7f71c2800700 1 -- 172.21.15.6:0/1261006765 shutdown_connections 2024-05-09T04:50:59.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:50:59.721+0000 7f71c2800700 1 -- 172.21.15.6:0/1261006765 wait complete. 2024-05-09T04:51:00.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:59 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/1261006765' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-09T04:51:00.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:50:59 smithi186 ceph-mon[34668]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-09T04:51:00.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:59 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/1261006765' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-09T04:51:00.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:50:59 smithi169 ceph-mon[34075]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-09T04:51:00.117 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:59 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1261006765' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-09T04:51:00.117 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:50:59 smithi006 ceph-mon[29835]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-09T04:51:00.589 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-09T04:51:00.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:00 smithi006 ceph-mon[29835]: pgmap v381: 129 pgs: 1 activating, 1 peering, 127 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:51:00.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:00 smithi006 ceph-mon[29835]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-09T04:51:00.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:00 smithi006 ceph-mon[29835]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-09T04:51:00.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:00 smithi006 ceph-mon[29835]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-09T04:51:00.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:00 smithi006 ceph-mon[29835]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T04:51:00.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:00 smithi006 ceph-mon[29835]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-05-09T04:51:00.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:00 smithi006 ceph-mon[29835]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-05-09T04:51:01.055 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:51:01.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:00 smithi186 ceph-mon[34668]: pgmap v381: 129 pgs: 1 activating, 1 peering, 127 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:51:01.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:00 smithi186 ceph-mon[34668]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-09T04:51:01.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:00 smithi186 ceph-mon[34668]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-09T04:51:01.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:00 smithi186 ceph-mon[34668]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-09T04:51:01.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:00 smithi186 ceph-mon[34668]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T04:51:01.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:00 smithi186 ceph-mon[34668]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-05-09T04:51:01.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:00 smithi186 ceph-mon[34668]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-05-09T04:51:01.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:00 smithi169 ceph-mon[34075]: pgmap v381: 129 pgs: 1 activating, 1 peering, 127 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:51:01.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:00 smithi169 ceph-mon[34075]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-09T04:51:01.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:00 smithi169 ceph-mon[34075]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-09T04:51:01.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:00 smithi169 ceph-mon[34075]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-09T04:51:01.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:00 smithi169 ceph-mon[34075]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T04:51:01.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:00 smithi169 ceph-mon[34075]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-05-09T04:51:01.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:00 smithi169 ceph-mon[34075]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-05-09T04:51:01.866 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:01 smithi169 ceph-mon[34075]: mds.? [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] up:active 2024-05-09T04:51:01.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:01 smithi169 ceph-mon[34075]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-09T04:51:01.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:01 smithi169 ceph-mon[34075]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-05-09T04:51:01.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:01 smithi169 ceph-mon[34075]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:creating} 9 up:standby 2024-05-09T04:51:01.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:01 smithi169 ceph-mon[34075]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-05-09T04:51:01.867 INFO:journalctl@ceph.mds.b.smithi169.stdout:May 09 04:51:01 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-b[64939]: 2024-05-09T04:51:01.602+0000 7f0e0841d700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-09T04:51:01.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:01 smithi006 ceph-mon[29835]: mds.? [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] up:active 2024-05-09T04:51:01.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:01 smithi006 ceph-mon[29835]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-09T04:51:01.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:01 smithi006 ceph-mon[29835]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-05-09T04:51:01.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:01 smithi006 ceph-mon[29835]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:creating} 9 up:standby 2024-05-09T04:51:01.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:01 smithi006 ceph-mon[29835]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-05-09T04:51:02.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:01 smithi186 ceph-mon[34668]: mds.? [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] up:active 2024-05-09T04:51:02.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:01 smithi186 ceph-mon[34668]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-09T04:51:02.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:01 smithi186 ceph-mon[34668]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-05-09T04:51:02.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:01 smithi186 ceph-mon[34668]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:creating} 9 up:standby 2024-05-09T04:51:02.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:01 smithi186 ceph-mon[34668]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-05-09T04:51:02.834 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:02 smithi006 ceph-mon[29835]: pgmap v382: 129 pgs: 1 activating, 1 peering, 127 active+clean; 583 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 3 op/s 2024-05-09T04:51:02.834 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:02 smithi006 ceph-mon[29835]: mds.? [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] up:active 2024-05-09T04:51:02.834 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:02 smithi006 ceph-mon[29835]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-09T04:51:02.834 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:02 smithi006 ceph-mon[29835]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 9 up:standby 2024-05-09T04:51:02.834 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:02 smithi006 ceph-mon[29835]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:creating} 8 up:standby 2024-05-09T04:51:02.835 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:02 smithi006 ceph-mon[29835]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-05-09T04:51:03.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:02 smithi186 ceph-mon[34668]: pgmap v382: 129 pgs: 1 activating, 1 peering, 127 active+clean; 583 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 3 op/s 2024-05-09T04:51:03.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:02 smithi186 ceph-mon[34668]: mds.? [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] up:active 2024-05-09T04:51:03.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:02 smithi186 ceph-mon[34668]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-09T04:51:03.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:02 smithi186 ceph-mon[34668]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 9 up:standby 2024-05-09T04:51:03.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:02 smithi186 ceph-mon[34668]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:creating} 8 up:standby 2024-05-09T04:51:03.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:02 smithi186 ceph-mon[34668]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-05-09T04:51:03.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:02 smithi169 ceph-mon[34075]: pgmap v382: 129 pgs: 1 activating, 1 peering, 127 active+clean; 583 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 3 op/s 2024-05-09T04:51:03.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:02 smithi169 ceph-mon[34075]: mds.? [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] up:active 2024-05-09T04:51:03.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:02 smithi169 ceph-mon[34075]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-09T04:51:03.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:02 smithi169 ceph-mon[34075]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 9 up:standby 2024-05-09T04:51:03.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:02 smithi169 ceph-mon[34075]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:creating} 8 up:standby 2024-05-09T04:51:03.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:02 smithi169 ceph-mon[34075]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-05-09T04:51:03.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.439+0000 7eff22600700 1 -- 172.21.15.6:0/2833942433 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff1c075d30 msgr2=0x7eff1c076140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:03.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.439+0000 7eff22600700 1 --2- 172.21.15.6:0/2833942433 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff1c075d30 0x7eff1c076140 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00b0a0 tx=0x7eff0c00b3b0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.439+0000 7eff22600700 1 -- 172.21.15.6:0/2833942433 shutdown_connections 2024-05-09T04:51:03.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.439+0000 7eff22600700 1 --2- 172.21.15.6:0/2833942433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff1c110ba0 0x7eff1c113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.439+0000 7eff22600700 1 --2- 172.21.15.6:0/2833942433 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff1c075d30 0x7eff1c076140 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.439+0000 7eff22600700 1 --2- 172.21.15.6:0/2833942433 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff1c076680 0x7eff1c110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.439+0000 7eff22600700 1 -- 172.21.15.6:0/2833942433 >> 172.21.15.6:0/2833942433 conn(0x7eff1c100040 msgr2=0x7eff1c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:03.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.440+0000 7eff22600700 1 -- 172.21.15.6:0/2833942433 shutdown_connections 2024-05-09T04:51:03.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.440+0000 7eff22600700 1 -- 172.21.15.6:0/2833942433 wait complete. 2024-05-09T04:51:03.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.441+0000 7eff22600700 1 Processor -- start 2024-05-09T04:51:03.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.442+0000 7eff22600700 1 -- start start 2024-05-09T04:51:03.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.442+0000 7eff22600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff1c075d30 0x7eff1c19d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:03.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.442+0000 7eff22600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff1c076680 0x7eff1c19d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:03.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.442+0000 7eff22600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff1c110ba0 0x7eff1c1a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:03.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.442+0000 7eff22600700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c116690 con 0x7eff1c076680 2024-05-09T04:51:03.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.442+0000 7eff22600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c116510 con 0x7eff1c110ba0 2024-05-09T04:51:03.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.442+0000 7eff22600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c116810 con 0x7eff1c075d30 2024-05-09T04:51:03.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.443+0000 7eff1be00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff1c110ba0 0x7eff1c1a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:03.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.443+0000 7eff1b400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff1c075d30 0x7eff1c19d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:03.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.443+0000 7eff1be00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff1c110ba0 0x7eff1c1a1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:35302/0 (socket says 172.21.15.6:35302) 2024-05-09T04:51:03.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.443+0000 7eff1be00700 1 -- 172.21.15.6:0/2366263922 learned_addr learned my addr 172.21.15.6:0/2366263922 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:51:03.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.443+0000 7eff12a00700 1 --2- 172.21.15.6:0/2366263922 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff1c076680 0x7eff1c19d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:03.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.443+0000 7eff1be00700 1 -- 172.21.15.6:0/2366263922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff1c075d30 msgr2=0x7eff1c19d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:03.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.443+0000 7eff1be00700 1 --2- 172.21.15.6:0/2366263922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff1c075d30 0x7eff1c19d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.443+0000 7eff1be00700 1 -- 172.21.15.6:0/2366263922 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff1c076680 msgr2=0x7eff1c19d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:03.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.443+0000 7eff1be00700 1 --2- 172.21.15.6:0/2366263922 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff1c076680 0x7eff1c19d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.443+0000 7eff1be00700 1 -- 172.21.15.6:0/2366263922 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff1c1a1ef0 con 0x7eff1c110ba0 2024-05-09T04:51:03.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.443+0000 7eff1be00700 1 --2- 172.21.15.6:0/2366263922 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff1c110ba0 0x7eff1c1a1850 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7eff1400e970 tx=0x7eff1400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:03.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.444+0000 7eff18c00700 1 -- 172.21.15.6:0/2366263922 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff14004a30 con 0x7eff1c110ba0 2024-05-09T04:51:03.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.444+0000 7eff18c00700 1 -- 172.21.15.6:0/2366263922 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff140054d0 con 0x7eff1c110ba0 2024-05-09T04:51:03.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.444+0000 7eff18c00700 1 -- 172.21.15.6:0/2366263922 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1400cd00 con 0x7eff1c110ba0 2024-05-09T04:51:03.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.444+0000 7eff22600700 1 -- 172.21.15.6:0/2366263922 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff1c1a21e0 con 0x7eff1c110ba0 2024-05-09T04:51:03.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.444+0000 7eff22600700 1 -- 172.21.15.6:0/2366263922 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff1c1a9b00 con 0x7eff1c110ba0 2024-05-09T04:51:03.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.445+0000 7eff22600700 1 -- 172.21.15.6:0/2366263922 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff1c0512c0 con 0x7eff1c110ba0 2024-05-09T04:51:03.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.447+0000 7eff18c00700 1 -- 172.21.15.6:0/2366263922 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7eff14004fe0 con 0x7eff1c110ba0 2024-05-09T04:51:03.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.449+0000 7eff18c00700 1 --2- 172.21.15.6:0/2366263922 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7eff000b2900 0x7eff000b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:03.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.449+0000 7eff18c00700 1 -- 172.21.15.6:0/2366263922 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7eff140e2640 con 0x7eff1c110ba0 2024-05-09T04:51:03.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.449+0000 7eff1b400700 1 --2- 172.21.15.6:0/2366263922 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7eff000b2900 0x7eff000b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:03.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.450+0000 7eff1b400700 1 --2- 172.21.15.6:0/2366263922 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7eff000b2900 0x7eff000b4e10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7eff0c009900 tx=0x7eff0c00b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:03.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.453+0000 7eff18c00700 1 -- 172.21.15.6:0/2366263922 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7eff140aa010 con 0x7eff1c110ba0 2024-05-09T04:51:03.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.747+0000 7eff22600700 1 -- 172.21.15.6:0/2366263922 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7eff1c02e5f0 con 0x7eff1c110ba0 2024-05-09T04:51:03.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.773+0000 7eff18c00700 1 -- 172.21.15.6:0/2366263922 <== mon.2 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v28) ==== 121+0+0 (secure 0 0 0) 0x7eff140a9910 con 0x7eff1c110ba0 2024-05-09T04:51:03.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.777+0000 7eff22600700 1 -- 172.21.15.6:0/2366263922 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7eff000b2900 msgr2=0x7eff000b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:03.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.777+0000 7eff22600700 1 --2- 172.21.15.6:0/2366263922 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7eff000b2900 0x7eff000b4e10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7eff0c009900 tx=0x7eff0c00b700 comp rx=0 tx=0).stop 2024-05-09T04:51:03.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.778+0000 7eff22600700 1 -- 172.21.15.6:0/2366263922 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff1c110ba0 msgr2=0x7eff1c1a1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:03.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.778+0000 7eff22600700 1 --2- 172.21.15.6:0/2366263922 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff1c110ba0 0x7eff1c1a1850 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7eff1400e970 tx=0x7eff1400ec80 comp rx=0 tx=0).stop 2024-05-09T04:51:03.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.778+0000 7eff22600700 1 -- 172.21.15.6:0/2366263922 shutdown_connections 2024-05-09T04:51:03.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.778+0000 7eff22600700 1 --2- 172.21.15.6:0/2366263922 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7eff000b2900 0x7eff000b4e10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.778+0000 7eff22600700 1 --2- 172.21.15.6:0/2366263922 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff1c110ba0 0x7eff1c1a1850 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.778+0000 7eff22600700 1 --2- 172.21.15.6:0/2366263922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff1c075d30 0x7eff1c19d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.778+0000 7eff22600700 1 --2- 172.21.15.6:0/2366263922 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff1c076680 0x7eff1c19d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.778+0000 7eff22600700 1 -- 172.21.15.6:0/2366263922 >> 172.21.15.6:0/2366263922 conn(0x7eff1c100040 msgr2=0x7eff1c100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:03.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.779+0000 7eff22600700 1 -- 172.21.15.6:0/2366263922 shutdown_connections 2024-05-09T04:51:03.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:03.779+0000 7eff22600700 1 -- 172.21.15.6:0/2366263922 wait complete. 2024-05-09T04:51:04.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:03 smithi186 ceph-mon[34668]: mds.? [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] up:active 2024-05-09T04:51:04.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:03 smithi186 ceph-mon[34668]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-09T04:51:04.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:03 smithi186 ceph-mon[34668]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-09T04:51:04.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:03 smithi186 ceph-mon[34668]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:active} 8 up:standby 2024-05-09T04:51:04.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:03 smithi186 ceph-mon[34668]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-09T04:51:04.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:03 smithi186 ceph-mon[34668]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-05-09T04:51:04.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:03 smithi169 ceph-mon[34075]: mds.? [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] up:active 2024-05-09T04:51:04.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:03 smithi169 ceph-mon[34075]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-09T04:51:04.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:03 smithi169 ceph-mon[34075]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-09T04:51:04.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:03 smithi169 ceph-mon[34075]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:active} 8 up:standby 2024-05-09T04:51:04.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:03 smithi169 ceph-mon[34075]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-09T04:51:04.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:03 smithi169 ceph-mon[34075]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-05-09T04:51:04.167 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:03 smithi006 ceph-mon[29835]: mds.? [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] up:active 2024-05-09T04:51:04.167 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:03 smithi006 ceph-mon[29835]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-09T04:51:04.167 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:03 smithi006 ceph-mon[29835]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-09T04:51:04.167 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:03 smithi006 ceph-mon[29835]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:active} 8 up:standby 2024-05-09T04:51:04.167 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:03 smithi006 ceph-mon[29835]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-09T04:51:04.167 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:03 smithi006 ceph-mon[29835]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-05-09T04:51:04.592 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-09T04:51:04.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:04 smithi006 ceph-mon[29835]: pgmap v383: 129 pgs: 129 active+clean; 585 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 11 op/s 2024-05-09T04:51:04.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:04 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/2366263922' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T04:51:04.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:04 smithi006 ceph-mon[29835]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-05-09T04:51:04.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:04 smithi006 ceph-mon[29835]: Cluster is now healthy 2024-05-09T04:51:04.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:04 smithi006 ceph-mon[29835]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T04:51:04.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:04 smithi006 ceph-mon[29835]: mds.? [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] up:active 2024-05-09T04:51:04.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:04 smithi006 ceph-mon[29835]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-09T04:51:04.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:04 smithi006 ceph-mon[29835]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-09T04:51:04.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:04 smithi006 ceph-mon[29835]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T04:51:05.096 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:51:05.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:04 smithi186 ceph-mon[34668]: pgmap v383: 129 pgs: 129 active+clean; 585 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 11 op/s 2024-05-09T04:51:05.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:04 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/2366263922' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T04:51:05.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:04 smithi186 ceph-mon[34668]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-05-09T04:51:05.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:04 smithi186 ceph-mon[34668]: Cluster is now healthy 2024-05-09T04:51:05.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:04 smithi186 ceph-mon[34668]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T04:51:05.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:04 smithi186 ceph-mon[34668]: mds.? [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] up:active 2024-05-09T04:51:05.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:04 smithi186 ceph-mon[34668]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-09T04:51:05.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:04 smithi186 ceph-mon[34668]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-09T04:51:05.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:04 smithi186 ceph-mon[34668]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T04:51:05.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:04 smithi169 ceph-mon[34075]: pgmap v383: 129 pgs: 129 active+clean; 585 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 11 op/s 2024-05-09T04:51:05.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:04 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/2366263922' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T04:51:05.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:04 smithi169 ceph-mon[34075]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-05-09T04:51:05.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:04 smithi169 ceph-mon[34075]: Cluster is now healthy 2024-05-09T04:51:05.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:04 smithi169 ceph-mon[34075]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T04:51:05.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:04 smithi169 ceph-mon[34075]: mds.? [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] up:active 2024-05-09T04:51:05.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:04 smithi169 ceph-mon[34075]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-09T04:51:05.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:04 smithi169 ceph-mon[34075]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-09T04:51:05.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:04 smithi169 ceph-mon[34075]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T04:51:06.867 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:06 smithi169 ceph-mon[34075]: pgmap v384: 129 pgs: 129 active+clean; 585 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 3.4 KiB/s wr, 14 op/s 2024-05-09T04:51:06.867 INFO:journalctl@ceph.mds.b.smithi169.stdout:May 09 04:51:06 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-b[64939]: 2024-05-09T04:51:06.602+0000 7f0e0841d700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-05-09T04:51:06.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:06 smithi006 ceph-mon[29835]: pgmap v384: 129 pgs: 129 active+clean; 585 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 3.4 KiB/s wr, 14 op/s 2024-05-09T04:51:07.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:06 smithi186 ceph-mon[34668]: pgmap v384: 129 pgs: 129 active+clean; 585 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 3.4 KiB/s wr, 14 op/s 2024-05-09T04:51:07.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.280+0000 7fc999c00700 1 -- 172.21.15.6:0/1451454927 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc994106ec0 msgr2=0x7fc994113350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:07.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.280+0000 7fc999c00700 1 --2- 172.21.15.6:0/1451454927 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc994106ec0 0x7fc994113350 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00b0a0 tx=0x7fc98c00b3b0 comp rx=0 tx=0).stop 2024-05-09T04:51:07.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.280+0000 7fc999c00700 1 -- 172.21.15.6:0/1451454927 shutdown_connections 2024-05-09T04:51:07.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.280+0000 7fc999c00700 1 --2- 172.21.15.6:0/1451454927 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc994106ec0 0x7fc994113350 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:07.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.280+0000 7fc999c00700 1 --2- 172.21.15.6:0/1451454927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc99410c5b0 0x7fc99410c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:07.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.280+0000 7fc999c00700 1 --2- 172.21.15.6:0/1451454927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc994106570 0x7fc994106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:07.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.280+0000 7fc999c00700 1 -- 172.21.15.6:0/1451454927 >> 172.21.15.6:0/1451454927 conn(0x7fc994102090 msgr2=0x7fc9941044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:07.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.281+0000 7fc999c00700 1 -- 172.21.15.6:0/1451454927 shutdown_connections 2024-05-09T04:51:07.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.281+0000 7fc999c00700 1 -- 172.21.15.6:0/1451454927 wait complete. 2024-05-09T04:51:07.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.282+0000 7fc999c00700 1 Processor -- start 2024-05-09T04:51:07.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.282+0000 7fc999c00700 1 -- start start 2024-05-09T04:51:07.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.282+0000 7fc999c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc994106570 0x7fc99419cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:07.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.282+0000 7fc999c00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc994106ec0 0x7fc99419d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:07.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.283+0000 7fc999c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc99410c5b0 0x7fc9941a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:07.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.283+0000 7fc999c00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc9941166e0 con 0x7fc99410c5b0 2024-05-09T04:51:07.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.283+0000 7fc999c00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc994116560 con 0x7fc994106ec0 2024-05-09T04:51:07.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.283+0000 7fc999c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc994116860 con 0x7fc994106570 2024-05-09T04:51:07.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.283+0000 7fc993400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc99410c5b0 0x7fc9941a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:07.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.283+0000 7fc993400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc99410c5b0 0x7fc9941a17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51282/0 (socket says 172.21.15.6:51282) 2024-05-09T04:51:07.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.283+0000 7fc993400700 1 -- 172.21.15.6:0/4272154203 learned_addr learned my addr 172.21.15.6:0/4272154203 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:51:07.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.283+0000 7fc992a00700 1 --2- 172.21.15.6:0/4272154203 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc994106570 0x7fc99419cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:07.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.283+0000 7fc98be00700 1 --2- 172.21.15.6:0/4272154203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc994106ec0 0x7fc99419d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:07.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.283+0000 7fc993400700 1 -- 172.21.15.6:0/4272154203 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc994106570 msgr2=0x7fc99419cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:07.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.283+0000 7fc993400700 1 --2- 172.21.15.6:0/4272154203 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc994106570 0x7fc99419cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:07.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.283+0000 7fc993400700 1 -- 172.21.15.6:0/4272154203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc994106ec0 msgr2=0x7fc99419d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:07.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.283+0000 7fc993400700 1 --2- 172.21.15.6:0/4272154203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc994106ec0 0x7fc99419d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:07.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.283+0000 7fc993400700 1 -- 172.21.15.6:0/4272154203 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9941a1e80 con 0x7fc99410c5b0 2024-05-09T04:51:07.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.284+0000 7fc993400700 1 --2- 172.21.15.6:0/4272154203 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc99410c5b0 0x7fc9941a17e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00b780 tx=0x7fc98c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:07.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.284+0000 7fc98b400700 1 -- 172.21.15.6:0/4272154203 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c021070 con 0x7fc99410c5b0 2024-05-09T04:51:07.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.284+0000 7fc999c00700 1 -- 172.21.15.6:0/4272154203 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9941a2110 con 0x7fc99410c5b0 2024-05-09T04:51:07.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.284+0000 7fc98b400700 1 -- 172.21.15.6:0/4272154203 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc98c007e30 con 0x7fc99410c5b0 2024-05-09T04:51:07.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.284+0000 7fc98b400700 1 -- 172.21.15.6:0/4272154203 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c004700 con 0x7fc99410c5b0 2024-05-09T04:51:07.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.284+0000 7fc999c00700 1 -- 172.21.15.6:0/4272154203 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9941a9a90 con 0x7fc99410c5b0 2024-05-09T04:51:07.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.286+0000 7fc999c00700 1 -- 172.21.15.6:0/4272154203 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc99410e970 con 0x7fc99410c5b0 2024-05-09T04:51:07.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.288+0000 7fc98b400700 1 -- 172.21.15.6:0/4272154203 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fc98c007980 con 0x7fc99410c5b0 2024-05-09T04:51:07.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.290+0000 7fc98b400700 1 --2- 172.21.15.6:0/4272154203 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fc9780b2880 0x7fc9780b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:07.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.290+0000 7fc98b400700 1 -- 172.21.15.6:0/4272154203 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fc98c00f020 con 0x7fc99410c5b0 2024-05-09T04:51:07.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.290+0000 7fc992a00700 1 --2- 172.21.15.6:0/4272154203 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fc9780b2880 0x7fc9780b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:07.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.291+0000 7fc992a00700 1 --2- 172.21.15.6:0/4272154203 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fc9780b2880 0x7fc9780b4d90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc9941079e0 tx=0x7fc984009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:07.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.295+0000 7fc98b400700 1 -- 172.21.15.6:0/4272154203 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc98c02a020 con 0x7fc99410c5b0 2024-05-09T04:51:07.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.587+0000 7fc999c00700 1 -- 172.21.15.6:0/4272154203 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fc9940512c0 con 0x7fc99410c5b0 2024-05-09T04:51:07.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.770+0000 7fc98b400700 1 -- 172.21.15.6:0/4272154203 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v29) ==== 115+0+0 (secure 0 0 0) 0x7fc98c0b10b0 con 0x7fc99410c5b0 2024-05-09T04:51:07.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.776+0000 7fc999c00700 1 -- 172.21.15.6:0/4272154203 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fc9780b2880 msgr2=0x7fc9780b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:07.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.776+0000 7fc999c00700 1 --2- 172.21.15.6:0/4272154203 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fc9780b2880 0x7fc9780b4d90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc9941079e0 tx=0x7fc984009260 comp rx=0 tx=0).stop 2024-05-09T04:51:07.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.776+0000 7fc999c00700 1 -- 172.21.15.6:0/4272154203 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc99410c5b0 msgr2=0x7fc9941a17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:07.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.776+0000 7fc999c00700 1 --2- 172.21.15.6:0/4272154203 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc99410c5b0 0x7fc9941a17e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00b780 tx=0x7fc98c004d00 comp rx=0 tx=0).stop 2024-05-09T04:51:07.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.777+0000 7fc999c00700 1 -- 172.21.15.6:0/4272154203 shutdown_connections 2024-05-09T04:51:07.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.777+0000 7fc999c00700 1 --2- 172.21.15.6:0/4272154203 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc99410c5b0 0x7fc9941a17e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:07.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.777+0000 7fc999c00700 1 --2- 172.21.15.6:0/4272154203 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc994106570 0x7fc99419cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:07.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.777+0000 7fc999c00700 1 --2- 172.21.15.6:0/4272154203 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fc9780b2880 0x7fc9780b4d90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:07.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.777+0000 7fc999c00700 1 --2- 172.21.15.6:0/4272154203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc994106ec0 0x7fc99419d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:07.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.777+0000 7fc999c00700 1 -- 172.21.15.6:0/4272154203 >> 172.21.15.6:0/4272154203 conn(0x7fc994102090 msgr2=0x7fc994104780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:07.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.777+0000 7fc999c00700 1 -- 172.21.15.6:0/4272154203 shutdown_connections 2024-05-09T04:51:07.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:07.777+0000 7fc999c00700 1 -- 172.21.15.6:0/4272154203 wait complete. 2024-05-09T04:51:07.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:07 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/4272154203' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-09T04:51:08.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:07 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/4272154203' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-09T04:51:08.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:07 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/4272154203' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-09T04:51:08.618 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-05-09T04:51:08.618 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-09T04:51:08.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:08 smithi006 ceph-mon[29835]: pgmap v385: 129 pgs: 129 active+clean; 587 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.8 KiB/s wr, 20 op/s 2024-05-09T04:51:08.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:08 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/4272154203' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-09T04:51:08.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:08 smithi006 ceph-mon[29835]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T04:51:09.087 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:51:09.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:08 smithi186 ceph-mon[34668]: pgmap v385: 129 pgs: 129 active+clean; 587 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.8 KiB/s wr, 20 op/s 2024-05-09T04:51:09.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:08 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/4272154203' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-09T04:51:09.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:08 smithi186 ceph-mon[34668]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T04:51:09.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:08 smithi169 ceph-mon[34075]: pgmap v385: 129 pgs: 129 active+clean; 587 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.8 KiB/s wr, 20 op/s 2024-05-09T04:51:09.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:08 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/4272154203' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-09T04:51:09.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:08 smithi169 ceph-mon[34075]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T04:51:11.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:10 smithi186 ceph-mon[34668]: pgmap v386: 129 pgs: 129 active+clean; 587 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-09T04:51:11.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:10 smithi169 ceph-mon[34075]: pgmap v386: 129 pgs: 129 active+clean; 587 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-09T04:51:11.122 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:10 smithi006 ceph-mon[29835]: pgmap v386: 129 pgs: 129 active+clean; 587 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-09T04:51:11.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.373+0000 7efeb3200700 1 -- 172.21.15.6:0/4190818983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efeac10e730 msgr2=0x7efeac110c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:11.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.373+0000 7efeb3200700 1 --2- 172.21.15.6:0/4190818983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efeac10e730 0x7efeac110c10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efea800b0a0 tx=0x7efea800b3b0 comp rx=0 tx=0).stop 2024-05-09T04:51:11.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.373+0000 7efeb3200700 1 -- 172.21.15.6:0/4190818983 shutdown_connections 2024-05-09T04:51:11.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.373+0000 7efeb3200700 1 --2- 172.21.15.6:0/4190818983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efeac10e730 0x7efeac110c10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:11.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.373+0000 7efeb3200700 1 --2- 172.21.15.6:0/4190818983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeac105350 0x7efeac105760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:11.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.373+0000 7efeb3200700 1 --2- 172.21.15.6:0/4190818983 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efeac105ca0 0x7efeac10e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:11.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.373+0000 7efeb3200700 1 -- 172.21.15.6:0/4190818983 >> 172.21.15.6:0/4190818983 conn(0x7efeac078ac0 msgr2=0x7efeac078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:11.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.373+0000 7efeb3200700 1 -- 172.21.15.6:0/4190818983 shutdown_connections 2024-05-09T04:51:11.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.373+0000 7efeb3200700 1 -- 172.21.15.6:0/4190818983 wait complete. 2024-05-09T04:51:11.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.375+0000 7efeb3200700 1 Processor -- start 2024-05-09T04:51:11.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.375+0000 7efeb3200700 1 -- start start 2024-05-09T04:51:11.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.375+0000 7efeb3200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efeac105350 0x7efeac1a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:11.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.375+0000 7efeb3200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efeac105ca0 0x7efeac1a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:11.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.375+0000 7efeb3200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeac10e730 0x7efeac19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:11.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.375+0000 7efeb3200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7efeac114260 con 0x7efeac105350 2024-05-09T04:51:11.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.375+0000 7efeb3200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efeac1140e0 con 0x7efeac10e730 2024-05-09T04:51:11.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.375+0000 7efeb3200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7efeac1143e0 con 0x7efeac105ca0 2024-05-09T04:51:11.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.376+0000 7efeb1e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efeac105350 0x7efeac1a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:11.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.376+0000 7efeb1400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efeac105ca0 0x7efeac1a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:11.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.376+0000 7efeb1e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efeac105350 0x7efeac1a07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51294/0 (socket says 172.21.15.6:51294) 2024-05-09T04:51:11.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.376+0000 7efeb1e00700 1 -- 172.21.15.6:0/3482228965 learned_addr learned my addr 172.21.15.6:0/3482228965 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:51:11.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.376+0000 7efeb2800700 1 --2- 172.21.15.6:0/3482228965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeac10e730 0x7efeac19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:11.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.376+0000 7efeb2800700 1 -- 172.21.15.6:0/3482228965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efeac105ca0 msgr2=0x7efeac1a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:11.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.376+0000 7efeb2800700 1 --2- 172.21.15.6:0/3482228965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efeac105ca0 0x7efeac1a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:11.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.376+0000 7efeb2800700 1 -- 172.21.15.6:0/3482228965 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efeac105350 msgr2=0x7efeac1a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:11.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.376+0000 7efeb2800700 1 --2- 172.21.15.6:0/3482228965 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efeac105350 0x7efeac1a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:11.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.376+0000 7efeb2800700 1 -- 172.21.15.6:0/3482228965 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeac19b070 con 0x7efeac10e730 2024-05-09T04:51:11.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.377+0000 7efeb2800700 1 --2- 172.21.15.6:0/3482228965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeac10e730 0x7efeac19a9c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efea800b070 tx=0x7efea8004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:11.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.377+0000 7efea6a00700 1 -- 172.21.15.6:0/3482228965 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea8021070 con 0x7efeac10e730 2024-05-09T04:51:11.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.377+0000 7efeb3200700 1 -- 172.21.15.6:0/3482228965 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeac19b300 con 0x7efeac10e730 2024-05-09T04:51:11.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.377+0000 7efea6a00700 1 -- 172.21.15.6:0/3482228965 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efea8007e30 con 0x7efeac10e730 2024-05-09T04:51:11.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.377+0000 7efea6a00700 1 -- 172.21.15.6:0/3482228965 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea80045d0 con 0x7efeac10e730 2024-05-09T04:51:11.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.377+0000 7efeb3200700 1 -- 172.21.15.6:0/3482228965 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeac1a7630 con 0x7efeac10e730 2024-05-09T04:51:11.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.378+0000 7efeb3200700 1 -- 172.21.15.6:0/3482228965 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeac06be20 con 0x7efeac10e730 2024-05-09T04:51:11.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.381+0000 7efea6a00700 1 -- 172.21.15.6:0/3482228965 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7efea8003c00 con 0x7efeac10e730 2024-05-09T04:51:11.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.382+0000 7efea6a00700 1 --2- 172.21.15.6:0/3482228965 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7efe940b2880 0x7efe940b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:11.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.382+0000 7efea6a00700 1 -- 172.21.15.6:0/3482228965 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7efea800f020 con 0x7efeac10e730 2024-05-09T04:51:11.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.383+0000 7efeb1e00700 1 --2- 172.21.15.6:0/3482228965 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7efe940b2880 0x7efe940b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:11.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.383+0000 7efeb1e00700 1 --2- 172.21.15.6:0/3482228965 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7efe940b2880 0x7efe940b4d90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efeac19bb00 tx=0x7efea0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:11.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.387+0000 7efea6a00700 1 -- 172.21.15.6:0/3482228965 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efea802a020 con 0x7efeac10e730 2024-05-09T04:51:11.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.658+0000 7efeb3200700 1 -- 172.21.15.6:0/3482228965 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7efeac19bef0 con 0x7efe940b2880 2024-05-09T04:51:11.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.703+0000 7efea6a00700 1 -- 172.21.15.6:0/3482228965 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efeac19bef0 con 0x7efe940b2880 2024-05-09T04:51:11.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.706+0000 7efeb3200700 1 -- 172.21.15.6:0/3482228965 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7efe940b2880 msgr2=0x7efe940b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:11.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.706+0000 7efeb3200700 1 --2- 172.21.15.6:0/3482228965 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7efe940b2880 0x7efe940b4d90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efeac19bb00 tx=0x7efea0009260 comp rx=0 tx=0).stop 2024-05-09T04:51:11.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.706+0000 7efeb3200700 1 -- 172.21.15.6:0/3482228965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeac10e730 msgr2=0x7efeac19a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:11.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.706+0000 7efeb3200700 1 --2- 172.21.15.6:0/3482228965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeac10e730 0x7efeac19a9c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efea800b070 tx=0x7efea8004d00 comp rx=0 tx=0).stop 2024-05-09T04:51:11.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.707+0000 7efeb3200700 1 -- 172.21.15.6:0/3482228965 shutdown_connections 2024-05-09T04:51:11.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.707+0000 7efeb3200700 1 --2- 172.21.15.6:0/3482228965 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7efe940b2880 0x7efe940b4d90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:11.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.707+0000 7efeb3200700 1 --2- 172.21.15.6:0/3482228965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeac10e730 0x7efeac19a9c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:11.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.707+0000 7efeb3200700 1 --2- 172.21.15.6:0/3482228965 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efeac105350 0x7efeac1a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:11.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.707+0000 7efeb3200700 1 --2- 172.21.15.6:0/3482228965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efeac105ca0 0x7efeac1a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:11.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.707+0000 7efeb3200700 1 -- 172.21.15.6:0/3482228965 >> 172.21.15.6:0/3482228965 conn(0x7efeac078ac0 msgr2=0x7efeac0ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:11.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.707+0000 7efeb3200700 1 -- 172.21.15.6:0/3482228965 shutdown_connections 2024-05-09T04:51:11.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:11.707+0000 7efeb3200700 1 -- 172.21.15.6:0/3482228965 wait complete. 2024-05-09T04:51:11.948 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:51:11 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:51:11.672+0000 7f9397400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T04:51:11.948 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:51:11 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:51:11.672+0000 7f9397400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T04:51:11.949 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:51:11 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:51:11.672+0000 7f9397400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T04:51:11.949 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:51:11 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:51:11.672+0000 7f9397400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T04:51:11.949 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:51:11 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:51:11.672+0000 7f9397400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T04:51:12.586 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-09T04:51:13.075 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:51:13.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:12 smithi186 ceph-mon[34668]: pgmap v387: 129 pgs: 129 active+clean; 587 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.0 KiB/s wr, 21 op/s 2024-05-09T04:51:13.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:12 smithi186 ceph-mon[34668]: from='client.24719 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:51:13.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:12 smithi186 ceph-mon[34668]: mgrmap e17: x(active, since 10m), standbys: z, y 2024-05-09T04:51:13.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:12 smithi169 ceph-mon[34075]: pgmap v387: 129 pgs: 129 active+clean; 587 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.0 KiB/s wr, 21 op/s 2024-05-09T04:51:13.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:12 smithi169 ceph-mon[34075]: from='client.24719 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:51:13.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:12 smithi169 ceph-mon[34075]: mgrmap e17: x(active, since 10m), standbys: z, y 2024-05-09T04:51:13.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:12 smithi006 ceph-mon[29835]: pgmap v387: 129 pgs: 129 active+clean; 587 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.0 KiB/s wr, 21 op/s 2024-05-09T04:51:13.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:12 smithi006 ceph-mon[29835]: from='client.24719 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:51:13.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:12 smithi006 ceph-mon[29835]: mgrmap e17: x(active, since 10m), standbys: z, y 2024-05-09T04:51:14.944 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:14 smithi006 ceph-mon[29835]: pgmap v388: 129 pgs: 129 active+clean; 587 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.0 KiB/s wr, 20 op/s 2024-05-09T04:51:15.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:14 smithi186 ceph-mon[34668]: pgmap v388: 129 pgs: 129 active+clean; 587 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.0 KiB/s wr, 20 op/s 2024-05-09T04:51:15.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:14 smithi169 ceph-mon[34075]: pgmap v388: 129 pgs: 129 active+clean; 587 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.0 KiB/s wr, 20 op/s 2024-05-09T04:51:15.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.311+0000 7fe5f6a00700 1 -- 172.21.15.6:0/56308554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5f0110ba0 msgr2=0x7fe5f0113000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:15.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.311+0000 7fe5f6a00700 1 --2- 172.21.15.6:0/56308554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5f0110ba0 0x7fe5f0113000 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800b0a0 tx=0x7fe5e800b3b0 comp rx=0 tx=0).stop 2024-05-09T04:51:15.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.311+0000 7fe5f6a00700 1 -- 172.21.15.6:0/56308554 shutdown_connections 2024-05-09T04:51:15.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.311+0000 7fe5f6a00700 1 --2- 172.21.15.6:0/56308554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5f0110ba0 0x7fe5f0113000 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:15.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.311+0000 7fe5f6a00700 1 --2- 172.21.15.6:0/56308554 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe5f0075d30 0x7fe5f0076140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:15.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.311+0000 7fe5f6a00700 1 --2- 172.21.15.6:0/56308554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5f0076680 0x7fe5f0110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:15.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.311+0000 7fe5f6a00700 1 -- 172.21.15.6:0/56308554 >> 172.21.15.6:0/56308554 conn(0x7fe5f0100040 msgr2=0x7fe5f0102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:15.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.311+0000 7fe5f6a00700 1 -- 172.21.15.6:0/56308554 shutdown_connections 2024-05-09T04:51:15.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.311+0000 7fe5f6a00700 1 -- 172.21.15.6:0/56308554 wait complete. 2024-05-09T04:51:15.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.313+0000 7fe5f6a00700 1 Processor -- start 2024-05-09T04:51:15.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.313+0000 7fe5f6a00700 1 -- start start 2024-05-09T04:51:15.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.314+0000 7fe5f6a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe5f0075d30 0x7fe5f019d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:15.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.314+0000 7fe5f6a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5f0076680 0x7fe5f019d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:15.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.314+0000 7fe5f6a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5f0110ba0 0x7fe5f01a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:15.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.314+0000 7fe5f6a00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f01166d0 con 0x7fe5f0075d30 2024-05-09T04:51:15.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.314+0000 7fe5f6a00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f0116550 con 0x7fe5f0110ba0 2024-05-09T04:51:15.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.314+0000 7fe5f6a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f0116850 con 0x7fe5f0076680 2024-05-09T04:51:15.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.314+0000 7fe5efe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe5f0075d30 0x7fe5f019d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:15.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.314+0000 7fe5ef400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5f0076680 0x7fe5f019d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:15.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.315+0000 7fe5f4a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5f0110ba0 0x7fe5f01a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:15.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.315+0000 7fe5efe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe5f0075d30 0x7fe5f019d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51326/0 (socket says 172.21.15.6:51326) 2024-05-09T04:51:15.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.315+0000 7fe5efe00700 1 -- 172.21.15.6:0/2629565200 learned_addr learned my addr 172.21.15.6:0/2629565200 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:51:15.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.315+0000 7fe5f4a00700 1 -- 172.21.15.6:0/2629565200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5f0076680 msgr2=0x7fe5f019d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:15.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.315+0000 7fe5f4a00700 1 --2- 172.21.15.6:0/2629565200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5f0076680 0x7fe5f019d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:15.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.315+0000 7fe5f4a00700 1 -- 172.21.15.6:0/2629565200 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe5f0075d30 msgr2=0x7fe5f019d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:15.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.315+0000 7fe5f4a00700 1 --2- 172.21.15.6:0/2629565200 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe5f0075d30 0x7fe5f019d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:15.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.315+0000 7fe5f4a00700 1 -- 172.21.15.6:0/2629565200 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5f01a1ef0 con 0x7fe5f0110ba0 2024-05-09T04:51:15.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.315+0000 7fe5ef400700 1 --2- 172.21.15.6:0/2629565200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5f0076680 0x7fe5f019d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:51:15.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.315+0000 7fe5f4a00700 1 --2- 172.21.15.6:0/2629565200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5f0110ba0 0x7fe5f01a1850 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800b070 tx=0x7fe5e8004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:15.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.316+0000 7fe5ecc00700 1 -- 172.21.15.6:0/2629565200 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8021070 con 0x7fe5f0110ba0 2024-05-09T04:51:15.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.316+0000 7fe5f6a00700 1 -- 172.21.15.6:0/2629565200 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5f01a2180 con 0x7fe5f0110ba0 2024-05-09T04:51:15.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.316+0000 7fe5ecc00700 1 -- 172.21.15.6:0/2629565200 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5e8007e30 con 0x7fe5f0110ba0 2024-05-09T04:51:15.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.316+0000 7fe5ecc00700 1 -- 172.21.15.6:0/2629565200 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e80045f0 con 0x7fe5f0110ba0 2024-05-09T04:51:15.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.316+0000 7fe5f6a00700 1 -- 172.21.15.6:0/2629565200 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5f01a9b00 con 0x7fe5f0110ba0 2024-05-09T04:51:15.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.317+0000 7fe5f6a00700 1 -- 172.21.15.6:0/2629565200 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5f00512c0 con 0x7fe5f0110ba0 2024-05-09T04:51:15.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.320+0000 7fe5ecc00700 1 -- 172.21.15.6:0/2629565200 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fe5e8007980 con 0x7fe5f0110ba0 2024-05-09T04:51:15.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.321+0000 7fe5ecc00700 1 --2- 172.21.15.6:0/2629565200 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe5d40b2910 0x7fe5d40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:15.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.321+0000 7fe5ecc00700 1 -- 172.21.15.6:0/2629565200 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fe5e800f020 con 0x7fe5f0110ba0 2024-05-09T04:51:15.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.321+0000 7fe5efe00700 1 --2- 172.21.15.6:0/2629565200 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe5d40b2910 0x7fe5d40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:15.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.322+0000 7fe5efe00700 1 --2- 172.21.15.6:0/2629565200 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe5d40b2910 0x7fe5d40b4e20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0009d20 tx=0x7fe5e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:15.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.323+0000 7fe5ecc00700 1 -- 172.21.15.6:0/2629565200 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe5e802a020 con 0x7fe5f0110ba0 2024-05-09T04:51:15.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.598+0000 7fe5f6a00700 1 -- 172.21.15.6:0/2629565200 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe5f019e380 con 0x7fe5d40b2910 2024-05-09T04:51:15.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.653+0000 7fe5ecc00700 1 -- 172.21.15.6:0/2629565200 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe5f019e380 con 0x7fe5d40b2910 2024-05-09T04:51:15.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.656+0000 7fe5f6a00700 1 -- 172.21.15.6:0/2629565200 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe5d40b2910 msgr2=0x7fe5d40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:15.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.656+0000 7fe5f6a00700 1 --2- 172.21.15.6:0/2629565200 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe5d40b2910 0x7fe5d40b4e20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0009d20 tx=0x7fe5e0009260 comp rx=0 tx=0).stop 2024-05-09T04:51:15.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.656+0000 7fe5f6a00700 1 -- 172.21.15.6:0/2629565200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5f0110ba0 msgr2=0x7fe5f01a1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:15.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.656+0000 7fe5f6a00700 1 --2- 172.21.15.6:0/2629565200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5f0110ba0 0x7fe5f01a1850 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800b070 tx=0x7fe5e8004c80 comp rx=0 tx=0).stop 2024-05-09T04:51:15.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.657+0000 7fe5f6a00700 1 -- 172.21.15.6:0/2629565200 shutdown_connections 2024-05-09T04:51:15.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.657+0000 7fe5f6a00700 1 --2- 172.21.15.6:0/2629565200 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe5d40b2910 0x7fe5d40b4e20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:15.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.657+0000 7fe5f6a00700 1 --2- 172.21.15.6:0/2629565200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5f0110ba0 0x7fe5f01a1850 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:15.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.657+0000 7fe5f6a00700 1 --2- 172.21.15.6:0/2629565200 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe5f0075d30 0x7fe5f019d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:15.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.657+0000 7fe5f6a00700 1 --2- 172.21.15.6:0/2629565200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5f0076680 0x7fe5f019d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:15.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.657+0000 7fe5f6a00700 1 -- 172.21.15.6:0/2629565200 >> 172.21.15.6:0/2629565200 conn(0x7fe5f0100040 msgr2=0x7fe5f0100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:15.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.657+0000 7fe5f6a00700 1 -- 172.21.15.6:0/2629565200 shutdown_connections 2024-05-09T04:51:15.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:15.657+0000 7fe5f6a00700 1 -- 172.21.15.6:0/2629565200 wait complete. 2024-05-09T04:51:16.566 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-09T04:51:17.050 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:51:17.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:16 smithi186 ceph-mon[34668]: pgmap v389: 129 pgs: 129 active+clean; 593 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.5 KiB/s wr, 17 op/s 2024-05-09T04:51:17.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:16 smithi186 ceph-mon[34668]: from='client.24731 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:51:17.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:16 smithi169 ceph-mon[34075]: pgmap v389: 129 pgs: 129 active+clean; 593 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.5 KiB/s wr, 17 op/s 2024-05-09T04:51:17.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:16 smithi169 ceph-mon[34075]: from='client.24731 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:51:17.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:16 smithi006 ceph-mon[29835]: pgmap v389: 129 pgs: 129 active+clean; 593 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.5 KiB/s wr, 17 op/s 2024-05-09T04:51:17.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:16 smithi006 ceph-mon[29835]: from='client.24731 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:51:18.901 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:18 smithi006 ceph-mon[29835]: pgmap v390: 129 pgs: 129 active+clean; 594 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.5 KiB/s wr, 15 op/s 2024-05-09T04:51:19.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:18 smithi186 ceph-mon[34668]: pgmap v390: 129 pgs: 129 active+clean; 594 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.5 KiB/s wr, 15 op/s 2024-05-09T04:51:19.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:18 smithi169 ceph-mon[34075]: pgmap v390: 129 pgs: 129 active+clean; 594 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.5 KiB/s wr, 15 op/s 2024-05-09T04:51:19.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.288+0000 7f9e3f200700 1 -- 172.21.15.6:0/4260367943 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e40104400 msgr2=0x7f9e40104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:19.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.288+0000 7f9e3f200700 1 --2- 172.21.15.6:0/4260367943 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e40104400 0x7f9e40104810 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9e300099c0 tx=0x7f9e30009cd0 comp rx=0 tx=0).stop 2024-05-09T04:51:19.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.288+0000 7f9e3f200700 1 -- 172.21.15.6:0/4260367943 shutdown_connections 2024-05-09T04:51:19.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.288+0000 7f9e3f200700 1 --2- 172.21.15.6:0/4260367943 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e40109680 0x7f9e40110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:19.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.288+0000 7f9e3f200700 1 --2- 172.21.15.6:0/4260367943 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e40104400 0x7f9e40104810 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:19.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.288+0000 7f9e3f200700 1 --2- 172.21.15.6:0/4260367943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e40108d30 0x7f9e40109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:19.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.288+0000 7f9e3f200700 1 -- 172.21.15.6:0/4260367943 >> 172.21.15.6:0/4260367943 conn(0x7f9e40079d10 msgr2=0x7f9e4007a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:19.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.288+0000 7f9e3f200700 1 -- 172.21.15.6:0/4260367943 shutdown_connections 2024-05-09T04:51:19.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.289+0000 7f9e3f200700 1 -- 172.21.15.6:0/4260367943 wait complete. 2024-05-09T04:51:19.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.290+0000 7f9e3f200700 1 Processor -- start 2024-05-09T04:51:19.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.291+0000 7f9e3f200700 1 -- start start 2024-05-09T04:51:19.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.291+0000 7f9e3f200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e40104400 0x7f9e4019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:19.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.291+0000 7f9e3f200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e40108d30 0x7f9e4019d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:19.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.291+0000 7f9e3f200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e40109680 0x7f9e401a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:19.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.291+0000 7f9e3f200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9e401142b0 con 0x7f9e40109680 2024-05-09T04:51:19.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.291+0000 7f9e3f200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9e40114130 con 0x7f9e40104400 2024-05-09T04:51:19.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.291+0000 7f9e3f200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9e40114430 con 0x7f9e40108d30 2024-05-09T04:51:19.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.291+0000 7f9e3e800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e40109680 0x7f9e401a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:19.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.291+0000 7f9e37e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e40108d30 0x7f9e4019d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:19.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.291+0000 7f9e3de00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e40104400 0x7f9e4019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:19.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.292+0000 7f9e37e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e40108d30 0x7f9e4019d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:57116/0 (socket says 172.21.15.6:57116) 2024-05-09T04:51:19.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.292+0000 7f9e37e00700 1 -- 172.21.15.6:0/322596712 learned_addr learned my addr 172.21.15.6:0/322596712 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:51:19.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.292+0000 7f9e37e00700 1 -- 172.21.15.6:0/322596712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e40104400 msgr2=0x7f9e4019ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:19.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.292+0000 7f9e37e00700 1 --2- 172.21.15.6:0/322596712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e40104400 0x7f9e4019ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:19.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.292+0000 7f9e37e00700 1 -- 172.21.15.6:0/322596712 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e40109680 msgr2=0x7f9e401a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:19.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.292+0000 7f9e37e00700 1 --2- 172.21.15.6:0/322596712 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e40109680 0x7f9e401a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:19.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.292+0000 7f9e37e00700 1 -- 172.21.15.6:0/322596712 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e401a1cf0 con 0x7f9e40108d30 2024-05-09T04:51:19.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.292+0000 7f9e3de00700 1 --2- 172.21.15.6:0/322596712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e40104400 0x7f9e4019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:51:19.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.292+0000 7f9e37e00700 1 --2- 172.21.15.6:0/322596712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e40108d30 0x7f9e4019d3c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9e2800e970 tx=0x7f9e2800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:19.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.293+0000 7f9e36a00700 1 -- 172.21.15.6:0/322596712 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e28004a30 con 0x7f9e40108d30 2024-05-09T04:51:19.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.293+0000 7f9e36a00700 1 -- 172.21.15.6:0/322596712 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e2800cc90 con 0x7f9e40108d30 2024-05-09T04:51:19.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.293+0000 7f9e3f200700 1 -- 172.21.15.6:0/322596712 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e401a1fe0 con 0x7f9e40108d30 2024-05-09T04:51:19.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.293+0000 7f9e36a00700 1 -- 172.21.15.6:0/322596712 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e28005610 con 0x7f9e40108d30 2024-05-09T04:51:19.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.293+0000 7f9e3f200700 1 -- 172.21.15.6:0/322596712 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e401a9900 con 0x7f9e40108d30 2024-05-09T04:51:19.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.294+0000 7f9e3f200700 1 -- 172.21.15.6:0/322596712 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e400512c0 con 0x7f9e40108d30 2024-05-09T04:51:19.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.297+0000 7f9e36a00700 1 -- 172.21.15.6:0/322596712 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f9e2800c7e0 con 0x7f9e40108d30 2024-05-09T04:51:19.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.298+0000 7f9e36a00700 1 --2- 172.21.15.6:0/322596712 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9e240b2910 0x7f9e240b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:19.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.298+0000 7f9e36a00700 1 -- 172.21.15.6:0/322596712 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f9e280e2b90 con 0x7f9e40108d30 2024-05-09T04:51:19.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.298+0000 7f9e3de00700 1 --2- 172.21.15.6:0/322596712 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9e240b2910 0x7f9e240b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:19.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.299+0000 7f9e3de00700 1 --2- 172.21.15.6:0/322596712 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9e240b2910 0x7f9e240b4e20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9e30000c00 tx=0x7f9e3001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:19.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.300+0000 7f9e36a00700 1 -- 172.21.15.6:0/322596712 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9e28054df0 con 0x7f9e40108d30 2024-05-09T04:51:19.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.574+0000 7f9e3f200700 1 -- 172.21.15.6:0/322596712 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f9e401023f0 con 0x7f9e240b2910 2024-05-09T04:51:19.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.589+0000 7f9e36a00700 1 -- 172.21.15.6:0/322596712 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f9e401023f0 con 0x7f9e240b2910 2024-05-09T04:51:19.591 INFO:teuthology.orchestra.run.smithi006.stdout:/volumes/qa/sv_0/e0d00f0f-a37a-403a-b0ee-5d1f9374ad94 2024-05-09T04:51:19.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.592+0000 7f9e3f200700 1 -- 172.21.15.6:0/322596712 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9e240b2910 msgr2=0x7f9e240b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:19.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.592+0000 7f9e3f200700 1 --2- 172.21.15.6:0/322596712 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9e240b2910 0x7f9e240b4e20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9e30000c00 tx=0x7f9e3001a040 comp rx=0 tx=0).stop 2024-05-09T04:51:19.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.592+0000 7f9e3f200700 1 -- 172.21.15.6:0/322596712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e40108d30 msgr2=0x7f9e4019d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:19.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.592+0000 7f9e3f200700 1 --2- 172.21.15.6:0/322596712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e40108d30 0x7f9e4019d3c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9e2800e970 tx=0x7f9e2800ec80 comp rx=0 tx=0).stop 2024-05-09T04:51:19.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.593+0000 7f9e3f200700 1 -- 172.21.15.6:0/322596712 shutdown_connections 2024-05-09T04:51:19.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.593+0000 7f9e3f200700 1 --2- 172.21.15.6:0/322596712 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e40109680 0x7f9e401a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:19.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.593+0000 7f9e3f200700 1 --2- 172.21.15.6:0/322596712 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9e240b2910 0x7f9e240b4e20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:19.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.593+0000 7f9e3f200700 1 --2- 172.21.15.6:0/322596712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e40104400 0x7f9e4019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:19.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.593+0000 7f9e3f200700 1 --2- 172.21.15.6:0/322596712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e40108d30 0x7f9e4019d3c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:19.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.593+0000 7f9e3f200700 1 -- 172.21.15.6:0/322596712 >> 172.21.15.6:0/322596712 conn(0x7f9e40079d10 msgr2=0x7f9e40110d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:19.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.593+0000 7f9e3f200700 1 -- 172.21.15.6:0/322596712 shutdown_connections 2024-05-09T04:51:19.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:19.593+0000 7f9e3f200700 1 -- 172.21.15.6:0/322596712 wait complete. 2024-05-09T04:51:20.595 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-09T04:51:20.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:20 smithi006 ceph-mon[29835]: from='client.24721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:51:20.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:20 smithi006 ceph-mon[29835]: pgmap v391: 129 pgs: 129 active+clean; 594 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-09T04:51:21.042 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:51:21.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:20 smithi186 ceph-mon[34668]: from='client.24721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:51:21.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:20 smithi186 ceph-mon[34668]: pgmap v391: 129 pgs: 129 active+clean; 594 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-09T04:51:21.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:20 smithi169 ceph-mon[34075]: from='client.24721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:51:21.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:20 smithi169 ceph-mon[34075]: pgmap v391: 129 pgs: 129 active+clean; 594 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-09T04:51:22.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:22 smithi006 ceph-mon[29835]: pgmap v392: 129 pgs: 129 active+clean; 612 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-05-09T04:51:23.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:22 smithi186 ceph-mon[34668]: pgmap v392: 129 pgs: 129 active+clean; 612 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-05-09T04:51:23.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:22 smithi169 ceph-mon[34075]: pgmap v392: 129 pgs: 129 active+clean; 612 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-05-09T04:51:23.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.564+0000 7f26e7200700 1 -- 172.21.15.6:0/2452534517 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e8075d30 msgr2=0x7f26e8076140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:23.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.564+0000 7f26e7200700 1 --2- 172.21.15.6:0/2452534517 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e8075d30 0x7f26e8076140 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f26d8009990 tx=0x7f26d8009ca0 comp rx=0 tx=0).stop 2024-05-09T04:51:23.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.564+0000 7f26e7200700 1 -- 172.21.15.6:0/2452534517 shutdown_connections 2024-05-09T04:51:23.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.564+0000 7f26e7200700 1 --2- 172.21.15.6:0/2452534517 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26e81109b0 0x7f26e8112e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:23.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.564+0000 7f26e7200700 1 --2- 172.21.15.6:0/2452534517 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e8075d30 0x7f26e8076140 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:23.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.564+0000 7f26e7200700 1 --2- 172.21.15.6:0/2452534517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26e8076680 0x7f26e8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:23.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.564+0000 7f26e7200700 1 -- 172.21.15.6:0/2452534517 >> 172.21.15.6:0/2452534517 conn(0x7f26e8079d10 msgr2=0x7f26e807a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:23.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.564+0000 7f26e7200700 1 -- 172.21.15.6:0/2452534517 shutdown_connections 2024-05-09T04:51:23.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.565+0000 7f26e7200700 1 -- 172.21.15.6:0/2452534517 wait complete. 2024-05-09T04:51:23.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.566+0000 7f26e7200700 1 Processor -- start 2024-05-09T04:51:23.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.566+0000 7f26e7200700 1 -- start start 2024-05-09T04:51:23.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.566+0000 7f26e7200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26e8075d30 0x7f26e819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:23.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.566+0000 7f26e7200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26e8076680 0x7f26e819d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:23.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.567+0000 7f26e7200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e81109b0 0x7f26e81a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:23.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.567+0000 7f26e7200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f26e8116540 con 0x7f26e81109b0 2024-05-09T04:51:23.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.567+0000 7f26e7200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f26e81163c0 con 0x7f26e8076680 2024-05-09T04:51:23.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.567+0000 7f26e7200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f26e81166c0 con 0x7f26e8075d30 2024-05-09T04:51:23.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.567+0000 7f26e5e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26e8075d30 0x7f26e819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:23.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.567+0000 7f26e5e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26e8075d30 0x7f26e819cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:57144/0 (socket says 172.21.15.6:57144) 2024-05-09T04:51:23.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.567+0000 7f26e5e00700 1 -- 172.21.15.6:0/3389041371 learned_addr learned my addr 172.21.15.6:0/3389041371 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:51:23.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.567+0000 7f26dfe00700 1 --2- 172.21.15.6:0/3389041371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26e8076680 0x7f26e819d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:23.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.567+0000 7f26e6800700 1 --2- 172.21.15.6:0/3389041371 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e81109b0 0x7f26e81a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:23.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.567+0000 7f26e5e00700 1 -- 172.21.15.6:0/3389041371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26e8076680 msgr2=0x7f26e819d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:23.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.567+0000 7f26e5e00700 1 --2- 172.21.15.6:0/3389041371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26e8076680 0x7f26e819d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:23.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.567+0000 7f26e5e00700 1 -- 172.21.15.6:0/3389041371 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e81109b0 msgr2=0x7f26e81a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:23.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.567+0000 7f26e5e00700 1 --2- 172.21.15.6:0/3389041371 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e81109b0 0x7f26e81a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:23.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.567+0000 7f26e5e00700 1 -- 172.21.15.6:0/3389041371 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e81a1de0 con 0x7f26e8075d30 2024-05-09T04:51:23.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.568+0000 7f26e5e00700 1 --2- 172.21.15.6:0/3389041371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26e8075d30 0x7f26e819cf10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f26d8000c00 tx=0x7f26d80057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:23.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.568+0000 7f26dea00700 1 -- 172.21.15.6:0/3389041371 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d8027070 con 0x7f26e8075d30 2024-05-09T04:51:23.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.568+0000 7f26e7200700 1 -- 172.21.15.6:0/3389041371 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e81a2070 con 0x7f26e8075d30 2024-05-09T04:51:23.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.568+0000 7f26dea00700 1 -- 172.21.15.6:0/3389041371 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26d8022470 con 0x7f26e8075d30 2024-05-09T04:51:23.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.568+0000 7f26dea00700 1 -- 172.21.15.6:0/3389041371 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d8017650 con 0x7f26e8075d30 2024-05-09T04:51:23.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.568+0000 7f26e7200700 1 -- 172.21.15.6:0/3389041371 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e81a99f0 con 0x7f26e8075d30 2024-05-09T04:51:23.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.569+0000 7f26e7200700 1 -- 172.21.15.6:0/3389041371 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26e80772a0 con 0x7f26e8075d30 2024-05-09T04:51:23.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.572+0000 7f26dea00700 1 -- 172.21.15.6:0/3389041371 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f26d802a070 con 0x7f26e8075d30 2024-05-09T04:51:23.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.574+0000 7f26dea00700 1 --2- 172.21.15.6:0/3389041371 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f26cc0b2910 0x7f26cc0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:23.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.574+0000 7f26dea00700 1 -- 172.21.15.6:0/3389041371 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f26d80e36c0 con 0x7f26e8075d30 2024-05-09T04:51:23.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.574+0000 7f26dfe00700 1 --2- 172.21.15.6:0/3389041371 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f26cc0b2910 0x7f26cc0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:23.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.575+0000 7f26dfe00700 1 --2- 172.21.15.6:0/3389041371 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f26cc0b2910 0x7f26cc0b4e20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f26e807a7a0 tx=0x7f26d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:23.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.576+0000 7f26dea00700 1 -- 172.21.15.6:0/3389041371 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f26d8013070 con 0x7f26e8075d30 2024-05-09T04:51:23.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.857+0000 7f26e7200700 1 -- 172.21.15.6:0/3389041371 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f26e8063c00 con 0x7f26cc0b2910 2024-05-09T04:51:23.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.905+0000 7f26dea00700 1 -- 172.21.15.6:0/3389041371 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f26e8063c00 con 0x7f26cc0b2910 2024-05-09T04:51:23.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.909+0000 7f26e7200700 1 -- 172.21.15.6:0/3389041371 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f26cc0b2910 msgr2=0x7f26cc0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:23.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.909+0000 7f26e7200700 1 --2- 172.21.15.6:0/3389041371 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f26cc0b2910 0x7f26cc0b4e20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f26e807a7a0 tx=0x7f26d0009260 comp rx=0 tx=0).stop 2024-05-09T04:51:23.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.909+0000 7f26e7200700 1 -- 172.21.15.6:0/3389041371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26e8075d30 msgr2=0x7f26e819cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:23.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.909+0000 7f26e7200700 1 --2- 172.21.15.6:0/3389041371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26e8075d30 0x7f26e819cf10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f26d8000c00 tx=0x7f26d80057c0 comp rx=0 tx=0).stop 2024-05-09T04:51:23.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.909+0000 7f26e7200700 1 -- 172.21.15.6:0/3389041371 shutdown_connections 2024-05-09T04:51:23.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.909+0000 7f26e7200700 1 --2- 172.21.15.6:0/3389041371 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e81109b0 0x7f26e81a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:23.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.909+0000 7f26e7200700 1 --2- 172.21.15.6:0/3389041371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26e8075d30 0x7f26e819cf10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:23.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.909+0000 7f26e7200700 1 --2- 172.21.15.6:0/3389041371 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f26cc0b2910 0x7f26cc0b4e20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:23.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.909+0000 7f26e7200700 1 --2- 172.21.15.6:0/3389041371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26e8076680 0x7f26e819d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:23.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.909+0000 7f26e7200700 1 -- 172.21.15.6:0/3389041371 >> 172.21.15.6:0/3389041371 conn(0x7f26e8079d10 msgr2=0x7f26e8100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:23.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.909+0000 7f26e7200700 1 -- 172.21.15.6:0/3389041371 shutdown_connections 2024-05-09T04:51:23.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:23.909+0000 7f26e7200700 1 -- 172.21.15.6:0/3389041371 wait complete. 2024-05-09T04:51:24.761 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-09T04:51:24.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:24 smithi006 ceph-mon[29835]: pgmap v393: 129 pgs: 129 active+clean; 612 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-05-09T04:51:24.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:24 smithi006 ceph-mon[29835]: from='client.24727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:51:25.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:24 smithi186 ceph-mon[34668]: pgmap v393: 129 pgs: 129 active+clean; 612 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-05-09T04:51:25.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:24 smithi186 ceph-mon[34668]: from='client.24727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:51:25.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:24 smithi169 ceph-mon[34075]: pgmap v393: 129 pgs: 129 active+clean; 612 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-05-09T04:51:25.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:24 smithi169 ceph-mon[34075]: from='client.24727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:51:25.240 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:51:26.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:26 smithi006 ceph-mon[29835]: pgmap v394: 129 pgs: 129 active+clean; 630 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2024-05-09T04:51:27.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:26 smithi186 ceph-mon[34668]: pgmap v394: 129 pgs: 129 active+clean; 630 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2024-05-09T04:51:27.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:26 smithi169 ceph-mon[34075]: pgmap v394: 129 pgs: 129 active+clean; 630 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2024-05-09T04:51:27.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.608+0000 7f016a400700 1 -- 172.21.15.6:0/3359481945 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0164110970 msgr2=0x7f0164110d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:27.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.608+0000 7f016a400700 1 --2- 172.21.15.6:0/3359481945 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0164110970 0x7f0164110d80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0154009990 tx=0x7f0154009ca0 comp rx=0 tx=0).stop 2024-05-09T04:51:27.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.608+0000 7f016a400700 1 -- 172.21.15.6:0/3359481945 shutdown_connections 2024-05-09T04:51:27.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.608+0000 7f016a400700 1 --2- 172.21.15.6:0/3359481945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f016410b2e0 0x7f0164117810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:27.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.608+0000 7f016a400700 1 --2- 172.21.15.6:0/3359481945 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0164110970 0x7f0164110d80 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:27.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.608+0000 7f016a400700 1 --2- 172.21.15.6:0/3359481945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f016410a990 0x7f016410ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:27.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.608+0000 7f016a400700 1 -- 172.21.15.6:0/3359481945 >> 172.21.15.6:0/3359481945 conn(0x7f0164077c80 msgr2=0x7f01640780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:27.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.609+0000 7f016a400700 1 -- 172.21.15.6:0/3359481945 shutdown_connections 2024-05-09T04:51:27.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.609+0000 7f016a400700 1 -- 172.21.15.6:0/3359481945 wait complete. 2024-05-09T04:51:27.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.610+0000 7f016a400700 1 Processor -- start 2024-05-09T04:51:27.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.611+0000 7f016a400700 1 -- start start 2024-05-09T04:51:27.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.611+0000 7f016a400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f016410a990 0x7f01641a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:27.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.611+0000 7f016a400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f016410b2e0 0x7f01641a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:27.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.611+0000 7f016a400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0164110970 0x7f01641a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:27.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.611+0000 7f016a400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f016411aac0 con 0x7f0164110970 2024-05-09T04:51:27.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.611+0000 7f016a400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f016411a940 con 0x7f016410b2e0 2024-05-09T04:51:27.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.611+0000 7f016a400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f016411ac40 con 0x7f016410a990 2024-05-09T04:51:27.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.612+0000 7f0163400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f016410a990 0x7f01641a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:27.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.612+0000 7f0163e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0164110970 0x7f01641a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:27.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.612+0000 7f0163400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f016410a990 0x7f01641a1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:53072/0 (socket says 172.21.15.6:53072) 2024-05-09T04:51:27.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.612+0000 7f0163e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0164110970 0x7f01641a5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:47008/0 (socket says 172.21.15.6:47008) 2024-05-09T04:51:27.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.612+0000 7f0163400700 1 -- 172.21.15.6:0/270673274 learned_addr learned my addr 172.21.15.6:0/270673274 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:51:27.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.612+0000 7f0162a00700 1 --2- 172.21.15.6:0/270673274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f016410b2e0 0x7f01641a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:27.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.612+0000 7f0163400700 1 -- 172.21.15.6:0/270673274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f016410b2e0 msgr2=0x7f01641a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:27.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.612+0000 7f0163400700 1 --2- 172.21.15.6:0/270673274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f016410b2e0 0x7f01641a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:27.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.612+0000 7f0163400700 1 -- 172.21.15.6:0/270673274 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0164110970 msgr2=0x7f01641a5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:27.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.612+0000 7f0163400700 1 --2- 172.21.15.6:0/270673274 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0164110970 0x7f01641a5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:27.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.612+0000 7f0163400700 1 -- 172.21.15.6:0/270673274 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01641a6400 con 0x7f016410a990 2024-05-09T04:51:27.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.613+0000 7f0163400700 1 --2- 172.21.15.6:0/270673274 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f016410a990 0x7f01641a1530 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0154000c00 tx=0x7f01540047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:27.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.613+0000 7f015be00700 1 -- 172.21.15.6:0/270673274 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0154026070 con 0x7f016410a990 2024-05-09T04:51:27.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.613+0000 7f016a400700 1 -- 172.21.15.6:0/270673274 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01641a6690 con 0x7f016410a990 2024-05-09T04:51:27.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.613+0000 7f015be00700 1 -- 172.21.15.6:0/270673274 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0154021410 con 0x7f016410a990 2024-05-09T04:51:27.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.613+0000 7f015be00700 1 -- 172.21.15.6:0/270673274 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0154017650 con 0x7f016410a990 2024-05-09T04:51:27.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.613+0000 7f016a400700 1 -- 172.21.15.6:0/270673274 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01641ae010 con 0x7f016410a990 2024-05-09T04:51:27.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.614+0000 7f016a400700 1 -- 172.21.15.6:0/270673274 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f016410c080 con 0x7f016410a990 2024-05-09T04:51:27.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.617+0000 7f015be00700 1 -- 172.21.15.6:0/270673274 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f0154029070 con 0x7f016410a990 2024-05-09T04:51:27.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.618+0000 7f015be00700 1 --2- 172.21.15.6:0/270673274 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f01480b2910 0x7f01480b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:27.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.619+0000 7f015be00700 1 -- 172.21.15.6:0/270673274 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f01540e37a0 con 0x7f016410a990 2024-05-09T04:51:27.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.619+0000 7f0162a00700 1 --2- 172.21.15.6:0/270673274 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f01480b2910 0x7f01480b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:27.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.620+0000 7f0162a00700 1 --2- 172.21.15.6:0/270673274 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f01480b2910 0x7f01480b4e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f016407b0d0 tx=0x7f0150009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:27.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.621+0000 7f015be00700 1 -- 172.21.15.6:0/270673274 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0154013070 con 0x7f016410a990 2024-05-09T04:51:27.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.906+0000 7f016a400700 1 -- 172.21.15.6:0/270673274 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f01641a2890 con 0x7f01480b2910 2024-05-09T04:51:27.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.919+0000 7f015be00700 1 -- 172.21.15.6:0/270673274 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f01641a2890 con 0x7f01480b2910 2024-05-09T04:51:27.920 INFO:teuthology.orchestra.run.smithi006.stdout:/volumes/qa/sv_1/f303924b-50ca-4c64-949c-577236452acd 2024-05-09T04:51:27.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.922+0000 7f016a400700 1 -- 172.21.15.6:0/270673274 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f01480b2910 msgr2=0x7f01480b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:27.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.922+0000 7f016a400700 1 --2- 172.21.15.6:0/270673274 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f01480b2910 0x7f01480b4e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f016407b0d0 tx=0x7f0150009260 comp rx=0 tx=0).stop 2024-05-09T04:51:27.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.922+0000 7f016a400700 1 -- 172.21.15.6:0/270673274 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f016410a990 msgr2=0x7f01641a1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:27.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.922+0000 7f016a400700 1 --2- 172.21.15.6:0/270673274 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f016410a990 0x7f01641a1530 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0154000c00 tx=0x7f01540047b0 comp rx=0 tx=0).stop 2024-05-09T04:51:27.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.922+0000 7f016a400700 1 -- 172.21.15.6:0/270673274 shutdown_connections 2024-05-09T04:51:27.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.922+0000 7f016a400700 1 --2- 172.21.15.6:0/270673274 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0164110970 0x7f01641a5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:27.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.922+0000 7f016a400700 1 --2- 172.21.15.6:0/270673274 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f016410a990 0x7f01641a1530 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:27.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.922+0000 7f016a400700 1 --2- 172.21.15.6:0/270673274 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f01480b2910 0x7f01480b4e20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:27.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.922+0000 7f016a400700 1 --2- 172.21.15.6:0/270673274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f016410b2e0 0x7f01641a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:27.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.922+0000 7f016a400700 1 -- 172.21.15.6:0/270673274 >> 172.21.15.6:0/270673274 conn(0x7f0164077c80 msgr2=0x7f016407bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:27.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.922+0000 7f016a400700 1 -- 172.21.15.6:0/270673274 shutdown_connections 2024-05-09T04:51:27.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:27.923+0000 7f016a400700 1 -- 172.21.15.6:0/270673274 wait complete. 2024-05-09T04:51:28.752 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:51:28.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:28 smithi006 ceph-mon[29835]: pgmap v395: 129 pgs: 129 active+clean; 630 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.8 KiB/s wr, 11 op/s 2024-05-09T04:51:28.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:28 smithi006 ceph-mon[29835]: from='client.24733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:51:29.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:28 smithi186 ceph-mon[34668]: pgmap v395: 129 pgs: 129 active+clean; 630 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.8 KiB/s wr, 11 op/s 2024-05-09T04:51:29.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:28 smithi186 ceph-mon[34668]: from='client.24733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:51:29.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:28 smithi169 ceph-mon[34075]: pgmap v395: 129 pgs: 129 active+clean; 630 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.8 KiB/s wr, 11 op/s 2024-05-09T04:51:29.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:28 smithi169 ceph-mon[34075]: from='client.24733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:51:29.229 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:51:31.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:30 smithi186 ceph-mon[34668]: pgmap v396: 129 pgs: 129 active+clean; 630 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.8 KiB/s wr, 11 op/s 2024-05-09T04:51:31.108 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:30 smithi006 ceph-mon[29835]: pgmap v396: 129 pgs: 129 active+clean; 630 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.8 KiB/s wr, 11 op/s 2024-05-09T04:51:31.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:30 smithi169 ceph-mon[34075]: pgmap v396: 129 pgs: 129 active+clean; 630 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.8 KiB/s wr, 11 op/s 2024-05-09T04:51:31.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.516+0000 7f25a7000700 1 -- 172.21.15.6:0/56474138 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25a010b8a0 msgr2=0x7f25a0113200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:31.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.516+0000 7f25a7000700 1 --2- 172.21.15.6:0/56474138 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25a010b8a0 0x7f25a0113200 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f259800b0a0 tx=0x7f259800b3b0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.516+0000 7f25a7000700 1 -- 172.21.15.6:0/56474138 shutdown_connections 2024-05-09T04:51:31.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.516+0000 7f25a7000700 1 --2- 172.21.15.6:0/56474138 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25a010b8a0 0x7f25a0113200 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.516+0000 7f25a7000700 1 --2- 172.21.15.6:0/56474138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25a0106620 0x7f25a0106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.516+0000 7f25a7000700 1 --2- 172.21.15.6:0/56474138 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25a010af50 0x7f25a010b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.516+0000 7f25a7000700 1 -- 172.21.15.6:0/56474138 >> 172.21.15.6:0/56474138 conn(0x7f25a0102030 msgr2=0x7f25a0104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:31.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.516+0000 7f25a7000700 1 -- 172.21.15.6:0/56474138 shutdown_connections 2024-05-09T04:51:31.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.516+0000 7f25a7000700 1 -- 172.21.15.6:0/56474138 wait complete. 2024-05-09T04:51:31.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.518+0000 7f25a7000700 1 Processor -- start 2024-05-09T04:51:31.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.518+0000 7f25a7000700 1 -- start start 2024-05-09T04:51:31.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.518+0000 7f25a7000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25a0106620 0x7f25a019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:31.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.518+0000 7f25a7000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25a010af50 0x7f25a019d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:31.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.518+0000 7f25a7000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25a010b8a0 0x7f25a01a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:31.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.518+0000 7f25a7000700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f25a0116520 con 0x7f25a010af50 2024-05-09T04:51:31.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.518+0000 7f25a7000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f25a01163a0 con 0x7f25a0106620 2024-05-09T04:51:31.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.518+0000 7f25a7000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f25a01166a0 con 0x7f25a010b8a0 2024-05-09T04:51:31.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.519+0000 7f25a5200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25a010af50 0x7f25a019d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:31.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.519+0000 7f25a5c00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25a0106620 0x7f25a019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:31.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.519+0000 7f25a5c00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25a0106620 0x7f25a019cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:33572/0 (socket says 172.21.15.6:33572) 2024-05-09T04:51:31.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.519+0000 7f25a5c00700 1 -- 172.21.15.6:0/3614480309 learned_addr learned my addr 172.21.15.6:0/3614480309 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:51:31.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.519+0000 7f25a6600700 1 --2- 172.21.15.6:0/3614480309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25a010b8a0 0x7f25a01a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:31.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.519+0000 7f25a6600700 1 -- 172.21.15.6:0/3614480309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25a0106620 msgr2=0x7f25a019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:31.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.519+0000 7f25a6600700 1 --2- 172.21.15.6:0/3614480309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25a0106620 0x7f25a019cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.519+0000 7f25a6600700 1 -- 172.21.15.6:0/3614480309 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25a010af50 msgr2=0x7f25a019d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:31.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.519+0000 7f25a6600700 1 --2- 172.21.15.6:0/3614480309 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25a010af50 0x7f25a019d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.519+0000 7f25a6600700 1 -- 172.21.15.6:0/3614480309 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25a01a1c80 con 0x7f25a010b8a0 2024-05-09T04:51:31.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.520+0000 7f25a6600700 1 --2- 172.21.15.6:0/3614480309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25a010b8a0 0x7f25a01a15e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f259800b070 tx=0x7f2598004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:31.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.520+0000 7f259e000700 1 -- 172.21.15.6:0/3614480309 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2598021070 con 0x7f25a010b8a0 2024-05-09T04:51:31.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.520+0000 7f259e000700 1 -- 172.21.15.6:0/3614480309 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2598007b80 con 0x7f25a010b8a0 2024-05-09T04:51:31.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.520+0000 7f259e000700 1 -- 172.21.15.6:0/3614480309 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2598005040 con 0x7f25a010b8a0 2024-05-09T04:51:31.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.520+0000 7f25a7000700 1 -- 172.21.15.6:0/3614480309 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25a01a1f10 con 0x7f25a010b8a0 2024-05-09T04:51:31.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.520+0000 7f25a7000700 1 -- 172.21.15.6:0/3614480309 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25a01a2400 con 0x7f25a010b8a0 2024-05-09T04:51:31.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.522+0000 7f25a7000700 1 -- 172.21.15.6:0/3614480309 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25a00512c0 con 0x7f25a010b8a0 2024-05-09T04:51:31.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.524+0000 7f259e000700 1 -- 172.21.15.6:0/3614480309 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f2598024070 con 0x7f25a010b8a0 2024-05-09T04:51:31.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.526+0000 7f259e000700 1 --2- 172.21.15.6:0/3614480309 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f25880b2910 0x7f25880b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:31.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.526+0000 7f259e000700 1 -- 172.21.15.6:0/3614480309 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f259800f020 con 0x7f25a010b8a0 2024-05-09T04:51:31.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.526+0000 7f25a5c00700 1 --2- 172.21.15.6:0/3614480309 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f25880b2910 0x7f25880b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:31.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.527+0000 7f25a5c00700 1 --2- 172.21.15.6:0/3614480309 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f25880b2910 0x7f25880b4e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2594009d20 tx=0x7f2594009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:31.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.528+0000 7f259e000700 1 -- 172.21.15.6:0/3614480309 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f259802b020 con 0x7f25a010b8a0 2024-05-09T04:51:31.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.828+0000 7f25a7000700 1 -- 172.21.15.6:0/3614480309 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f25a002e5f0 con 0x7f25a010b8a0 2024-05-09T04:51:31.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.829+0000 7f259e000700 1 -- 172.21.15.6:0/3614480309 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11927 (secure 0 0 0) 0x7f25980ab7c0 con 0x7f25a010b8a0 2024-05-09T04:51:31.830 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:51:31.830 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/162094599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":162094599},{"type":"v1","addr":"172.21.15.6:6835","nonce":162094599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/589294363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":589294363},{"type":"v1","addr":"172.21.15.6:6841","nonce":589294363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:49:36.920376+0000","modified":"2024-05-09T04:51:07.589004+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24554,"mds_1":24530,"mds_2":24512,"mds_3":24511,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24442":{"gid":24442,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6833/2977145206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2977145206},{"type":"v1","addr":"172.21.15.186:6833","nonce":2977145206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6833/133345948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":133345948},{"type":"v1","addr":"172.21.15.169:6833","nonce":133345948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.6:6837/1281521088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":1281521088},{"type":"v1","addr":"172.21.15.6:6837","nonce":1281521088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2268322007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2268322007},{"type":"v1","addr":"172.21.15.186:6835","nonce":2268322007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/3811005052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3811005052},{"type":"v1","addr":"172.21.15.169:6835","nonce":3811005052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.186:6837/1647607118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1647607118},{"type":"v1","addr":"172.21.15.186:6837","nonce":1647607118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":32,"addr":"172.21.15.186:6839/3954883105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":3954883105},{"type":"v1","addr":"172.21.15.186:6839","nonce":3954883105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.169:6837/1845715294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1845715294},{"type":"v1","addr":"172.21.15.169:6837","nonce":1845715294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.6:6839/2780306492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":2780306492},{"type":"v1","addr":"172.21.15.6:6839","nonce":2780306492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.169:6839/614388771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":614388771},{"type":"v1","addr":"172.21.15.169:6839","nonce":614388771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24493,24511,24530,24554,24512]},"id":1}]} 2024-05-09T04:51:31.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.832+0000 7f25a7000700 1 -- 172.21.15.6:0/3614480309 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f25880b2910 msgr2=0x7f25880b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:31.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.832+0000 7f25a7000700 1 --2- 172.21.15.6:0/3614480309 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f25880b2910 0x7f25880b4e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2594009d20 tx=0x7f2594009260 comp rx=0 tx=0).stop 2024-05-09T04:51:31.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.832+0000 7f25a7000700 1 -- 172.21.15.6:0/3614480309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25a010b8a0 msgr2=0x7f25a01a15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:31.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.832+0000 7f25a7000700 1 --2- 172.21.15.6:0/3614480309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25a010b8a0 0x7f25a01a15e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f259800b070 tx=0x7f2598004c80 comp rx=0 tx=0).stop 2024-05-09T04:51:31.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.832+0000 7f25a7000700 1 -- 172.21.15.6:0/3614480309 shutdown_connections 2024-05-09T04:51:31.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.832+0000 7f25a7000700 1 --2- 172.21.15.6:0/3614480309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25a010b8a0 0x7f25a01a15e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.832+0000 7f25a7000700 1 --2- 172.21.15.6:0/3614480309 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f25880b2910 0x7f25880b4e20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.832+0000 7f25a7000700 1 --2- 172.21.15.6:0/3614480309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25a0106620 0x7f25a019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.832+0000 7f25a7000700 1 --2- 172.21.15.6:0/3614480309 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25a010af50 0x7f25a019d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.832+0000 7f25a7000700 1 -- 172.21.15.6:0/3614480309 >> 172.21.15.6:0/3614480309 conn(0x7f25a0102030 msgr2=0x7f25a010a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:31.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.832+0000 7f25a7000700 1 -- 172.21.15.6:0/3614480309 shutdown_connections 2024-05-09T04:51:31.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:31.832+0000 7f25a7000700 1 -- 172.21.15.6:0/3614480309 wait complete. 2024-05-09T04:51:31.834 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 29 2024-05-09T04:51:31.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:31 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/3614480309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:51:32.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:31 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/3614480309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:51:32.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:31 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/3614480309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:51:32.693 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T04:51:32.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:32 smithi006 ceph-mon[29835]: pgmap v397: 129 pgs: 129 active+clean; 637 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2024-05-09T04:51:32.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:32 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:51:32.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:32 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:51:32.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:32 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:51:33.077 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:32 smithi186 ceph-mon[34668]: pgmap v397: 129 pgs: 129 active+clean; 637 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2024-05-09T04:51:33.077 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:32 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:51:33.077 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:32 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:51:33.077 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:32 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:51:33.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:32 smithi169 ceph-mon[34075]: pgmap v397: 129 pgs: 129 active+clean; 637 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2024-05-09T04:51:33.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:32 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:51:33.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:32 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:51:33.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:32 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:51:33.141 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:51:34.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:33 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:51:34.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:33 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:51:34.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:33 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:51:34.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:33 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:51:34.176 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:33 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:51:34.176 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:33 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:51:34.176 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:33 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:51:34.176 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:33 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:51:34.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:33 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:51:34.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:33 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:51:34.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:33 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:51:34.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:33 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:51:35.052 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:34 smithi006 ceph-mon[29835]: pgmap v398: 129 pgs: 129 active+clean; 637 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-05-09T04:51:35.053 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:34 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:51:35.053 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:34 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:51:35.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:34 smithi186 ceph-mon[34668]: pgmap v398: 129 pgs: 129 active+clean; 637 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-05-09T04:51:35.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:34 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:51:35.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:34 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:51:35.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:34 smithi169 ceph-mon[34075]: pgmap v398: 129 pgs: 129 active+clean; 637 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-05-09T04:51:35.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:34 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:51:35.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:34 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:51:35.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.404+0000 7fd36ea00700 1 -- 172.21.15.6:0/3571160958 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd36810b110 msgr2=0x7fd368117620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:35.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.404+0000 7fd36ea00700 1 --2- 172.21.15.6:0/3571160958 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd36810b110 0x7fd368117620 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd36000b0a0 tx=0x7fd36000b3b0 comp rx=0 tx=0).stop 2024-05-09T04:51:35.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.404+0000 7fd36ea00700 1 -- 172.21.15.6:0/3571160958 shutdown_connections 2024-05-09T04:51:35.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.404+0000 7fd36ea00700 1 --2- 172.21.15.6:0/3571160958 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd36810b110 0x7fd368117620 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:35.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.404+0000 7fd36ea00700 1 --2- 172.21.15.6:0/3571160958 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd36810f6c0 0x7fd36810fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:35.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.404+0000 7fd36ea00700 1 --2- 172.21.15.6:0/3571160958 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd36810a7c0 0x7fd36810abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:35.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.404+0000 7fd36ea00700 1 -- 172.21.15.6:0/3571160958 >> 172.21.15.6:0/3571160958 conn(0x7fd368076b00 msgr2=0x7fd368076f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:35.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.404+0000 7fd36ea00700 1 -- 172.21.15.6:0/3571160958 shutdown_connections 2024-05-09T04:51:35.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.405+0000 7fd36ea00700 1 -- 172.21.15.6:0/3571160958 wait complete. 2024-05-09T04:51:35.407 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.406+0000 7fd36ea00700 1 Processor -- start 2024-05-09T04:51:35.407 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.406+0000 7fd36ea00700 1 -- start start 2024-05-09T04:51:35.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.407+0000 7fd36ea00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd36810a7c0 0x7fd3681a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:35.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.407+0000 7fd36ea00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd36810b110 0x7fd3681a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:35.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.407+0000 7fd36ea00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd36810f6c0 0x7fd3681a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:35.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.407+0000 7fd36ea00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd36811a8f0 con 0x7fd36810a7c0 2024-05-09T04:51:35.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.407+0000 7fd36ea00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd36811a770 con 0x7fd36810f6c0 2024-05-09T04:51:35.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.407+0000 7fd36ea00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd36811aa70 con 0x7fd36810b110 2024-05-09T04:51:35.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.407+0000 7fd36d600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd36810a7c0 0x7fd3681a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:35.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.407+0000 7fd36d600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd36810a7c0 0x7fd3681a1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:47038/0 (socket says 172.21.15.6:47038) 2024-05-09T04:51:35.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.407+0000 7fd36d600700 1 -- 172.21.15.6:0/2127461093 learned_addr learned my addr 172.21.15.6:0/2127461093 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:51:35.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.407+0000 7fd36cc00700 1 --2- 172.21.15.6:0/2127461093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd36810b110 0x7fd3681a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:35.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.407+0000 7fd36e000700 1 --2- 172.21.15.6:0/2127461093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd36810f6c0 0x7fd3681a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:35.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.408+0000 7fd36d600700 1 -- 172.21.15.6:0/2127461093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd36810b110 msgr2=0x7fd3681a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:35.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.408+0000 7fd36d600700 1 --2- 172.21.15.6:0/2127461093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd36810b110 0x7fd3681a17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:35.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.408+0000 7fd36d600700 1 -- 172.21.15.6:0/2127461093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd36810f6c0 msgr2=0x7fd3681a5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:35.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.408+0000 7fd36d600700 1 --2- 172.21.15.6:0/2127461093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd36810f6c0 0x7fd3681a5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:35.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.408+0000 7fd36d600700 1 -- 172.21.15.6:0/2127461093 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3681a60a0 con 0x7fd36810a7c0 2024-05-09T04:51:35.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.408+0000 7fd36d600700 1 --2- 172.21.15.6:0/2127461093 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd36810a7c0 0x7fd3681a1220 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00e9a0 tx=0x7fd35c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:35.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.408+0000 7fd366000700 1 -- 172.21.15.6:0/2127461093 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c00ca10 con 0x7fd36810a7c0 2024-05-09T04:51:35.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.408+0000 7fd366000700 1 -- 172.21.15.6:0/2127461093 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd35c00cbb0 con 0x7fd36810a7c0 2024-05-09T04:51:35.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.408+0000 7fd366000700 1 -- 172.21.15.6:0/2127461093 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c018680 con 0x7fd36810a7c0 2024-05-09T04:51:35.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.408+0000 7fd36ea00700 1 -- 172.21.15.6:0/2127461093 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3681a6390 con 0x7fd36810a7c0 2024-05-09T04:51:35.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.409+0000 7fd36ea00700 1 -- 172.21.15.6:0/2127461093 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3681adcb0 con 0x7fd36810a7c0 2024-05-09T04:51:35.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.410+0000 7fd36ea00700 1 -- 172.21.15.6:0/2127461093 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3680512c0 con 0x7fd36810a7c0 2024-05-09T04:51:35.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.412+0000 7fd366000700 1 -- 172.21.15.6:0/2127461093 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fd35c010430 con 0x7fd36810a7c0 2024-05-09T04:51:35.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.414+0000 7fd366000700 1 --2- 172.21.15.6:0/2127461093 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd3500b2910 0x7fd3500b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:35.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.414+0000 7fd366000700 1 -- 172.21.15.6:0/2127461093 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fd35c0e2b90 con 0x7fd36810a7c0 2024-05-09T04:51:35.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.414+0000 7fd36cc00700 1 --2- 172.21.15.6:0/2127461093 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd3500b2910 0x7fd3500b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:35.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.415+0000 7fd36cc00700 1 --2- 172.21.15.6:0/2127461093 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd3500b2910 0x7fd3500b4e20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd368077ec0 tx=0x7fd354009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:35.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.417+0000 7fd366000700 1 -- 172.21.15.6:0/2127461093 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd35c0aa370 con 0x7fd36810a7c0 2024-05-09T04:51:35.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.664+0000 7fd36ea00700 1 -- 172.21.15.6:0/2127461093 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd36802e5f0 con 0x7fd36810a7c0 2024-05-09T04:51:35.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.668+0000 7fd366000700 1 -- 172.21.15.6:0/2127461093 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24659 (secure 0 0 0) 0x7fd35c0b23f0 con 0x7fd36810a7c0 2024-05-09T04:51:35.670 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:51:35.670 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":89,"fsid":"f74d813c-0dbd-11ef-bc97-c7b262605968","created":"2024-05-09T04:39:35.287470+0000","modified":"2024-05-09T04:50:57.686719+0000","last_up_change":"2024-05-09T04:48:16.919842+0000","last_in_change":"2024-05-09T04:47:56.452645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:43:47.538306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:49:28.805447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T04:49:32.970287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T04:49:45.637000+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1b732fd3-e648-4b0d-aece-083bdaaa5d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6803","nonce":1738750914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6805","nonce":1738750914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6809","nonce":1738750914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6807","nonce":1738750914}]},"public_addr":"172.21.15.6:6803/1738750914","cluster_addr":"172.21.15.6:6805/1738750914","heartbeat_back_addr":"172.21.15.6:6809/1738750914","heartbeat_front_addr":"172.21.15.6:6807/1738750914","state":["exists","up"]},{"osd":1,"uuid":"9770c07d-4243-43bf-a0cd-9d2ac558556c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6801","nonce":2552207737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6803","nonce":2552207737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6807","nonce":2552207737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6805","nonce":2552207737}]},"public_addr":"172.21.15.169:6801/2552207737","cluster_addr":"172.21.15.169:6803/2552207737","heartbeat_back_addr":"172.21.15.169:6807/2552207737","heartbeat_front_addr":"172.21.15.169:6805/2552207737","state":["exists","up"]},{"osd":2,"uuid":"3f50ed89-b2a8-415e-b78d-8510d98d9368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6801","nonce":231435105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6803","nonce":231435105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6807","nonce":231435105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6805","nonce":231435105}]},"public_addr":"172.21.15.186:6801/231435105","cluster_addr":"172.21.15.186:6803/231435105","heartbeat_back_addr":"172.21.15.186:6807/231435105","heartbeat_front_addr":"172.21.15.186:6805/231435105","state":["exists","up"]},{"osd":3,"uuid":"afcbc36b-5669-4044-8dc5-2ebfe7970774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6811","nonce":2846613703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6813","nonce":2846613703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6817","nonce":2846613703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6815","nonce":2846613703}]},"public_addr":"172.21.15.6:6811/2846613703","cluster_addr":"172.21.15.6:6813/2846613703","heartbeat_back_addr":"172.21.15.6:6817/2846613703","heartbeat_front_addr":"172.21.15.6:6815/2846613703","state":["exists","up"]},{"osd":4,"uuid":"bb84cc5e-0297-4e23-be50-7ee2f7e59665","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6809","nonce":1928722453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6811","nonce":1928722453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6815","nonce":1928722453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6813","nonce":1928722453}]},"public_addr":"172.21.15.169:6809/1928722453","cluster_addr":"172.21.15.169:6811/1928722453","heartbeat_back_addr":"172.21.15.169:6815/1928722453","heartbeat_front_addr":"172.21.15.169:6813/1928722453","state":["exists","up"]},{"osd":5,"uuid":"830dc7f3-352e-4b01-8d4c-a9b68dcb40d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6809","nonce":3644445439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6811","nonce":3644445439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6815","nonce":3644445439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6813","nonce":3644445439}]},"public_addr":"172.21.15.186:6809/3644445439","cluster_addr":"172.21.15.186:6811/3644445439","heartbeat_back_addr":"172.21.15.186:6815/3644445439","heartbeat_front_addr":"172.21.15.186:6813/3644445439","state":["exists","up"]},{"osd":6,"uuid":"2e8478cb-c8c9-47ce-a3a5-48ffc959ab24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6819","nonce":3068252891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6821","nonce":3068252891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6825","nonce":3068252891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6823","nonce":3068252891}]},"public_addr":"172.21.15.6:6819/3068252891","cluster_addr":"172.21.15.6:6821/3068252891","heartbeat_back_addr":"172.21.15.6:6825/3068252891","heartbeat_front_addr":"172.21.15.6:6823/3068252891","state":["exists","up"]},{"osd":7,"uuid":"0f432ee2-42a4-4efa-9233-8e5a2fd65d10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6817","nonce":1303584711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6819","nonce":1303584711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6823","nonce":1303584711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6821","nonce":1303584711}]},"public_addr":"172.21.15.169:6817/1303584711","cluster_addr":"172.21.15.169:6819/1303584711","heartbeat_back_addr":"172.21.15.169:6823/1303584711","heartbeat_front_addr":"172.21.15.169:6821/1303584711","state":["exists","up"]},{"osd":8,"uuid":"cf5dca81-131b-48e7-9417-d01d4be47316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6817","nonce":3475163771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6819","nonce":3475163771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6823","nonce":3475163771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6821","nonce":3475163771}]},"public_addr":"172.21.15.186:6817/3475163771","cluster_addr":"172.21.15.186:6819/3475163771","heartbeat_back_addr":"172.21.15.186:6823/3475163771","heartbeat_front_addr":"172.21.15.186:6821/3475163771","state":["exists","up"]},{"osd":9,"uuid":"bca7bedd-1a57-4465-9033-064ca468add2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6827","nonce":3590990973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6829","nonce":3590990973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6833","nonce":3590990973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6831","nonce":3590990973}]},"public_addr":"172.21.15.6:6827/3590990973","cluster_addr":"172.21.15.6:6829/3590990973","heartbeat_back_addr":"172.21.15.6:6833/3590990973","heartbeat_front_addr":"172.21.15.6:6831/3590990973","state":["exists","up"]},{"osd":10,"uuid":"97b0e058-64f8-42ee-9b99-3ef8e39c8a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6825","nonce":61007341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6827","nonce":61007341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6831","nonce":61007341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6829","nonce":61007341}]},"public_addr":"172.21.15.169:6825/61007341","cluster_addr":"172.21.15.169:6827/61007341","heartbeat_back_addr":"172.21.15.169:6831/61007341","heartbeat_front_addr":"172.21.15.169:6829/61007341","state":["exists","up"]},{"osd":11,"uuid":"7f2220b0-24cf-48b2-95ae-1d66e00e57fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6825","nonce":1138940441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6827","nonce":1138940441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6831","nonce":1138940441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6829","nonce":1138940441}]},"public_addr":"172.21.15.186:6825/1138940441","cluster_addr":"172.21.15.186:6827/1138940441","heartbeat_back_addr":"172.21.15.186:6831/1138940441","heartbeat_front_addr":"172.21.15.186:6829/1138940441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:53.766295+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:20.195243+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:44.153812+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:20.472086+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:48.415371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:13.795415+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:50.236398+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:18.066623+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:43.781009+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:47:20.084719+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:47:48.219901+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:48:14.965536+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/3742411767":"2024-05-10T04:40:55.357057+0000","172.21.15.6:0/2410771702":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6801/937713898":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6800/937713898":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6801/3021503778":"2024-05-10T04:40:19.639151+0000","172.21.15.6:6800/3021503778":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/2559510850":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/3810474743":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/4133970555":"2024-05-10T04:40:55.357057+0000","172.21.15.6:0/664817743":"2024-05-10T04:40:19.639151+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:51:35.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.673+0000 7fd36ea00700 1 -- 172.21.15.6:0/2127461093 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd3500b2910 msgr2=0x7fd3500b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:35.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.673+0000 7fd36ea00700 1 --2- 172.21.15.6:0/2127461093 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd3500b2910 0x7fd3500b4e20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd368077ec0 tx=0x7fd354009260 comp rx=0 tx=0).stop 2024-05-09T04:51:35.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.673+0000 7fd36ea00700 1 -- 172.21.15.6:0/2127461093 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd36810a7c0 msgr2=0x7fd3681a1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:35.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.673+0000 7fd36ea00700 1 --2- 172.21.15.6:0/2127461093 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd36810a7c0 0x7fd3681a1220 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00e9a0 tx=0x7fd35c00ecb0 comp rx=0 tx=0).stop 2024-05-09T04:51:35.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.673+0000 7fd36ea00700 1 -- 172.21.15.6:0/2127461093 shutdown_connections 2024-05-09T04:51:35.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.673+0000 7fd36ea00700 1 --2- 172.21.15.6:0/2127461093 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd3500b2910 0x7fd3500b4e20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:35.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.673+0000 7fd36ea00700 1 --2- 172.21.15.6:0/2127461093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd36810f6c0 0x7fd3681a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:35.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.673+0000 7fd36ea00700 1 --2- 172.21.15.6:0/2127461093 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd36810a7c0 0x7fd3681a1220 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:35.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.673+0000 7fd36ea00700 1 --2- 172.21.15.6:0/2127461093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd36810b110 0x7fd3681a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:35.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.673+0000 7fd36ea00700 1 -- 172.21.15.6:0/2127461093 >> 172.21.15.6:0/2127461093 conn(0x7fd368076b00 msgr2=0x7fd368108860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:35.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.673+0000 7fd36ea00700 1 -- 172.21.15.6:0/2127461093 shutdown_connections 2024-05-09T04:51:35.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:35.674+0000 7fd36ea00700 1 -- 172.21.15.6:0/2127461093 wait complete. 2024-05-09T04:51:35.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:35 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/2127461093' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:51:36.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:35 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/2127461093' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:51:36.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:35 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/2127461093' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:51:36.859 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-09T04:51:36.859 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T04:51:37.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:36 smithi006 ceph-mon[29835]: pgmap v399: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-05-09T04:51:37.312 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:51:37.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:36 smithi186 ceph-mon[34668]: pgmap v399: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-05-09T04:51:37.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:36 smithi169 ceph-mon[34075]: pgmap v399: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-05-09T04:51:38.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:38 smithi186 ceph-mon[34668]: pgmap v400: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-05-09T04:51:38.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:38 smithi169 ceph-mon[34075]: pgmap v400: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-05-09T04:51:38.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:38 smithi006 ceph-mon[29835]: pgmap v400: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-05-09T04:51:39.527 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:39 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:51:39.527 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:39 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:51:39.538 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:39 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:51:39.538 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:39 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:51:39.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.536+0000 7fcd13800700 1 -- 172.21.15.6:0/1021550537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd0c1045f0 msgr2=0x7fcd0c104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:39.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.536+0000 7fcd13800700 1 --2- 172.21.15.6:0/1021550537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd0c1045f0 0x7fcd0c104a00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcd000099c0 tx=0x7fcd00009cd0 comp rx=0 tx=0).stop 2024-05-09T04:51:39.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.537+0000 7fcd13800700 1 -- 172.21.15.6:0/1021550537 shutdown_connections 2024-05-09T04:51:39.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.537+0000 7fcd13800700 1 --2- 172.21.15.6:0/1021550537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd0c109870 0x7fcd0c1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:39.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.537+0000 7fcd13800700 1 --2- 172.21.15.6:0/1021550537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd0c1045f0 0x7fcd0c104a00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:39.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.537+0000 7fcd13800700 1 --2- 172.21.15.6:0/1021550537 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcd0c108f20 0x7fcd0c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:39.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.537+0000 7fcd13800700 1 -- 172.21.15.6:0/1021550537 >> 172.21.15.6:0/1021550537 conn(0x7fcd0c100040 msgr2=0x7fcd0c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:39.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.537+0000 7fcd13800700 1 -- 172.21.15.6:0/1021550537 shutdown_connections 2024-05-09T04:51:39.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.537+0000 7fcd13800700 1 -- 172.21.15.6:0/1021550537 wait complete. 2024-05-09T04:51:39.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.538+0000 7fcd13800700 1 Processor -- start 2024-05-09T04:51:39.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.538+0000 7fcd13800700 1 -- start start 2024-05-09T04:51:39.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.539+0000 7fcd13800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcd0c1045f0 0x7fcd0c19d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:39.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.539+0000 7fcd13800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd0c108f20 0x7fcd0c19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:39.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.539+0000 7fcd13800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd0c109870 0x7fcd0c1a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:39.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.539+0000 7fcd13800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c1144f0 con 0x7fcd0c1045f0 2024-05-09T04:51:39.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.539+0000 7fcd13800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c114370 con 0x7fcd0c108f20 2024-05-09T04:51:39.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.539+0000 7fcd13800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c114670 con 0x7fcd0c109870 2024-05-09T04:51:39.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.539+0000 7fcd11800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd0c109870 0x7fcd0c1a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:39.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.539+0000 7fcd10e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcd0c1045f0 0x7fcd0c19d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:39.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.539+0000 7fcd11800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd0c109870 0x7fcd0c1a1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:51108/0 (socket says 172.21.15.6:51108) 2024-05-09T04:51:39.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.539+0000 7fcd11800700 1 -- 172.21.15.6:0/198300776 learned_addr learned my addr 172.21.15.6:0/198300776 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:51:39.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.539+0000 7fcd0be00700 1 --2- 172.21.15.6:0/198300776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd0c108f20 0x7fcd0c19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:39.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.540+0000 7fcd11800700 1 -- 172.21.15.6:0/198300776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd0c108f20 msgr2=0x7fcd0c19d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:39.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.540+0000 7fcd11800700 1 --2- 172.21.15.6:0/198300776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd0c108f20 0x7fcd0c19d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:39.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.540+0000 7fcd11800700 1 -- 172.21.15.6:0/198300776 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcd0c1045f0 msgr2=0x7fcd0c19d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:39.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.540+0000 7fcd11800700 1 --2- 172.21.15.6:0/198300776 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcd0c1045f0 0x7fcd0c19d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:39.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.540+0000 7fcd11800700 1 -- 172.21.15.6:0/198300776 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd0c1a1f20 con 0x7fcd0c109870 2024-05-09T04:51:39.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.540+0000 7fcd11800700 1 --2- 172.21.15.6:0/198300776 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd0c109870 0x7fcd0c1a1880 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcd0400ef30 tx=0x7fcd0400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:39.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.540+0000 7fcd09600700 1 -- 172.21.15.6:0/198300776 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd040092e0 con 0x7fcd0c109870 2024-05-09T04:51:39.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.540+0000 7fcd09600700 1 -- 172.21.15.6:0/198300776 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd04004bb0 con 0x7fcd0c109870 2024-05-09T04:51:39.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.540+0000 7fcd13800700 1 -- 172.21.15.6:0/198300776 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd0c1a21b0 con 0x7fcd0c109870 2024-05-09T04:51:39.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.540+0000 7fcd09600700 1 -- 172.21.15.6:0/198300776 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd04004020 con 0x7fcd0c109870 2024-05-09T04:51:39.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.541+0000 7fcd13800700 1 -- 172.21.15.6:0/198300776 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd0c1a9b30 con 0x7fcd0c109870 2024-05-09T04:51:39.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.542+0000 7fcd13800700 1 -- 172.21.15.6:0/198300776 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd0c0512c0 con 0x7fcd0c109870 2024-05-09T04:51:39.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.544+0000 7fcd09600700 1 -- 172.21.15.6:0/198300776 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fcd0401b030 con 0x7fcd0c109870 2024-05-09T04:51:39.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.546+0000 7fcd09600700 1 --2- 172.21.15.6:0/198300776 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fccf40b2990 0x7fccf40b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:39.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.546+0000 7fcd09600700 1 -- 172.21.15.6:0/198300776 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fcd040e2160 con 0x7fcd0c109870 2024-05-09T04:51:39.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.546+0000 7fcd10e00700 1 --2- 172.21.15.6:0/198300776 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fccf40b2990 0x7fccf40b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:39.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.547+0000 7fcd10e00700 1 --2- 172.21.15.6:0/198300776 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fccf40b2990 0x7fccf40b4ea0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcd00000c00 tx=0x7fcd0001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:39.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.550+0000 7fcd09600700 1 -- 172.21.15.6:0/198300776 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcd04022020 con 0x7fcd0c109870 2024-05-09T04:51:39.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:39 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:51:39.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:39 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:51:39.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.801+0000 7fcd13800700 1 -- 172.21.15.6:0/198300776 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcd0c110bb0 con 0x7fccf40b2990 2024-05-09T04:51:39.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.824+0000 7fcd09600700 1 -- 172.21.15.6:0/198300776 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340430 (secure 0 0 0) 0x7fcd0007b1e0 con 0x7fccf40b2990 2024-05-09T04:51:39.829 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:51:39.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.833+0000 7fcd13800700 1 -- 172.21.15.6:0/198300776 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fccf40b2990 msgr2=0x7fccf40b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:39.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.833+0000 7fcd13800700 1 --2- 172.21.15.6:0/198300776 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fccf40b2990 0x7fccf40b4ea0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcd00000c00 tx=0x7fcd0001a040 comp rx=0 tx=0).stop 2024-05-09T04:51:39.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.833+0000 7fcd13800700 1 -- 172.21.15.6:0/198300776 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd0c109870 msgr2=0x7fcd0c1a1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:39.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.833+0000 7fcd13800700 1 --2- 172.21.15.6:0/198300776 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd0c109870 0x7fcd0c1a1880 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcd0400ef30 tx=0x7fcd0400c3a0 comp rx=0 tx=0).stop 2024-05-09T04:51:39.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.833+0000 7fcd13800700 1 -- 172.21.15.6:0/198300776 shutdown_connections 2024-05-09T04:51:39.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.833+0000 7fcd13800700 1 --2- 172.21.15.6:0/198300776 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd0c109870 0x7fcd0c1a1880 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:39.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.833+0000 7fcd13800700 1 --2- 172.21.15.6:0/198300776 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcd0c1045f0 0x7fcd0c19d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:39.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.833+0000 7fcd13800700 1 --2- 172.21.15.6:0/198300776 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fccf40b2990 0x7fccf40b4ea0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:39.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.833+0000 7fcd13800700 1 --2- 172.21.15.6:0/198300776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd0c108f20 0x7fcd0c19d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:39.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.833+0000 7fcd13800700 1 -- 172.21.15.6:0/198300776 >> 172.21.15.6:0/198300776 conn(0x7fcd0c100040 msgr2=0x7fcd0c1104a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:39.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.833+0000 7fcd13800700 1 -- 172.21.15.6:0/198300776 shutdown_connections 2024-05-09T04:51:39.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:39.833+0000 7fcd13800700 1 -- 172.21.15.6:0/198300776 wait complete. 2024-05-09T04:51:39.835 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-05-09T04:51:40.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:40 smithi186 ceph-mon[34668]: pgmap v401: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-09T04:51:40.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:40 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:51:40.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:40 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:51:40.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:40 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:51:40.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:40 smithi186 ceph-mon[34668]: from='client.24751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:51:40.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:40 smithi169 ceph-mon[34075]: pgmap v401: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-09T04:51:40.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:40 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:51:40.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:40 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:51:40.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:40 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:51:40.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:40 smithi169 ceph-mon[34075]: from='client.24751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:51:40.617 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:40 smithi006 ceph-mon[29835]: pgmap v401: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-09T04:51:40.618 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:40 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:51:40.618 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:40 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:51:40.618 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:40 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:51:40.618 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:40 smithi006 ceph-mon[29835]: from='client.24751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:51:40.668 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":401,"stamp":"2024-05-09T04:51:39.594571+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656643,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":448,"num_read_kb":322,"num_write":373,"num_write_kb":2289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":308,"ondisk_log_size":308,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":367452,"kb_used_data":14140,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124574372,"statfs":{"total":1151940427776,"available":1151564156928,"internally_reserved":0,"allocated":14479360,"data_stored":8068737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19137,"internal_metadata":361674047},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11048,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":70,"num_write":15,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005868"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.817091+0000","last_change":"2024-05-09T04:49:34.144921+0000","last_active":"2024-05-09T04:50:57.817091+0000","last_peered":"2024-05-09T04:50:57.817091+0000","last_clean":"2024-05-09T04:50:57.817091+0000","last_became_active":"2024-05-09T04:49:34.144275+0000","last_became_peered":"2024-05-09T04:49:34.144275+0000","last_unstale":"2024-05-09T04:50:57.817091+0000","last_undegraded":"2024-05-09T04:50:57.817091+0000","last_fullsized":"2024-05-09T04:50:57.817091+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:00:31.377738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.204322+0000","last_change":"2024-05-09T04:49:34.158057+0000","last_active":"2024-05-09T04:50:58.204322+0000","last_peered":"2024-05-09T04:50:58.204322+0000","last_clean":"2024-05-09T04:50:58.204322+0000","last_became_active":"2024-05-09T04:49:34.157129+0000","last_became_peered":"2024-05-09T04:49:34.157129+0000","last_unstale":"2024-05-09T04:50:58.204322+0000","last_undegraded":"2024-05-09T04:50:58.204322+0000","last_fullsized":"2024-05-09T04:50:58.204322+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:49.360309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203840+0000","last_change":"2024-05-09T04:49:34.145519+0000","last_active":"2024-05-09T04:50:58.203840+0000","last_peered":"2024-05-09T04:50:58.203840+0000","last_clean":"2024-05-09T04:50:58.203840+0000","last_became_active":"2024-05-09T04:49:34.144501+0000","last_became_peered":"2024-05-09T04:49:34.144501+0000","last_unstale":"2024-05-09T04:50:58.203840+0000","last_undegraded":"2024-05-09T04:50:58.203840+0000","last_fullsized":"2024-05-09T04:50:58.203840+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:05:53.727520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203281+0000","last_change":"2024-05-09T04:49:34.134383+0000","last_active":"2024-05-09T04:50:58.203281+0000","last_peered":"2024-05-09T04:50:58.203281+0000","last_clean":"2024-05-09T04:50:58.203281+0000","last_became_active":"2024-05-09T04:49:34.134049+0000","last_became_peered":"2024-05-09T04:49:34.134049+0000","last_unstale":"2024-05-09T04:50:58.203281+0000","last_undegraded":"2024-05-09T04:50:58.203281+0000","last_fullsized":"2024-05-09T04:50:58.203281+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:36:02.548188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.201604+0000","last_change":"2024-05-09T04:49:34.157897+0000","last_active":"2024-05-09T04:50:58.201604+0000","last_peered":"2024-05-09T04:50:58.201604+0000","last_clean":"2024-05-09T04:50:58.201604+0000","last_became_active":"2024-05-09T04:49:34.157516+0000","last_became_peered":"2024-05-09T04:49:34.157516+0000","last_unstale":"2024-05-09T04:50:58.201604+0000","last_undegraded":"2024-05-09T04:50:58.201604+0000","last_fullsized":"2024-05-09T04:50:58.201604+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:33:25.444155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.202150+0000","last_change":"2024-05-09T04:49:34.146389+0000","last_active":"2024-05-09T04:50:58.202150+0000","last_peered":"2024-05-09T04:50:58.202150+0000","last_clean":"2024-05-09T04:50:58.202150+0000","last_became_active":"2024-05-09T04:49:34.145994+0000","last_became_peered":"2024-05-09T04:49:34.145994+0000","last_unstale":"2024-05-09T04:50:58.202150+0000","last_undegraded":"2024-05-09T04:50:58.202150+0000","last_fullsized":"2024-05-09T04:50:58.202150+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:59.094826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203055+0000","last_change":"2024-05-09T04:49:34.146428+0000","last_active":"2024-05-09T04:50:58.203055+0000","last_peered":"2024-05-09T04:50:58.203055+0000","last_clean":"2024-05-09T04:50:58.203055+0000","last_became_active":"2024-05-09T04:49:34.146045+0000","last_became_peered":"2024-05-09T04:49:34.146045+0000","last_unstale":"2024-05-09T04:50:58.203055+0000","last_undegraded":"2024-05-09T04:50:58.203055+0000","last_fullsized":"2024-05-09T04:50:58.203055+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:13.129474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700092+0000","last_change":"2024-05-09T04:49:34.135494+0000","last_active":"2024-05-09T04:50:57.700092+0000","last_peered":"2024-05-09T04:50:57.700092+0000","last_clean":"2024-05-09T04:50:57.700092+0000","last_became_active":"2024-05-09T04:49:34.135085+0000","last_became_peered":"2024-05-09T04:49:34.135085+0000","last_unstale":"2024-05-09T04:50:57.700092+0000","last_undegraded":"2024-05-09T04:50:57.700092+0000","last_fullsized":"2024-05-09T04:50:57.700092+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:57.023344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.952831+0000","last_change":"2024-05-09T04:49:34.131353+0000","last_active":"2024-05-09T04:50:57.952831+0000","last_peered":"2024-05-09T04:50:57.952831+0000","last_clean":"2024-05-09T04:50:57.952831+0000","last_became_active":"2024-05-09T04:49:34.130954+0000","last_became_peered":"2024-05-09T04:49:34.130954+0000","last_unstale":"2024-05-09T04:50:57.952831+0000","last_undegraded":"2024-05-09T04:50:57.952831+0000","last_fullsized":"2024-05-09T04:50:57.952831+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:04:46.947861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.204015+0000","last_change":"2024-05-09T04:49:34.134733+0000","last_active":"2024-05-09T04:50:58.204015+0000","last_peered":"2024-05-09T04:50:58.204015+0000","last_clean":"2024-05-09T04:50:58.204015+0000","last_became_active":"2024-05-09T04:49:34.134337+0000","last_became_peered":"2024-05-09T04:49:34.134337+0000","last_unstale":"2024-05-09T04:50:58.204015+0000","last_undegraded":"2024-05-09T04:50:58.204015+0000","last_fullsized":"2024-05-09T04:50:58.204015+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:05.452488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203646+0000","last_change":"2024-05-09T04:49:34.139143+0000","last_active":"2024-05-09T04:50:58.203646+0000","last_peered":"2024-05-09T04:50:58.203646+0000","last_clean":"2024-05-09T04:50:58.203646+0000","last_became_active":"2024-05-09T04:49:34.138749+0000","last_became_peered":"2024-05-09T04:49:34.138749+0000","last_unstale":"2024-05-09T04:50:58.203646+0000","last_undegraded":"2024-05-09T04:50:58.203646+0000","last_fullsized":"2024-05-09T04:50:58.203646+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:11.552063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203348+0000","last_change":"2024-05-09T04:49:34.134687+0000","last_active":"2024-05-09T04:50:58.203348+0000","last_peered":"2024-05-09T04:50:58.203348+0000","last_clean":"2024-05-09T04:50:58.203348+0000","last_became_active":"2024-05-09T04:49:34.134324+0000","last_became_peered":"2024-05-09T04:49:34.134324+0000","last_unstale":"2024-05-09T04:50:58.203348+0000","last_undegraded":"2024-05-09T04:50:58.203348+0000","last_fullsized":"2024-05-09T04:50:58.203348+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:11.667883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.087985+0000","last_change":"2024-05-09T04:49:34.153592+0000","last_active":"2024-05-09T04:50:58.087985+0000","last_peered":"2024-05-09T04:50:58.087985+0000","last_clean":"2024-05-09T04:50:58.087985+0000","last_became_active":"2024-05-09T04:49:34.153184+0000","last_became_peered":"2024-05-09T04:49:34.153184+0000","last_unstale":"2024-05-09T04:50:58.087985+0000","last_undegraded":"2024-05-09T04:50:58.087985+0000","last_fullsized":"2024-05-09T04:50:58.087985+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:36:56.784781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203005+0000","last_change":"2024-05-09T04:49:34.144046+0000","last_active":"2024-05-09T04:50:58.203005+0000","last_peered":"2024-05-09T04:50:58.203005+0000","last_clean":"2024-05-09T04:50:58.203005+0000","last_became_active":"2024-05-09T04:49:34.141004+0000","last_became_peered":"2024-05-09T04:49:34.141004+0000","last_unstale":"2024-05-09T04:50:58.203005+0000","last_undegraded":"2024-05-09T04:50:58.203005+0000","last_fullsized":"2024-05-09T04:50:58.203005+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:09:12.274750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203503+0000","last_change":"2024-05-09T04:49:34.147063+0000","last_active":"2024-05-09T04:50:58.203503+0000","last_peered":"2024-05-09T04:50:58.203503+0000","last_clean":"2024-05-09T04:50:58.203503+0000","last_became_active":"2024-05-09T04:49:34.144934+0000","last_became_peered":"2024-05-09T04:49:34.144934+0000","last_unstale":"2024-05-09T04:50:58.203503+0000","last_undegraded":"2024-05-09T04:50:58.203503+0000","last_fullsized":"2024-05-09T04:50:58.203503+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:47:59.125451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.951922+0000","last_change":"2024-05-09T04:49:34.141070+0000","last_active":"2024-05-09T04:50:57.951922+0000","last_peered":"2024-05-09T04:50:57.951922+0000","last_clean":"2024-05-09T04:50:57.951922+0000","last_became_active":"2024-05-09T04:49:34.140618+0000","last_became_peered":"2024-05-09T04:49:34.140618+0000","last_unstale":"2024-05-09T04:50:57.951922+0000","last_undegraded":"2024-05-09T04:50:57.951922+0000","last_fullsized":"2024-05-09T04:50:57.951922+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:02.435768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.817013+0000","last_change":"2024-05-09T04:49:34.144434+0000","last_active":"2024-05-09T04:50:57.817013+0000","last_peered":"2024-05-09T04:50:57.817013+0000","last_clean":"2024-05-09T04:50:57.817013+0000","last_became_active":"2024-05-09T04:49:34.144052+0000","last_became_peered":"2024-05-09T04:49:34.144052+0000","last_unstale":"2024-05-09T04:50:57.817013+0000","last_undegraded":"2024-05-09T04:50:57.817013+0000","last_fullsized":"2024-05-09T04:50:57.817013+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:05:26.141627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.707290+0000","last_change":"2024-05-09T04:49:34.148681+0000","last_active":"2024-05-09T04:50:57.707290+0000","last_peered":"2024-05-09T04:50:57.707290+0000","last_clean":"2024-05-09T04:50:57.707290+0000","last_became_active":"2024-05-09T04:49:34.145277+0000","last_became_peered":"2024-05-09T04:49:34.145277+0000","last_unstale":"2024-05-09T04:50:57.707290+0000","last_undegraded":"2024-05-09T04:50:57.707290+0000","last_fullsized":"2024-05-09T04:50:57.707290+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:09.546998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700816+0000","last_change":"2024-05-09T04:49:34.153229+0000","last_active":"2024-05-09T04:50:57.700816+0000","last_peered":"2024-05-09T04:50:57.700816+0000","last_clean":"2024-05-09T04:50:57.700816+0000","last_became_active":"2024-05-09T04:49:34.152869+0000","last_became_peered":"2024-05-09T04:49:34.152869+0000","last_unstale":"2024-05-09T04:50:57.700816+0000","last_undegraded":"2024-05-09T04:50:57.700816+0000","last_fullsized":"2024-05-09T04:50:57.700816+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:54.350842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700306+0000","last_change":"2024-05-09T04:49:56.710411+0000","last_active":"2024-05-09T04:50:57.700306+0000","last_peered":"2024-05-09T04:50:57.700306+0000","last_clean":"2024-05-09T04:50:57.700306+0000","last_became_active":"2024-05-09T04:49:56.709903+0000","last_became_peered":"2024-05-09T04:49:56.709903+0000","last_unstale":"2024-05-09T04:50:57.700306+0000","last_undegraded":"2024-05-09T04:50:57.700306+0000","last_fullsized":"2024-05-09T04:50:57.700306+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:53:44.216648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,10,0],"acting":[2,3,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":90,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:36.859856+0000","last_change":"2024-05-09T04:49:30.061766+0000","last_active":"2024-05-09T04:51:36.859856+0000","last_peered":"2024-05-09T04:51:36.859856+0000","last_clean":"2024-05-09T04:51:36.859856+0000","last_became_active":"2024-05-09T04:49:30.060545+0000","last_became_peered":"2024-05-09T04:49:30.060545+0000","last_unstale":"2024-05-09T04:51:36.859856+0000","last_undegraded":"2024-05-09T04:51:36.859856+0000","last_fullsized":"2024-05-09T04:51:36.859856+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:15:03.857107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.706063+0000","last_change":"2024-05-09T04:49:34.140706+0000","last_active":"2024-05-09T04:50:57.706063+0000","last_peered":"2024-05-09T04:50:57.706063+0000","last_clean":"2024-05-09T04:50:57.706063+0000","last_became_active":"2024-05-09T04:49:34.139032+0000","last_became_peered":"2024-05-09T04:49:34.139032+0000","last_unstale":"2024-05-09T04:50:57.706063+0000","last_undegraded":"2024-05-09T04:50:57.706063+0000","last_fullsized":"2024-05-09T04:50:57.706063+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:19.878523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.201633+0000","last_change":"2024-05-09T04:49:47.674254+0000","last_active":"2024-05-09T04:50:58.201633+0000","last_peered":"2024-05-09T04:50:58.201633+0000","last_clean":"2024-05-09T04:50:58.201633+0000","last_became_active":"2024-05-09T04:49:47.673476+0000","last_became_peered":"2024-05-09T04:49:47.673476+0000","last_unstale":"2024-05-09T04:50:58.201633+0000","last_undegraded":"2024-05-09T04:50:58.201633+0000","last_fullsized":"2024-05-09T04:50:58.201633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:41:31.709210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:00.747787+0000","last_change":"2024-05-09T04:49:30.154162+0000","last_active":"2024-05-09T04:51:00.747787+0000","last_peered":"2024-05-09T04:51:00.747787+0000","last_clean":"2024-05-09T04:51:00.747787+0000","last_became_active":"2024-05-09T04:49:30.152869+0000","last_became_peered":"2024-05-09T04:49:30.152869+0000","last_unstale":"2024-05-09T04:51:00.747787+0000","last_undegraded":"2024-05-09T04:51:00.747787+0000","last_fullsized":"2024-05-09T04:51:00.747787+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:43.433033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.204656+0000","last_change":"2024-05-09T04:49:34.157729+0000","last_active":"2024-05-09T04:50:58.204656+0000","last_peered":"2024-05-09T04:50:58.204656+0000","last_clean":"2024-05-09T04:50:58.204656+0000","last_became_active":"2024-05-09T04:49:34.155040+0000","last_became_peered":"2024-05-09T04:49:34.155040+0000","last_unstale":"2024-05-09T04:50:58.204656+0000","last_undegraded":"2024-05-09T04:50:58.204656+0000","last_fullsized":"2024-05-09T04:50:58.204656+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:43:30.104394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.202270+0000","last_change":"2024-05-09T04:49:47.679665+0000","last_active":"2024-05-09T04:50:58.202270+0000","last_peered":"2024-05-09T04:50:58.202270+0000","last_clean":"2024-05-09T04:50:58.202270+0000","last_became_active":"2024-05-09T04:49:47.679257+0000","last_became_peered":"2024-05-09T04:49:47.679257+0000","last_unstale":"2024-05-09T04:50:58.202270+0000","last_undegraded":"2024-05-09T04:50:58.202270+0000","last_fullsized":"2024-05-09T04:50:58.202270+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:53.111205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:03.775933+0000","last_change":"2024-05-09T04:49:30.047386+0000","last_active":"2024-05-09T04:51:03.775933+0000","last_peered":"2024-05-09T04:51:03.775933+0000","last_clean":"2024-05-09T04:51:03.775933+0000","last_became_active":"2024-05-09T04:49:30.046947+0000","last_became_peered":"2024-05-09T04:49:30.046947+0000","last_unstale":"2024-05-09T04:51:03.775933+0000","last_undegraded":"2024-05-09T04:51:03.775933+0000","last_fullsized":"2024-05-09T04:51:03.775933+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:46.861354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.952368+0000","last_change":"2024-05-09T04:49:34.153320+0000","last_active":"2024-05-09T04:50:57.952368+0000","last_peered":"2024-05-09T04:50:57.952368+0000","last_clean":"2024-05-09T04:50:57.952368+0000","last_became_active":"2024-05-09T04:49:34.152854+0000","last_became_peered":"2024-05-09T04:49:34.152854+0000","last_unstale":"2024-05-09T04:50:57.952368+0000","last_undegraded":"2024-05-09T04:50:57.952368+0000","last_fullsized":"2024-05-09T04:50:57.952368+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:40.801340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.202310+0000","last_change":"2024-05-09T04:49:47.668475+0000","last_active":"2024-05-09T04:50:58.202310+0000","last_peered":"2024-05-09T04:50:58.202310+0000","last_clean":"2024-05-09T04:50:58.202310+0000","last_became_active":"2024-05-09T04:49:47.667909+0000","last_became_peered":"2024-05-09T04:49:47.667909+0000","last_unstale":"2024-05-09T04:50:58.202310+0000","last_undegraded":"2024-05-09T04:50:58.202310+0000","last_fullsized":"2024-05-09T04:50:58.202310+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:39:27.687468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:02.769105+0000","last_change":"2024-05-09T04:49:30.048659+0000","last_active":"2024-05-09T04:51:02.769105+0000","last_peered":"2024-05-09T04:51:02.769105+0000","last_clean":"2024-05-09T04:51:02.769105+0000","last_became_active":"2024-05-09T04:49:30.048259+0000","last_became_peered":"2024-05-09T04:49:30.048259+0000","last_unstale":"2024-05-09T04:51:02.769105+0000","last_undegraded":"2024-05-09T04:51:02.769105+0000","last_fullsized":"2024-05-09T04:51:02.769105+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:37:46.012594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.032995+0000","last_change":"2024-05-09T04:49:34.131416+0000","last_active":"2024-05-09T04:50:58.032995+0000","last_peered":"2024-05-09T04:50:58.032995+0000","last_clean":"2024-05-09T04:50:58.032995+0000","last_became_active":"2024-05-09T04:49:34.130985+0000","last_became_peered":"2024-05-09T04:49:34.130985+0000","last_unstale":"2024-05-09T04:50:58.032995+0000","last_undegraded":"2024-05-09T04:50:58.032995+0000","last_fullsized":"2024-05-09T04:50:58.032995+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:02:43.228142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700616+0000","last_change":"2024-05-09T04:49:47.652247+0000","last_active":"2024-05-09T04:50:57.700616+0000","last_peered":"2024-05-09T04:50:57.700616+0000","last_clean":"2024-05-09T04:50:57.700616+0000","last_became_active":"2024-05-09T04:49:47.651747+0000","last_became_peered":"2024-05-09T04:49:47.651747+0000","last_unstale":"2024-05-09T04:50:57.700616+0000","last_undegraded":"2024-05-09T04:50:57.700616+0000","last_fullsized":"2024-05-09T04:50:57.700616+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:45.353027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":83,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:33.825305+0000","last_change":"2024-05-09T04:49:30.047421+0000","last_active":"2024-05-09T04:51:33.825305+0000","last_peered":"2024-05-09T04:51:33.825305+0000","last_clean":"2024-05-09T04:51:33.825305+0000","last_became_active":"2024-05-09T04:49:30.047007+0000","last_became_peered":"2024-05-09T04:49:30.047007+0000","last_unstale":"2024-05-09T04:51:33.825305+0000","last_undegraded":"2024-05-09T04:51:33.825305+0000","last_fullsized":"2024-05-09T04:51:33.825305+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:23.930326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.201477+0000","last_change":"2024-05-09T04:49:34.146605+0000","last_active":"2024-05-09T04:50:58.201477+0000","last_peered":"2024-05-09T04:50:58.201477+0000","last_clean":"2024-05-09T04:50:58.201477+0000","last_became_active":"2024-05-09T04:49:34.146205+0000","last_became_peered":"2024-05-09T04:49:34.146205+0000","last_unstale":"2024-05-09T04:50:58.201477+0000","last_undegraded":"2024-05-09T04:50:58.201477+0000","last_fullsized":"2024-05-09T04:50:58.201477+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:13:52.234951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700662+0000","last_change":"2024-05-09T04:49:47.673652+0000","last_active":"2024-05-09T04:50:57.700662+0000","last_peered":"2024-05-09T04:50:57.700662+0000","last_clean":"2024-05-09T04:50:57.700662+0000","last_became_active":"2024-05-09T04:49:47.673255+0000","last_became_peered":"2024-05-09T04:49:47.673255+0000","last_unstale":"2024-05-09T04:50:57.700662+0000","last_undegraded":"2024-05-09T04:50:57.700662+0000","last_fullsized":"2024-05-09T04:50:57.700662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:12:50.672212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:01.758883+0000","last_change":"2024-05-09T04:49:30.056089+0000","last_active":"2024-05-09T04:51:01.758883+0000","last_peered":"2024-05-09T04:51:01.758883+0000","last_clean":"2024-05-09T04:51:01.758883+0000","last_became_active":"2024-05-09T04:49:30.055635+0000","last_became_peered":"2024-05-09T04:49:30.055635+0000","last_unstale":"2024-05-09T04:51:01.758883+0000","last_undegraded":"2024-05-09T04:51:01.758883+0000","last_fullsized":"2024-05-09T04:51:01.758883+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:35:06.529058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.032803+0000","last_change":"2024-05-09T04:49:34.147628+0000","last_active":"2024-05-09T04:50:58.032803+0000","last_peered":"2024-05-09T04:50:58.032803+0000","last_clean":"2024-05-09T04:50:58.032803+0000","last_became_active":"2024-05-09T04:49:34.147208+0000","last_became_peered":"2024-05-09T04:49:34.147208+0000","last_unstale":"2024-05-09T04:50:58.032803+0000","last_undegraded":"2024-05-09T04:50:58.032803+0000","last_fullsized":"2024-05-09T04:50:58.032803+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:09:22.520468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.816867+0000","last_change":"2024-05-09T04:49:56.758366+0000","last_active":"2024-05-09T04:50:57.816867+0000","last_peered":"2024-05-09T04:50:57.816867+0000","last_clean":"2024-05-09T04:50:57.816867+0000","last_became_active":"2024-05-09T04:49:56.757763+0000","last_became_peered":"2024-05-09T04:49:56.757763+0000","last_unstale":"2024-05-09T04:50:57.816867+0000","last_undegraded":"2024-05-09T04:50:57.816867+0000","last_fullsized":"2024-05-09T04:50:57.816867+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:58.711764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:00.750291+0000","last_change":"2024-05-09T04:49:30.058805+0000","last_active":"2024-05-09T04:51:00.750291+0000","last_peered":"2024-05-09T04:51:00.750291+0000","last_clean":"2024-05-09T04:51:00.750291+0000","last_became_active":"2024-05-09T04:49:30.057278+0000","last_became_peered":"2024-05-09T04:49:30.057278+0000","last_unstale":"2024-05-09T04:51:00.750291+0000","last_undegraded":"2024-05-09T04:51:00.750291+0000","last_fullsized":"2024-05-09T04:51:00.750291+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:34.727781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700817+0000","last_change":"2024-05-09T04:49:34.148753+0000","last_active":"2024-05-09T04:50:57.700817+0000","last_peered":"2024-05-09T04:50:57.700817+0000","last_clean":"2024-05-09T04:50:57.700817+0000","last_became_active":"2024-05-09T04:49:34.148407+0000","last_became_peered":"2024-05-09T04:49:34.148407+0000","last_unstale":"2024-05-09T04:50:57.700817+0000","last_undegraded":"2024-05-09T04:50:57.700817+0000","last_fullsized":"2024-05-09T04:50:57.700817+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:39.602934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.204137+0000","last_change":"2024-05-09T04:49:47.672521+0000","last_active":"2024-05-09T04:50:58.204137+0000","last_peered":"2024-05-09T04:50:58.204137+0000","last_clean":"2024-05-09T04:50:58.204137+0000","last_became_active":"2024-05-09T04:49:47.671353+0000","last_became_peered":"2024-05-09T04:49:47.671353+0000","last_unstale":"2024-05-09T04:50:58.204137+0000","last_undegraded":"2024-05-09T04:50:58.204137+0000","last_fullsized":"2024-05-09T04:50:58.204137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:56:34.832651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:02.767997+0000","last_change":"2024-05-09T04:49:30.048589+0000","last_active":"2024-05-09T04:51:02.767997+0000","last_peered":"2024-05-09T04:51:02.767997+0000","last_clean":"2024-05-09T04:51:02.767997+0000","last_became_active":"2024-05-09T04:49:30.048241+0000","last_became_peered":"2024-05-09T04:49:30.048241+0000","last_unstale":"2024-05-09T04:51:02.767997+0000","last_undegraded":"2024-05-09T04:51:02.767997+0000","last_fullsized":"2024-05-09T04:51:02.767997+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:38:36.204539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.701592+0000","last_change":"2024-05-09T04:49:34.151201+0000","last_active":"2024-05-09T04:50:57.701592+0000","last_peered":"2024-05-09T04:50:57.701592+0000","last_clean":"2024-05-09T04:50:57.701592+0000","last_became_active":"2024-05-09T04:49:34.150505+0000","last_became_peered":"2024-05-09T04:49:34.150505+0000","last_unstale":"2024-05-09T04:50:57.701592+0000","last_undegraded":"2024-05-09T04:50:57.701592+0000","last_fullsized":"2024-05-09T04:50:57.701592+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:40.011516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700293+0000","last_change":"2024-05-09T04:49:47.653827+0000","last_active":"2024-05-09T04:50:57.700293+0000","last_peered":"2024-05-09T04:50:57.700293+0000","last_clean":"2024-05-09T04:50:57.700293+0000","last_became_active":"2024-05-09T04:49:47.652423+0000","last_became_peered":"2024-05-09T04:49:47.652423+0000","last_unstale":"2024-05-09T04:50:57.700293+0000","last_undegraded":"2024-05-09T04:50:57.700293+0000","last_fullsized":"2024-05-09T04:50:57.700293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:07:07.602659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":110,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:34.825858+0000","last_change":"2024-05-09T04:49:30.062849+0000","last_active":"2024-05-09T04:51:34.825858+0000","last_peered":"2024-05-09T04:51:34.825858+0000","last_clean":"2024-05-09T04:51:34.825858+0000","last_became_active":"2024-05-09T04:49:30.062426+0000","last_became_peered":"2024-05-09T04:49:30.062426+0000","last_unstale":"2024-05-09T04:51:34.825858+0000","last_undegraded":"2024-05-09T04:51:34.825858+0000","last_fullsized":"2024-05-09T04:51:34.825858+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:25:11.996208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":64,"num_read_kb":64,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.952785+0000","last_change":"2024-05-09T04:49:34.151145+0000","last_active":"2024-05-09T04:50:57.952785+0000","last_peered":"2024-05-09T04:50:57.952785+0000","last_clean":"2024-05-09T04:50:57.952785+0000","last_became_active":"2024-05-09T04:49:34.150745+0000","last_became_peered":"2024-05-09T04:49:34.150745+0000","last_unstale":"2024-05-09T04:50:57.952785+0000","last_undegraded":"2024-05-09T04:50:57.952785+0000","last_fullsized":"2024-05-09T04:50:57.952785+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:40.647454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700450+0000","last_change":"2024-05-09T04:49:47.672796+0000","last_active":"2024-05-09T04:50:57.700450+0000","last_peered":"2024-05-09T04:50:57.700450+0000","last_clean":"2024-05-09T04:50:57.700450+0000","last_became_active":"2024-05-09T04:49:47.667807+0000","last_became_peered":"2024-05-09T04:49:47.667807+0000","last_unstale":"2024-05-09T04:50:57.700450+0000","last_undegraded":"2024-05-09T04:50:57.700450+0000","last_fullsized":"2024-05-09T04:50:57.700450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:58.542973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:02.767370+0000","last_change":"2024-05-09T04:49:30.047265+0000","last_active":"2024-05-09T04:51:02.767370+0000","last_peered":"2024-05-09T04:51:02.767370+0000","last_clean":"2024-05-09T04:51:02.767370+0000","last_became_active":"2024-05-09T04:49:30.046888+0000","last_became_peered":"2024-05-09T04:49:30.046888+0000","last_unstale":"2024-05-09T04:51:02.767370+0000","last_undegraded":"2024-05-09T04:51:02.767370+0000","last_fullsized":"2024-05-09T04:51:02.767370+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:38:41.878678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.204534+0000","last_change":"2024-05-09T04:49:34.144447+0000","last_active":"2024-05-09T04:50:58.204534+0000","last_peered":"2024-05-09T04:50:58.204534+0000","last_clean":"2024-05-09T04:50:58.204534+0000","last_became_active":"2024-05-09T04:49:34.144073+0000","last_became_peered":"2024-05-09T04:49:34.144073+0000","last_unstale":"2024-05-09T04:50:58.204534+0000","last_undegraded":"2024-05-09T04:50:58.204534+0000","last_fullsized":"2024-05-09T04:50:58.204534+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:53:42.619552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.202416+0000","last_change":"2024-05-09T04:49:47.673364+0000","last_active":"2024-05-09T04:50:58.202416+0000","last_peered":"2024-05-09T04:50:58.202416+0000","last_clean":"2024-05-09T04:50:58.202416+0000","last_became_active":"2024-05-09T04:49:47.672712+0000","last_became_peered":"2024-05-09T04:49:47.672712+0000","last_unstale":"2024-05-09T04:50:58.202416+0000","last_undegraded":"2024-05-09T04:50:58.202416+0000","last_fullsized":"2024-05-09T04:50:58.202416+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:25:57.165777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:01.759351+0000","last_change":"2024-05-09T04:49:30.160924+0000","last_active":"2024-05-09T04:51:01.759351+0000","last_peered":"2024-05-09T04:51:01.759351+0000","last_clean":"2024-05-09T04:51:01.759351+0000","last_became_active":"2024-05-09T04:49:30.160479+0000","last_became_peered":"2024-05-09T04:49:30.160479+0000","last_unstale":"2024-05-09T04:51:01.759351+0000","last_undegraded":"2024-05-09T04:51:01.759351+0000","last_fullsized":"2024-05-09T04:51:01.759351+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:44:57.171441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.952383+0000","last_change":"2024-05-09T04:49:34.136514+0000","last_active":"2024-05-09T04:50:57.952383+0000","last_peered":"2024-05-09T04:50:57.952383+0000","last_clean":"2024-05-09T04:50:57.952383+0000","last_became_active":"2024-05-09T04:49:34.136121+0000","last_became_peered":"2024-05-09T04:49:34.136121+0000","last_unstale":"2024-05-09T04:50:57.952383+0000","last_undegraded":"2024-05-09T04:50:57.952383+0000","last_fullsized":"2024-05-09T04:50:57.952383+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:43:27.781720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700489+0000","last_change":"2024-05-09T04:49:47.673371+0000","last_active":"2024-05-09T04:50:57.700489+0000","last_peered":"2024-05-09T04:50:57.700489+0000","last_clean":"2024-05-09T04:50:57.700489+0000","last_became_active":"2024-05-09T04:49:47.669727+0000","last_became_peered":"2024-05-09T04:49:47.669727+0000","last_unstale":"2024-05-09T04:50:57.700489+0000","last_undegraded":"2024-05-09T04:50:57.700489+0000","last_fullsized":"2024-05-09T04:50:57.700489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:34:45.510934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'35","reported_seq":150,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:34.825815+0000","last_change":"2024-05-09T04:49:30.048359+0000","last_active":"2024-05-09T04:51:34.825815+0000","last_peered":"2024-05-09T04:51:34.825815+0000","last_clean":"2024-05-09T04:51:34.825815+0000","last_became_active":"2024-05-09T04:49:30.047949+0000","last_became_peered":"2024-05-09T04:49:30.047949+0000","last_unstale":"2024-05-09T04:51:34.825815+0000","last_undegraded":"2024-05-09T04:51:34.825815+0000","last_fullsized":"2024-05-09T04:51:34.825815+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:01:10.498997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58837,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":73,"num_read_kb":94,"num_write":37,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.201835+0000","last_change":"2024-05-09T04:49:34.132589+0000","last_active":"2024-05-09T04:50:58.201835+0000","last_peered":"2024-05-09T04:50:58.201835+0000","last_clean":"2024-05-09T04:50:58.201835+0000","last_became_active":"2024-05-09T04:49:34.132227+0000","last_became_peered":"2024-05-09T04:49:34.132227+0000","last_unstale":"2024-05-09T04:50:58.201835+0000","last_undegraded":"2024-05-09T04:50:58.201835+0000","last_fullsized":"2024-05-09T04:50:58.201835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:50.003878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.033454+0000","last_change":"2024-05-09T04:49:47.668236+0000","last_active":"2024-05-09T04:50:58.033454+0000","last_peered":"2024-05-09T04:50:58.033454+0000","last_clean":"2024-05-09T04:50:58.033454+0000","last_became_active":"2024-05-09T04:49:47.667779+0000","last_became_peered":"2024-05-09T04:49:47.667779+0000","last_unstale":"2024-05-09T04:50:58.033454+0000","last_undegraded":"2024-05-09T04:50:58.033454+0000","last_fullsized":"2024-05-09T04:50:58.033454+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:16.177888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:03.776077+0000","last_change":"2024-05-09T04:49:30.046145+0000","last_active":"2024-05-09T04:51:03.776077+0000","last_peered":"2024-05-09T04:51:03.776077+0000","last_clean":"2024-05-09T04:51:03.776077+0000","last_became_active":"2024-05-09T04:49:30.045478+0000","last_became_peered":"2024-05-09T04:49:30.045478+0000","last_unstale":"2024-05-09T04:51:03.776077+0000","last_undegraded":"2024-05-09T04:51:03.776077+0000","last_fullsized":"2024-05-09T04:51:03.776077+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:52:08.313984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.087326+0000","last_change":"2024-05-09T04:49:34.154349+0000","last_active":"2024-05-09T04:50:58.087326+0000","last_peered":"2024-05-09T04:50:58.087326+0000","last_clean":"2024-05-09T04:50:58.087326+0000","last_became_active":"2024-05-09T04:49:34.154011+0000","last_became_peered":"2024-05-09T04:49:34.154011+0000","last_unstale":"2024-05-09T04:50:58.087326+0000","last_undegraded":"2024-05-09T04:50:58.087326+0000","last_fullsized":"2024-05-09T04:50:58.087326+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:50:32.345863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.816645+0000","last_change":"2024-05-09T04:49:47.670170+0000","last_active":"2024-05-09T04:50:57.816645+0000","last_peered":"2024-05-09T04:50:57.816645+0000","last_clean":"2024-05-09T04:50:57.816645+0000","last_became_active":"2024-05-09T04:49:47.669705+0000","last_became_peered":"2024-05-09T04:49:47.669705+0000","last_unstale":"2024-05-09T04:50:57.816645+0000","last_undegraded":"2024-05-09T04:50:57.816645+0000","last_fullsized":"2024-05-09T04:50:57.816645+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:59:27.339811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:02.768392+0000","last_change":"2024-05-09T04:49:30.051159+0000","last_active":"2024-05-09T04:51:02.768392+0000","last_peered":"2024-05-09T04:51:02.768392+0000","last_clean":"2024-05-09T04:51:02.768392+0000","last_became_active":"2024-05-09T04:49:30.050108+0000","last_became_peered":"2024-05-09T04:49:30.050108+0000","last_unstale":"2024-05-09T04:51:02.768392+0000","last_undegraded":"2024-05-09T04:51:02.768392+0000","last_fullsized":"2024-05-09T04:51:02.768392+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:32:56.778567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.202482+0000","last_change":"2024-05-09T04:49:34.162751+0000","last_active":"2024-05-09T04:50:58.202482+0000","last_peered":"2024-05-09T04:50:58.202482+0000","last_clean":"2024-05-09T04:50:58.202482+0000","last_became_active":"2024-05-09T04:49:34.162336+0000","last_became_peered":"2024-05-09T04:49:34.162336+0000","last_unstale":"2024-05-09T04:50:58.202482+0000","last_undegraded":"2024-05-09T04:50:58.202482+0000","last_fullsized":"2024-05-09T04:50:58.202482+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:41.758074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.705812+0000","last_change":"2024-05-09T04:49:47.680955+0000","last_active":"2024-05-09T04:50:57.705812+0000","last_peered":"2024-05-09T04:50:57.705812+0000","last_clean":"2024-05-09T04:50:57.705812+0000","last_became_active":"2024-05-09T04:49:47.680041+0000","last_became_peered":"2024-05-09T04:49:47.680041+0000","last_unstale":"2024-05-09T04:50:57.705812+0000","last_undegraded":"2024-05-09T04:50:57.705812+0000","last_fullsized":"2024-05-09T04:50:57.705812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:46.297611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":83,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:34.857107+0000","last_change":"2024-05-09T04:49:30.045857+0000","last_active":"2024-05-09T04:51:34.857107+0000","last_peered":"2024-05-09T04:51:34.857107+0000","last_clean":"2024-05-09T04:51:34.857107+0000","last_became_active":"2024-05-09T04:49:30.045404+0000","last_became_peered":"2024-05-09T04:49:30.045404+0000","last_unstale":"2024-05-09T04:51:34.857107+0000","last_undegraded":"2024-05-09T04:51:34.857107+0000","last_fullsized":"2024-05-09T04:51:34.857107+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:44:19.473449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203360+0000","last_change":"2024-05-09T04:49:34.131734+0000","last_active":"2024-05-09T04:50:58.203360+0000","last_peered":"2024-05-09T04:50:58.203360+0000","last_clean":"2024-05-09T04:50:58.203360+0000","last_became_active":"2024-05-09T04:49:34.131396+0000","last_became_peered":"2024-05-09T04:49:34.131396+0000","last_unstale":"2024-05-09T04:50:58.203360+0000","last_undegraded":"2024-05-09T04:50:58.203360+0000","last_fullsized":"2024-05-09T04:50:58.203360+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:21.080651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.088080+0000","last_change":"2024-05-09T04:49:56.710429+0000","last_active":"2024-05-09T04:50:58.088080+0000","last_peered":"2024-05-09T04:50:58.088080+0000","last_clean":"2024-05-09T04:50:58.088080+0000","last_became_active":"2024-05-09T04:49:56.708497+0000","last_became_peered":"2024-05-09T04:49:56.708497+0000","last_unstale":"2024-05-09T04:50:58.088080+0000","last_undegraded":"2024-05-09T04:50:58.088080+0000","last_fullsized":"2024-05-09T04:50:58.088080+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:36.281300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:02.770396+0000","last_change":"2024-05-09T04:49:56.718731+0000","last_active":"2024-05-09T04:51:02.770396+0000","last_peered":"2024-05-09T04:51:02.770396+0000","last_clean":"2024-05-09T04:51:02.770396+0000","last_became_active":"2024-05-09T04:49:56.709958+0000","last_became_peered":"2024-05-09T04:49:56.709958+0000","last_unstale":"2024-05-09T04:51:02.770396+0000","last_undegraded":"2024-05-09T04:51:02.770396+0000","last_fullsized":"2024-05-09T04:51:02.770396+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:11.003906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.032698+0000","last_change":"2024-05-09T04:49:34.131705+0000","last_active":"2024-05-09T04:50:58.032698+0000","last_peered":"2024-05-09T04:50:58.032698+0000","last_clean":"2024-05-09T04:50:58.032698+0000","last_became_active":"2024-05-09T04:49:34.131287+0000","last_became_peered":"2024-05-09T04:49:34.131287+0000","last_unstale":"2024-05-09T04:50:58.032698+0000","last_undegraded":"2024-05-09T04:50:58.032698+0000","last_fullsized":"2024-05-09T04:50:58.032698+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:10:24.914832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.032517+0000","last_change":"2024-05-09T04:49:47.663861+0000","last_active":"2024-05-09T04:50:58.032517+0000","last_peered":"2024-05-09T04:50:58.032517+0000","last_clean":"2024-05-09T04:50:58.032517+0000","last_became_active":"2024-05-09T04:49:47.663374+0000","last_became_peered":"2024-05-09T04:49:47.663374+0000","last_unstale":"2024-05-09T04:50:58.032517+0000","last_undegraded":"2024-05-09T04:50:58.032517+0000","last_fullsized":"2024-05-09T04:50:58.032517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:08.063040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:02.741895+0000","last_change":"2024-05-09T04:49:30.044183+0000","last_active":"2024-05-09T04:51:02.741895+0000","last_peered":"2024-05-09T04:51:02.741895+0000","last_clean":"2024-05-09T04:51:02.741895+0000","last_became_active":"2024-05-09T04:49:30.043608+0000","last_became_peered":"2024-05-09T04:49:30.043608+0000","last_unstale":"2024-05-09T04:51:02.741895+0000","last_undegraded":"2024-05-09T04:51:02.741895+0000","last_fullsized":"2024-05-09T04:51:02.741895+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:48:57.788970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.816866+0000","last_change":"2024-05-09T04:49:34.133941+0000","last_active":"2024-05-09T04:50:57.816866+0000","last_peered":"2024-05-09T04:50:57.816866+0000","last_clean":"2024-05-09T04:50:57.816866+0000","last_became_active":"2024-05-09T04:49:34.133550+0000","last_became_peered":"2024-05-09T04:49:34.133550+0000","last_unstale":"2024-05-09T04:50:57.816866+0000","last_undegraded":"2024-05-09T04:50:57.816866+0000","last_fullsized":"2024-05-09T04:50:57.816866+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:45:14.703565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.201372+0000","last_change":"2024-05-09T04:49:47.901085+0000","last_active":"2024-05-09T04:50:58.201372+0000","last_peered":"2024-05-09T04:50:58.201372+0000","last_clean":"2024-05-09T04:50:58.201372+0000","last_became_active":"2024-05-09T04:49:47.900444+0000","last_became_peered":"2024-05-09T04:49:47.900444+0000","last_unstale":"2024-05-09T04:50:58.201372+0000","last_undegraded":"2024-05-09T04:50:58.201372+0000","last_fullsized":"2024-05-09T04:50:58.201372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:44.195683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:01.759239+0000","last_change":"2024-05-09T04:49:30.154060+0000","last_active":"2024-05-09T04:51:01.759239+0000","last_peered":"2024-05-09T04:51:01.759239+0000","last_clean":"2024-05-09T04:51:01.759239+0000","last_became_active":"2024-05-09T04:49:30.152556+0000","last_became_peered":"2024-05-09T04:49:30.152556+0000","last_unstale":"2024-05-09T04:51:01.759239+0000","last_undegraded":"2024-05-09T04:51:01.759239+0000","last_fullsized":"2024-05-09T04:51:01.759239+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:27.056251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203823+0000","last_change":"2024-05-09T04:49:34.153128+0000","last_active":"2024-05-09T04:50:58.203823+0000","last_peered":"2024-05-09T04:50:58.203823+0000","last_clean":"2024-05-09T04:50:58.203823+0000","last_became_active":"2024-05-09T04:49:34.152734+0000","last_became_peered":"2024-05-09T04:49:34.152734+0000","last_unstale":"2024-05-09T04:50:58.203823+0000","last_undegraded":"2024-05-09T04:50:58.203823+0000","last_fullsized":"2024-05-09T04:50:58.203823+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:54:06.920877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.087648+0000","last_change":"2024-05-09T04:49:47.670263+0000","last_active":"2024-05-09T04:50:58.087648+0000","last_peered":"2024-05-09T04:50:58.087648+0000","last_clean":"2024-05-09T04:50:58.087648+0000","last_became_active":"2024-05-09T04:49:47.665743+0000","last_became_peered":"2024-05-09T04:49:47.665743+0000","last_unstale":"2024-05-09T04:50:58.087648+0000","last_undegraded":"2024-05-09T04:50:58.087648+0000","last_fullsized":"2024-05-09T04:50:58.087648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:27.980226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":85,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:37.863575+0000","last_change":"2024-05-09T04:49:30.056119+0000","last_active":"2024-05-09T04:51:37.863575+0000","last_peered":"2024-05-09T04:51:37.863575+0000","last_clean":"2024-05-09T04:51:37.863575+0000","last_became_active":"2024-05-09T04:49:30.055660+0000","last_became_peered":"2024-05-09T04:49:30.055660+0000","last_unstale":"2024-05-09T04:51:37.863575+0000","last_undegraded":"2024-05-09T04:51:37.863575+0000","last_fullsized":"2024-05-09T04:51:37.863575+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:21.513879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.202444+0000","last_change":"2024-05-09T04:49:56.764303+0000","last_active":"2024-05-09T04:50:58.202444+0000","last_peered":"2024-05-09T04:50:58.202444+0000","last_clean":"2024-05-09T04:50:58.202444+0000","last_became_active":"2024-05-09T04:49:56.763854+0000","last_became_peered":"2024-05-09T04:49:56.763854+0000","last_unstale":"2024-05-09T04:50:58.202444+0000","last_undegraded":"2024-05-09T04:50:58.202444+0000","last_fullsized":"2024-05-09T04:50:58.202444+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:06.086602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.088039+0000","last_change":"2024-05-09T04:49:47.900866+0000","last_active":"2024-05-09T04:50:58.088039+0000","last_peered":"2024-05-09T04:50:58.088039+0000","last_clean":"2024-05-09T04:50:58.088039+0000","last_became_active":"2024-05-09T04:49:47.900332+0000","last_became_peered":"2024-05-09T04:49:47.900332+0000","last_unstale":"2024-05-09T04:50:58.088039+0000","last_undegraded":"2024-05-09T04:50:58.088039+0000","last_fullsized":"2024-05-09T04:50:58.088039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:06:41.448262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.032608+0000","last_change":"2024-05-09T04:49:56.702559+0000","last_active":"2024-05-09T04:50:58.032608+0000","last_peered":"2024-05-09T04:50:58.032608+0000","last_clean":"2024-05-09T04:50:58.032608+0000","last_became_active":"2024-05-09T04:49:56.702135+0000","last_became_peered":"2024-05-09T04:49:56.702135+0000","last_unstale":"2024-05-09T04:50:58.032608+0000","last_undegraded":"2024-05-09T04:50:58.032608+0000","last_fullsized":"2024-05-09T04:50:58.032608+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:43:05.848239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":83,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:36.860709+0000","last_change":"2024-05-09T04:49:30.158050+0000","last_active":"2024-05-09T04:51:36.860709+0000","last_peered":"2024-05-09T04:51:36.860709+0000","last_clean":"2024-05-09T04:51:36.860709+0000","last_became_active":"2024-05-09T04:49:30.156719+0000","last_became_peered":"2024-05-09T04:49:30.156719+0000","last_unstale":"2024-05-09T04:51:36.860709+0000","last_undegraded":"2024-05-09T04:51:36.860709+0000","last_fullsized":"2024-05-09T04:51:36.860709+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:47.963111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203562+0000","last_change":"2024-05-09T04:49:47.673594+0000","last_active":"2024-05-09T04:50:58.203562+0000","last_peered":"2024-05-09T04:50:58.203562+0000","last_clean":"2024-05-09T04:50:58.203562+0000","last_became_active":"2024-05-09T04:49:47.672671+0000","last_became_peered":"2024-05-09T04:49:47.672671+0000","last_unstale":"2024-05-09T04:50:58.203562+0000","last_undegraded":"2024-05-09T04:50:58.203562+0000","last_fullsized":"2024-05-09T04:50:58.203562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:58:56.078180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.035054+0000","last_change":"2024-05-09T04:50:58.035054+0000","last_active":"2024-05-09T04:50:58.035054+0000","last_peered":"2024-05-09T04:50:58.035054+0000","last_clean":"2024-05-09T04:50:58.035054+0000","last_became_active":"2024-05-09T04:50:58.034297+0000","last_became_peered":"2024-05-09T04:50:58.034297+0000","last_unstale":"2024-05-09T04:50:58.035054+0000","last_undegraded":"2024-05-09T04:50:58.035054+0000","last_fullsized":"2024-05-09T04:50:58.035054+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:10:56.827950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:03.777300+0000","last_change":"2024-05-09T04:49:30.060224+0000","last_active":"2024-05-09T04:51:03.777300+0000","last_peered":"2024-05-09T04:51:03.777300+0000","last_clean":"2024-05-09T04:51:03.777300+0000","last_became_active":"2024-05-09T04:49:30.059830+0000","last_became_peered":"2024-05-09T04:49:30.059830+0000","last_unstale":"2024-05-09T04:51:03.777300+0000","last_undegraded":"2024-05-09T04:51:03.777300+0000","last_fullsized":"2024-05-09T04:51:03.777300+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:02.925226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":116,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203093+0000","last_change":"2024-05-09T04:47:53.050380+0000","last_active":"2024-05-09T04:50:58.203093+0000","last_peered":"2024-05-09T04:50:58.203093+0000","last_clean":"2024-05-09T04:50:58.203093+0000","last_became_active":"2024-05-09T04:47:53.044058+0000","last_became_peered":"2024-05-09T04:47:53.044058+0000","last_unstale":"2024-05-09T04:50:58.203093+0000","last_undegraded":"2024-05-09T04:50:58.203093+0000","last_fullsized":"2024-05-09T04:50:58.203093+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:43:48.067159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:43:48.067159+0000","last_clean_scrub_stamp":"2024-05-09T04:43:48.067159+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:17:18.714607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.033258+0000","last_change":"2024-05-09T04:49:47.901114+0000","last_active":"2024-05-09T04:50:58.033258+0000","last_peered":"2024-05-09T04:50:58.033258+0000","last_clean":"2024-05-09T04:50:58.033258+0000","last_became_active":"2024-05-09T04:49:47.900450+0000","last_became_peered":"2024-05-09T04:49:47.900450+0000","last_unstale":"2024-05-09T04:50:58.033258+0000","last_undegraded":"2024-05-09T04:50:58.033258+0000","last_fullsized":"2024-05-09T04:50:58.033258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:25:44.517789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.087043+0000","last_change":"2024-05-09T04:49:34.147027+0000","last_active":"2024-05-09T04:50:58.087043+0000","last_peered":"2024-05-09T04:50:58.087043+0000","last_clean":"2024-05-09T04:50:58.087043+0000","last_became_active":"2024-05-09T04:49:34.146672+0000","last_became_peered":"2024-05-09T04:49:34.146672+0000","last_unstale":"2024-05-09T04:50:58.087043+0000","last_undegraded":"2024-05-09T04:50:58.087043+0000","last_fullsized":"2024-05-09T04:50:58.087043+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:15:44.847304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:02.768703+0000","last_change":"2024-05-09T04:49:30.043877+0000","last_active":"2024-05-09T04:51:02.768703+0000","last_peered":"2024-05-09T04:51:02.768703+0000","last_clean":"2024-05-09T04:51:02.768703+0000","last_became_active":"2024-05-09T04:49:30.043513+0000","last_became_peered":"2024-05-09T04:49:30.043513+0000","last_unstale":"2024-05-09T04:51:02.768703+0000","last_undegraded":"2024-05-09T04:51:02.768703+0000","last_fullsized":"2024-05-09T04:51:02.768703+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:16.501692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.816530+0000","last_change":"2024-05-09T04:49:47.668605+0000","last_active":"2024-05-09T04:50:57.816530+0000","last_peered":"2024-05-09T04:50:57.816530+0000","last_clean":"2024-05-09T04:50:57.816530+0000","last_became_active":"2024-05-09T04:49:47.668095+0000","last_became_peered":"2024-05-09T04:49:47.668095+0000","last_unstale":"2024-05-09T04:50:57.816530+0000","last_undegraded":"2024-05-09T04:50:57.816530+0000","last_fullsized":"2024-05-09T04:50:57.816530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:39:30.508618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:03.777515+0000","last_change":"2024-05-09T04:49:30.159554+0000","last_active":"2024-05-09T04:51:03.777515+0000","last_peered":"2024-05-09T04:51:03.777515+0000","last_clean":"2024-05-09T04:51:03.777515+0000","last_became_active":"2024-05-09T04:49:30.159266+0000","last_became_peered":"2024-05-09T04:49:30.159266+0000","last_unstale":"2024-05-09T04:51:03.777515+0000","last_undegraded":"2024-05-09T04:51:03.777515+0000","last_fullsized":"2024-05-09T04:51:03.777515+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:42:04.252862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.088153+0000","last_change":"2024-05-09T04:49:34.150688+0000","last_active":"2024-05-09T04:50:58.088153+0000","last_peered":"2024-05-09T04:50:58.088153+0000","last_clean":"2024-05-09T04:50:58.088153+0000","last_became_active":"2024-05-09T04:49:34.150319+0000","last_became_peered":"2024-05-09T04:49:34.150319+0000","last_unstale":"2024-05-09T04:50:58.088153+0000","last_undegraded":"2024-05-09T04:50:58.088153+0000","last_fullsized":"2024-05-09T04:50:58.088153+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:35:04.832761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700490+0000","last_change":"2024-05-09T04:49:47.902323+0000","last_active":"2024-05-09T04:50:57.700490+0000","last_peered":"2024-05-09T04:50:57.700490+0000","last_clean":"2024-05-09T04:50:57.700490+0000","last_became_active":"2024-05-09T04:49:47.901389+0000","last_became_peered":"2024-05-09T04:49:47.901389+0000","last_unstale":"2024-05-09T04:50:57.700490+0000","last_undegraded":"2024-05-09T04:50:57.700490+0000","last_fullsized":"2024-05-09T04:50:57.700490+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:32.615944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'15","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:30.179228+0000","last_change":"2024-05-09T04:49:30.061172+0000","last_active":"2024-05-09T04:51:30.179228+0000","last_peered":"2024-05-09T04:51:30.179228+0000","last_clean":"2024-05-09T04:51:30.179228+0000","last_became_active":"2024-05-09T04:49:30.060680+0000","last_became_peered":"2024-05-09T04:49:30.060680+0000","last_unstale":"2024-05-09T04:51:30.179228+0000","last_undegraded":"2024-05-09T04:51:30.179228+0000","last_fullsized":"2024-05-09T04:51:30.179228+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:45:44.649124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.704460+0000","last_change":"2024-05-09T04:49:34.130319+0000","last_active":"2024-05-09T04:50:57.704460+0000","last_peered":"2024-05-09T04:50:57.704460+0000","last_clean":"2024-05-09T04:50:57.704460+0000","last_became_active":"2024-05-09T04:49:34.129610+0000","last_became_peered":"2024-05-09T04:49:34.129610+0000","last_unstale":"2024-05-09T04:50:57.704460+0000","last_undegraded":"2024-05-09T04:50:57.704460+0000","last_fullsized":"2024-05-09T04:50:57.704460+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:45.505221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700194+0000","last_change":"2024-05-09T04:49:47.647513+0000","last_active":"2024-05-09T04:50:57.700194+0000","last_peered":"2024-05-09T04:50:57.700194+0000","last_clean":"2024-05-09T04:50:57.700194+0000","last_became_active":"2024-05-09T04:49:47.647069+0000","last_became_peered":"2024-05-09T04:49:47.647069+0000","last_unstale":"2024-05-09T04:50:57.700194+0000","last_undegraded":"2024-05-09T04:50:57.700194+0000","last_fullsized":"2024-05-09T04:50:57.700194+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:05:27.658919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:02.772145+0000","last_change":"2024-05-09T04:49:30.350850+0000","last_active":"2024-05-09T04:51:02.772145+0000","last_peered":"2024-05-09T04:51:02.772145+0000","last_clean":"2024-05-09T04:51:02.772145+0000","last_became_active":"2024-05-09T04:49:30.350431+0000","last_became_peered":"2024-05-09T04:49:30.350431+0000","last_unstale":"2024-05-09T04:51:02.772145+0000","last_undegraded":"2024-05-09T04:51:02.772145+0000","last_fullsized":"2024-05-09T04:51:02.772145+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:53:05.388338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.204464+0000","last_change":"2024-05-09T04:49:34.153630+0000","last_active":"2024-05-09T04:50:58.204464+0000","last_peered":"2024-05-09T04:50:58.204464+0000","last_clean":"2024-05-09T04:50:58.204464+0000","last_became_active":"2024-05-09T04:49:34.152045+0000","last_became_peered":"2024-05-09T04:49:34.152045+0000","last_unstale":"2024-05-09T04:50:58.204464+0000","last_undegraded":"2024-05-09T04:50:58.204464+0000","last_fullsized":"2024-05-09T04:50:58.204464+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:32:35.991731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.202751+0000","last_change":"2024-05-09T04:49:47.901256+0000","last_active":"2024-05-09T04:50:58.202751+0000","last_peered":"2024-05-09T04:50:58.202751+0000","last_clean":"2024-05-09T04:50:58.202751+0000","last_became_active":"2024-05-09T04:49:47.900413+0000","last_became_peered":"2024-05-09T04:49:47.900413+0000","last_unstale":"2024-05-09T04:50:58.202751+0000","last_undegraded":"2024-05-09T04:50:58.202751+0000","last_fullsized":"2024-05-09T04:50:58.202751+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:46:11.845220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:59.742438+0000","last_change":"2024-05-09T04:49:30.152273+0000","last_active":"2024-05-09T04:50:59.742438+0000","last_peered":"2024-05-09T04:50:59.742438+0000","last_clean":"2024-05-09T04:50:59.742438+0000","last_became_active":"2024-05-09T04:49:30.151973+0000","last_became_peered":"2024-05-09T04:49:30.151973+0000","last_unstale":"2024-05-09T04:50:59.742438+0000","last_undegraded":"2024-05-09T04:50:59.742438+0000","last_fullsized":"2024-05-09T04:50:59.742438+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:14:52.912588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.201324+0000","last_change":"2024-05-09T04:49:34.126829+0000","last_active":"2024-05-09T04:50:58.201324+0000","last_peered":"2024-05-09T04:50:58.201324+0000","last_clean":"2024-05-09T04:50:58.201324+0000","last_became_active":"2024-05-09T04:49:34.126430+0000","last_became_peered":"2024-05-09T04:49:34.126430+0000","last_unstale":"2024-05-09T04:50:58.201324+0000","last_undegraded":"2024-05-09T04:50:58.201324+0000","last_fullsized":"2024-05-09T04:50:58.201324+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:38.247148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.705899+0000","last_change":"2024-05-09T04:49:47.675292+0000","last_active":"2024-05-09T04:50:57.705899+0000","last_peered":"2024-05-09T04:50:57.705899+0000","last_clean":"2024-05-09T04:50:57.705899+0000","last_became_active":"2024-05-09T04:49:47.674802+0000","last_became_peered":"2024-05-09T04:49:47.674802+0000","last_unstale":"2024-05-09T04:50:57.705899+0000","last_undegraded":"2024-05-09T04:50:57.705899+0000","last_fullsized":"2024-05-09T04:50:57.705899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:47:21.988054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:02.769658+0000","last_change":"2024-05-09T04:49:30.057854+0000","last_active":"2024-05-09T04:51:02.769658+0000","last_peered":"2024-05-09T04:51:02.769658+0000","last_clean":"2024-05-09T04:51:02.769658+0000","last_became_active":"2024-05-09T04:49:30.056625+0000","last_became_peered":"2024-05-09T04:49:30.056625+0000","last_unstale":"2024-05-09T04:51:02.769658+0000","last_undegraded":"2024-05-09T04:51:02.769658+0000","last_fullsized":"2024-05-09T04:51:02.769658+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:51.272533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.816674+0000","last_change":"2024-05-09T04:49:34.144414+0000","last_active":"2024-05-09T04:50:57.816674+0000","last_peered":"2024-05-09T04:50:57.816674+0000","last_clean":"2024-05-09T04:50:57.816674+0000","last_became_active":"2024-05-09T04:49:34.144064+0000","last_became_peered":"2024-05-09T04:49:34.144064+0000","last_unstale":"2024-05-09T04:50:57.816674+0000","last_undegraded":"2024-05-09T04:50:57.816674+0000","last_fullsized":"2024-05-09T04:50:57.816674+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:44:40.991576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203040+0000","last_change":"2024-05-09T04:49:47.672956+0000","last_active":"2024-05-09T04:50:58.203040+0000","last_peered":"2024-05-09T04:50:58.203040+0000","last_clean":"2024-05-09T04:50:58.203040+0000","last_became_active":"2024-05-09T04:49:47.672513+0000","last_became_peered":"2024-05-09T04:49:47.672513+0000","last_unstale":"2024-05-09T04:50:58.203040+0000","last_undegraded":"2024-05-09T04:50:58.203040+0000","last_fullsized":"2024-05-09T04:50:58.203040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:32.250039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:01.758882+0000","last_change":"2024-05-09T04:49:56.704058+0000","last_active":"2024-05-09T04:51:01.758882+0000","last_peered":"2024-05-09T04:51:01.758882+0000","last_clean":"2024-05-09T04:51:01.758882+0000","last_became_active":"2024-05-09T04:49:56.703589+0000","last_became_peered":"2024-05-09T04:49:56.703589+0000","last_unstale":"2024-05-09T04:51:01.758882+0000","last_undegraded":"2024-05-09T04:51:01.758882+0000","last_fullsized":"2024-05-09T04:51:01.758882+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:54.596200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,1],"acting":[11,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.702507+0000","last_change":"2024-05-09T04:49:34.159321+0000","last_active":"2024-05-09T04:50:57.702507+0000","last_peered":"2024-05-09T04:50:57.702507+0000","last_clean":"2024-05-09T04:50:57.702507+0000","last_became_active":"2024-05-09T04:49:34.157918+0000","last_became_peered":"2024-05-09T04:49:34.157918+0000","last_unstale":"2024-05-09T04:50:57.702507+0000","last_undegraded":"2024-05-09T04:50:57.702507+0000","last_fullsized":"2024-05-09T04:50:57.702507+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:33:59.659905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.202786+0000","last_change":"2024-05-09T04:49:47.900692+0000","last_active":"2024-05-09T04:50:58.202786+0000","last_peered":"2024-05-09T04:50:58.202786+0000","last_clean":"2024-05-09T04:50:58.202786+0000","last_became_active":"2024-05-09T04:49:47.900095+0000","last_became_peered":"2024-05-09T04:49:47.900095+0000","last_unstale":"2024-05-09T04:50:58.202786+0000","last_undegraded":"2024-05-09T04:50:58.202786+0000","last_fullsized":"2024-05-09T04:50:58.202786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:08.632408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:59.742219+0000","last_change":"2024-05-09T04:49:30.048768+0000","last_active":"2024-05-09T04:50:59.742219+0000","last_peered":"2024-05-09T04:50:59.742219+0000","last_clean":"2024-05-09T04:50:59.742219+0000","last_became_active":"2024-05-09T04:49:30.048235+0000","last_became_peered":"2024-05-09T04:49:30.048235+0000","last_unstale":"2024-05-09T04:50:59.742219+0000","last_undegraded":"2024-05-09T04:50:59.742219+0000","last_fullsized":"2024-05-09T04:50:59.742219+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:51:16.988726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.202529+0000","last_change":"2024-05-09T04:49:34.142110+0000","last_active":"2024-05-09T04:50:58.202529+0000","last_peered":"2024-05-09T04:50:58.202529+0000","last_clean":"2024-05-09T04:50:58.202529+0000","last_became_active":"2024-05-09T04:49:34.141709+0000","last_became_peered":"2024-05-09T04:49:34.141709+0000","last_unstale":"2024-05-09T04:50:58.202529+0000","last_undegraded":"2024-05-09T04:50:58.202529+0000","last_fullsized":"2024-05-09T04:50:58.202529+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:16.365915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.952024+0000","last_change":"2024-05-09T04:49:47.667784+0000","last_active":"2024-05-09T04:50:57.952024+0000","last_peered":"2024-05-09T04:50:57.952024+0000","last_clean":"2024-05-09T04:50:57.952024+0000","last_became_active":"2024-05-09T04:49:47.667316+0000","last_became_peered":"2024-05-09T04:49:47.667316+0000","last_unstale":"2024-05-09T04:50:57.952024+0000","last_undegraded":"2024-05-09T04:50:57.952024+0000","last_fullsized":"2024-05-09T04:50:57.952024+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:04:20.233287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":83,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:35.827733+0000","last_change":"2024-05-09T04:49:30.351231+0000","last_active":"2024-05-09T04:51:35.827733+0000","last_peered":"2024-05-09T04:51:35.827733+0000","last_clean":"2024-05-09T04:51:35.827733+0000","last_became_active":"2024-05-09T04:49:30.350817+0000","last_became_peered":"2024-05-09T04:49:30.350817+0000","last_unstale":"2024-05-09T04:51:35.827733+0000","last_undegraded":"2024-05-09T04:51:35.827733+0000","last_fullsized":"2024-05-09T04:51:35.827733+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:51.234300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":36,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203752+0000","last_change":"2024-05-09T04:49:34.129625+0000","last_active":"2024-05-09T04:50:58.203752+0000","last_peered":"2024-05-09T04:50:58.203752+0000","last_clean":"2024-05-09T04:50:58.203752+0000","last_became_active":"2024-05-09T04:49:34.129215+0000","last_became_peered":"2024-05-09T04:49:34.129215+0000","last_unstale":"2024-05-09T04:50:58.203752+0000","last_undegraded":"2024-05-09T04:50:58.203752+0000","last_fullsized":"2024-05-09T04:50:58.203752+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:26.247116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203783+0000","last_change":"2024-05-09T04:49:47.682819+0000","last_active":"2024-05-09T04:50:58.203783+0000","last_peered":"2024-05-09T04:50:58.203783+0000","last_clean":"2024-05-09T04:50:58.203783+0000","last_became_active":"2024-05-09T04:49:47.682331+0000","last_became_peered":"2024-05-09T04:49:47.682331+0000","last_unstale":"2024-05-09T04:50:58.203783+0000","last_undegraded":"2024-05-09T04:50:58.203783+0000","last_fullsized":"2024-05-09T04:50:58.203783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:03.852975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:02.768779+0000","last_change":"2024-05-09T04:49:30.050247+0000","last_active":"2024-05-09T04:51:02.768779+0000","last_peered":"2024-05-09T04:51:02.768779+0000","last_clean":"2024-05-09T04:51:02.768779+0000","last_became_active":"2024-05-09T04:49:30.049835+0000","last_became_peered":"2024-05-09T04:49:30.049835+0000","last_unstale":"2024-05-09T04:51:02.768779+0000","last_undegraded":"2024-05-09T04:51:02.768779+0000","last_fullsized":"2024-05-09T04:51:02.768779+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:52.591186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203356+0000","last_change":"2024-05-09T04:49:34.145406+0000","last_active":"2024-05-09T04:50:58.203356+0000","last_peered":"2024-05-09T04:50:58.203356+0000","last_clean":"2024-05-09T04:50:58.203356+0000","last_became_active":"2024-05-09T04:49:34.144946+0000","last_became_peered":"2024-05-09T04:49:34.144946+0000","last_unstale":"2024-05-09T04:50:58.203356+0000","last_undegraded":"2024-05-09T04:50:58.203356+0000","last_fullsized":"2024-05-09T04:50:58.203356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:43:04.704215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.201310+0000","last_change":"2024-05-09T04:49:56.758844+0000","last_active":"2024-05-09T04:50:58.201310+0000","last_peered":"2024-05-09T04:50:58.201310+0000","last_clean":"2024-05-09T04:50:58.201310+0000","last_became_active":"2024-05-09T04:49:56.758335+0000","last_became_peered":"2024-05-09T04:49:56.758335+0000","last_unstale":"2024-05-09T04:50:58.201310+0000","last_undegraded":"2024-05-09T04:50:58.201310+0000","last_fullsized":"2024-05-09T04:50:58.201310+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:39:33.344225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":57,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:02.770650+0000","last_change":"2024-05-09T04:49:56.766262+0000","last_active":"2024-05-09T04:51:02.770650+0000","last_peered":"2024-05-09T04:51:02.770650+0000","last_clean":"2024-05-09T04:51:02.770650+0000","last_became_active":"2024-05-09T04:49:56.758765+0000","last_became_peered":"2024-05-09T04:49:56.758765+0000","last_unstale":"2024-05-09T04:51:02.770650+0000","last_undegraded":"2024-05-09T04:51:02.770650+0000","last_fullsized":"2024-05-09T04:51:02.770650+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:44:18.271729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203752+0000","last_change":"2024-05-09T04:49:34.134957+0000","last_active":"2024-05-09T04:50:58.203752+0000","last_peered":"2024-05-09T04:50:58.203752+0000","last_clean":"2024-05-09T04:50:58.203752+0000","last_became_active":"2024-05-09T04:49:34.134544+0000","last_became_peered":"2024-05-09T04:49:34.134544+0000","last_unstale":"2024-05-09T04:50:58.203752+0000","last_undegraded":"2024-05-09T04:50:58.203752+0000","last_fullsized":"2024-05-09T04:50:58.203752+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:21:54.855905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.816693+0000","last_change":"2024-05-09T04:49:34.138112+0000","last_active":"2024-05-09T04:50:57.816693+0000","last_peered":"2024-05-09T04:50:57.816693+0000","last_clean":"2024-05-09T04:50:57.816693+0000","last_became_active":"2024-05-09T04:49:34.137716+0000","last_became_peered":"2024-05-09T04:49:34.137716+0000","last_unstale":"2024-05-09T04:50:57.816693+0000","last_undegraded":"2024-05-09T04:50:57.816693+0000","last_fullsized":"2024-05-09T04:50:57.816693+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:17.028262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.204287+0000","last_change":"2024-05-09T04:50:58.204287+0000","last_active":"2024-05-09T04:50:58.204287+0000","last_peered":"2024-05-09T04:50:58.204287+0000","last_clean":"2024-05-09T04:50:58.204287+0000","last_became_active":"2024-05-09T04:50:58.203894+0000","last_became_peered":"2024-05-09T04:50:58.203894+0000","last_unstale":"2024-05-09T04:50:58.204287+0000","last_undegraded":"2024-05-09T04:50:58.204287+0000","last_fullsized":"2024-05-09T04:50:58.204287+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:02.214347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700272+0000","last_change":"2024-05-09T04:49:34.147742+0000","last_active":"2024-05-09T04:50:57.700272+0000","last_peered":"2024-05-09T04:50:57.700272+0000","last_clean":"2024-05-09T04:50:57.700272+0000","last_became_active":"2024-05-09T04:49:34.147301+0000","last_became_peered":"2024-05-09T04:49:34.147301+0000","last_unstale":"2024-05-09T04:50:57.700272+0000","last_undegraded":"2024-05-09T04:50:57.700272+0000","last_fullsized":"2024-05-09T04:50:57.700272+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:36:55.371212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.204297+0000","last_change":"2024-05-09T04:49:34.163599+0000","last_active":"2024-05-09T04:50:58.204297+0000","last_peered":"2024-05-09T04:50:58.204297+0000","last_clean":"2024-05-09T04:50:58.204297+0000","last_became_active":"2024-05-09T04:49:34.162898+0000","last_became_peered":"2024-05-09T04:49:34.162898+0000","last_unstale":"2024-05-09T04:50:58.204297+0000","last_undegraded":"2024-05-09T04:50:58.204297+0000","last_fullsized":"2024-05-09T04:50:58.204297+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:55:57.213901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.088033+0000","last_change":"2024-05-09T04:49:34.151648+0000","last_active":"2024-05-09T04:50:58.088033+0000","last_peered":"2024-05-09T04:50:58.088033+0000","last_clean":"2024-05-09T04:50:58.088033+0000","last_became_active":"2024-05-09T04:49:34.148701+0000","last_became_peered":"2024-05-09T04:49:34.148701+0000","last_unstale":"2024-05-09T04:50:58.088033+0000","last_undegraded":"2024-05-09T04:50:58.088033+0000","last_fullsized":"2024-05-09T04:50:58.088033+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:54:30.122252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.816937+0000","last_change":"2024-05-09T04:49:34.136679+0000","last_active":"2024-05-09T04:50:57.816937+0000","last_peered":"2024-05-09T04:50:57.816937+0000","last_clean":"2024-05-09T04:50:57.816937+0000","last_became_active":"2024-05-09T04:49:34.136306+0000","last_became_peered":"2024-05-09T04:49:34.136306+0000","last_unstale":"2024-05-09T04:50:57.816937+0000","last_undegraded":"2024-05-09T04:50:57.816937+0000","last_fullsized":"2024-05-09T04:50:57.816937+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:51:39.476564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700949+0000","last_change":"2024-05-09T04:49:34.130769+0000","last_active":"2024-05-09T04:50:57.700949+0000","last_peered":"2024-05-09T04:50:57.700949+0000","last_clean":"2024-05-09T04:50:57.700949+0000","last_became_active":"2024-05-09T04:49:34.130362+0000","last_became_peered":"2024-05-09T04:49:34.130362+0000","last_unstale":"2024-05-09T04:50:57.700949+0000","last_undegraded":"2024-05-09T04:50:57.700949+0000","last_fullsized":"2024-05-09T04:50:57.700949+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:54.942259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.704942+0000","last_change":"2024-05-09T04:49:34.138054+0000","last_active":"2024-05-09T04:50:57.704942+0000","last_peered":"2024-05-09T04:50:57.704942+0000","last_clean":"2024-05-09T04:50:57.704942+0000","last_became_active":"2024-05-09T04:49:34.136111+0000","last_became_peered":"2024-05-09T04:49:34.136111+0000","last_unstale":"2024-05-09T04:50:57.704942+0000","last_undegraded":"2024-05-09T04:50:57.704942+0000","last_fullsized":"2024-05-09T04:50:57.704942+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:15:27.870874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.032338+0000","last_change":"2024-05-09T04:49:34.146370+0000","last_active":"2024-05-09T04:50:58.032338+0000","last_peered":"2024-05-09T04:50:58.032338+0000","last_clean":"2024-05-09T04:50:58.032338+0000","last_became_active":"2024-05-09T04:49:34.145948+0000","last_became_peered":"2024-05-09T04:49:34.145948+0000","last_unstale":"2024-05-09T04:50:58.032338+0000","last_undegraded":"2024-05-09T04:50:58.032338+0000","last_fullsized":"2024-05-09T04:50:58.032338+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:14.826168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.701358+0000","last_change":"2024-05-09T04:49:34.138028+0000","last_active":"2024-05-09T04:50:57.701358+0000","last_peered":"2024-05-09T04:50:57.701358+0000","last_clean":"2024-05-09T04:50:57.701358+0000","last_became_active":"2024-05-09T04:49:34.137558+0000","last_became_peered":"2024-05-09T04:49:34.137558+0000","last_unstale":"2024-05-09T04:50:57.701358+0000","last_undegraded":"2024-05-09T04:50:57.701358+0000","last_fullsized":"2024-05-09T04:50:57.701358+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:50:16.258945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.952521+0000","last_change":"2024-05-09T04:49:34.137946+0000","last_active":"2024-05-09T04:50:57.952521+0000","last_peered":"2024-05-09T04:50:57.952521+0000","last_clean":"2024-05-09T04:50:57.952521+0000","last_became_active":"2024-05-09T04:49:34.137498+0000","last_became_peered":"2024-05-09T04:49:34.137498+0000","last_unstale":"2024-05-09T04:50:57.952521+0000","last_undegraded":"2024-05-09T04:50:57.952521+0000","last_fullsized":"2024-05-09T04:50:57.952521+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:36:54.473689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.087619+0000","last_change":"2024-05-09T04:49:34.162103+0000","last_active":"2024-05-09T04:50:58.087619+0000","last_peered":"2024-05-09T04:50:58.087619+0000","last_clean":"2024-05-09T04:50:58.087619+0000","last_became_active":"2024-05-09T04:49:34.161681+0000","last_became_peered":"2024-05-09T04:49:34.161681+0000","last_unstale":"2024-05-09T04:50:58.087619+0000","last_undegraded":"2024-05-09T04:50:58.087619+0000","last_fullsized":"2024-05-09T04:50:58.087619+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:36:10.460401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.816788+0000","last_change":"2024-05-09T04:49:34.136538+0000","last_active":"2024-05-09T04:50:57.816788+0000","last_peered":"2024-05-09T04:50:57.816788+0000","last_clean":"2024-05-09T04:50:57.816788+0000","last_became_active":"2024-05-09T04:49:34.134903+0000","last_became_peered":"2024-05-09T04:49:34.134903+0000","last_unstale":"2024-05-09T04:50:57.816788+0000","last_undegraded":"2024-05-09T04:50:57.816788+0000","last_fullsized":"2024-05-09T04:50:57.816788+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:44:48.673600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66041,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":372,"num_read_kb":258,"num_write":248,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":206653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678057,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":510251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874287,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37012,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708140,"statfs":{"total":95995035648,"available":95957135360,"internally_reserved":0,"allocated":1716224,"data_stored":1156183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037812,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":514165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234044,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":510335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430273,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1642496,"data_stored":1107335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593799,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":505573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574949,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":569705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738473,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":507029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1114112,"data_stored":570316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149785,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":505281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953556,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":510038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790029,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1650688,"data_stored":1102526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T04:51:40.676 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T04:51:41.210 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:51:42.929 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:42 smithi006 ceph-mon[29835]: pgmap v402: 129 pgs: 129 active+clean; 645 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-05-09T04:51:43.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:42 smithi186 ceph-mon[34668]: pgmap v402: 129 pgs: 129 active+clean; 645 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-05-09T04:51:43.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:42 smithi169 ceph-mon[34075]: pgmap v402: 129 pgs: 129 active+clean; 645 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-05-09T04:51:43.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.546+0000 7f1379e00700 1 -- 172.21.15.6:0/567111209 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1374106580 msgr2=0x7f1374106990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:43.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.546+0000 7f1379e00700 1 --2- 172.21.15.6:0/567111209 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1374106580 0x7f1374106990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f135c009990 tx=0x7f135c009ca0 comp rx=0 tx=0).stop 2024-05-09T04:51:43.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.546+0000 7f1379e00700 1 -- 172.21.15.6:0/567111209 shutdown_connections 2024-05-09T04:51:43.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.546+0000 7f1379e00700 1 --2- 172.21.15.6:0/567111209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1374106ed0 0x7f1374113360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:43.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.546+0000 7f1379e00700 1 --2- 172.21.15.6:0/567111209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f137410c5c0 0x7f137410c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:43.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.546+0000 7f1379e00700 1 --2- 172.21.15.6:0/567111209 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1374106580 0x7f1374106990 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:43.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.546+0000 7f1379e00700 1 -- 172.21.15.6:0/567111209 >> 172.21.15.6:0/567111209 conn(0x7f1374102080 msgr2=0x7f13741044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:43.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.547+0000 7f1379e00700 1 -- 172.21.15.6:0/567111209 shutdown_connections 2024-05-09T04:51:43.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.547+0000 7f1379e00700 1 -- 172.21.15.6:0/567111209 wait complete. 2024-05-09T04:51:43.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.548+0000 7f1379e00700 1 Processor -- start 2024-05-09T04:51:43.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.549+0000 7f1379e00700 1 -- start start 2024-05-09T04:51:43.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.549+0000 7f1379e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1374106580 0x7f137419d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:43.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.549+0000 7f1379e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1374106ed0 0x7f137419d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:43.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.549+0000 7f1379e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f137410c5c0 0x7f13741a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:43.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.549+0000 7f1379e00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f13741166f0 con 0x7f1374106ed0 2024-05-09T04:51:43.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.549+0000 7f1379e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1374116570 con 0x7f137410c5c0 2024-05-09T04:51:43.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.549+0000 7f1379e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1374116870 con 0x7f1374106580 2024-05-09T04:51:43.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.550+0000 7f136a000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1374106ed0 0x7f137419d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:43.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.550+0000 7f136a000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1374106ed0 0x7f137419d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33804/0 (socket says 172.21.15.6:33804) 2024-05-09T04:51:43.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.550+0000 7f136a000700 1 -- 172.21.15.6:0/2429434550 learned_addr learned my addr 172.21.15.6:0/2429434550 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:51:43.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.550+0000 7f1372a00700 1 --2- 172.21.15.6:0/2429434550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1374106580 0x7f137419d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:43.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.550+0000 7f1373400700 1 --2- 172.21.15.6:0/2429434550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f137410c5c0 0x7f13741a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:43.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.550+0000 7f136a000700 1 -- 172.21.15.6:0/2429434550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1374106580 msgr2=0x7f137419d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:43.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.550+0000 7f136a000700 1 --2- 172.21.15.6:0/2429434550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1374106580 0x7f137419d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:43.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.550+0000 7f136a000700 1 -- 172.21.15.6:0/2429434550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f137410c5c0 msgr2=0x7f13741a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:43.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.550+0000 7f136a000700 1 --2- 172.21.15.6:0/2429434550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f137410c5c0 0x7f13741a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:43.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.550+0000 7f136a000700 1 -- 172.21.15.6:0/2429434550 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13741a1f30 con 0x7f1374106ed0 2024-05-09T04:51:43.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.550+0000 7f1373400700 1 --2- 172.21.15.6:0/2429434550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f137410c5c0 0x7f13741a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:51:43.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.551+0000 7f136a000700 1 --2- 172.21.15.6:0/2429434550 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1374106ed0 0x7f137419d600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f135c0098e0 tx=0x7f135c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:43.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.551+0000 7f1378a00700 1 -- 172.21.15.6:0/2429434550 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c027070 con 0x7f1374106ed0 2024-05-09T04:51:43.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.551+0000 7f1378a00700 1 -- 172.21.15.6:0/2429434550 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f135c022470 con 0x7f1374106ed0 2024-05-09T04:51:43.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.551+0000 7f1378a00700 1 -- 172.21.15.6:0/2429434550 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c017650 con 0x7f1374106ed0 2024-05-09T04:51:43.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.551+0000 7f1379e00700 1 -- 172.21.15.6:0/2429434550 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13741a21c0 con 0x7f1374106ed0 2024-05-09T04:51:43.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.551+0000 7f1379e00700 1 -- 172.21.15.6:0/2429434550 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13741a2650 con 0x7f1374106ed0 2024-05-09T04:51:43.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.552+0000 7f1379e00700 1 -- 172.21.15.6:0/2429434550 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13740512c0 con 0x7f1374106ed0 2024-05-09T04:51:43.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.555+0000 7f1378a00700 1 -- 172.21.15.6:0/2429434550 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f135c02a070 con 0x7f1374106ed0 2024-05-09T04:51:43.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.558+0000 7f1378a00700 1 --2- 172.21.15.6:0/2429434550 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f13580b2990 0x7f13580b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:43.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.558+0000 7f1378a00700 1 -- 172.21.15.6:0/2429434550 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f135c0e3520 con 0x7f1374106ed0 2024-05-09T04:51:43.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.558+0000 7f1372a00700 1 --2- 172.21.15.6:0/2429434550 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f13580b2990 0x7f13580b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:43.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.559+0000 7f1372a00700 1 --2- 172.21.15.6:0/2429434550 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f13580b2990 0x7f13580b4ea0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f13741079f0 tx=0x7f136400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:43.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.561+0000 7f1378a00700 1 -- 172.21.15.6:0/2429434550 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f135c013070 con 0x7f1374106ed0 2024-05-09T04:51:43.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.796+0000 7f1379e00700 1 -- 172.21.15.6:0/2429434550 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1374063c00 con 0x7f13580b2990 2024-05-09T04:51:43.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.826+0000 7f1378a00700 1 -- 172.21.15.6:0/2429434550 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340405 (secure 0 0 0) 0x7f1374063c00 con 0x7f13580b2990 2024-05-09T04:51:43.833 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:51:43.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.835+0000 7f1379e00700 1 -- 172.21.15.6:0/2429434550 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f13580b2990 msgr2=0x7f13580b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:43.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.835+0000 7f1379e00700 1 --2- 172.21.15.6:0/2429434550 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f13580b2990 0x7f13580b4ea0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f13741079f0 tx=0x7f136400b040 comp rx=0 tx=0).stop 2024-05-09T04:51:43.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.835+0000 7f1379e00700 1 -- 172.21.15.6:0/2429434550 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1374106ed0 msgr2=0x7f137419d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:43.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.835+0000 7f1379e00700 1 --2- 172.21.15.6:0/2429434550 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1374106ed0 0x7f137419d600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f135c0098e0 tx=0x7f135c0057c0 comp rx=0 tx=0).stop 2024-05-09T04:51:43.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.835+0000 7f1379e00700 1 -- 172.21.15.6:0/2429434550 shutdown_connections 2024-05-09T04:51:43.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.835+0000 7f1379e00700 1 --2- 172.21.15.6:0/2429434550 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f13580b2990 0x7f13580b4ea0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:43.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.835+0000 7f1379e00700 1 --2- 172.21.15.6:0/2429434550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f137410c5c0 0x7f13741a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:43.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.835+0000 7f1379e00700 1 --2- 172.21.15.6:0/2429434550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1374106580 0x7f137419d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:43.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.835+0000 7f1379e00700 1 --2- 172.21.15.6:0/2429434550 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1374106ed0 0x7f137419d600 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:43.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.835+0000 7f1379e00700 1 -- 172.21.15.6:0/2429434550 >> 172.21.15.6:0/2429434550 conn(0x7f1374102080 msgr2=0x7f13741048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:43.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.835+0000 7f1379e00700 1 -- 172.21.15.6:0/2429434550 shutdown_connections 2024-05-09T04:51:43.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:43.835+0000 7f1379e00700 1 -- 172.21.15.6:0/2429434550 wait complete. 2024-05-09T04:51:43.838 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-05-09T04:51:44.766 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":403,"stamp":"2024-05-09T04:51:43.596798+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660908,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":494,"num_read_kb":352,"num_write":374,"num_write_kb":2294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":309,"ondisk_log_size":309,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":367456,"kb_used_data":14144,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124574368,"statfs":{"total":1151940427776,"available":1151564152832,"internally_reserved":0,"allocated":14483456,"data_stored":8075278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19137,"internal_metadata":361674047},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8713,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":116,"num_read_kb":69,"num_write":9,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006272"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.817119+0000","last_change":"2024-05-09T04:49:34.144921+0000","last_active":"2024-05-09T04:50:57.817119+0000","last_peered":"2024-05-09T04:50:57.817119+0000","last_clean":"2024-05-09T04:50:57.817119+0000","last_became_active":"2024-05-09T04:49:34.144275+0000","last_became_peered":"2024-05-09T04:49:34.144275+0000","last_unstale":"2024-05-09T04:50:57.817119+0000","last_undegraded":"2024-05-09T04:50:57.817119+0000","last_fullsized":"2024-05-09T04:50:57.817119+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:00:31.377738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.204384+0000","last_change":"2024-05-09T04:49:34.158057+0000","last_active":"2024-05-09T04:50:58.204384+0000","last_peered":"2024-05-09T04:50:58.204384+0000","last_clean":"2024-05-09T04:50:58.204384+0000","last_became_active":"2024-05-09T04:49:34.157129+0000","last_became_peered":"2024-05-09T04:49:34.157129+0000","last_unstale":"2024-05-09T04:50:58.204384+0000","last_undegraded":"2024-05-09T04:50:58.204384+0000","last_fullsized":"2024-05-09T04:50:58.204384+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:49.360309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203901+0000","last_change":"2024-05-09T04:49:34.145519+0000","last_active":"2024-05-09T04:50:58.203901+0000","last_peered":"2024-05-09T04:50:58.203901+0000","last_clean":"2024-05-09T04:50:58.203901+0000","last_became_active":"2024-05-09T04:49:34.144501+0000","last_became_peered":"2024-05-09T04:49:34.144501+0000","last_unstale":"2024-05-09T04:50:58.203901+0000","last_undegraded":"2024-05-09T04:50:58.203901+0000","last_fullsized":"2024-05-09T04:50:58.203901+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:05:53.727520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203281+0000","last_change":"2024-05-09T04:49:34.134383+0000","last_active":"2024-05-09T04:50:58.203281+0000","last_peered":"2024-05-09T04:50:58.203281+0000","last_clean":"2024-05-09T04:50:58.203281+0000","last_became_active":"2024-05-09T04:49:34.134049+0000","last_became_peered":"2024-05-09T04:49:34.134049+0000","last_unstale":"2024-05-09T04:50:58.203281+0000","last_undegraded":"2024-05-09T04:50:58.203281+0000","last_fullsized":"2024-05-09T04:50:58.203281+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:36:02.548188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.201604+0000","last_change":"2024-05-09T04:49:34.157897+0000","last_active":"2024-05-09T04:50:58.201604+0000","last_peered":"2024-05-09T04:50:58.201604+0000","last_clean":"2024-05-09T04:50:58.201604+0000","last_became_active":"2024-05-09T04:49:34.157516+0000","last_became_peered":"2024-05-09T04:49:34.157516+0000","last_unstale":"2024-05-09T04:50:58.201604+0000","last_undegraded":"2024-05-09T04:50:58.201604+0000","last_fullsized":"2024-05-09T04:50:58.201604+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:33:25.444155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.202150+0000","last_change":"2024-05-09T04:49:34.146389+0000","last_active":"2024-05-09T04:50:58.202150+0000","last_peered":"2024-05-09T04:50:58.202150+0000","last_clean":"2024-05-09T04:50:58.202150+0000","last_became_active":"2024-05-09T04:49:34.145994+0000","last_became_peered":"2024-05-09T04:49:34.145994+0000","last_unstale":"2024-05-09T04:50:58.202150+0000","last_undegraded":"2024-05-09T04:50:58.202150+0000","last_fullsized":"2024-05-09T04:50:58.202150+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:59.094826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203055+0000","last_change":"2024-05-09T04:49:34.146428+0000","last_active":"2024-05-09T04:50:58.203055+0000","last_peered":"2024-05-09T04:50:58.203055+0000","last_clean":"2024-05-09T04:50:58.203055+0000","last_became_active":"2024-05-09T04:49:34.146045+0000","last_became_peered":"2024-05-09T04:49:34.146045+0000","last_unstale":"2024-05-09T04:50:58.203055+0000","last_undegraded":"2024-05-09T04:50:58.203055+0000","last_fullsized":"2024-05-09T04:50:58.203055+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:13.129474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700092+0000","last_change":"2024-05-09T04:49:34.135494+0000","last_active":"2024-05-09T04:50:57.700092+0000","last_peered":"2024-05-09T04:50:57.700092+0000","last_clean":"2024-05-09T04:50:57.700092+0000","last_became_active":"2024-05-09T04:49:34.135085+0000","last_became_peered":"2024-05-09T04:49:34.135085+0000","last_unstale":"2024-05-09T04:50:57.700092+0000","last_undegraded":"2024-05-09T04:50:57.700092+0000","last_fullsized":"2024-05-09T04:50:57.700092+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:57.023344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.952831+0000","last_change":"2024-05-09T04:49:34.131353+0000","last_active":"2024-05-09T04:50:57.952831+0000","last_peered":"2024-05-09T04:50:57.952831+0000","last_clean":"2024-05-09T04:50:57.952831+0000","last_became_active":"2024-05-09T04:49:34.130954+0000","last_became_peered":"2024-05-09T04:49:34.130954+0000","last_unstale":"2024-05-09T04:50:57.952831+0000","last_undegraded":"2024-05-09T04:50:57.952831+0000","last_fullsized":"2024-05-09T04:50:57.952831+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:04:46.947861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.204015+0000","last_change":"2024-05-09T04:49:34.134733+0000","last_active":"2024-05-09T04:50:58.204015+0000","last_peered":"2024-05-09T04:50:58.204015+0000","last_clean":"2024-05-09T04:50:58.204015+0000","last_became_active":"2024-05-09T04:49:34.134337+0000","last_became_peered":"2024-05-09T04:49:34.134337+0000","last_unstale":"2024-05-09T04:50:58.204015+0000","last_undegraded":"2024-05-09T04:50:58.204015+0000","last_fullsized":"2024-05-09T04:50:58.204015+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:05.452488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203646+0000","last_change":"2024-05-09T04:49:34.139143+0000","last_active":"2024-05-09T04:50:58.203646+0000","last_peered":"2024-05-09T04:50:58.203646+0000","last_clean":"2024-05-09T04:50:58.203646+0000","last_became_active":"2024-05-09T04:49:34.138749+0000","last_became_peered":"2024-05-09T04:49:34.138749+0000","last_unstale":"2024-05-09T04:50:58.203646+0000","last_undegraded":"2024-05-09T04:50:58.203646+0000","last_fullsized":"2024-05-09T04:50:58.203646+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:11.552063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203348+0000","last_change":"2024-05-09T04:49:34.134687+0000","last_active":"2024-05-09T04:50:58.203348+0000","last_peered":"2024-05-09T04:50:58.203348+0000","last_clean":"2024-05-09T04:50:58.203348+0000","last_became_active":"2024-05-09T04:49:34.134324+0000","last_became_peered":"2024-05-09T04:49:34.134324+0000","last_unstale":"2024-05-09T04:50:58.203348+0000","last_undegraded":"2024-05-09T04:50:58.203348+0000","last_fullsized":"2024-05-09T04:50:58.203348+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:11.667883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.087985+0000","last_change":"2024-05-09T04:49:34.153592+0000","last_active":"2024-05-09T04:50:58.087985+0000","last_peered":"2024-05-09T04:50:58.087985+0000","last_clean":"2024-05-09T04:50:58.087985+0000","last_became_active":"2024-05-09T04:49:34.153184+0000","last_became_peered":"2024-05-09T04:49:34.153184+0000","last_unstale":"2024-05-09T04:50:58.087985+0000","last_undegraded":"2024-05-09T04:50:58.087985+0000","last_fullsized":"2024-05-09T04:50:58.087985+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:36:56.784781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203005+0000","last_change":"2024-05-09T04:49:34.144046+0000","last_active":"2024-05-09T04:50:58.203005+0000","last_peered":"2024-05-09T04:50:58.203005+0000","last_clean":"2024-05-09T04:50:58.203005+0000","last_became_active":"2024-05-09T04:49:34.141004+0000","last_became_peered":"2024-05-09T04:49:34.141004+0000","last_unstale":"2024-05-09T04:50:58.203005+0000","last_undegraded":"2024-05-09T04:50:58.203005+0000","last_fullsized":"2024-05-09T04:50:58.203005+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:09:12.274750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203503+0000","last_change":"2024-05-09T04:49:34.147063+0000","last_active":"2024-05-09T04:50:58.203503+0000","last_peered":"2024-05-09T04:50:58.203503+0000","last_clean":"2024-05-09T04:50:58.203503+0000","last_became_active":"2024-05-09T04:49:34.144934+0000","last_became_peered":"2024-05-09T04:49:34.144934+0000","last_unstale":"2024-05-09T04:50:58.203503+0000","last_undegraded":"2024-05-09T04:50:58.203503+0000","last_fullsized":"2024-05-09T04:50:58.203503+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:47:59.125451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.951922+0000","last_change":"2024-05-09T04:49:34.141070+0000","last_active":"2024-05-09T04:50:57.951922+0000","last_peered":"2024-05-09T04:50:57.951922+0000","last_clean":"2024-05-09T04:50:57.951922+0000","last_became_active":"2024-05-09T04:49:34.140618+0000","last_became_peered":"2024-05-09T04:49:34.140618+0000","last_unstale":"2024-05-09T04:50:57.951922+0000","last_undegraded":"2024-05-09T04:50:57.951922+0000","last_fullsized":"2024-05-09T04:50:57.951922+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:02.435768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.817013+0000","last_change":"2024-05-09T04:49:34.144434+0000","last_active":"2024-05-09T04:50:57.817013+0000","last_peered":"2024-05-09T04:50:57.817013+0000","last_clean":"2024-05-09T04:50:57.817013+0000","last_became_active":"2024-05-09T04:49:34.144052+0000","last_became_peered":"2024-05-09T04:49:34.144052+0000","last_unstale":"2024-05-09T04:50:57.817013+0000","last_undegraded":"2024-05-09T04:50:57.817013+0000","last_fullsized":"2024-05-09T04:50:57.817013+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:05:26.141627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.707290+0000","last_change":"2024-05-09T04:49:34.148681+0000","last_active":"2024-05-09T04:50:57.707290+0000","last_peered":"2024-05-09T04:50:57.707290+0000","last_clean":"2024-05-09T04:50:57.707290+0000","last_became_active":"2024-05-09T04:49:34.145277+0000","last_became_peered":"2024-05-09T04:49:34.145277+0000","last_unstale":"2024-05-09T04:50:57.707290+0000","last_undegraded":"2024-05-09T04:50:57.707290+0000","last_fullsized":"2024-05-09T04:50:57.707290+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:09.546998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700816+0000","last_change":"2024-05-09T04:49:34.153229+0000","last_active":"2024-05-09T04:50:57.700816+0000","last_peered":"2024-05-09T04:50:57.700816+0000","last_clean":"2024-05-09T04:50:57.700816+0000","last_became_active":"2024-05-09T04:49:34.152869+0000","last_became_peered":"2024-05-09T04:49:34.152869+0000","last_unstale":"2024-05-09T04:50:57.700816+0000","last_undegraded":"2024-05-09T04:50:57.700816+0000","last_fullsized":"2024-05-09T04:50:57.700816+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:54.350842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700306+0000","last_change":"2024-05-09T04:49:56.710411+0000","last_active":"2024-05-09T04:50:57.700306+0000","last_peered":"2024-05-09T04:50:57.700306+0000","last_clean":"2024-05-09T04:50:57.700306+0000","last_became_active":"2024-05-09T04:49:56.709903+0000","last_became_peered":"2024-05-09T04:49:56.709903+0000","last_unstale":"2024-05-09T04:50:57.700306+0000","last_undegraded":"2024-05-09T04:50:57.700306+0000","last_fullsized":"2024-05-09T04:50:57.700306+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:53:44.216648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,10,0],"acting":[2,3,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":95,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:40.868759+0000","last_change":"2024-05-09T04:49:30.061766+0000","last_active":"2024-05-09T04:51:40.868759+0000","last_peered":"2024-05-09T04:51:40.868759+0000","last_clean":"2024-05-09T04:51:40.868759+0000","last_became_active":"2024-05-09T04:49:30.060545+0000","last_became_peered":"2024-05-09T04:49:30.060545+0000","last_unstale":"2024-05-09T04:51:40.868759+0000","last_undegraded":"2024-05-09T04:51:40.868759+0000","last_fullsized":"2024-05-09T04:51:40.868759+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:15:03.857107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.706063+0000","last_change":"2024-05-09T04:49:34.140706+0000","last_active":"2024-05-09T04:50:57.706063+0000","last_peered":"2024-05-09T04:50:57.706063+0000","last_clean":"2024-05-09T04:50:57.706063+0000","last_became_active":"2024-05-09T04:49:34.139032+0000","last_became_peered":"2024-05-09T04:49:34.139032+0000","last_unstale":"2024-05-09T04:50:57.706063+0000","last_undegraded":"2024-05-09T04:50:57.706063+0000","last_fullsized":"2024-05-09T04:50:57.706063+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:19.878523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.201633+0000","last_change":"2024-05-09T04:49:47.674254+0000","last_active":"2024-05-09T04:50:58.201633+0000","last_peered":"2024-05-09T04:50:58.201633+0000","last_clean":"2024-05-09T04:50:58.201633+0000","last_became_active":"2024-05-09T04:49:47.673476+0000","last_became_peered":"2024-05-09T04:49:47.673476+0000","last_unstale":"2024-05-09T04:50:58.201633+0000","last_undegraded":"2024-05-09T04:50:58.201633+0000","last_fullsized":"2024-05-09T04:50:58.201633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:41:31.709210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:00.747787+0000","last_change":"2024-05-09T04:49:30.154162+0000","last_active":"2024-05-09T04:51:00.747787+0000","last_peered":"2024-05-09T04:51:00.747787+0000","last_clean":"2024-05-09T04:51:00.747787+0000","last_became_active":"2024-05-09T04:49:30.152869+0000","last_became_peered":"2024-05-09T04:49:30.152869+0000","last_unstale":"2024-05-09T04:51:00.747787+0000","last_undegraded":"2024-05-09T04:51:00.747787+0000","last_fullsized":"2024-05-09T04:51:00.747787+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:43.433033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.204811+0000","last_change":"2024-05-09T04:49:34.157729+0000","last_active":"2024-05-09T04:50:58.204811+0000","last_peered":"2024-05-09T04:50:58.204811+0000","last_clean":"2024-05-09T04:50:58.204811+0000","last_became_active":"2024-05-09T04:49:34.155040+0000","last_became_peered":"2024-05-09T04:49:34.155040+0000","last_unstale":"2024-05-09T04:50:58.204811+0000","last_undegraded":"2024-05-09T04:50:58.204811+0000","last_fullsized":"2024-05-09T04:50:58.204811+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:43:30.104394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.202270+0000","last_change":"2024-05-09T04:49:47.679665+0000","last_active":"2024-05-09T04:50:58.202270+0000","last_peered":"2024-05-09T04:50:58.202270+0000","last_clean":"2024-05-09T04:50:58.202270+0000","last_became_active":"2024-05-09T04:49:47.679257+0000","last_became_peered":"2024-05-09T04:49:47.679257+0000","last_unstale":"2024-05-09T04:50:58.202270+0000","last_undegraded":"2024-05-09T04:50:58.202270+0000","last_fullsized":"2024-05-09T04:50:58.202270+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:53.111205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:03.775933+0000","last_change":"2024-05-09T04:49:30.047386+0000","last_active":"2024-05-09T04:51:03.775933+0000","last_peered":"2024-05-09T04:51:03.775933+0000","last_clean":"2024-05-09T04:51:03.775933+0000","last_became_active":"2024-05-09T04:49:30.046947+0000","last_became_peered":"2024-05-09T04:49:30.046947+0000","last_unstale":"2024-05-09T04:51:03.775933+0000","last_undegraded":"2024-05-09T04:51:03.775933+0000","last_fullsized":"2024-05-09T04:51:03.775933+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:46.861354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.952368+0000","last_change":"2024-05-09T04:49:34.153320+0000","last_active":"2024-05-09T04:50:57.952368+0000","last_peered":"2024-05-09T04:50:57.952368+0000","last_clean":"2024-05-09T04:50:57.952368+0000","last_became_active":"2024-05-09T04:49:34.152854+0000","last_became_peered":"2024-05-09T04:49:34.152854+0000","last_unstale":"2024-05-09T04:50:57.952368+0000","last_undegraded":"2024-05-09T04:50:57.952368+0000","last_fullsized":"2024-05-09T04:50:57.952368+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:40.801340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.202310+0000","last_change":"2024-05-09T04:49:47.668475+0000","last_active":"2024-05-09T04:50:58.202310+0000","last_peered":"2024-05-09T04:50:58.202310+0000","last_clean":"2024-05-09T04:50:58.202310+0000","last_became_active":"2024-05-09T04:49:47.667909+0000","last_became_peered":"2024-05-09T04:49:47.667909+0000","last_unstale":"2024-05-09T04:50:58.202310+0000","last_undegraded":"2024-05-09T04:50:58.202310+0000","last_fullsized":"2024-05-09T04:50:58.202310+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:39:27.687468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:02.769105+0000","last_change":"2024-05-09T04:49:30.048659+0000","last_active":"2024-05-09T04:51:02.769105+0000","last_peered":"2024-05-09T04:51:02.769105+0000","last_clean":"2024-05-09T04:51:02.769105+0000","last_became_active":"2024-05-09T04:49:30.048259+0000","last_became_peered":"2024-05-09T04:49:30.048259+0000","last_unstale":"2024-05-09T04:51:02.769105+0000","last_undegraded":"2024-05-09T04:51:02.769105+0000","last_fullsized":"2024-05-09T04:51:02.769105+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:37:46.012594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.032995+0000","last_change":"2024-05-09T04:49:34.131416+0000","last_active":"2024-05-09T04:50:58.032995+0000","last_peered":"2024-05-09T04:50:58.032995+0000","last_clean":"2024-05-09T04:50:58.032995+0000","last_became_active":"2024-05-09T04:49:34.130985+0000","last_became_peered":"2024-05-09T04:49:34.130985+0000","last_unstale":"2024-05-09T04:50:58.032995+0000","last_undegraded":"2024-05-09T04:50:58.032995+0000","last_fullsized":"2024-05-09T04:50:58.032995+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:02:43.228142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700616+0000","last_change":"2024-05-09T04:49:47.652247+0000","last_active":"2024-05-09T04:50:57.700616+0000","last_peered":"2024-05-09T04:50:57.700616+0000","last_clean":"2024-05-09T04:50:57.700616+0000","last_became_active":"2024-05-09T04:49:47.651747+0000","last_became_peered":"2024-05-09T04:49:47.651747+0000","last_unstale":"2024-05-09T04:50:57.700616+0000","last_undegraded":"2024-05-09T04:50:57.700616+0000","last_fullsized":"2024-05-09T04:50:57.700616+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:45.353027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":88,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:38.833017+0000","last_change":"2024-05-09T04:49:30.047421+0000","last_active":"2024-05-09T04:51:38.833017+0000","last_peered":"2024-05-09T04:51:38.833017+0000","last_clean":"2024-05-09T04:51:38.833017+0000","last_became_active":"2024-05-09T04:49:30.047007+0000","last_became_peered":"2024-05-09T04:49:30.047007+0000","last_unstale":"2024-05-09T04:51:38.833017+0000","last_undegraded":"2024-05-09T04:51:38.833017+0000","last_fullsized":"2024-05-09T04:51:38.833017+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:23.930326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.201477+0000","last_change":"2024-05-09T04:49:34.146605+0000","last_active":"2024-05-09T04:50:58.201477+0000","last_peered":"2024-05-09T04:50:58.201477+0000","last_clean":"2024-05-09T04:50:58.201477+0000","last_became_active":"2024-05-09T04:49:34.146205+0000","last_became_peered":"2024-05-09T04:49:34.146205+0000","last_unstale":"2024-05-09T04:50:58.201477+0000","last_undegraded":"2024-05-09T04:50:58.201477+0000","last_fullsized":"2024-05-09T04:50:58.201477+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:13:52.234951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700662+0000","last_change":"2024-05-09T04:49:47.673652+0000","last_active":"2024-05-09T04:50:57.700662+0000","last_peered":"2024-05-09T04:50:57.700662+0000","last_clean":"2024-05-09T04:50:57.700662+0000","last_became_active":"2024-05-09T04:49:47.673255+0000","last_became_peered":"2024-05-09T04:49:47.673255+0000","last_unstale":"2024-05-09T04:50:57.700662+0000","last_undegraded":"2024-05-09T04:50:57.700662+0000","last_fullsized":"2024-05-09T04:50:57.700662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:12:50.672212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:01.758883+0000","last_change":"2024-05-09T04:49:30.056089+0000","last_active":"2024-05-09T04:51:01.758883+0000","last_peered":"2024-05-09T04:51:01.758883+0000","last_clean":"2024-05-09T04:51:01.758883+0000","last_became_active":"2024-05-09T04:49:30.055635+0000","last_became_peered":"2024-05-09T04:49:30.055635+0000","last_unstale":"2024-05-09T04:51:01.758883+0000","last_undegraded":"2024-05-09T04:51:01.758883+0000","last_fullsized":"2024-05-09T04:51:01.758883+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:35:06.529058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.032803+0000","last_change":"2024-05-09T04:49:34.147628+0000","last_active":"2024-05-09T04:50:58.032803+0000","last_peered":"2024-05-09T04:50:58.032803+0000","last_clean":"2024-05-09T04:50:58.032803+0000","last_became_active":"2024-05-09T04:49:34.147208+0000","last_became_peered":"2024-05-09T04:49:34.147208+0000","last_unstale":"2024-05-09T04:50:58.032803+0000","last_undegraded":"2024-05-09T04:50:58.032803+0000","last_fullsized":"2024-05-09T04:50:58.032803+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:09:22.520468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.816867+0000","last_change":"2024-05-09T04:49:56.758366+0000","last_active":"2024-05-09T04:50:57.816867+0000","last_peered":"2024-05-09T04:50:57.816867+0000","last_clean":"2024-05-09T04:50:57.816867+0000","last_became_active":"2024-05-09T04:49:56.757763+0000","last_became_peered":"2024-05-09T04:49:56.757763+0000","last_unstale":"2024-05-09T04:50:57.816867+0000","last_undegraded":"2024-05-09T04:50:57.816867+0000","last_fullsized":"2024-05-09T04:50:57.816867+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:58.711764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:00.750291+0000","last_change":"2024-05-09T04:49:30.058805+0000","last_active":"2024-05-09T04:51:00.750291+0000","last_peered":"2024-05-09T04:51:00.750291+0000","last_clean":"2024-05-09T04:51:00.750291+0000","last_became_active":"2024-05-09T04:49:30.057278+0000","last_became_peered":"2024-05-09T04:49:30.057278+0000","last_unstale":"2024-05-09T04:51:00.750291+0000","last_undegraded":"2024-05-09T04:51:00.750291+0000","last_fullsized":"2024-05-09T04:51:00.750291+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:34.727781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700817+0000","last_change":"2024-05-09T04:49:34.148753+0000","last_active":"2024-05-09T04:50:57.700817+0000","last_peered":"2024-05-09T04:50:57.700817+0000","last_clean":"2024-05-09T04:50:57.700817+0000","last_became_active":"2024-05-09T04:49:34.148407+0000","last_became_peered":"2024-05-09T04:49:34.148407+0000","last_unstale":"2024-05-09T04:50:57.700817+0000","last_undegraded":"2024-05-09T04:50:57.700817+0000","last_fullsized":"2024-05-09T04:50:57.700817+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:39.602934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.204137+0000","last_change":"2024-05-09T04:49:47.672521+0000","last_active":"2024-05-09T04:50:58.204137+0000","last_peered":"2024-05-09T04:50:58.204137+0000","last_clean":"2024-05-09T04:50:58.204137+0000","last_became_active":"2024-05-09T04:49:47.671353+0000","last_became_peered":"2024-05-09T04:49:47.671353+0000","last_unstale":"2024-05-09T04:50:58.204137+0000","last_undegraded":"2024-05-09T04:50:58.204137+0000","last_fullsized":"2024-05-09T04:50:58.204137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:56:34.832651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:02.767997+0000","last_change":"2024-05-09T04:49:30.048589+0000","last_active":"2024-05-09T04:51:02.767997+0000","last_peered":"2024-05-09T04:51:02.767997+0000","last_clean":"2024-05-09T04:51:02.767997+0000","last_became_active":"2024-05-09T04:49:30.048241+0000","last_became_peered":"2024-05-09T04:49:30.048241+0000","last_unstale":"2024-05-09T04:51:02.767997+0000","last_undegraded":"2024-05-09T04:51:02.767997+0000","last_fullsized":"2024-05-09T04:51:02.767997+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:38:36.204539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.701592+0000","last_change":"2024-05-09T04:49:34.151201+0000","last_active":"2024-05-09T04:50:57.701592+0000","last_peered":"2024-05-09T04:50:57.701592+0000","last_clean":"2024-05-09T04:50:57.701592+0000","last_became_active":"2024-05-09T04:49:34.150505+0000","last_became_peered":"2024-05-09T04:49:34.150505+0000","last_unstale":"2024-05-09T04:50:57.701592+0000","last_undegraded":"2024-05-09T04:50:57.701592+0000","last_fullsized":"2024-05-09T04:50:57.701592+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:40.011516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700293+0000","last_change":"2024-05-09T04:49:47.653827+0000","last_active":"2024-05-09T04:50:57.700293+0000","last_peered":"2024-05-09T04:50:57.700293+0000","last_clean":"2024-05-09T04:50:57.700293+0000","last_became_active":"2024-05-09T04:49:47.652423+0000","last_became_peered":"2024-05-09T04:49:47.652423+0000","last_unstale":"2024-05-09T04:50:57.700293+0000","last_undegraded":"2024-05-09T04:50:57.700293+0000","last_fullsized":"2024-05-09T04:50:57.700293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:07:07.602659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":120,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:39.833780+0000","last_change":"2024-05-09T04:49:30.062849+0000","last_active":"2024-05-09T04:51:39.833780+0000","last_peered":"2024-05-09T04:51:39.833780+0000","last_clean":"2024-05-09T04:51:39.833780+0000","last_became_active":"2024-05-09T04:49:30.062426+0000","last_became_peered":"2024-05-09T04:49:30.062426+0000","last_unstale":"2024-05-09T04:51:39.833780+0000","last_undegraded":"2024-05-09T04:51:39.833780+0000","last_fullsized":"2024-05-09T04:51:39.833780+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:25:11.996208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.952785+0000","last_change":"2024-05-09T04:49:34.151145+0000","last_active":"2024-05-09T04:50:57.952785+0000","last_peered":"2024-05-09T04:50:57.952785+0000","last_clean":"2024-05-09T04:50:57.952785+0000","last_became_active":"2024-05-09T04:49:34.150745+0000","last_became_peered":"2024-05-09T04:49:34.150745+0000","last_unstale":"2024-05-09T04:50:57.952785+0000","last_undegraded":"2024-05-09T04:50:57.952785+0000","last_fullsized":"2024-05-09T04:50:57.952785+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:40.647454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700450+0000","last_change":"2024-05-09T04:49:47.672796+0000","last_active":"2024-05-09T04:50:57.700450+0000","last_peered":"2024-05-09T04:50:57.700450+0000","last_clean":"2024-05-09T04:50:57.700450+0000","last_became_active":"2024-05-09T04:49:47.667807+0000","last_became_peered":"2024-05-09T04:49:47.667807+0000","last_unstale":"2024-05-09T04:50:57.700450+0000","last_undegraded":"2024-05-09T04:50:57.700450+0000","last_fullsized":"2024-05-09T04:50:57.700450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:58.542973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:02.767370+0000","last_change":"2024-05-09T04:49:30.047265+0000","last_active":"2024-05-09T04:51:02.767370+0000","last_peered":"2024-05-09T04:51:02.767370+0000","last_clean":"2024-05-09T04:51:02.767370+0000","last_became_active":"2024-05-09T04:49:30.046888+0000","last_became_peered":"2024-05-09T04:49:30.046888+0000","last_unstale":"2024-05-09T04:51:02.767370+0000","last_undegraded":"2024-05-09T04:51:02.767370+0000","last_fullsized":"2024-05-09T04:51:02.767370+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:38:41.878678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.204534+0000","last_change":"2024-05-09T04:49:34.144447+0000","last_active":"2024-05-09T04:50:58.204534+0000","last_peered":"2024-05-09T04:50:58.204534+0000","last_clean":"2024-05-09T04:50:58.204534+0000","last_became_active":"2024-05-09T04:49:34.144073+0000","last_became_peered":"2024-05-09T04:49:34.144073+0000","last_unstale":"2024-05-09T04:50:58.204534+0000","last_undegraded":"2024-05-09T04:50:58.204534+0000","last_fullsized":"2024-05-09T04:50:58.204534+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:53:42.619552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.202416+0000","last_change":"2024-05-09T04:49:47.673364+0000","last_active":"2024-05-09T04:50:58.202416+0000","last_peered":"2024-05-09T04:50:58.202416+0000","last_clean":"2024-05-09T04:50:58.202416+0000","last_became_active":"2024-05-09T04:49:47.672712+0000","last_became_peered":"2024-05-09T04:49:47.672712+0000","last_unstale":"2024-05-09T04:50:58.202416+0000","last_undegraded":"2024-05-09T04:50:58.202416+0000","last_fullsized":"2024-05-09T04:50:58.202416+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:25:57.165777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:01.759351+0000","last_change":"2024-05-09T04:49:30.160924+0000","last_active":"2024-05-09T04:51:01.759351+0000","last_peered":"2024-05-09T04:51:01.759351+0000","last_clean":"2024-05-09T04:51:01.759351+0000","last_became_active":"2024-05-09T04:49:30.160479+0000","last_became_peered":"2024-05-09T04:49:30.160479+0000","last_unstale":"2024-05-09T04:51:01.759351+0000","last_undegraded":"2024-05-09T04:51:01.759351+0000","last_fullsized":"2024-05-09T04:51:01.759351+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:44:57.171441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.952383+0000","last_change":"2024-05-09T04:49:34.136514+0000","last_active":"2024-05-09T04:50:57.952383+0000","last_peered":"2024-05-09T04:50:57.952383+0000","last_clean":"2024-05-09T04:50:57.952383+0000","last_became_active":"2024-05-09T04:49:34.136121+0000","last_became_peered":"2024-05-09T04:49:34.136121+0000","last_unstale":"2024-05-09T04:50:57.952383+0000","last_undegraded":"2024-05-09T04:50:57.952383+0000","last_fullsized":"2024-05-09T04:50:57.952383+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:43:27.781720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700489+0000","last_change":"2024-05-09T04:49:47.673371+0000","last_active":"2024-05-09T04:50:57.700489+0000","last_peered":"2024-05-09T04:50:57.700489+0000","last_clean":"2024-05-09T04:50:57.700489+0000","last_became_active":"2024-05-09T04:49:47.669727+0000","last_became_peered":"2024-05-09T04:49:47.669727+0000","last_unstale":"2024-05-09T04:50:57.700489+0000","last_undegraded":"2024-05-09T04:50:57.700489+0000","last_fullsized":"2024-05-09T04:50:57.700489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:34:45.510934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'36","reported_seq":162,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:39.833792+0000","last_change":"2024-05-09T04:49:30.048359+0000","last_active":"2024-05-09T04:51:39.833792+0000","last_peered":"2024-05-09T04:51:39.833792+0000","last_clean":"2024-05-09T04:51:39.833792+0000","last_became_active":"2024-05-09T04:49:30.047949+0000","last_became_peered":"2024-05-09T04:49:30.047949+0000","last_unstale":"2024-05-09T04:51:39.833792+0000","last_undegraded":"2024-05-09T04:51:39.833792+0000","last_fullsized":"2024-05-09T04:51:39.833792+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:01:10.498997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63102,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":84,"num_read_kb":104,"num_write":38,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.201835+0000","last_change":"2024-05-09T04:49:34.132589+0000","last_active":"2024-05-09T04:50:58.201835+0000","last_peered":"2024-05-09T04:50:58.201835+0000","last_clean":"2024-05-09T04:50:58.201835+0000","last_became_active":"2024-05-09T04:49:34.132227+0000","last_became_peered":"2024-05-09T04:49:34.132227+0000","last_unstale":"2024-05-09T04:50:58.201835+0000","last_undegraded":"2024-05-09T04:50:58.201835+0000","last_fullsized":"2024-05-09T04:50:58.201835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:50.003878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.033454+0000","last_change":"2024-05-09T04:49:47.668236+0000","last_active":"2024-05-09T04:50:58.033454+0000","last_peered":"2024-05-09T04:50:58.033454+0000","last_clean":"2024-05-09T04:50:58.033454+0000","last_became_active":"2024-05-09T04:49:47.667779+0000","last_became_peered":"2024-05-09T04:49:47.667779+0000","last_unstale":"2024-05-09T04:50:58.033454+0000","last_undegraded":"2024-05-09T04:50:58.033454+0000","last_fullsized":"2024-05-09T04:50:58.033454+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:16.177888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:03.776077+0000","last_change":"2024-05-09T04:49:30.046145+0000","last_active":"2024-05-09T04:51:03.776077+0000","last_peered":"2024-05-09T04:51:03.776077+0000","last_clean":"2024-05-09T04:51:03.776077+0000","last_became_active":"2024-05-09T04:49:30.045478+0000","last_became_peered":"2024-05-09T04:49:30.045478+0000","last_unstale":"2024-05-09T04:51:03.776077+0000","last_undegraded":"2024-05-09T04:51:03.776077+0000","last_fullsized":"2024-05-09T04:51:03.776077+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:52:08.313984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.087326+0000","last_change":"2024-05-09T04:49:34.154349+0000","last_active":"2024-05-09T04:50:58.087326+0000","last_peered":"2024-05-09T04:50:58.087326+0000","last_clean":"2024-05-09T04:50:58.087326+0000","last_became_active":"2024-05-09T04:49:34.154011+0000","last_became_peered":"2024-05-09T04:49:34.154011+0000","last_unstale":"2024-05-09T04:50:58.087326+0000","last_undegraded":"2024-05-09T04:50:58.087326+0000","last_fullsized":"2024-05-09T04:50:58.087326+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:50:32.345863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.816645+0000","last_change":"2024-05-09T04:49:47.670170+0000","last_active":"2024-05-09T04:50:57.816645+0000","last_peered":"2024-05-09T04:50:57.816645+0000","last_clean":"2024-05-09T04:50:57.816645+0000","last_became_active":"2024-05-09T04:49:47.669705+0000","last_became_peered":"2024-05-09T04:49:47.669705+0000","last_unstale":"2024-05-09T04:50:57.816645+0000","last_undegraded":"2024-05-09T04:50:57.816645+0000","last_fullsized":"2024-05-09T04:50:57.816645+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:59:27.339811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:02.768392+0000","last_change":"2024-05-09T04:49:30.051159+0000","last_active":"2024-05-09T04:51:02.768392+0000","last_peered":"2024-05-09T04:51:02.768392+0000","last_clean":"2024-05-09T04:51:02.768392+0000","last_became_active":"2024-05-09T04:49:30.050108+0000","last_became_peered":"2024-05-09T04:49:30.050108+0000","last_unstale":"2024-05-09T04:51:02.768392+0000","last_undegraded":"2024-05-09T04:51:02.768392+0000","last_fullsized":"2024-05-09T04:51:02.768392+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:32:56.778567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.202482+0000","last_change":"2024-05-09T04:49:34.162751+0000","last_active":"2024-05-09T04:50:58.202482+0000","last_peered":"2024-05-09T04:50:58.202482+0000","last_clean":"2024-05-09T04:50:58.202482+0000","last_became_active":"2024-05-09T04:49:34.162336+0000","last_became_peered":"2024-05-09T04:49:34.162336+0000","last_unstale":"2024-05-09T04:50:58.202482+0000","last_undegraded":"2024-05-09T04:50:58.202482+0000","last_fullsized":"2024-05-09T04:50:58.202482+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:41.758074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.705812+0000","last_change":"2024-05-09T04:49:47.680955+0000","last_active":"2024-05-09T04:50:57.705812+0000","last_peered":"2024-05-09T04:50:57.705812+0000","last_clean":"2024-05-09T04:50:57.705812+0000","last_became_active":"2024-05-09T04:49:47.680041+0000","last_became_peered":"2024-05-09T04:49:47.680041+0000","last_unstale":"2024-05-09T04:50:57.705812+0000","last_undegraded":"2024-05-09T04:50:57.705812+0000","last_fullsized":"2024-05-09T04:50:57.705812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:46.297611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":88,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:39.867558+0000","last_change":"2024-05-09T04:49:30.045857+0000","last_active":"2024-05-09T04:51:39.867558+0000","last_peered":"2024-05-09T04:51:39.867558+0000","last_clean":"2024-05-09T04:51:39.867558+0000","last_became_active":"2024-05-09T04:49:30.045404+0000","last_became_peered":"2024-05-09T04:49:30.045404+0000","last_unstale":"2024-05-09T04:51:39.867558+0000","last_undegraded":"2024-05-09T04:51:39.867558+0000","last_fullsized":"2024-05-09T04:51:39.867558+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:44:19.473449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":42,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203360+0000","last_change":"2024-05-09T04:49:34.131734+0000","last_active":"2024-05-09T04:50:58.203360+0000","last_peered":"2024-05-09T04:50:58.203360+0000","last_clean":"2024-05-09T04:50:58.203360+0000","last_became_active":"2024-05-09T04:49:34.131396+0000","last_became_peered":"2024-05-09T04:49:34.131396+0000","last_unstale":"2024-05-09T04:50:58.203360+0000","last_undegraded":"2024-05-09T04:50:58.203360+0000","last_fullsized":"2024-05-09T04:50:58.203360+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:21.080651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.088080+0000","last_change":"2024-05-09T04:49:56.710429+0000","last_active":"2024-05-09T04:50:58.088080+0000","last_peered":"2024-05-09T04:50:58.088080+0000","last_clean":"2024-05-09T04:50:58.088080+0000","last_became_active":"2024-05-09T04:49:56.708497+0000","last_became_peered":"2024-05-09T04:49:56.708497+0000","last_unstale":"2024-05-09T04:50:58.088080+0000","last_undegraded":"2024-05-09T04:50:58.088080+0000","last_fullsized":"2024-05-09T04:50:58.088080+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:36.281300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:02.770396+0000","last_change":"2024-05-09T04:49:56.718731+0000","last_active":"2024-05-09T04:51:02.770396+0000","last_peered":"2024-05-09T04:51:02.770396+0000","last_clean":"2024-05-09T04:51:02.770396+0000","last_became_active":"2024-05-09T04:49:56.709958+0000","last_became_peered":"2024-05-09T04:49:56.709958+0000","last_unstale":"2024-05-09T04:51:02.770396+0000","last_undegraded":"2024-05-09T04:51:02.770396+0000","last_fullsized":"2024-05-09T04:51:02.770396+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:11.003906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.032698+0000","last_change":"2024-05-09T04:49:34.131705+0000","last_active":"2024-05-09T04:50:58.032698+0000","last_peered":"2024-05-09T04:50:58.032698+0000","last_clean":"2024-05-09T04:50:58.032698+0000","last_became_active":"2024-05-09T04:49:34.131287+0000","last_became_peered":"2024-05-09T04:49:34.131287+0000","last_unstale":"2024-05-09T04:50:58.032698+0000","last_undegraded":"2024-05-09T04:50:58.032698+0000","last_fullsized":"2024-05-09T04:50:58.032698+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:10:24.914832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.032517+0000","last_change":"2024-05-09T04:49:47.663861+0000","last_active":"2024-05-09T04:50:58.032517+0000","last_peered":"2024-05-09T04:50:58.032517+0000","last_clean":"2024-05-09T04:50:58.032517+0000","last_became_active":"2024-05-09T04:49:47.663374+0000","last_became_peered":"2024-05-09T04:49:47.663374+0000","last_unstale":"2024-05-09T04:50:58.032517+0000","last_undegraded":"2024-05-09T04:50:58.032517+0000","last_fullsized":"2024-05-09T04:50:58.032517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:08.063040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:02.741895+0000","last_change":"2024-05-09T04:49:30.044183+0000","last_active":"2024-05-09T04:51:02.741895+0000","last_peered":"2024-05-09T04:51:02.741895+0000","last_clean":"2024-05-09T04:51:02.741895+0000","last_became_active":"2024-05-09T04:49:30.043608+0000","last_became_peered":"2024-05-09T04:49:30.043608+0000","last_unstale":"2024-05-09T04:51:02.741895+0000","last_undegraded":"2024-05-09T04:51:02.741895+0000","last_fullsized":"2024-05-09T04:51:02.741895+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:48:57.788970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.816866+0000","last_change":"2024-05-09T04:49:34.133941+0000","last_active":"2024-05-09T04:50:57.816866+0000","last_peered":"2024-05-09T04:50:57.816866+0000","last_clean":"2024-05-09T04:50:57.816866+0000","last_became_active":"2024-05-09T04:49:34.133550+0000","last_became_peered":"2024-05-09T04:49:34.133550+0000","last_unstale":"2024-05-09T04:50:57.816866+0000","last_undegraded":"2024-05-09T04:50:57.816866+0000","last_fullsized":"2024-05-09T04:50:57.816866+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:45:14.703565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.201372+0000","last_change":"2024-05-09T04:49:47.901085+0000","last_active":"2024-05-09T04:50:58.201372+0000","last_peered":"2024-05-09T04:50:58.201372+0000","last_clean":"2024-05-09T04:50:58.201372+0000","last_became_active":"2024-05-09T04:49:47.900444+0000","last_became_peered":"2024-05-09T04:49:47.900444+0000","last_unstale":"2024-05-09T04:50:58.201372+0000","last_undegraded":"2024-05-09T04:50:58.201372+0000","last_fullsized":"2024-05-09T04:50:58.201372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:44.195683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:01.759239+0000","last_change":"2024-05-09T04:49:30.154060+0000","last_active":"2024-05-09T04:51:01.759239+0000","last_peered":"2024-05-09T04:51:01.759239+0000","last_clean":"2024-05-09T04:51:01.759239+0000","last_became_active":"2024-05-09T04:49:30.152556+0000","last_became_peered":"2024-05-09T04:49:30.152556+0000","last_unstale":"2024-05-09T04:51:01.759239+0000","last_undegraded":"2024-05-09T04:51:01.759239+0000","last_fullsized":"2024-05-09T04:51:01.759239+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:27.056251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203888+0000","last_change":"2024-05-09T04:49:34.153128+0000","last_active":"2024-05-09T04:50:58.203888+0000","last_peered":"2024-05-09T04:50:58.203888+0000","last_clean":"2024-05-09T04:50:58.203888+0000","last_became_active":"2024-05-09T04:49:34.152734+0000","last_became_peered":"2024-05-09T04:49:34.152734+0000","last_unstale":"2024-05-09T04:50:58.203888+0000","last_undegraded":"2024-05-09T04:50:58.203888+0000","last_fullsized":"2024-05-09T04:50:58.203888+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:54:06.920877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.087648+0000","last_change":"2024-05-09T04:49:47.670263+0000","last_active":"2024-05-09T04:50:58.087648+0000","last_peered":"2024-05-09T04:50:58.087648+0000","last_clean":"2024-05-09T04:50:58.087648+0000","last_became_active":"2024-05-09T04:49:47.665743+0000","last_became_peered":"2024-05-09T04:49:47.665743+0000","last_unstale":"2024-05-09T04:50:58.087648+0000","last_undegraded":"2024-05-09T04:50:58.087648+0000","last_fullsized":"2024-05-09T04:50:58.087648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:27.980226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":85,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:37.863575+0000","last_change":"2024-05-09T04:49:30.056119+0000","last_active":"2024-05-09T04:51:37.863575+0000","last_peered":"2024-05-09T04:51:37.863575+0000","last_clean":"2024-05-09T04:51:37.863575+0000","last_became_active":"2024-05-09T04:49:30.055660+0000","last_became_peered":"2024-05-09T04:49:30.055660+0000","last_unstale":"2024-05-09T04:51:37.863575+0000","last_undegraded":"2024-05-09T04:51:37.863575+0000","last_fullsized":"2024-05-09T04:51:37.863575+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:21.513879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.202492+0000","last_change":"2024-05-09T04:49:56.764303+0000","last_active":"2024-05-09T04:50:58.202492+0000","last_peered":"2024-05-09T04:50:58.202492+0000","last_clean":"2024-05-09T04:50:58.202492+0000","last_became_active":"2024-05-09T04:49:56.763854+0000","last_became_peered":"2024-05-09T04:49:56.763854+0000","last_unstale":"2024-05-09T04:50:58.202492+0000","last_undegraded":"2024-05-09T04:50:58.202492+0000","last_fullsized":"2024-05-09T04:50:58.202492+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:06.086602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.088039+0000","last_change":"2024-05-09T04:49:47.900866+0000","last_active":"2024-05-09T04:50:58.088039+0000","last_peered":"2024-05-09T04:50:58.088039+0000","last_clean":"2024-05-09T04:50:58.088039+0000","last_became_active":"2024-05-09T04:49:47.900332+0000","last_became_peered":"2024-05-09T04:49:47.900332+0000","last_unstale":"2024-05-09T04:50:58.088039+0000","last_undegraded":"2024-05-09T04:50:58.088039+0000","last_fullsized":"2024-05-09T04:50:58.088039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:06:41.448262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.032608+0000","last_change":"2024-05-09T04:49:56.702559+0000","last_active":"2024-05-09T04:50:58.032608+0000","last_peered":"2024-05-09T04:50:58.032608+0000","last_clean":"2024-05-09T04:50:58.032608+0000","last_became_active":"2024-05-09T04:49:56.702135+0000","last_became_peered":"2024-05-09T04:49:56.702135+0000","last_unstale":"2024-05-09T04:50:58.032608+0000","last_undegraded":"2024-05-09T04:50:58.032608+0000","last_fullsized":"2024-05-09T04:50:58.032608+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:43:05.848239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":88,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:40.868598+0000","last_change":"2024-05-09T04:49:30.158050+0000","last_active":"2024-05-09T04:51:40.868598+0000","last_peered":"2024-05-09T04:51:40.868598+0000","last_clean":"2024-05-09T04:51:40.868598+0000","last_became_active":"2024-05-09T04:49:30.156719+0000","last_became_peered":"2024-05-09T04:49:30.156719+0000","last_unstale":"2024-05-09T04:51:40.868598+0000","last_undegraded":"2024-05-09T04:51:40.868598+0000","last_fullsized":"2024-05-09T04:51:40.868598+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:47.963111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203562+0000","last_change":"2024-05-09T04:49:47.673594+0000","last_active":"2024-05-09T04:50:58.203562+0000","last_peered":"2024-05-09T04:50:58.203562+0000","last_clean":"2024-05-09T04:50:58.203562+0000","last_became_active":"2024-05-09T04:49:47.672671+0000","last_became_peered":"2024-05-09T04:49:47.672671+0000","last_unstale":"2024-05-09T04:50:58.203562+0000","last_undegraded":"2024-05-09T04:50:58.203562+0000","last_fullsized":"2024-05-09T04:50:58.203562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:58:56.078180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.035054+0000","last_change":"2024-05-09T04:50:58.035054+0000","last_active":"2024-05-09T04:50:58.035054+0000","last_peered":"2024-05-09T04:50:58.035054+0000","last_clean":"2024-05-09T04:50:58.035054+0000","last_became_active":"2024-05-09T04:50:58.034297+0000","last_became_peered":"2024-05-09T04:50:58.034297+0000","last_unstale":"2024-05-09T04:50:58.035054+0000","last_undegraded":"2024-05-09T04:50:58.035054+0000","last_fullsized":"2024-05-09T04:50:58.035054+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:10:56.827950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:03.777300+0000","last_change":"2024-05-09T04:49:30.060224+0000","last_active":"2024-05-09T04:51:03.777300+0000","last_peered":"2024-05-09T04:51:03.777300+0000","last_clean":"2024-05-09T04:51:03.777300+0000","last_became_active":"2024-05-09T04:49:30.059830+0000","last_became_peered":"2024-05-09T04:49:30.059830+0000","last_unstale":"2024-05-09T04:51:03.777300+0000","last_undegraded":"2024-05-09T04:51:03.777300+0000","last_fullsized":"2024-05-09T04:51:03.777300+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:02.925226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":116,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203093+0000","last_change":"2024-05-09T04:47:53.050380+0000","last_active":"2024-05-09T04:50:58.203093+0000","last_peered":"2024-05-09T04:50:58.203093+0000","last_clean":"2024-05-09T04:50:58.203093+0000","last_became_active":"2024-05-09T04:47:53.044058+0000","last_became_peered":"2024-05-09T04:47:53.044058+0000","last_unstale":"2024-05-09T04:50:58.203093+0000","last_undegraded":"2024-05-09T04:50:58.203093+0000","last_fullsized":"2024-05-09T04:50:58.203093+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:43:48.067159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:43:48.067159+0000","last_clean_scrub_stamp":"2024-05-09T04:43:48.067159+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:17:18.714607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.033258+0000","last_change":"2024-05-09T04:49:47.901114+0000","last_active":"2024-05-09T04:50:58.033258+0000","last_peered":"2024-05-09T04:50:58.033258+0000","last_clean":"2024-05-09T04:50:58.033258+0000","last_became_active":"2024-05-09T04:49:47.900450+0000","last_became_peered":"2024-05-09T04:49:47.900450+0000","last_unstale":"2024-05-09T04:50:58.033258+0000","last_undegraded":"2024-05-09T04:50:58.033258+0000","last_fullsized":"2024-05-09T04:50:58.033258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:25:44.517789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.087115+0000","last_change":"2024-05-09T04:49:34.147027+0000","last_active":"2024-05-09T04:50:58.087115+0000","last_peered":"2024-05-09T04:50:58.087115+0000","last_clean":"2024-05-09T04:50:58.087115+0000","last_became_active":"2024-05-09T04:49:34.146672+0000","last_became_peered":"2024-05-09T04:49:34.146672+0000","last_unstale":"2024-05-09T04:50:58.087115+0000","last_undegraded":"2024-05-09T04:50:58.087115+0000","last_fullsized":"2024-05-09T04:50:58.087115+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:15:44.847304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:02.768703+0000","last_change":"2024-05-09T04:49:30.043877+0000","last_active":"2024-05-09T04:51:02.768703+0000","last_peered":"2024-05-09T04:51:02.768703+0000","last_clean":"2024-05-09T04:51:02.768703+0000","last_became_active":"2024-05-09T04:49:30.043513+0000","last_became_peered":"2024-05-09T04:49:30.043513+0000","last_unstale":"2024-05-09T04:51:02.768703+0000","last_undegraded":"2024-05-09T04:51:02.768703+0000","last_fullsized":"2024-05-09T04:51:02.768703+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:16.501692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.816530+0000","last_change":"2024-05-09T04:49:47.668605+0000","last_active":"2024-05-09T04:50:57.816530+0000","last_peered":"2024-05-09T04:50:57.816530+0000","last_clean":"2024-05-09T04:50:57.816530+0000","last_became_active":"2024-05-09T04:49:47.668095+0000","last_became_peered":"2024-05-09T04:49:47.668095+0000","last_unstale":"2024-05-09T04:50:57.816530+0000","last_undegraded":"2024-05-09T04:50:57.816530+0000","last_fullsized":"2024-05-09T04:50:57.816530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:39:30.508618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:03.777515+0000","last_change":"2024-05-09T04:49:30.159554+0000","last_active":"2024-05-09T04:51:03.777515+0000","last_peered":"2024-05-09T04:51:03.777515+0000","last_clean":"2024-05-09T04:51:03.777515+0000","last_became_active":"2024-05-09T04:49:30.159266+0000","last_became_peered":"2024-05-09T04:49:30.159266+0000","last_unstale":"2024-05-09T04:51:03.777515+0000","last_undegraded":"2024-05-09T04:51:03.777515+0000","last_fullsized":"2024-05-09T04:51:03.777515+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:42:04.252862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.088272+0000","last_change":"2024-05-09T04:49:34.150688+0000","last_active":"2024-05-09T04:50:58.088272+0000","last_peered":"2024-05-09T04:50:58.088272+0000","last_clean":"2024-05-09T04:50:58.088272+0000","last_became_active":"2024-05-09T04:49:34.150319+0000","last_became_peered":"2024-05-09T04:49:34.150319+0000","last_unstale":"2024-05-09T04:50:58.088272+0000","last_undegraded":"2024-05-09T04:50:58.088272+0000","last_fullsized":"2024-05-09T04:50:58.088272+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:35:04.832761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700490+0000","last_change":"2024-05-09T04:49:47.902323+0000","last_active":"2024-05-09T04:50:57.700490+0000","last_peered":"2024-05-09T04:50:57.700490+0000","last_clean":"2024-05-09T04:50:57.700490+0000","last_became_active":"2024-05-09T04:49:47.901389+0000","last_became_peered":"2024-05-09T04:49:47.901389+0000","last_unstale":"2024-05-09T04:50:57.700490+0000","last_undegraded":"2024-05-09T04:50:57.700490+0000","last_fullsized":"2024-05-09T04:50:57.700490+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:32.615944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'15","reported_seq":57,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:34.178268+0000","last_change":"2024-05-09T04:49:30.061172+0000","last_active":"2024-05-09T04:51:34.178268+0000","last_peered":"2024-05-09T04:51:34.178268+0000","last_clean":"2024-05-09T04:51:34.178268+0000","last_became_active":"2024-05-09T04:49:30.060680+0000","last_became_peered":"2024-05-09T04:49:30.060680+0000","last_unstale":"2024-05-09T04:51:34.178268+0000","last_undegraded":"2024-05-09T04:51:34.178268+0000","last_fullsized":"2024-05-09T04:51:34.178268+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:45:44.649124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.704460+0000","last_change":"2024-05-09T04:49:34.130319+0000","last_active":"2024-05-09T04:50:57.704460+0000","last_peered":"2024-05-09T04:50:57.704460+0000","last_clean":"2024-05-09T04:50:57.704460+0000","last_became_active":"2024-05-09T04:49:34.129610+0000","last_became_peered":"2024-05-09T04:49:34.129610+0000","last_unstale":"2024-05-09T04:50:57.704460+0000","last_undegraded":"2024-05-09T04:50:57.704460+0000","last_fullsized":"2024-05-09T04:50:57.704460+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:45.505221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700194+0000","last_change":"2024-05-09T04:49:47.647513+0000","last_active":"2024-05-09T04:50:57.700194+0000","last_peered":"2024-05-09T04:50:57.700194+0000","last_clean":"2024-05-09T04:50:57.700194+0000","last_became_active":"2024-05-09T04:49:47.647069+0000","last_became_peered":"2024-05-09T04:49:47.647069+0000","last_unstale":"2024-05-09T04:50:57.700194+0000","last_undegraded":"2024-05-09T04:50:57.700194+0000","last_fullsized":"2024-05-09T04:50:57.700194+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:05:27.658919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:03.775689+0000","last_change":"2024-05-09T04:49:30.350850+0000","last_active":"2024-05-09T04:51:03.775689+0000","last_peered":"2024-05-09T04:51:03.775689+0000","last_clean":"2024-05-09T04:51:03.775689+0000","last_became_active":"2024-05-09T04:49:30.350431+0000","last_became_peered":"2024-05-09T04:49:30.350431+0000","last_unstale":"2024-05-09T04:51:03.775689+0000","last_undegraded":"2024-05-09T04:51:03.775689+0000","last_fullsized":"2024-05-09T04:51:03.775689+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:53:05.388338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.204464+0000","last_change":"2024-05-09T04:49:34.153630+0000","last_active":"2024-05-09T04:50:58.204464+0000","last_peered":"2024-05-09T04:50:58.204464+0000","last_clean":"2024-05-09T04:50:58.204464+0000","last_became_active":"2024-05-09T04:49:34.152045+0000","last_became_peered":"2024-05-09T04:49:34.152045+0000","last_unstale":"2024-05-09T04:50:58.204464+0000","last_undegraded":"2024-05-09T04:50:58.204464+0000","last_fullsized":"2024-05-09T04:50:58.204464+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:32:35.991731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.202751+0000","last_change":"2024-05-09T04:49:47.901256+0000","last_active":"2024-05-09T04:50:58.202751+0000","last_peered":"2024-05-09T04:50:58.202751+0000","last_clean":"2024-05-09T04:50:58.202751+0000","last_became_active":"2024-05-09T04:49:47.900413+0000","last_became_peered":"2024-05-09T04:49:47.900413+0000","last_unstale":"2024-05-09T04:50:58.202751+0000","last_undegraded":"2024-05-09T04:50:58.202751+0000","last_fullsized":"2024-05-09T04:50:58.202751+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:46:11.845220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:59.742698+0000","last_change":"2024-05-09T04:49:30.152273+0000","last_active":"2024-05-09T04:50:59.742698+0000","last_peered":"2024-05-09T04:50:59.742698+0000","last_clean":"2024-05-09T04:50:59.742698+0000","last_became_active":"2024-05-09T04:49:30.151973+0000","last_became_peered":"2024-05-09T04:49:30.151973+0000","last_unstale":"2024-05-09T04:50:59.742698+0000","last_undegraded":"2024-05-09T04:50:59.742698+0000","last_fullsized":"2024-05-09T04:50:59.742698+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:14:52.912588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.201324+0000","last_change":"2024-05-09T04:49:34.126829+0000","last_active":"2024-05-09T04:50:58.201324+0000","last_peered":"2024-05-09T04:50:58.201324+0000","last_clean":"2024-05-09T04:50:58.201324+0000","last_became_active":"2024-05-09T04:49:34.126430+0000","last_became_peered":"2024-05-09T04:49:34.126430+0000","last_unstale":"2024-05-09T04:50:58.201324+0000","last_undegraded":"2024-05-09T04:50:58.201324+0000","last_fullsized":"2024-05-09T04:50:58.201324+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:38.247148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.705899+0000","last_change":"2024-05-09T04:49:47.675292+0000","last_active":"2024-05-09T04:50:57.705899+0000","last_peered":"2024-05-09T04:50:57.705899+0000","last_clean":"2024-05-09T04:50:57.705899+0000","last_became_active":"2024-05-09T04:49:47.674802+0000","last_became_peered":"2024-05-09T04:49:47.674802+0000","last_unstale":"2024-05-09T04:50:57.705899+0000","last_undegraded":"2024-05-09T04:50:57.705899+0000","last_fullsized":"2024-05-09T04:50:57.705899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:47:21.988054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:02.769886+0000","last_change":"2024-05-09T04:49:30.057854+0000","last_active":"2024-05-09T04:51:02.769886+0000","last_peered":"2024-05-09T04:51:02.769886+0000","last_clean":"2024-05-09T04:51:02.769886+0000","last_became_active":"2024-05-09T04:49:30.056625+0000","last_became_peered":"2024-05-09T04:49:30.056625+0000","last_unstale":"2024-05-09T04:51:02.769886+0000","last_undegraded":"2024-05-09T04:51:02.769886+0000","last_fullsized":"2024-05-09T04:51:02.769886+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:51.272533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.816674+0000","last_change":"2024-05-09T04:49:34.144414+0000","last_active":"2024-05-09T04:50:57.816674+0000","last_peered":"2024-05-09T04:50:57.816674+0000","last_clean":"2024-05-09T04:50:57.816674+0000","last_became_active":"2024-05-09T04:49:34.144064+0000","last_became_peered":"2024-05-09T04:49:34.144064+0000","last_unstale":"2024-05-09T04:50:57.816674+0000","last_undegraded":"2024-05-09T04:50:57.816674+0000","last_fullsized":"2024-05-09T04:50:57.816674+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:44:40.991576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203040+0000","last_change":"2024-05-09T04:49:47.672956+0000","last_active":"2024-05-09T04:50:58.203040+0000","last_peered":"2024-05-09T04:50:58.203040+0000","last_clean":"2024-05-09T04:50:58.203040+0000","last_became_active":"2024-05-09T04:49:47.672513+0000","last_became_peered":"2024-05-09T04:49:47.672513+0000","last_unstale":"2024-05-09T04:50:58.203040+0000","last_undegraded":"2024-05-09T04:50:58.203040+0000","last_fullsized":"2024-05-09T04:50:58.203040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:32.250039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:01.758882+0000","last_change":"2024-05-09T04:49:56.704058+0000","last_active":"2024-05-09T04:51:01.758882+0000","last_peered":"2024-05-09T04:51:01.758882+0000","last_clean":"2024-05-09T04:51:01.758882+0000","last_became_active":"2024-05-09T04:49:56.703589+0000","last_became_peered":"2024-05-09T04:49:56.703589+0000","last_unstale":"2024-05-09T04:51:01.758882+0000","last_undegraded":"2024-05-09T04:51:01.758882+0000","last_fullsized":"2024-05-09T04:51:01.758882+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:54.596200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,1],"acting":[11,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.702507+0000","last_change":"2024-05-09T04:49:34.159321+0000","last_active":"2024-05-09T04:50:57.702507+0000","last_peered":"2024-05-09T04:50:57.702507+0000","last_clean":"2024-05-09T04:50:57.702507+0000","last_became_active":"2024-05-09T04:49:34.157918+0000","last_became_peered":"2024-05-09T04:49:34.157918+0000","last_unstale":"2024-05-09T04:50:57.702507+0000","last_undegraded":"2024-05-09T04:50:57.702507+0000","last_fullsized":"2024-05-09T04:50:57.702507+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:33:59.659905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.202786+0000","last_change":"2024-05-09T04:49:47.900692+0000","last_active":"2024-05-09T04:50:58.202786+0000","last_peered":"2024-05-09T04:50:58.202786+0000","last_clean":"2024-05-09T04:50:58.202786+0000","last_became_active":"2024-05-09T04:49:47.900095+0000","last_became_peered":"2024-05-09T04:49:47.900095+0000","last_unstale":"2024-05-09T04:50:58.202786+0000","last_undegraded":"2024-05-09T04:50:58.202786+0000","last_fullsized":"2024-05-09T04:50:58.202786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:08.632408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:59.742219+0000","last_change":"2024-05-09T04:49:30.048768+0000","last_active":"2024-05-09T04:50:59.742219+0000","last_peered":"2024-05-09T04:50:59.742219+0000","last_clean":"2024-05-09T04:50:59.742219+0000","last_became_active":"2024-05-09T04:49:30.048235+0000","last_became_peered":"2024-05-09T04:49:30.048235+0000","last_unstale":"2024-05-09T04:50:59.742219+0000","last_undegraded":"2024-05-09T04:50:59.742219+0000","last_fullsized":"2024-05-09T04:50:59.742219+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:51:16.988726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.202529+0000","last_change":"2024-05-09T04:49:34.142110+0000","last_active":"2024-05-09T04:50:58.202529+0000","last_peered":"2024-05-09T04:50:58.202529+0000","last_clean":"2024-05-09T04:50:58.202529+0000","last_became_active":"2024-05-09T04:49:34.141709+0000","last_became_peered":"2024-05-09T04:49:34.141709+0000","last_unstale":"2024-05-09T04:50:58.202529+0000","last_undegraded":"2024-05-09T04:50:58.202529+0000","last_fullsized":"2024-05-09T04:50:58.202529+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:16.365915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.952024+0000","last_change":"2024-05-09T04:49:47.667784+0000","last_active":"2024-05-09T04:50:57.952024+0000","last_peered":"2024-05-09T04:50:57.952024+0000","last_clean":"2024-05-09T04:50:57.952024+0000","last_became_active":"2024-05-09T04:49:47.667316+0000","last_became_peered":"2024-05-09T04:49:47.667316+0000","last_unstale":"2024-05-09T04:50:57.952024+0000","last_undegraded":"2024-05-09T04:50:57.952024+0000","last_fullsized":"2024-05-09T04:50:57.952024+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:04:20.233287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":88,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:40.836087+0000","last_change":"2024-05-09T04:49:30.351231+0000","last_active":"2024-05-09T04:51:40.836087+0000","last_peered":"2024-05-09T04:51:40.836087+0000","last_clean":"2024-05-09T04:51:40.836087+0000","last_became_active":"2024-05-09T04:49:30.350817+0000","last_became_peered":"2024-05-09T04:49:30.350817+0000","last_unstale":"2024-05-09T04:51:40.836087+0000","last_undegraded":"2024-05-09T04:51:40.836087+0000","last_fullsized":"2024-05-09T04:51:40.836087+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:51.234300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":41,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203752+0000","last_change":"2024-05-09T04:49:34.129625+0000","last_active":"2024-05-09T04:50:58.203752+0000","last_peered":"2024-05-09T04:50:58.203752+0000","last_clean":"2024-05-09T04:50:58.203752+0000","last_became_active":"2024-05-09T04:49:34.129215+0000","last_became_peered":"2024-05-09T04:49:34.129215+0000","last_unstale":"2024-05-09T04:50:58.203752+0000","last_undegraded":"2024-05-09T04:50:58.203752+0000","last_fullsized":"2024-05-09T04:50:58.203752+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:26.247116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203783+0000","last_change":"2024-05-09T04:49:47.682819+0000","last_active":"2024-05-09T04:50:58.203783+0000","last_peered":"2024-05-09T04:50:58.203783+0000","last_clean":"2024-05-09T04:50:58.203783+0000","last_became_active":"2024-05-09T04:49:47.682331+0000","last_became_peered":"2024-05-09T04:49:47.682331+0000","last_unstale":"2024-05-09T04:50:58.203783+0000","last_undegraded":"2024-05-09T04:50:58.203783+0000","last_fullsized":"2024-05-09T04:50:58.203783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:03.852975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:02.768779+0000","last_change":"2024-05-09T04:49:30.050247+0000","last_active":"2024-05-09T04:51:02.768779+0000","last_peered":"2024-05-09T04:51:02.768779+0000","last_clean":"2024-05-09T04:51:02.768779+0000","last_became_active":"2024-05-09T04:49:30.049835+0000","last_became_peered":"2024-05-09T04:49:30.049835+0000","last_unstale":"2024-05-09T04:51:02.768779+0000","last_undegraded":"2024-05-09T04:51:02.768779+0000","last_fullsized":"2024-05-09T04:51:02.768779+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:52.591186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203356+0000","last_change":"2024-05-09T04:49:34.145406+0000","last_active":"2024-05-09T04:50:58.203356+0000","last_peered":"2024-05-09T04:50:58.203356+0000","last_clean":"2024-05-09T04:50:58.203356+0000","last_became_active":"2024-05-09T04:49:34.144946+0000","last_became_peered":"2024-05-09T04:49:34.144946+0000","last_unstale":"2024-05-09T04:50:58.203356+0000","last_undegraded":"2024-05-09T04:50:58.203356+0000","last_fullsized":"2024-05-09T04:50:58.203356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:43:04.704215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.201310+0000","last_change":"2024-05-09T04:49:56.758844+0000","last_active":"2024-05-09T04:50:58.201310+0000","last_peered":"2024-05-09T04:50:58.201310+0000","last_clean":"2024-05-09T04:50:58.201310+0000","last_became_active":"2024-05-09T04:49:56.758335+0000","last_became_peered":"2024-05-09T04:49:56.758335+0000","last_unstale":"2024-05-09T04:50:58.201310+0000","last_undegraded":"2024-05-09T04:50:58.201310+0000","last_fullsized":"2024-05-09T04:50:58.201310+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:39:33.344225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":57,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:02.770650+0000","last_change":"2024-05-09T04:49:56.766262+0000","last_active":"2024-05-09T04:51:02.770650+0000","last_peered":"2024-05-09T04:51:02.770650+0000","last_clean":"2024-05-09T04:51:02.770650+0000","last_became_active":"2024-05-09T04:49:56.758765+0000","last_became_peered":"2024-05-09T04:49:56.758765+0000","last_unstale":"2024-05-09T04:51:02.770650+0000","last_undegraded":"2024-05-09T04:51:02.770650+0000","last_fullsized":"2024-05-09T04:51:02.770650+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:44:18.271729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203752+0000","last_change":"2024-05-09T04:49:34.134957+0000","last_active":"2024-05-09T04:50:58.203752+0000","last_peered":"2024-05-09T04:50:58.203752+0000","last_clean":"2024-05-09T04:50:58.203752+0000","last_became_active":"2024-05-09T04:49:34.134544+0000","last_became_peered":"2024-05-09T04:49:34.134544+0000","last_unstale":"2024-05-09T04:50:58.203752+0000","last_undegraded":"2024-05-09T04:50:58.203752+0000","last_fullsized":"2024-05-09T04:50:58.203752+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:21:54.855905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.816770+0000","last_change":"2024-05-09T04:49:34.138112+0000","last_active":"2024-05-09T04:50:57.816770+0000","last_peered":"2024-05-09T04:50:57.816770+0000","last_clean":"2024-05-09T04:50:57.816770+0000","last_became_active":"2024-05-09T04:49:34.137716+0000","last_became_peered":"2024-05-09T04:49:34.137716+0000","last_unstale":"2024-05-09T04:50:57.816770+0000","last_undegraded":"2024-05-09T04:50:57.816770+0000","last_fullsized":"2024-05-09T04:50:57.816770+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:17.028262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.204750+0000","last_change":"2024-05-09T04:50:58.204287+0000","last_active":"2024-05-09T04:50:58.204750+0000","last_peered":"2024-05-09T04:50:58.204750+0000","last_clean":"2024-05-09T04:50:58.204750+0000","last_became_active":"2024-05-09T04:50:58.203894+0000","last_became_peered":"2024-05-09T04:50:58.203894+0000","last_unstale":"2024-05-09T04:50:58.204750+0000","last_undegraded":"2024-05-09T04:50:58.204750+0000","last_fullsized":"2024-05-09T04:50:58.204750+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:02.214347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700344+0000","last_change":"2024-05-09T04:49:34.147742+0000","last_active":"2024-05-09T04:50:57.700344+0000","last_peered":"2024-05-09T04:50:57.700344+0000","last_clean":"2024-05-09T04:50:57.700344+0000","last_became_active":"2024-05-09T04:49:34.147301+0000","last_became_peered":"2024-05-09T04:49:34.147301+0000","last_unstale":"2024-05-09T04:50:57.700344+0000","last_undegraded":"2024-05-09T04:50:57.700344+0000","last_fullsized":"2024-05-09T04:50:57.700344+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:36:55.371212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.204452+0000","last_change":"2024-05-09T04:49:34.163599+0000","last_active":"2024-05-09T04:50:58.204452+0000","last_peered":"2024-05-09T04:50:58.204452+0000","last_clean":"2024-05-09T04:50:58.204452+0000","last_became_active":"2024-05-09T04:49:34.162898+0000","last_became_peered":"2024-05-09T04:49:34.162898+0000","last_unstale":"2024-05-09T04:50:58.204452+0000","last_undegraded":"2024-05-09T04:50:58.204452+0000","last_fullsized":"2024-05-09T04:50:58.204452+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:55:57.213901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.088033+0000","last_change":"2024-05-09T04:49:34.151648+0000","last_active":"2024-05-09T04:50:58.088033+0000","last_peered":"2024-05-09T04:50:58.088033+0000","last_clean":"2024-05-09T04:50:58.088033+0000","last_became_active":"2024-05-09T04:49:34.148701+0000","last_became_peered":"2024-05-09T04:49:34.148701+0000","last_unstale":"2024-05-09T04:50:58.088033+0000","last_undegraded":"2024-05-09T04:50:58.088033+0000","last_fullsized":"2024-05-09T04:50:58.088033+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:54:30.122252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.816937+0000","last_change":"2024-05-09T04:49:34.136679+0000","last_active":"2024-05-09T04:50:57.816937+0000","last_peered":"2024-05-09T04:50:57.816937+0000","last_clean":"2024-05-09T04:50:57.816937+0000","last_became_active":"2024-05-09T04:49:34.136306+0000","last_became_peered":"2024-05-09T04:49:34.136306+0000","last_unstale":"2024-05-09T04:50:57.816937+0000","last_undegraded":"2024-05-09T04:50:57.816937+0000","last_fullsized":"2024-05-09T04:50:57.816937+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:51:39.476564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700949+0000","last_change":"2024-05-09T04:49:34.130769+0000","last_active":"2024-05-09T04:50:57.700949+0000","last_peered":"2024-05-09T04:50:57.700949+0000","last_clean":"2024-05-09T04:50:57.700949+0000","last_became_active":"2024-05-09T04:49:34.130362+0000","last_became_peered":"2024-05-09T04:49:34.130362+0000","last_unstale":"2024-05-09T04:50:57.700949+0000","last_undegraded":"2024-05-09T04:50:57.700949+0000","last_fullsized":"2024-05-09T04:50:57.700949+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:54.942259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.704942+0000","last_change":"2024-05-09T04:49:34.138054+0000","last_active":"2024-05-09T04:50:57.704942+0000","last_peered":"2024-05-09T04:50:57.704942+0000","last_clean":"2024-05-09T04:50:57.704942+0000","last_became_active":"2024-05-09T04:49:34.136111+0000","last_became_peered":"2024-05-09T04:49:34.136111+0000","last_unstale":"2024-05-09T04:50:57.704942+0000","last_undegraded":"2024-05-09T04:50:57.704942+0000","last_fullsized":"2024-05-09T04:50:57.704942+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:15:27.870874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.032338+0000","last_change":"2024-05-09T04:49:34.146370+0000","last_active":"2024-05-09T04:50:58.032338+0000","last_peered":"2024-05-09T04:50:58.032338+0000","last_clean":"2024-05-09T04:50:58.032338+0000","last_became_active":"2024-05-09T04:49:34.145948+0000","last_became_peered":"2024-05-09T04:49:34.145948+0000","last_unstale":"2024-05-09T04:50:58.032338+0000","last_undegraded":"2024-05-09T04:50:58.032338+0000","last_fullsized":"2024-05-09T04:50:58.032338+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:14.826168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.701358+0000","last_change":"2024-05-09T04:49:34.138028+0000","last_active":"2024-05-09T04:50:57.701358+0000","last_peered":"2024-05-09T04:50:57.701358+0000","last_clean":"2024-05-09T04:50:57.701358+0000","last_became_active":"2024-05-09T04:49:34.137558+0000","last_became_peered":"2024-05-09T04:49:34.137558+0000","last_unstale":"2024-05-09T04:50:57.701358+0000","last_undegraded":"2024-05-09T04:50:57.701358+0000","last_fullsized":"2024-05-09T04:50:57.701358+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:50:16.258945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.952521+0000","last_change":"2024-05-09T04:49:34.137946+0000","last_active":"2024-05-09T04:50:57.952521+0000","last_peered":"2024-05-09T04:50:57.952521+0000","last_clean":"2024-05-09T04:50:57.952521+0000","last_became_active":"2024-05-09T04:49:34.137498+0000","last_became_peered":"2024-05-09T04:49:34.137498+0000","last_unstale":"2024-05-09T04:50:57.952521+0000","last_undegraded":"2024-05-09T04:50:57.952521+0000","last_fullsized":"2024-05-09T04:50:57.952521+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:36:54.473689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.087619+0000","last_change":"2024-05-09T04:49:34.162103+0000","last_active":"2024-05-09T04:50:58.087619+0000","last_peered":"2024-05-09T04:50:58.087619+0000","last_clean":"2024-05-09T04:50:58.087619+0000","last_became_active":"2024-05-09T04:49:34.161681+0000","last_became_peered":"2024-05-09T04:49:34.161681+0000","last_unstale":"2024-05-09T04:50:58.087619+0000","last_undegraded":"2024-05-09T04:50:58.087619+0000","last_fullsized":"2024-05-09T04:50:58.087619+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:36:10.460401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.816788+0000","last_change":"2024-05-09T04:49:34.136538+0000","last_active":"2024-05-09T04:50:57.816788+0000","last_peered":"2024-05-09T04:50:57.816788+0000","last_clean":"2024-05-09T04:50:57.816788+0000","last_became_active":"2024-05-09T04:49:34.134903+0000","last_became_peered":"2024-05-09T04:49:34.134903+0000","last_unstale":"2024-05-09T04:50:57.816788+0000","last_undegraded":"2024-05-09T04:50:57.816788+0000","last_fullsized":"2024-05-09T04:50:57.816788+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:44:48.673600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70306,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":418,"num_read_kb":288,"num_write":249,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":213194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":215,"ondisk_log_size":215,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678058,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":510251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874288,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37016,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708136,"statfs":{"total":95995035648,"available":95957131264,"internally_reserved":0,"allocated":1720320,"data_stored":1160448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037813,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":514165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234045,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":510335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430273,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1642496,"data_stored":1107335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593800,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":505573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":570843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738474,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":507029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1114112,"data_stored":571454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149786,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":505281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953557,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":510038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790030,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1650688,"data_stored":1102526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T04:51:44.774 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-09T04:51:44.774 INFO:tasks.cephadm:Setting up client nodes... 2024-05-09T04:51:44.775 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-09T04:51:44.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:44 smithi006 ceph-mon[29835]: pgmap v403: 129 pgs: 129 active+clean; 645 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-09T04:51:44.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:44 smithi006 ceph-mon[29835]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:51:45.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:44 smithi186 ceph-mon[34668]: pgmap v403: 129 pgs: 129 active+clean; 645 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-09T04:51:45.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:44 smithi186 ceph-mon[34668]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:51:45.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:44 smithi169 ceph-mon[34075]: pgmap v403: 129 pgs: 129 active+clean; 645 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-09T04:51:45.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:44 smithi169 ceph-mon[34075]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:51:45.289 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:51:46.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:46 smithi006 ceph-mon[29835]: pgmap v404: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-09T04:51:47.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:46 smithi186 ceph-mon[34668]: pgmap v404: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-09T04:51:47.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:46 smithi169 ceph-mon[34075]: pgmap v404: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-09T04:51:47.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.618+0000 7f6f1ca00700 1 -- 172.21.15.6:0/1680024205 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f18104360 msgr2=0x7f6f18104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:47.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.618+0000 7f6f1ca00700 1 --2- 172.21.15.6:0/1680024205 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f18104360 0x7f6f18104770 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6f080098f0 tx=0x7f6f08009c00 comp rx=0 tx=0).stop 2024-05-09T04:51:47.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.618+0000 7f6f1ca00700 1 -- 172.21.15.6:0/1680024205 shutdown_connections 2024-05-09T04:51:47.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.618+0000 7f6f1ca00700 1 --2- 172.21.15.6:0/1680024205 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f18105e30 0x7f6f1810c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:47.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.618+0000 7f6f1ca00700 1 --2- 172.21.15.6:0/1680024205 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f18104360 0x7f6f18104770 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:47.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.618+0000 7f6f1ca00700 1 --2- 172.21.15.6:0/1680024205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f18105490 0x7f6f181058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:47.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.618+0000 7f6f1ca00700 1 -- 172.21.15.6:0/1680024205 >> 172.21.15.6:0/1680024205 conn(0x7f6f180ff9c0 msgr2=0x7f6f18101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:47.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.618+0000 7f6f1ca00700 1 -- 172.21.15.6:0/1680024205 shutdown_connections 2024-05-09T04:51:47.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.619+0000 7f6f1ca00700 1 -- 172.21.15.6:0/1680024205 wait complete. 2024-05-09T04:51:47.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.620+0000 7f6f1ca00700 1 Processor -- start 2024-05-09T04:51:47.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.620+0000 7f6f1ca00700 1 -- start start 2024-05-09T04:51:47.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.621+0000 7f6f1ca00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f18104360 0x7f6f1819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:47.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.621+0000 7f6f1ca00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f18105490 0x7f6f1819d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:47.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.621+0000 7f6f1ca00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f18105e30 0x7f6f181a4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:47.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.621+0000 7f6f1ca00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1810fdf0 con 0x7f6f18105e30 2024-05-09T04:51:47.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.621+0000 7f6f1ca00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1810fc70 con 0x7f6f18104360 2024-05-09T04:51:47.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.621+0000 7f6f1ca00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1810ff70 con 0x7f6f18105490 2024-05-09T04:51:47.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.621+0000 7f6f15400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f18105490 0x7f6f1819d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:47.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.621+0000 7f6f15e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f18104360 0x7f6f1819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:47.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.621+0000 7f6f16800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f18105e30 0x7f6f181a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:47.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.621+0000 7f6f15400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f18105490 0x7f6f1819d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:49378/0 (socket says 172.21.15.6:49378) 2024-05-09T04:51:47.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.621+0000 7f6f15400700 1 -- 172.21.15.6:0/2439552253 learned_addr learned my addr 172.21.15.6:0/2439552253 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:51:47.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.621+0000 7f6f15e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f18104360 0x7f6f1819cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:49630/0 (socket says 172.21.15.6:49630) 2024-05-09T04:51:47.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.622+0000 7f6f15e00700 1 -- 172.21.15.6:0/2439552253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f18105490 msgr2=0x7f6f1819d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:47.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.622+0000 7f6f15e00700 1 --2- 172.21.15.6:0/2439552253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f18105490 0x7f6f1819d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:47.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.622+0000 7f6f15e00700 1 -- 172.21.15.6:0/2439552253 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f18105e30 msgr2=0x7f6f181a4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:47.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.622+0000 7f6f15e00700 1 --2- 172.21.15.6:0/2439552253 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f18105e30 0x7f6f181a4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:47.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.622+0000 7f6f15e00700 1 -- 172.21.15.6:0/2439552253 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f181a4780 con 0x7f6f18104360 2024-05-09T04:51:47.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.622+0000 7f6f15e00700 1 --2- 172.21.15.6:0/2439552253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f18104360 0x7f6f1819cc20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6f08009230 tx=0x7f6f08010f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:47.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.622+0000 7f6f0ea00700 1 -- 172.21.15.6:0/2439552253 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f080050c0 con 0x7f6f18104360 2024-05-09T04:51:47.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.622+0000 7f6f0ea00700 1 -- 172.21.15.6:0/2439552253 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f0800b560 con 0x7f6f18104360 2024-05-09T04:51:47.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.622+0000 7f6f1ca00700 1 -- 172.21.15.6:0/2439552253 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f181a4a10 con 0x7f6f18104360 2024-05-09T04:51:47.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.622+0000 7f6f0ea00700 1 -- 172.21.15.6:0/2439552253 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f080039b0 con 0x7f6f18104360 2024-05-09T04:51:47.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.623+0000 7f6f1ca00700 1 -- 172.21.15.6:0/2439552253 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f181a4ec0 con 0x7f6f18104360 2024-05-09T04:51:47.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.624+0000 7f6f1ca00700 1 -- 172.21.15.6:0/2439552253 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f180512c0 con 0x7f6f18104360 2024-05-09T04:51:47.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.627+0000 7f6f0ea00700 1 -- 172.21.15.6:0/2439552253 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f6f0800b710 con 0x7f6f18104360 2024-05-09T04:51:47.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.628+0000 7f6f0ea00700 1 --2- 172.21.15.6:0/2439552253 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f6efc0b2990 0x7f6efc0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:47.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.628+0000 7f6f0ea00700 1 -- 172.21.15.6:0/2439552253 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f6f080e3770 con 0x7f6f18104360 2024-05-09T04:51:47.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.628+0000 7f6f15400700 1 --2- 172.21.15.6:0/2439552253 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f6efc0b2990 0x7f6efc0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:47.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.629+0000 7f6f15400700 1 --2- 172.21.15.6:0/2439552253 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f6efc0b2990 0x7f6efc0b4ea0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6f1819dcd0 tx=0x7f6f04009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:47.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.631+0000 7f6f0ea00700 1 -- 172.21.15.6:0/2439552253 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6f08013030 con 0x7f6f18104360 2024-05-09T04:51:47.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.974+0000 7f6f1ca00700 1 -- 172.21.15.6:0/2439552253 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f6f181a5130 con 0x7f6f18104360 2024-05-09T04:51:47.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.985+0000 7f6f0ea00700 1 -- 172.21.15.6:0/2439552253 <== mon.2 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v33) ==== 170+0+59 (secure 0 0 0) 0x7f6f080afe20 con 0x7f6f18104360 2024-05-09T04:51:47.986 INFO:teuthology.orchestra.run.smithi006.stdout:[client.0] 2024-05-09T04:51:47.986 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQBjVjxmNIVGOhAAHUmFydJgv8DCX9FFqs4ROw== 2024-05-09T04:51:47.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.988+0000 7f6f1ca00700 1 -- 172.21.15.6:0/2439552253 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f6efc0b2990 msgr2=0x7f6efc0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:47.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.988+0000 7f6f1ca00700 1 --2- 172.21.15.6:0/2439552253 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f6efc0b2990 0x7f6efc0b4ea0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6f1819dcd0 tx=0x7f6f04009260 comp rx=0 tx=0).stop 2024-05-09T04:51:47.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.988+0000 7f6f1ca00700 1 -- 172.21.15.6:0/2439552253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f18104360 msgr2=0x7f6f1819cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:47.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.988+0000 7f6f1ca00700 1 --2- 172.21.15.6:0/2439552253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f18104360 0x7f6f1819cc20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6f08009230 tx=0x7f6f08010f00 comp rx=0 tx=0).stop 2024-05-09T04:51:47.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.989+0000 7f6f1ca00700 1 -- 172.21.15.6:0/2439552253 shutdown_connections 2024-05-09T04:51:47.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.989+0000 7f6f1ca00700 1 --2- 172.21.15.6:0/2439552253 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f18105e30 0x7f6f181a4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:47.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.989+0000 7f6f1ca00700 1 --2- 172.21.15.6:0/2439552253 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f6efc0b2990 0x7f6efc0b4ea0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:47.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.989+0000 7f6f1ca00700 1 --2- 172.21.15.6:0/2439552253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f18104360 0x7f6f1819cc20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:47.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.989+0000 7f6f1ca00700 1 --2- 172.21.15.6:0/2439552253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f18105490 0x7f6f1819d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:47.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.989+0000 7f6f1ca00700 1 -- 172.21.15.6:0/2439552253 >> 172.21.15.6:0/2439552253 conn(0x7f6f180ff9c0 msgr2=0x7f6f180759f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:47.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.989+0000 7f6f1ca00700 1 -- 172.21.15.6:0/2439552253 shutdown_connections 2024-05-09T04:51:47.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:47.989+0000 7f6f1ca00700 1 -- 172.21.15.6:0/2439552253 wait complete. 2024-05-09T04:51:48.829 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T04:51:48.829 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-09T04:51:48.829 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-09T04:51:48.935 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:48 smithi006 ceph-mon[29835]: pgmap v405: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-09T04:51:48.935 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:48 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/2439552253' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T04:51:48.935 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:48 smithi006 ceph-mon[29835]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T04:51:48.935 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:48 smithi006 ceph-mon[29835]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-09T04:51:48.947 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-09T04:51:48.947 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-09T04:51:48.947 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph mgr dump --format=json 2024-05-09T04:51:49.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:48 smithi186 ceph-mon[34668]: pgmap v405: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-09T04:51:49.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:48 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/2439552253' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T04:51:49.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:48 smithi186 ceph-mon[34668]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T04:51:49.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:48 smithi186 ceph-mon[34668]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-09T04:51:49.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:48 smithi169 ceph-mon[34075]: pgmap v405: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-09T04:51:49.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:48 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/2439552253' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T04:51:49.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:48 smithi169 ceph-mon[34075]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T04:51:49.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:48 smithi169 ceph-mon[34075]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-09T04:51:49.456 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:51:50.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:50 smithi186 ceph-mon[34668]: pgmap v406: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 596 B/s wr, 10 op/s 2024-05-09T04:51:50.937 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:50 smithi169 ceph-mon[34075]: pgmap v406: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 596 B/s wr, 10 op/s 2024-05-09T04:51:50.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:50 smithi006 ceph-mon[29835]: pgmap v406: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 596 B/s wr, 10 op/s 2024-05-09T04:51:51.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.835+0000 7f6287000700 1 -- 172.21.15.6:0/608520171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6280105e30 msgr2=0x7f628010e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:51.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.835+0000 7f6287000700 1 --2- 172.21.15.6:0/608520171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6280105e30 0x7f628010e380 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f626c009990 tx=0x7f626c009ca0 comp rx=0 tx=0).stop 2024-05-09T04:51:51.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.835+0000 7f6287000700 1 -- 172.21.15.6:0/608520171 shutdown_connections 2024-05-09T04:51:51.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.835+0000 7f6287000700 1 --2- 172.21.15.6:0/608520171 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f628010e910 0x7f6280110df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:51.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.835+0000 7f6287000700 1 --2- 172.21.15.6:0/608520171 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62801054e0 0x7f62801058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:51.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.835+0000 7f6287000700 1 --2- 172.21.15.6:0/608520171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6280105e30 0x7f628010e380 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:51.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.835+0000 7f6287000700 1 -- 172.21.15.6:0/608520171 >> 172.21.15.6:0/608520171 conn(0x7f62800fedc0 msgr2=0x7f62801011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:51.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.835+0000 7f6287000700 1 -- 172.21.15.6:0/608520171 shutdown_connections 2024-05-09T04:51:51.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.835+0000 7f6287000700 1 -- 172.21.15.6:0/608520171 wait complete. 2024-05-09T04:51:51.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.837+0000 7f6287000700 1 Processor -- start 2024-05-09T04:51:51.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.837+0000 7f6287000700 1 -- start start 2024-05-09T04:51:51.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.838+0000 7f6287000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f62801054e0 0x7f628019adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:51.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.838+0000 7f6287000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6280105e30 0x7f628019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:51.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.838+0000 7f6287000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f628010e910 0x7f628019f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:51.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.838+0000 7f6287000700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6280114460 con 0x7f62801054e0 2024-05-09T04:51:51.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.838+0000 7f6287000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f62801142e0 con 0x7f628010e910 2024-05-09T04:51:51.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.838+0000 7f6287000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f62801145e0 con 0x7f6280105e30 2024-05-09T04:51:51.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.839+0000 7f6285000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f628010e910 0x7f628019f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:51.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.839+0000 7f627fe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f62801054e0 0x7f628019adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:51.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.839+0000 7f6285000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f628010e910 0x7f628019f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:49646/0 (socket says 172.21.15.6:49646) 2024-05-09T04:51:51.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.839+0000 7f627f400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6280105e30 0x7f628019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:51.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.839+0000 7f6285000700 1 -- 172.21.15.6:0/1046418810 learned_addr learned my addr 172.21.15.6:0/1046418810 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:51:51.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.839+0000 7f6285000700 1 -- 172.21.15.6:0/1046418810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6280105e30 msgr2=0x7f628019b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:51.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.839+0000 7f6285000700 1 --2- 172.21.15.6:0/1046418810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6280105e30 0x7f628019b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:51.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.839+0000 7f6285000700 1 -- 172.21.15.6:0/1046418810 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f62801054e0 msgr2=0x7f628019adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:51.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.839+0000 7f6285000700 1 --2- 172.21.15.6:0/1046418810 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f62801054e0 0x7f628019adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:51.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.839+0000 7f6285000700 1 -- 172.21.15.6:0/1046418810 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f628019fc80 con 0x7f628010e910 2024-05-09T04:51:51.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.839+0000 7f6285000700 1 --2- 172.21.15.6:0/1046418810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f628010e910 0x7f628019f5e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f627800ef30 tx=0x7f627800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:51.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.840+0000 7f627cc00700 1 -- 172.21.15.6:0/1046418810 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62780092e0 con 0x7f628010e910 2024-05-09T04:51:51.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.840+0000 7f627cc00700 1 -- 172.21.15.6:0/1046418810 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6278004bb0 con 0x7f628010e910 2024-05-09T04:51:51.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.840+0000 7f6287000700 1 -- 172.21.15.6:0/1046418810 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f628019ff10 con 0x7f628010e910 2024-05-09T04:51:51.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.840+0000 7f6287000700 1 -- 172.21.15.6:0/1046418810 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62801a03d0 con 0x7f628010e910 2024-05-09T04:51:51.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.840+0000 7f627cc00700 1 -- 172.21.15.6:0/1046418810 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6278004020 con 0x7f628010e910 2024-05-09T04:51:51.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.841+0000 7f6287000700 1 -- 172.21.15.6:0/1046418810 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62800512c0 con 0x7f628010e910 2024-05-09T04:51:51.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.843+0000 7f627cc00700 1 -- 172.21.15.6:0/1046418810 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f627801b030 con 0x7f628010e910 2024-05-09T04:51:51.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.845+0000 7f627cc00700 1 --2- 172.21.15.6:0/1046418810 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f62640b2910 0x7f62640b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:51.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.845+0000 7f627cc00700 1 -- 172.21.15.6:0/1046418810 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f62780e2ba0 con 0x7f628010e910 2024-05-09T04:51:51.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.845+0000 7f627fe00700 1 --2- 172.21.15.6:0/1046418810 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f62640b2910 0x7f62640b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:51.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.846+0000 7f627fe00700 1 --2- 172.21.15.6:0/1046418810 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f62640b2910 0x7f62640b4e20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6270009c60 tx=0x7f6270009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:51.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:51.852+0000 7f627cc00700 1 -- 172.21.15.6:0/1046418810 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6278022020 con 0x7f628010e910 2024-05-09T04:51:52.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:52.154+0000 7f6287000700 1 -- 172.21.15.6:0/1046418810 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f628002e5f0 con 0x7f628010e910 2024-05-09T04:51:52.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:52.161+0000 7f627cc00700 1 -- 172.21.15.6:0/1046418810 <== mon.2 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288638 (secure 0 0 0) 0x7f62780e2e20 con 0x7f628010e910 2024-05-09T04:51:52.163 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:51:52.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:52.167+0000 7f6287000700 1 -- 172.21.15.6:0/1046418810 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f62640b2910 msgr2=0x7f62640b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:52.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:52.167+0000 7f6287000700 1 --2- 172.21.15.6:0/1046418810 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f62640b2910 0x7f62640b4e20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6270009c60 tx=0x7f6270009260 comp rx=0 tx=0).stop 2024-05-09T04:51:52.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:52.167+0000 7f6287000700 1 -- 172.21.15.6:0/1046418810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f628010e910 msgr2=0x7f628019f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:52.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:52.167+0000 7f6287000700 1 --2- 172.21.15.6:0/1046418810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f628010e910 0x7f628019f5e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f627800ef30 tx=0x7f627800c3a0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:52.167+0000 7f6287000700 1 -- 172.21.15.6:0/1046418810 shutdown_connections 2024-05-09T04:51:52.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:52.167+0000 7f6287000700 1 --2- 172.21.15.6:0/1046418810 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f62640b2910 0x7f62640b4e20 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:52.167+0000 7f6287000700 1 --2- 172.21.15.6:0/1046418810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f628010e910 0x7f628019f5e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:52.167+0000 7f6287000700 1 --2- 172.21.15.6:0/1046418810 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f62801054e0 0x7f628019adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:52.167+0000 7f6287000700 1 --2- 172.21.15.6:0/1046418810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6280105e30 0x7f628019b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:52.167+0000 7f6287000700 1 -- 172.21.15.6:0/1046418810 >> 172.21.15.6:0/1046418810 conn(0x7f62800fedc0 msgr2=0x7f62800ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:52.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:52.167+0000 7f6287000700 1 -- 172.21.15.6:0/1046418810 shutdown_connections 2024-05-09T04:51:52.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:52.168+0000 7f6287000700 1 -- 172.21.15.6:0/1046418810 wait complete. 2024-05-09T04:51:52.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:52 smithi006 ceph-mon[29835]: pgmap v407: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 596 B/s wr, 11 op/s 2024-05-09T04:51:52.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:52 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1046418810' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-09T04:51:53.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:52 smithi186 ceph-mon[34668]: pgmap v407: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 596 B/s wr, 11 op/s 2024-05-09T04:51:53.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:52 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/1046418810' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-09T04:51:53.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:52 smithi169 ceph-mon[34075]: pgmap v407: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 596 B/s wr, 11 op/s 2024-05-09T04:51:53.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:52 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/1046418810' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-09T04:51:53.139 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":2997423326},{"type":"v1","addr":"172.21.15.6:6801","nonce":2997423326}]},"active_addr":"172.21.15.6:6801/2997423326","active_change":"2024-05-09T04:40:55.357191+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.6:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":3701138119}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":1900352833}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":2114267993}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":3649951131}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":2975738599}]}]} 2024-05-09T04:51:53.144 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-09T04:51:53.145 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-09T04:51:53.145 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T04:51:53.678 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:51:54.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:54 smithi006 ceph-mon[29835]: pgmap v408: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-09T04:51:55.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:54 smithi186 ceph-mon[34668]: pgmap v408: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-09T04:51:55.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:54 smithi169 ceph-mon[34075]: pgmap v408: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-09T04:51:55.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:55.991+0000 7f93e8400700 1 -- 172.21.15.6:0/1708062146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93e0106570 msgr2=0x7f93e0106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:55.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:55.991+0000 7f93e8400700 1 --2- 172.21.15.6:0/1708062146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93e0106570 0x7f93e0106980 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f93cc0098f0 tx=0x7f93cc009c00 comp rx=0 tx=0).stop 2024-05-09T04:51:55.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:55.991+0000 7f93e8400700 1 -- 172.21.15.6:0/1708062146 shutdown_connections 2024-05-09T04:51:55.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:55.991+0000 7f93e8400700 1 --2- 172.21.15.6:0/1708062146 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93e0106ec0 0x7f93e0113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:55.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:55.991+0000 7f93e8400700 1 --2- 172.21.15.6:0/1708062146 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93e010c5b0 0x7f93e010c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:55.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:55.991+0000 7f93e8400700 1 --2- 172.21.15.6:0/1708062146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93e0106570 0x7f93e0106980 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:55.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:55.991+0000 7f93e8400700 1 -- 172.21.15.6:0/1708062146 >> 172.21.15.6:0/1708062146 conn(0x7f93e0102050 msgr2=0x7f93e01044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:55.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:55.991+0000 7f93e8400700 1 -- 172.21.15.6:0/1708062146 shutdown_connections 2024-05-09T04:51:55.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:55.991+0000 7f93e8400700 1 -- 172.21.15.6:0/1708062146 wait complete. 2024-05-09T04:51:55.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:55.993+0000 7f93e8400700 1 Processor -- start 2024-05-09T04:51:55.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:55.993+0000 7f93e8400700 1 -- start start 2024-05-09T04:51:55.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:55.993+0000 7f93e8400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93e0106570 0x7f93e019d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:55.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:55.993+0000 7f93e8400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93e0106ec0 0x7f93e019d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:55.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:55.993+0000 7f93e8400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93e010c5b0 0x7f93e01a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:55.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:55.993+0000 7f93e8400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f93e0116700 con 0x7f93e0106ec0 2024-05-09T04:51:55.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:55.993+0000 7f93e8400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f93e0116580 con 0x7f93e0106570 2024-05-09T04:51:55.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:55.993+0000 7f93e8400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f93e0116880 con 0x7f93e010c5b0 2024-05-09T04:51:55.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:55.994+0000 7f93e5a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93e0106570 0x7f93e019d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:55.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:55.994+0000 7f93e5000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93e0106ec0 0x7f93e019d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:55.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:55.994+0000 7f93e6400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93e010c5b0 0x7f93e01a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:55.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:55.994+0000 7f93e5a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93e0106570 0x7f93e019d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:49666/0 (socket says 172.21.15.6:49666) 2024-05-09T04:51:55.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:55.994+0000 7f93e5a00700 1 -- 172.21.15.6:0/1339681808 learned_addr learned my addr 172.21.15.6:0/1339681808 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:51:55.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:55.994+0000 7f93e5000700 1 -- 172.21.15.6:0/1339681808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93e010c5b0 msgr2=0x7f93e01a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:55.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:55.994+0000 7f93e5000700 1 --2- 172.21.15.6:0/1339681808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93e010c5b0 0x7f93e01a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:55.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:55.994+0000 7f93e5000700 1 -- 172.21.15.6:0/1339681808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93e0106570 msgr2=0x7f93e019d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:55.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:55.994+0000 7f93e5000700 1 --2- 172.21.15.6:0/1339681808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93e0106570 0x7f93e019d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:55.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:55.994+0000 7f93e5000700 1 -- 172.21.15.6:0/1339681808 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e01a1f80 con 0x7f93e0106ec0 2024-05-09T04:51:55.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:55.995+0000 7f93e5000700 1 --2- 172.21.15.6:0/1339681808 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93e0106ec0 0x7f93e019d650 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f93cc009230 tx=0x7f93cc00f970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:55.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:55.995+0000 7f93de000700 1 -- 172.21.15.6:0/1339681808 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc005040 con 0x7f93e0106ec0 2024-05-09T04:51:55.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:55.995+0000 7f93e8400700 1 -- 172.21.15.6:0/1339681808 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93e01a2210 con 0x7f93e0106ec0 2024-05-09T04:51:55.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:55.995+0000 7f93de000700 1 -- 172.21.15.6:0/1339681808 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93cc005690 con 0x7f93e0106ec0 2024-05-09T04:51:55.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:55.995+0000 7f93de000700 1 -- 172.21.15.6:0/1339681808 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc003990 con 0x7f93e0106ec0 2024-05-09T04:51:55.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:55.995+0000 7f93e8400700 1 -- 172.21.15.6:0/1339681808 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93e01a26c0 con 0x7f93e0106ec0 2024-05-09T04:51:55.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:55.996+0000 7f93e8400700 1 -- 172.21.15.6:0/1339681808 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93e00512c0 con 0x7f93e0106ec0 2024-05-09T04:51:56.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:56.001+0000 7f93de000700 1 -- 172.21.15.6:0/1339681808 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f93cc00b7f0 con 0x7f93e0106ec0 2024-05-09T04:51:56.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:56.002+0000 7f93de000700 1 --2- 172.21.15.6:0/1339681808 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f93c80b2910 0x7f93c80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:56.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:56.003+0000 7f93de000700 1 -- 172.21.15.6:0/1339681808 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f93cc0e3750 con 0x7f93e0106ec0 2024-05-09T04:51:56.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:56.003+0000 7f93e5a00700 1 --2- 172.21.15.6:0/1339681808 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f93c80b2910 0x7f93c80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:56.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:56.004+0000 7f93e5a00700 1 --2- 172.21.15.6:0/1339681808 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f93c80b2910 0x7f93c80b4e20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f93d4009d00 tx=0x7f93d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:56.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:56.005+0000 7f93de000700 1 -- 172.21.15.6:0/1339681808 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f93cc013030 con 0x7f93e0106ec0 2024-05-09T04:51:56.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:56.244+0000 7f93e8400700 1 -- 172.21.15.6:0/1339681808 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f93e002e5f0 con 0x7f93e0106ec0 2024-05-09T04:51:56.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:56.248+0000 7f93de000700 1 -- 172.21.15.6:0/1339681808 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24659 (secure 0 0 0) 0x7f93cc0aafc0 con 0x7f93e0106ec0 2024-05-09T04:51:56.249 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:51:56.250 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":89,"fsid":"f74d813c-0dbd-11ef-bc97-c7b262605968","created":"2024-05-09T04:39:35.287470+0000","modified":"2024-05-09T04:50:57.686719+0000","last_up_change":"2024-05-09T04:48:16.919842+0000","last_in_change":"2024-05-09T04:47:56.452645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:43:47.538306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:49:28.805447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T04:49:32.970287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T04:49:45.637000+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1b732fd3-e648-4b0d-aece-083bdaaa5d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6803","nonce":1738750914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6805","nonce":1738750914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6809","nonce":1738750914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6807","nonce":1738750914}]},"public_addr":"172.21.15.6:6803/1738750914","cluster_addr":"172.21.15.6:6805/1738750914","heartbeat_back_addr":"172.21.15.6:6809/1738750914","heartbeat_front_addr":"172.21.15.6:6807/1738750914","state":["exists","up"]},{"osd":1,"uuid":"9770c07d-4243-43bf-a0cd-9d2ac558556c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6801","nonce":2552207737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6803","nonce":2552207737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6807","nonce":2552207737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6805","nonce":2552207737}]},"public_addr":"172.21.15.169:6801/2552207737","cluster_addr":"172.21.15.169:6803/2552207737","heartbeat_back_addr":"172.21.15.169:6807/2552207737","heartbeat_front_addr":"172.21.15.169:6805/2552207737","state":["exists","up"]},{"osd":2,"uuid":"3f50ed89-b2a8-415e-b78d-8510d98d9368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6801","nonce":231435105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6803","nonce":231435105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6807","nonce":231435105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6805","nonce":231435105}]},"public_addr":"172.21.15.186:6801/231435105","cluster_addr":"172.21.15.186:6803/231435105","heartbeat_back_addr":"172.21.15.186:6807/231435105","heartbeat_front_addr":"172.21.15.186:6805/231435105","state":["exists","up"]},{"osd":3,"uuid":"afcbc36b-5669-4044-8dc5-2ebfe7970774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6811","nonce":2846613703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6813","nonce":2846613703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6817","nonce":2846613703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6815","nonce":2846613703}]},"public_addr":"172.21.15.6:6811/2846613703","cluster_addr":"172.21.15.6:6813/2846613703","heartbeat_back_addr":"172.21.15.6:6817/2846613703","heartbeat_front_addr":"172.21.15.6:6815/2846613703","state":["exists","up"]},{"osd":4,"uuid":"bb84cc5e-0297-4e23-be50-7ee2f7e59665","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6809","nonce":1928722453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6811","nonce":1928722453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6815","nonce":1928722453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6813","nonce":1928722453}]},"public_addr":"172.21.15.169:6809/1928722453","cluster_addr":"172.21.15.169:6811/1928722453","heartbeat_back_addr":"172.21.15.169:6815/1928722453","heartbeat_front_addr":"172.21.15.169:6813/1928722453","state":["exists","up"]},{"osd":5,"uuid":"830dc7f3-352e-4b01-8d4c-a9b68dcb40d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6809","nonce":3644445439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6811","nonce":3644445439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6815","nonce":3644445439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6813","nonce":3644445439}]},"public_addr":"172.21.15.186:6809/3644445439","cluster_addr":"172.21.15.186:6811/3644445439","heartbeat_back_addr":"172.21.15.186:6815/3644445439","heartbeat_front_addr":"172.21.15.186:6813/3644445439","state":["exists","up"]},{"osd":6,"uuid":"2e8478cb-c8c9-47ce-a3a5-48ffc959ab24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6819","nonce":3068252891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6821","nonce":3068252891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6825","nonce":3068252891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6823","nonce":3068252891}]},"public_addr":"172.21.15.6:6819/3068252891","cluster_addr":"172.21.15.6:6821/3068252891","heartbeat_back_addr":"172.21.15.6:6825/3068252891","heartbeat_front_addr":"172.21.15.6:6823/3068252891","state":["exists","up"]},{"osd":7,"uuid":"0f432ee2-42a4-4efa-9233-8e5a2fd65d10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6817","nonce":1303584711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6819","nonce":1303584711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6823","nonce":1303584711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6821","nonce":1303584711}]},"public_addr":"172.21.15.169:6817/1303584711","cluster_addr":"172.21.15.169:6819/1303584711","heartbeat_back_addr":"172.21.15.169:6823/1303584711","heartbeat_front_addr":"172.21.15.169:6821/1303584711","state":["exists","up"]},{"osd":8,"uuid":"cf5dca81-131b-48e7-9417-d01d4be47316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6817","nonce":3475163771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6819","nonce":3475163771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6823","nonce":3475163771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6821","nonce":3475163771}]},"public_addr":"172.21.15.186:6817/3475163771","cluster_addr":"172.21.15.186:6819/3475163771","heartbeat_back_addr":"172.21.15.186:6823/3475163771","heartbeat_front_addr":"172.21.15.186:6821/3475163771","state":["exists","up"]},{"osd":9,"uuid":"bca7bedd-1a57-4465-9033-064ca468add2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6827","nonce":3590990973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6829","nonce":3590990973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6833","nonce":3590990973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6831","nonce":3590990973}]},"public_addr":"172.21.15.6:6827/3590990973","cluster_addr":"172.21.15.6:6829/3590990973","heartbeat_back_addr":"172.21.15.6:6833/3590990973","heartbeat_front_addr":"172.21.15.6:6831/3590990973","state":["exists","up"]},{"osd":10,"uuid":"97b0e058-64f8-42ee-9b99-3ef8e39c8a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6825","nonce":61007341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6827","nonce":61007341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6831","nonce":61007341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6829","nonce":61007341}]},"public_addr":"172.21.15.169:6825/61007341","cluster_addr":"172.21.15.169:6827/61007341","heartbeat_back_addr":"172.21.15.169:6831/61007341","heartbeat_front_addr":"172.21.15.169:6829/61007341","state":["exists","up"]},{"osd":11,"uuid":"7f2220b0-24cf-48b2-95ae-1d66e00e57fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6825","nonce":1138940441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6827","nonce":1138940441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6831","nonce":1138940441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6829","nonce":1138940441}]},"public_addr":"172.21.15.186:6825/1138940441","cluster_addr":"172.21.15.186:6827/1138940441","heartbeat_back_addr":"172.21.15.186:6831/1138940441","heartbeat_front_addr":"172.21.15.186:6829/1138940441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:53.766295+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:20.195243+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:44.153812+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:20.472086+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:48.415371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:13.795415+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:50.236398+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:18.066623+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:43.781009+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:47:20.084719+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:47:48.219901+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:48:14.965536+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/3742411767":"2024-05-10T04:40:55.357057+0000","172.21.15.6:0/2410771702":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6801/937713898":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6800/937713898":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6801/3021503778":"2024-05-10T04:40:19.639151+0000","172.21.15.6:6800/3021503778":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/2559510850":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/3810474743":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/4133970555":"2024-05-10T04:40:55.357057+0000","172.21.15.6:0/664817743":"2024-05-10T04:40:19.639151+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:51:56.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:56.251+0000 7f93e8400700 1 -- 172.21.15.6:0/1339681808 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f93c80b2910 msgr2=0x7f93c80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:56.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:56.251+0000 7f93e8400700 1 --2- 172.21.15.6:0/1339681808 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f93c80b2910 0x7f93c80b4e20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f93d4009d00 tx=0x7f93d4009260 comp rx=0 tx=0).stop 2024-05-09T04:51:56.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:56.251+0000 7f93e8400700 1 -- 172.21.15.6:0/1339681808 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93e0106ec0 msgr2=0x7f93e019d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:56.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:56.251+0000 7f93e8400700 1 --2- 172.21.15.6:0/1339681808 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93e0106ec0 0x7f93e019d650 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f93cc009230 tx=0x7f93cc00f970 comp rx=0 tx=0).stop 2024-05-09T04:51:56.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:56.252+0000 7f93e8400700 1 -- 172.21.15.6:0/1339681808 shutdown_connections 2024-05-09T04:51:56.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:56.252+0000 7f93e8400700 1 --2- 172.21.15.6:0/1339681808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93e010c5b0 0x7f93e01a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:56.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:56.252+0000 7f93e8400700 1 --2- 172.21.15.6:0/1339681808 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f93c80b2910 0x7f93c80b4e20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:56.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:56.252+0000 7f93e8400700 1 --2- 172.21.15.6:0/1339681808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93e0106570 0x7f93e019d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:56.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:56.252+0000 7f93e8400700 1 --2- 172.21.15.6:0/1339681808 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93e0106ec0 0x7f93e019d650 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:56.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:56.252+0000 7f93e8400700 1 -- 172.21.15.6:0/1339681808 >> 172.21.15.6:0/1339681808 conn(0x7f93e0102050 msgr2=0x7f93e0103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:56.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:56.252+0000 7f93e8400700 1 -- 172.21.15.6:0/1339681808 shutdown_connections 2024-05-09T04:51:56.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:56.252+0000 7f93e8400700 1 -- 172.21.15.6:0/1339681808 wait complete. 2024-05-09T04:51:56.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:56 smithi006 ceph-mon[29835]: pgmap v409: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-05-09T04:51:56.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:56 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1339681808' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:51:57.068 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-09T04:51:57.068 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T04:51:57.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:56 smithi186 ceph-mon[34668]: pgmap v409: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-05-09T04:51:57.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:56 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/1339681808' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:51:57.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:56 smithi169 ceph-mon[34075]: pgmap v409: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-05-09T04:51:57.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:56 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/1339681808' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:51:57.535 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:51:58.825 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:51:58 smithi006 ceph-mon[29835]: pgmap v410: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:51:59.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:51:58 smithi186 ceph-mon[34668]: pgmap v410: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:51:59.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:51:58 smithi169 ceph-mon[34075]: pgmap v410: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:51:59.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.752+0000 7f1090400700 1 -- 172.21.15.6:0/2363131274 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1088076680 msgr2=0x7f1088110660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:59.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.752+0000 7f1090400700 1 --2- 172.21.15.6:0/2363131274 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1088076680 0x7f1088110660 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1074009990 tx=0x7f1074009ca0 comp rx=0 tx=0).stop 2024-05-09T04:51:59.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.752+0000 7f1090400700 1 -- 172.21.15.6:0/2363131274 shutdown_connections 2024-05-09T04:51:59.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.752+0000 7f1090400700 1 --2- 172.21.15.6:0/2363131274 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1088110ba0 0x7f1088113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:59.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.752+0000 7f1090400700 1 --2- 172.21.15.6:0/2363131274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1088075d30 0x7f1088076140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:59.754 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.752+0000 7f1090400700 1 --2- 172.21.15.6:0/2363131274 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1088076680 0x7f1088110660 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:59.754 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.752+0000 7f1090400700 1 -- 172.21.15.6:0/2363131274 >> 172.21.15.6:0/2363131274 conn(0x7f1088100040 msgr2=0x7f1088102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:59.754 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.752+0000 7f1090400700 1 -- 172.21.15.6:0/2363131274 shutdown_connections 2024-05-09T04:51:59.754 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.752+0000 7f1090400700 1 -- 172.21.15.6:0/2363131274 wait complete. 2024-05-09T04:51:59.754 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.754+0000 7f1090400700 1 Processor -- start 2024-05-09T04:51:59.754 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.754+0000 7f1090400700 1 -- start start 2024-05-09T04:51:59.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.754+0000 7f1090400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1088075d30 0x7f108819d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:59.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.754+0000 7f1090400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1088076680 0x7f108819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:59.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.754+0000 7f1090400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1088110ba0 0x7f10881a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:59.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.754+0000 7f1090400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f10881166f0 con 0x7f1088110ba0 2024-05-09T04:51:59.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.754+0000 7f1090400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1088116570 con 0x7f1088076680 2024-05-09T04:51:59.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.754+0000 7f1090400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1088116870 con 0x7f1088075d30 2024-05-09T04:51:59.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.755+0000 7f108da00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1088075d30 0x7f108819d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:59.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.755+0000 7f108da00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1088075d30 0x7f108819d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:33790/0 (socket says 172.21.15.6:33790) 2024-05-09T04:51:59.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.755+0000 7f108da00700 1 -- 172.21.15.6:0/2509804305 learned_addr learned my addr 172.21.15.6:0/2509804305 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:51:59.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.755+0000 7f108e400700 1 --2- 172.21.15.6:0/2509804305 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1088110ba0 0x7f10881a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:59.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.755+0000 7f108d000700 1 --2- 172.21.15.6:0/2509804305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1088076680 0x7f108819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:59.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.755+0000 7f108da00700 1 -- 172.21.15.6:0/2509804305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1088076680 msgr2=0x7f108819d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:59.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.755+0000 7f108da00700 1 --2- 172.21.15.6:0/2509804305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1088076680 0x7f108819d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:59.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.755+0000 7f108da00700 1 -- 172.21.15.6:0/2509804305 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1088110ba0 msgr2=0x7f10881a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:59.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.755+0000 7f108da00700 1 --2- 172.21.15.6:0/2509804305 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1088110ba0 0x7f10881a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:59.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.755+0000 7f108da00700 1 -- 172.21.15.6:0/2509804305 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10881a1f70 con 0x7f1088075d30 2024-05-09T04:51:59.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.755+0000 7f108da00700 1 --2- 172.21.15.6:0/2509804305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1088075d30 0x7f108819d0a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f107c00d950 tx=0x7f107c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:59.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.756+0000 7f1086000700 1 -- 172.21.15.6:0/2509804305 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107c016070 con 0x7f1088075d30 2024-05-09T04:51:59.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.756+0000 7f1086000700 1 -- 172.21.15.6:0/2509804305 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f107c00fd40 con 0x7f1088075d30 2024-05-09T04:51:59.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.756+0000 7f1090400700 1 -- 172.21.15.6:0/2509804305 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10881a2260 con 0x7f1088075d30 2024-05-09T04:51:59.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.756+0000 7f1086000700 1 -- 172.21.15.6:0/2509804305 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107c00e3b0 con 0x7f1088075d30 2024-05-09T04:51:59.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.756+0000 7f1090400700 1 -- 172.21.15.6:0/2509804305 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10881a9b80 con 0x7f1088075d30 2024-05-09T04:51:59.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.757+0000 7f1090400700 1 -- 172.21.15.6:0/2509804305 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10880512c0 con 0x7f1088075d30 2024-05-09T04:51:59.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.760+0000 7f1086000700 1 -- 172.21.15.6:0/2509804305 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f107c019070 con 0x7f1088075d30 2024-05-09T04:51:59.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.761+0000 7f1086000700 1 --2- 172.21.15.6:0/2509804305 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f10700b2990 0x7f10700b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:59.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.761+0000 7f1086000700 1 -- 172.21.15.6:0/2509804305 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f107c0e2180 con 0x7f1088075d30 2024-05-09T04:51:59.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.761+0000 7f108d000700 1 --2- 172.21.15.6:0/2509804305 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f10700b2990 0x7f10700b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:59.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.762+0000 7f108d000700 1 --2- 172.21.15.6:0/2509804305 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f10700b2990 0x7f10700b4ea0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1074003800 tx=0x7f107400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:59.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:51:59.763+0000 7f1086000700 1 -- 172.21.15.6:0/2509804305 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f107c0a9960 con 0x7f1088075d30 2024-05-09T04:52:00.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:00.003+0000 7f1090400700 1 -- 172.21.15.6:0/2509804305 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f108802e5f0 con 0x7f1088075d30 2024-05-09T04:52:00.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:00.005+0000 7f1086000700 1 -- 172.21.15.6:0/2509804305 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24659 (secure 0 0 0) 0x7f107c0a9260 con 0x7f1088075d30 2024-05-09T04:52:00.007 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:52:00.008 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":89,"fsid":"f74d813c-0dbd-11ef-bc97-c7b262605968","created":"2024-05-09T04:39:35.287470+0000","modified":"2024-05-09T04:50:57.686719+0000","last_up_change":"2024-05-09T04:48:16.919842+0000","last_in_change":"2024-05-09T04:47:56.452645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:43:47.538306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:49:28.805447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T04:49:32.970287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T04:49:45.637000+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1b732fd3-e648-4b0d-aece-083bdaaa5d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6803","nonce":1738750914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6805","nonce":1738750914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6809","nonce":1738750914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6807","nonce":1738750914}]},"public_addr":"172.21.15.6:6803/1738750914","cluster_addr":"172.21.15.6:6805/1738750914","heartbeat_back_addr":"172.21.15.6:6809/1738750914","heartbeat_front_addr":"172.21.15.6:6807/1738750914","state":["exists","up"]},{"osd":1,"uuid":"9770c07d-4243-43bf-a0cd-9d2ac558556c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6801","nonce":2552207737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6803","nonce":2552207737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6807","nonce":2552207737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6805","nonce":2552207737}]},"public_addr":"172.21.15.169:6801/2552207737","cluster_addr":"172.21.15.169:6803/2552207737","heartbeat_back_addr":"172.21.15.169:6807/2552207737","heartbeat_front_addr":"172.21.15.169:6805/2552207737","state":["exists","up"]},{"osd":2,"uuid":"3f50ed89-b2a8-415e-b78d-8510d98d9368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6801","nonce":231435105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6803","nonce":231435105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6807","nonce":231435105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6805","nonce":231435105}]},"public_addr":"172.21.15.186:6801/231435105","cluster_addr":"172.21.15.186:6803/231435105","heartbeat_back_addr":"172.21.15.186:6807/231435105","heartbeat_front_addr":"172.21.15.186:6805/231435105","state":["exists","up"]},{"osd":3,"uuid":"afcbc36b-5669-4044-8dc5-2ebfe7970774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6811","nonce":2846613703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6813","nonce":2846613703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6817","nonce":2846613703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6815","nonce":2846613703}]},"public_addr":"172.21.15.6:6811/2846613703","cluster_addr":"172.21.15.6:6813/2846613703","heartbeat_back_addr":"172.21.15.6:6817/2846613703","heartbeat_front_addr":"172.21.15.6:6815/2846613703","state":["exists","up"]},{"osd":4,"uuid":"bb84cc5e-0297-4e23-be50-7ee2f7e59665","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6809","nonce":1928722453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6811","nonce":1928722453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6815","nonce":1928722453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6813","nonce":1928722453}]},"public_addr":"172.21.15.169:6809/1928722453","cluster_addr":"172.21.15.169:6811/1928722453","heartbeat_back_addr":"172.21.15.169:6815/1928722453","heartbeat_front_addr":"172.21.15.169:6813/1928722453","state":["exists","up"]},{"osd":5,"uuid":"830dc7f3-352e-4b01-8d4c-a9b68dcb40d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6809","nonce":3644445439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6811","nonce":3644445439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6815","nonce":3644445439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6813","nonce":3644445439}]},"public_addr":"172.21.15.186:6809/3644445439","cluster_addr":"172.21.15.186:6811/3644445439","heartbeat_back_addr":"172.21.15.186:6815/3644445439","heartbeat_front_addr":"172.21.15.186:6813/3644445439","state":["exists","up"]},{"osd":6,"uuid":"2e8478cb-c8c9-47ce-a3a5-48ffc959ab24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6819","nonce":3068252891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6821","nonce":3068252891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6825","nonce":3068252891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6823","nonce":3068252891}]},"public_addr":"172.21.15.6:6819/3068252891","cluster_addr":"172.21.15.6:6821/3068252891","heartbeat_back_addr":"172.21.15.6:6825/3068252891","heartbeat_front_addr":"172.21.15.6:6823/3068252891","state":["exists","up"]},{"osd":7,"uuid":"0f432ee2-42a4-4efa-9233-8e5a2fd65d10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6817","nonce":1303584711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6819","nonce":1303584711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6823","nonce":1303584711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6821","nonce":1303584711}]},"public_addr":"172.21.15.169:6817/1303584711","cluster_addr":"172.21.15.169:6819/1303584711","heartbeat_back_addr":"172.21.15.169:6823/1303584711","heartbeat_front_addr":"172.21.15.169:6821/1303584711","state":["exists","up"]},{"osd":8,"uuid":"cf5dca81-131b-48e7-9417-d01d4be47316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6817","nonce":3475163771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6819","nonce":3475163771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6823","nonce":3475163771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6821","nonce":3475163771}]},"public_addr":"172.21.15.186:6817/3475163771","cluster_addr":"172.21.15.186:6819/3475163771","heartbeat_back_addr":"172.21.15.186:6823/3475163771","heartbeat_front_addr":"172.21.15.186:6821/3475163771","state":["exists","up"]},{"osd":9,"uuid":"bca7bedd-1a57-4465-9033-064ca468add2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6827","nonce":3590990973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6829","nonce":3590990973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6833","nonce":3590990973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6831","nonce":3590990973}]},"public_addr":"172.21.15.6:6827/3590990973","cluster_addr":"172.21.15.6:6829/3590990973","heartbeat_back_addr":"172.21.15.6:6833/3590990973","heartbeat_front_addr":"172.21.15.6:6831/3590990973","state":["exists","up"]},{"osd":10,"uuid":"97b0e058-64f8-42ee-9b99-3ef8e39c8a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6825","nonce":61007341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6827","nonce":61007341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6831","nonce":61007341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6829","nonce":61007341}]},"public_addr":"172.21.15.169:6825/61007341","cluster_addr":"172.21.15.169:6827/61007341","heartbeat_back_addr":"172.21.15.169:6831/61007341","heartbeat_front_addr":"172.21.15.169:6829/61007341","state":["exists","up"]},{"osd":11,"uuid":"7f2220b0-24cf-48b2-95ae-1d66e00e57fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6825","nonce":1138940441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6827","nonce":1138940441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6831","nonce":1138940441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6829","nonce":1138940441}]},"public_addr":"172.21.15.186:6825/1138940441","cluster_addr":"172.21.15.186:6827/1138940441","heartbeat_back_addr":"172.21.15.186:6831/1138940441","heartbeat_front_addr":"172.21.15.186:6829/1138940441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:53.766295+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:20.195243+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:44.153812+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:20.472086+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:48.415371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:13.795415+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:50.236398+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:18.066623+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:43.781009+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:47:20.084719+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:47:48.219901+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:48:14.965536+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/3742411767":"2024-05-10T04:40:55.357057+0000","172.21.15.6:0/2410771702":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6801/937713898":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6800/937713898":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6801/3021503778":"2024-05-10T04:40:19.639151+0000","172.21.15.6:6800/3021503778":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/2559510850":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/3810474743":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/4133970555":"2024-05-10T04:40:55.357057+0000","172.21.15.6:0/664817743":"2024-05-10T04:40:19.639151+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:52:00.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:00.009+0000 7f1090400700 1 -- 172.21.15.6:0/2509804305 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f10700b2990 msgr2=0x7f10700b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:00.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:00.009+0000 7f1090400700 1 --2- 172.21.15.6:0/2509804305 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f10700b2990 0x7f10700b4ea0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1074003800 tx=0x7f107400b540 comp rx=0 tx=0).stop 2024-05-09T04:52:00.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:00.009+0000 7f1090400700 1 -- 172.21.15.6:0/2509804305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1088075d30 msgr2=0x7f108819d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:00.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:00.009+0000 7f1090400700 1 --2- 172.21.15.6:0/2509804305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1088075d30 0x7f108819d0a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f107c00d950 tx=0x7f107c00dc60 comp rx=0 tx=0).stop 2024-05-09T04:52:00.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:00.010+0000 7f1090400700 1 -- 172.21.15.6:0/2509804305 shutdown_connections 2024-05-09T04:52:00.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:00.010+0000 7f1090400700 1 --2- 172.21.15.6:0/2509804305 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1088110ba0 0x7f10881a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:00.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:00.010+0000 7f1090400700 1 --2- 172.21.15.6:0/2509804305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1088075d30 0x7f108819d0a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:00.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:00.010+0000 7f1090400700 1 --2- 172.21.15.6:0/2509804305 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f10700b2990 0x7f10700b4ea0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:00.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:00.010+0000 7f1090400700 1 --2- 172.21.15.6:0/2509804305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1088076680 0x7f108819d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:00.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:00.010+0000 7f1090400700 1 -- 172.21.15.6:0/2509804305 >> 172.21.15.6:0/2509804305 conn(0x7f1088100040 msgr2=0x7f1088100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:00.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:00.010+0000 7f1090400700 1 -- 172.21.15.6:0/2509804305 shutdown_connections 2024-05-09T04:52:00.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:00.010+0000 7f1090400700 1 -- 172.21.15.6:0/2509804305 wait complete. 2024-05-09T04:52:00.895 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-09T04:52:00.895 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-09T04:52:00.896 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-09T04:52:00.896 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-09T04:52:00.896 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-09T04:52:00.897 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-09T04:52:00.897 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-09T04:52:00.897 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-09T04:52:00.898 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-09T04:52:00.898 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-09T04:52:00.899 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-09T04:52:00.899 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-09T04:52:01.019 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:00 smithi006 ceph-mon[29835]: pgmap v411: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:52:01.019 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:00 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/2509804305' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:52:01.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:00 smithi186 ceph-mon[34668]: pgmap v411: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:52:01.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:00 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/2509804305' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:52:01.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:00 smithi169 ceph-mon[34075]: pgmap v411: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:52:01.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:00 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/2509804305' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:52:01.937 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:01.949 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:01.959 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:02.029 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:02.130 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:02.131 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:02.131 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:02.132 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:02.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:02 smithi006 ceph-mon[29835]: pgmap v412: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:52:03.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:02 smithi186 ceph-mon[34668]: pgmap v412: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:52:03.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:02 smithi169 ceph-mon[34075]: pgmap v412: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:52:05.026 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:04 smithi006 ceph-mon[29835]: pgmap v413: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:52:05.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:04 smithi186 ceph-mon[34668]: pgmap v413: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:52:05.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:04 smithi169 ceph-mon[34075]: pgmap v413: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:52:05.487 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:05.820 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:05.821 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:06.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:06 smithi006 ceph-mon[29835]: pgmap v414: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:52:07.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:06 smithi186 ceph-mon[34668]: pgmap v414: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:52:07.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:06 smithi169 ceph-mon[34075]: pgmap v414: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:52:08.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.199+0000 7f9139c00700 1 -- 172.21.15.6:0/3605386658 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9134108740 msgr2=0x7f91341100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:08.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.199+0000 7f9139c00700 1 --2- 172.21.15.6:0/3605386658 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9134108740 0x7f91341100a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f912c00b0d0 tx=0x7f912c00b3e0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.199+0000 7f9139c00700 1 -- 172.21.15.6:0/3605386658 shutdown_connections 2024-05-09T04:52:08.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.199+0000 7f9139c00700 1 --2- 172.21.15.6:0/3605386658 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9134108740 0x7f91341100a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.199+0000 7f9139c00700 1 --2- 172.21.15.6:0/3605386658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91341034c0 0x7f91341038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.199+0000 7f9139c00700 1 --2- 172.21.15.6:0/3605386658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9134107df0 0x7f9134108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.199+0000 7f9139c00700 1 -- 172.21.15.6:0/3605386658 >> 172.21.15.6:0/3605386658 conn(0x7f91340fedc0 msgr2=0x7f91341011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:08.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.200+0000 7f9139c00700 1 -- 172.21.15.6:0/3605386658 shutdown_connections 2024-05-09T04:52:08.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.200+0000 7f9139c00700 1 -- 172.21.15.6:0/3605386658 wait complete. 2024-05-09T04:52:08.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.201+0000 7f9139c00700 1 Processor -- start 2024-05-09T04:52:08.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.201+0000 7f9139c00700 1 -- start start 2024-05-09T04:52:08.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.202+0000 7f9139c00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91341034c0 0x7f913419ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:08.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.202+0000 7f9139c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9134107df0 0x7f913419b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:08.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.202+0000 7f9139c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9134108740 0x7f913419f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:08.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.202+0000 7f9139c00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9134113370 con 0x7f9134108740 2024-05-09T04:52:08.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.202+0000 7f9139c00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f91341131f0 con 0x7f91341034c0 2024-05-09T04:52:08.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.202+0000 7f9139c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f91341134f0 con 0x7f9134107df0 2024-05-09T04:52:08.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.202+0000 7f9132a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91341034c0 0x7f913419ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:08.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.202+0000 7f9132a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91341034c0 0x7f913419ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:56892/0 (socket says 172.21.15.6:56892) 2024-05-09T04:52:08.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.202+0000 7f9132a00700 1 -- 172.21.15.6:0/2863331823 learned_addr learned my addr 172.21.15.6:0/2863331823 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:52:08.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.202+0000 7f9133400700 1 --2- 172.21.15.6:0/2863331823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9134108740 0x7f913419f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:08.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.202+0000 7f9132000700 1 --2- 172.21.15.6:0/2863331823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9134107df0 0x7f913419b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:08.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.203+0000 7f9132a00700 1 -- 172.21.15.6:0/2863331823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9134107df0 msgr2=0x7f913419b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:08.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.203+0000 7f9132a00700 1 --2- 172.21.15.6:0/2863331823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9134107df0 0x7f913419b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.203+0000 7f9132a00700 1 -- 172.21.15.6:0/2863331823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9134108740 msgr2=0x7f913419f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:08.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.203+0000 7f9132a00700 1 --2- 172.21.15.6:0/2863331823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9134108740 0x7f913419f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.203+0000 7f9132a00700 1 -- 172.21.15.6:0/2863331823 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f913419fc00 con 0x7f91341034c0 2024-05-09T04:52:08.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.203+0000 7f9132a00700 1 --2- 172.21.15.6:0/2863331823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91341034c0 0x7f913419ad30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f912400e9a0 tx=0x7f912400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:08.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.204+0000 7f912b400700 1 -- 172.21.15.6:0/2863331823 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912400ca10 con 0x7f91341034c0 2024-05-09T04:52:08.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.204+0000 7f912b400700 1 -- 172.21.15.6:0/2863331823 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f912400cbb0 con 0x7f91341034c0 2024-05-09T04:52:08.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.204+0000 7f9139c00700 1 -- 172.21.15.6:0/2863331823 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f913419fef0 con 0x7f91341034c0 2024-05-09T04:52:08.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.204+0000 7f912b400700 1 -- 172.21.15.6:0/2863331823 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9124018680 con 0x7f91341034c0 2024-05-09T04:52:08.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.204+0000 7f9139c00700 1 -- 172.21.15.6:0/2863331823 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91341a7810 con 0x7f91341034c0 2024-05-09T04:52:08.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.205+0000 7f9139c00700 1 -- 172.21.15.6:0/2863331823 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9134063f50 con 0x7f91341034c0 2024-05-09T04:52:08.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.208+0000 7f912b400700 1 -- 172.21.15.6:0/2863331823 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f9124010430 con 0x7f91341034c0 2024-05-09T04:52:08.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.210+0000 7f912b400700 1 --2- 172.21.15.6:0/2863331823 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f91180b2990 0x7f91180b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:08.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.210+0000 7f912b400700 1 -- 172.21.15.6:0/2863331823 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f91240e2e20 con 0x7f91341034c0 2024-05-09T04:52:08.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.211+0000 7f9132000700 1 --2- 172.21.15.6:0/2863331823 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f91180b2990 0x7f91180b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:08.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.211+0000 7f912b400700 1 --2- 172.21.15.6:0/2863331823 >> [v2:172.21.15.169:6800/2552207737,v1:172.21.15.169:6801/2552207737] conn(0x7f91180bdbc0 0x7f91180c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:08.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.211+0000 7f912b400700 1 -- 172.21.15.6:0/2863331823 --> [v2:172.21.15.169:6800/2552207737,v1:172.21.15.169:6801/2552207737] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9124024e70 con 0x7f91180bdbc0 2024-05-09T04:52:08.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.211+0000 7f912b400700 1 -- 172.21.15.6:0/2863331823 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f91240e3270 con 0x7f91341034c0 2024-05-09T04:52:08.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.211+0000 7f9133400700 1 --2- 172.21.15.6:0/2863331823 >> [v2:172.21.15.169:6800/2552207737,v1:172.21.15.169:6801/2552207737] conn(0x7f91180bdbc0 0x7f91180c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:08.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.212+0000 7f9132000700 1 --2- 172.21.15.6:0/2863331823 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f91180b2990 0x7f91180b4ea0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9134109900 tx=0x7f9120009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:08.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.212+0000 7f9133400700 1 --2- 172.21.15.6:0/2863331823 >> [v2:172.21.15.169:6800/2552207737,v1:172.21.15.169:6801/2552207737] conn(0x7f91180bdbc0 0x7f91180c00b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:08.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.213+0000 7f912b400700 1 -- 172.21.15.6:0/2863331823 <== osd.1 v2:172.21.15.169:6800/2552207737 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9124024e70 con 0x7f91180bdbc0 2024-05-09T04:52:08.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.239+0000 7f9139c00700 1 -- 172.21.15.6:0/2863331823 --> [v2:172.21.15.169:6800/2552207737,v1:172.21.15.169:6801/2552207737] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9134064350 con 0x7f91180bdbc0 2024-05-09T04:52:08.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.240+0000 7f912b400700 1 -- 172.21.15.6:0/2863331823 <== osd.1 v2:172.21.15.169:6800/2552207737 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9134064350 con 0x7f91180bdbc0 2024-05-09T04:52:08.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.240+0000 7f9139c00700 1 -- 172.21.15.6:0/2863331823 >> [v2:172.21.15.169:6800/2552207737,v1:172.21.15.169:6801/2552207737] conn(0x7f91180bdbc0 msgr2=0x7f91180c00b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:08.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.240+0000 7f9139c00700 1 --2- 172.21.15.6:0/2863331823 >> [v2:172.21.15.169:6800/2552207737,v1:172.21.15.169:6801/2552207737] conn(0x7f91180bdbc0 0x7f91180c00b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.241+0000 7f9139c00700 1 -- 172.21.15.6:0/2863331823 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f91180b2990 msgr2=0x7f91180b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:08.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.241+0000 7f9139c00700 1 --2- 172.21.15.6:0/2863331823 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f91180b2990 0x7f91180b4ea0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9134109900 tx=0x7f9120009260 comp rx=0 tx=0).stop 2024-05-09T04:52:08.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.241+0000 7f9139c00700 1 -- 172.21.15.6:0/2863331823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91341034c0 msgr2=0x7f913419ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:08.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.241+0000 7f9139c00700 1 --2- 172.21.15.6:0/2863331823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91341034c0 0x7f913419ad30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f912400e9a0 tx=0x7f912400ecb0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.241+0000 7f9133400700 1 -- 172.21.15.6:0/2863331823 reap_dead start 2024-05-09T04:52:08.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.241+0000 7f9139c00700 1 -- 172.21.15.6:0/2863331823 shutdown_connections 2024-05-09T04:52:08.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.241+0000 7f9139c00700 1 -- 172.21.15.6:0/2863331823 >> 172.21.15.6:0/2863331823 conn(0x7f91340fedc0 msgr2=0x7f913406b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:08.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.241+0000 7f9139c00700 1 -- 172.21.15.6:0/2863331823 shutdown_connections 2024-05-09T04:52:08.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:08.241+0000 7f9139c00700 1 -- 172.21.15.6:0/2863331823 wait complete. 2024-05-09T04:52:09.026 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:08 smithi006 ceph-mon[29835]: pgmap v415: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:52:09.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:08 smithi186 ceph-mon[34668]: pgmap v415: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:52:09.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:08 smithi169 ceph-mon[34075]: pgmap v415: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:52:09.838 INFO:teuthology.orchestra.run.smithi006.stdout:55834574956 2024-05-09T04:52:09.839 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-09T04:52:11.041 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:10 smithi006 ceph-mon[29835]: pgmap v416: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:52:11.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:10 smithi186 ceph-mon[34668]: pgmap v416: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:52:11.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:10 smithi169 ceph-mon[34075]: pgmap v416: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:52:11.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.395+0000 7fe7ba200700 1 -- 172.21.15.6:0/3881096808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7b4109870 msgr2=0x7fe7b41111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:11.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.395+0000 7fe7ba200700 1 --2- 172.21.15.6:0/3881096808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7b4109870 0x7fe7b41111d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac009f90 tx=0x7fe7ac00b330 comp rx=0 tx=0).stop 2024-05-09T04:52:11.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.396+0000 7fe7ba200700 1 -- 172.21.15.6:0/3881096808 shutdown_connections 2024-05-09T04:52:11.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.396+0000 7fe7ba200700 1 --2- 172.21.15.6:0/3881096808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7b4109870 0x7fe7b41111d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.396+0000 7fe7ba200700 1 --2- 172.21.15.6:0/3881096808 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7b41045f0 0x7fe7b4104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.396+0000 7fe7ba200700 1 --2- 172.21.15.6:0/3881096808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7b4108f20 0x7fe7b4109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.396+0000 7fe7ba200700 1 -- 172.21.15.6:0/3881096808 >> 172.21.15.6:0/3881096808 conn(0x7fe7b4100040 msgr2=0x7fe7b4102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:11.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.396+0000 7fe7ba200700 1 -- 172.21.15.6:0/3881096808 shutdown_connections 2024-05-09T04:52:11.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.396+0000 7fe7ba200700 1 -- 172.21.15.6:0/3881096808 wait complete. 2024-05-09T04:52:11.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.397+0000 7fe7ba200700 1 Processor -- start 2024-05-09T04:52:11.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.398+0000 7fe7ba200700 1 -- start start 2024-05-09T04:52:11.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.398+0000 7fe7ba200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7b41045f0 0x7fe7b419d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:11.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.398+0000 7fe7ba200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7b4108f20 0x7fe7b419d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:11.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.399+0000 7fe7ba200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7b4109870 0x7fe7b41a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:11.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.399+0000 7fe7ba200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b41144f0 con 0x7fe7b4109870 2024-05-09T04:52:11.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.399+0000 7fe7ba200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b4114370 con 0x7fe7b4108f20 2024-05-09T04:52:11.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.399+0000 7fe7ba200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b4114670 con 0x7fe7b41045f0 2024-05-09T04:52:11.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.399+0000 7fe7b3400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7b41045f0 0x7fe7b419d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:11.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.399+0000 7fe7b3e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7b4109870 0x7fe7b41a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:11.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.399+0000 7fe7b3400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7b41045f0 0x7fe7b419d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:57084/0 (socket says 172.21.15.6:57084) 2024-05-09T04:52:11.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.399+0000 7fe7b3400700 1 -- 172.21.15.6:0/3843389626 learned_addr learned my addr 172.21.15.6:0/3843389626 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:52:11.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.399+0000 7fe7b2a00700 1 --2- 172.21.15.6:0/3843389626 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7b4108f20 0x7fe7b419d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:11.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.400+0000 7fe7b3400700 1 -- 172.21.15.6:0/3843389626 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7b4108f20 msgr2=0x7fe7b419d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:11.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.400+0000 7fe7b3400700 1 --2- 172.21.15.6:0/3843389626 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7b4108f20 0x7fe7b419d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.400+0000 7fe7b3400700 1 -- 172.21.15.6:0/3843389626 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7b4109870 msgr2=0x7fe7b41a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:11.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.400+0000 7fe7b3400700 1 --2- 172.21.15.6:0/3843389626 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7b4109870 0x7fe7b41a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.400+0000 7fe7b3400700 1 -- 172.21.15.6:0/3843389626 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b41a1f90 con 0x7fe7b41045f0 2024-05-09T04:52:11.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.400+0000 7fe7b3e00700 1 --2- 172.21.15.6:0/3843389626 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7b4109870 0x7fe7b41a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:11.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.400+0000 7fe7b3400700 1 --2- 172.21.15.6:0/3843389626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7b41045f0 0x7fe7b419d0c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe7a400e9c0 tx=0x7fe7a400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:11.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.400+0000 7fe7abe00700 1 -- 172.21.15.6:0/3843389626 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4004a30 con 0x7fe7b41045f0 2024-05-09T04:52:11.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.401+0000 7fe7ba200700 1 -- 172.21.15.6:0/3843389626 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7b41a2280 con 0x7fe7b41045f0 2024-05-09T04:52:11.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.401+0000 7fe7abe00700 1 -- 172.21.15.6:0/3843389626 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7a400cc70 con 0x7fe7b41045f0 2024-05-09T04:52:11.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.401+0000 7fe7abe00700 1 -- 172.21.15.6:0/3843389626 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a40054a0 con 0x7fe7b41045f0 2024-05-09T04:52:11.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.401+0000 7fe7ba200700 1 -- 172.21.15.6:0/3843389626 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7b41a9ba0 con 0x7fe7b41045f0 2024-05-09T04:52:11.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.402+0000 7fe7ba200700 1 -- 172.21.15.6:0/3843389626 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe7b4076cd0 con 0x7fe7b41045f0 2024-05-09T04:52:11.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.405+0000 7fe7abe00700 1 -- 172.21.15.6:0/3843389626 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fe7a4004bd0 con 0x7fe7b41045f0 2024-05-09T04:52:11.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.406+0000 7fe7abe00700 1 --2- 172.21.15.6:0/3843389626 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe7980b2910 0x7fe7980b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:11.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.407+0000 7fe7abe00700 1 -- 172.21.15.6:0/3843389626 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fe7a40e2c90 con 0x7fe7b41045f0 2024-05-09T04:52:11.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.407+0000 7fe7b2a00700 1 --2- 172.21.15.6:0/3843389626 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe7980b2910 0x7fe7980b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:11.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.407+0000 7fe7abe00700 1 --2- 172.21.15.6:0/3843389626 >> [v2:172.21.15.169:6808/1928722453,v1:172.21.15.169:6809/1928722453] conn(0x7fe7980bdb40 0x7fe7980c0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:11.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.407+0000 7fe7abe00700 1 -- 172.21.15.6:0/3843389626 --> [v2:172.21.15.169:6808/1928722453,v1:172.21.15.169:6809/1928722453] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe7a400be50 con 0x7fe7980bdb40 2024-05-09T04:52:11.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.407+0000 7fe7abe00700 1 -- 172.21.15.6:0/3843389626 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fe7a40e3040 con 0x7fe7b41045f0 2024-05-09T04:52:11.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.407+0000 7fe7b3e00700 1 --2- 172.21.15.6:0/3843389626 >> [v2:172.21.15.169:6808/1928722453,v1:172.21.15.169:6809/1928722453] conn(0x7fe7980bdb40 0x7fe7980c0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:11.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.408+0000 7fe7b2a00700 1 --2- 172.21.15.6:0/3843389626 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe7980b2910 0x7fe7980b4e20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4103a00 tx=0x7fe7a000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:11.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.408+0000 7fe7b3e00700 1 --2- 172.21.15.6:0/3843389626 >> [v2:172.21.15.169:6808/1928722453,v1:172.21.15.169:6809/1928722453] conn(0x7fe7980bdb40 0x7fe7980c0030 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:11.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.409+0000 7fe7abe00700 1 -- 172.21.15.6:0/3843389626 <== osd.4 v2:172.21.15.169:6808/1928722453 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe7a400be50 con 0x7fe7980bdb40 2024-05-09T04:52:11.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.428+0000 7fe7ba200700 1 -- 172.21.15.6:0/3843389626 --> [v2:172.21.15.169:6808/1928722453,v1:172.21.15.169:6809/1928722453] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe7b4105aa0 con 0x7fe7980bdb40 2024-05-09T04:52:11.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.429+0000 7fe7abe00700 1 -- 172.21.15.6:0/3843389626 <== osd.4 v2:172.21.15.169:6808/1928722453 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe7b4105aa0 con 0x7fe7980bdb40 2024-05-09T04:52:11.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.429+0000 7fe7ba200700 1 -- 172.21.15.6:0/3843389626 >> [v2:172.21.15.169:6808/1928722453,v1:172.21.15.169:6809/1928722453] conn(0x7fe7980bdb40 msgr2=0x7fe7980c0030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:11.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.429+0000 7fe7ba200700 1 --2- 172.21.15.6:0/3843389626 >> [v2:172.21.15.169:6808/1928722453,v1:172.21.15.169:6809/1928722453] conn(0x7fe7980bdb40 0x7fe7980c0030 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.430+0000 7fe7ba200700 1 -- 172.21.15.6:0/3843389626 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe7980b2910 msgr2=0x7fe7980b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:11.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.430+0000 7fe7ba200700 1 --2- 172.21.15.6:0/3843389626 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe7980b2910 0x7fe7980b4e20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4103a00 tx=0x7fe7a000b410 comp rx=0 tx=0).stop 2024-05-09T04:52:11.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.430+0000 7fe7ba200700 1 -- 172.21.15.6:0/3843389626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7b41045f0 msgr2=0x7fe7b419d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:11.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.430+0000 7fe7ba200700 1 --2- 172.21.15.6:0/3843389626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7b41045f0 0x7fe7b419d0c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe7a400e9c0 tx=0x7fe7a400ecd0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.430+0000 7fe7b3e00700 1 -- 172.21.15.6:0/3843389626 reap_dead start 2024-05-09T04:52:11.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.430+0000 7fe7ba200700 1 -- 172.21.15.6:0/3843389626 shutdown_connections 2024-05-09T04:52:11.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.430+0000 7fe7ba200700 1 -- 172.21.15.6:0/3843389626 >> 172.21.15.6:0/3843389626 conn(0x7fe7b4100040 msgr2=0x7fe7b4108660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:11.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.430+0000 7fe7ba200700 1 -- 172.21.15.6:0/3843389626 shutdown_connections 2024-05-09T04:52:11.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:11.430+0000 7fe7ba200700 1 -- 172.21.15.6:0/3843389626 wait complete. 2024-05-09T04:52:13.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:12 smithi186 ceph-mon[34668]: pgmap v417: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:52:13.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:12 smithi169 ceph-mon[34075]: pgmap v417: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:52:13.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:12 smithi006 ceph-mon[29835]: pgmap v417: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:52:15.017 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:14 smithi006 ceph-mon[29835]: pgmap v418: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:52:15.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:14 smithi186 ceph-mon[34668]: pgmap v418: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:52:15.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:14 smithi169 ceph-mon[34075]: pgmap v418: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:52:16.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.006+0000 7f9a7ec00700 1 -- 172.21.15.6:0/3227103514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a78103550 msgr2=0x7f9a78103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:16.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.006+0000 7f9a7ec00700 1 --2- 172.21.15.6:0/3227103514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a78103550 0x7f9a78103960 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9a680099c0 tx=0x7f9a68009cd0 comp rx=0 tx=0).stop 2024-05-09T04:52:16.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.006+0000 7f9a7ec00700 1 -- 172.21.15.6:0/3227103514 shutdown_connections 2024-05-09T04:52:16.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.006+0000 7f9a7ec00700 1 --2- 172.21.15.6:0/3227103514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a78108780 0x7f9a781100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:16.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.006+0000 7f9a7ec00700 1 --2- 172.21.15.6:0/3227103514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a78103550 0x7f9a78103960 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:16.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.006+0000 7f9a7ec00700 1 --2- 172.21.15.6:0/3227103514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a78107e30 0x7f9a78108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:16.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.006+0000 7f9a7ec00700 1 -- 172.21.15.6:0/3227103514 >> 172.21.15.6:0/3227103514 conn(0x7f9a78078e00 msgr2=0x7f9a78079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:16.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.006+0000 7f9a7ec00700 1 -- 172.21.15.6:0/3227103514 shutdown_connections 2024-05-09T04:52:16.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.006+0000 7f9a7ec00700 1 -- 172.21.15.6:0/3227103514 wait complete. 2024-05-09T04:52:16.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.008+0000 7f9a7ec00700 1 Processor -- start 2024-05-09T04:52:16.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.008+0000 7f9a7ec00700 1 -- start start 2024-05-09T04:52:16.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.008+0000 7f9a7ec00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a78103550 0x7f9a7819d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:16.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.008+0000 7f9a7ec00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a78107e30 0x7f9a7819d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:16.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.008+0000 7f9a7ec00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a78108780 0x7f9a781a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:16.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.008+0000 7f9a7ec00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9a78113400 con 0x7f9a78108780 2024-05-09T04:52:16.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.008+0000 7f9a7ec00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9a78113280 con 0x7f9a78107e30 2024-05-09T04:52:16.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.008+0000 7f9a7ec00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9a78113580 con 0x7f9a78103550 2024-05-09T04:52:16.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.009+0000 7f9a77e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a78103550 0x7f9a7819d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:16.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.009+0000 7f9a7cc00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a78108780 0x7f9a781a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:16.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.009+0000 7f9a77e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a78103550 0x7f9a7819d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:57098/0 (socket says 172.21.15.6:57098) 2024-05-09T04:52:16.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.009+0000 7f9a77e00700 1 -- 172.21.15.6:0/4073004892 learned_addr learned my addr 172.21.15.6:0/4073004892 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:52:16.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.009+0000 7f9a7cc00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a78108780 0x7f9a781a1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39556/0 (socket says 172.21.15.6:39556) 2024-05-09T04:52:16.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.009+0000 7f9a77400700 1 --2- 172.21.15.6:0/4073004892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a78107e30 0x7f9a7819d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:16.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.009+0000 7f9a77e00700 1 -- 172.21.15.6:0/4073004892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a78107e30 msgr2=0x7f9a7819d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:16.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.009+0000 7f9a77e00700 1 --2- 172.21.15.6:0/4073004892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a78107e30 0x7f9a7819d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:16.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.009+0000 7f9a77e00700 1 -- 172.21.15.6:0/4073004892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a78108780 msgr2=0x7f9a781a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:16.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.009+0000 7f9a77e00700 1 --2- 172.21.15.6:0/4073004892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a78108780 0x7f9a781a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:16.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.009+0000 7f9a77e00700 1 -- 172.21.15.6:0/4073004892 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a781a1fc0 con 0x7f9a78103550 2024-05-09T04:52:16.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.010+0000 7f9a7cc00700 1 --2- 172.21.15.6:0/4073004892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a78108780 0x7f9a781a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:16.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.010+0000 7f9a77400700 1 --2- 172.21.15.6:0/4073004892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a78107e30 0x7f9a7819d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:16.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.010+0000 7f9a77e00700 1 --2- 172.21.15.6:0/4073004892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a78103550 0x7f9a7819d0f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9a68006010 tx=0x7f9a68022b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:16.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.010+0000 7f9a74c00700 1 -- 172.21.15.6:0/4073004892 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a680052c0 con 0x7f9a78103550 2024-05-09T04:52:16.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.010+0000 7f9a74c00700 1 -- 172.21.15.6:0/4073004892 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a68003c70 con 0x7f9a78103550 2024-05-09T04:52:16.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.010+0000 7f9a7ec00700 1 -- 172.21.15.6:0/4073004892 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a781a2250 con 0x7f9a78103550 2024-05-09T04:52:16.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.011+0000 7f9a74c00700 1 -- 172.21.15.6:0/4073004892 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6800f790 con 0x7f9a78103550 2024-05-09T04:52:16.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.012+0000 7f9a7ec00700 1 -- 172.21.15.6:0/4073004892 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a781a2730 con 0x7f9a78103550 2024-05-09T04:52:16.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.012+0000 7f9a7ec00700 1 -- 172.21.15.6:0/4073004892 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9a781026e0 con 0x7f9a78103550 2024-05-09T04:52:16.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.015+0000 7f9a74c00700 1 -- 172.21.15.6:0/4073004892 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f9a680037c0 con 0x7f9a78103550 2024-05-09T04:52:16.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.016+0000 7f9a74c00700 1 --2- 172.21.15.6:0/4073004892 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9a5c0b2910 0x7f9a5c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:16.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.016+0000 7f9a74c00700 1 -- 172.21.15.6:0/4073004892 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f9a680ebfe0 con 0x7f9a78103550 2024-05-09T04:52:16.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.017+0000 7f9a77400700 1 --2- 172.21.15.6:0/4073004892 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9a5c0b2910 0x7f9a5c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:16.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.017+0000 7f9a74c00700 1 --2- 172.21.15.6:0/4073004892 >> [v2:172.21.15.186:6808/3644445439,v1:172.21.15.186:6809/3644445439] conn(0x7f9a5c0bdb40 0x7f9a5c0c0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:16.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.017+0000 7f9a74c00700 1 -- 172.21.15.6:0/4073004892 --> [v2:172.21.15.186:6808/3644445439,v1:172.21.15.186:6809/3644445439] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9a68027e80 con 0x7f9a5c0bdb40 2024-05-09T04:52:16.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.017+0000 7f9a74c00700 1 -- 172.21.15.6:0/4073004892 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f9a680f3050 con 0x7f9a78103550 2024-05-09T04:52:16.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.017+0000 7f9a7cc00700 1 --2- 172.21.15.6:0/4073004892 >> [v2:172.21.15.186:6808/3644445439,v1:172.21.15.186:6809/3644445439] conn(0x7f9a5c0bdb40 0x7f9a5c0c0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:16.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.018+0000 7f9a77400700 1 --2- 172.21.15.6:0/4073004892 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9a5c0b2910 0x7f9a5c0b4e20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9a78076980 tx=0x7f9a64009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:16.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.018+0000 7f9a7cc00700 1 --2- 172.21.15.6:0/4073004892 >> [v2:172.21.15.186:6808/3644445439,v1:172.21.15.186:6809/3644445439] conn(0x7f9a5c0bdb40 0x7f9a5c0c0030 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:16.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.019+0000 7f9a74c00700 1 -- 172.21.15.6:0/4073004892 <== osd.5 v2:172.21.15.186:6808/3644445439 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9a68027e80 con 0x7f9a5c0bdb40 2024-05-09T04:52:16.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.037+0000 7f9a7ec00700 1 -- 172.21.15.6:0/4073004892 --> [v2:172.21.15.186:6808/3644445439,v1:172.21.15.186:6809/3644445439] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9a781049b0 con 0x7f9a5c0bdb40 2024-05-09T04:52:16.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.038+0000 7f9a74c00700 1 -- 172.21.15.6:0/4073004892 <== osd.5 v2:172.21.15.186:6808/3644445439 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9a781049b0 con 0x7f9a5c0bdb40 2024-05-09T04:52:16.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.039+0000 7f9a7ec00700 1 -- 172.21.15.6:0/4073004892 >> [v2:172.21.15.186:6808/3644445439,v1:172.21.15.186:6809/3644445439] conn(0x7f9a5c0bdb40 msgr2=0x7f9a5c0c0030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:16.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.039+0000 7f9a7ec00700 1 --2- 172.21.15.6:0/4073004892 >> [v2:172.21.15.186:6808/3644445439,v1:172.21.15.186:6809/3644445439] conn(0x7f9a5c0bdb40 0x7f9a5c0c0030 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:16.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.039+0000 7f9a7ec00700 1 -- 172.21.15.6:0/4073004892 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9a5c0b2910 msgr2=0x7f9a5c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:16.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.039+0000 7f9a7ec00700 1 --2- 172.21.15.6:0/4073004892 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9a5c0b2910 0x7f9a5c0b4e20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9a78076980 tx=0x7f9a64009260 comp rx=0 tx=0).stop 2024-05-09T04:52:16.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.039+0000 7f9a7ec00700 1 -- 172.21.15.6:0/4073004892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a78103550 msgr2=0x7f9a7819d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:16.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.039+0000 7f9a7ec00700 1 --2- 172.21.15.6:0/4073004892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a78103550 0x7f9a7819d0f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9a68006010 tx=0x7f9a68022b10 comp rx=0 tx=0).stop 2024-05-09T04:52:16.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.039+0000 7f9a7cc00700 1 -- 172.21.15.6:0/4073004892 reap_dead start 2024-05-09T04:52:16.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.039+0000 7f9a7ec00700 1 -- 172.21.15.6:0/4073004892 shutdown_connections 2024-05-09T04:52:16.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.039+0000 7f9a7ec00700 1 -- 172.21.15.6:0/4073004892 >> 172.21.15.6:0/4073004892 conn(0x7f9a78078e00 msgr2=0x7f9a78101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:16.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.039+0000 7f9a7ec00700 1 -- 172.21.15.6:0/4073004892 shutdown_connections 2024-05-09T04:52:16.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.039+0000 7f9a7ec00700 1 -- 172.21.15.6:0/4073004892 wait complete. 2024-05-09T04:52:16.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.190+0000 7f863f800700 1 -- 172.21.15.6:0/3543126043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f863810e8d0 msgr2=0x7f8638110db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:16.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.190+0000 7f863f800700 1 --2- 172.21.15.6:0/3543126043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f863810e8d0 0x7f8638110db0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f863000b0a0 tx=0x7f863000b3b0 comp rx=0 tx=0).stop 2024-05-09T04:52:16.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.190+0000 7f863f800700 1 -- 172.21.15.6:0/3543126043 shutdown_connections 2024-05-09T04:52:16.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.190+0000 7f863f800700 1 --2- 172.21.15.6:0/3543126043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f863810e8d0 0x7f8638110db0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:16.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.190+0000 7f863f800700 1 --2- 172.21.15.6:0/3543126043 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8638105460 0x7f8638105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:16.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.190+0000 7f863f800700 1 --2- 172.21.15.6:0/3543126043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8638105db0 0x7f863810e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:16.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.190+0000 7f863f800700 1 -- 172.21.15.6:0/3543126043 >> 172.21.15.6:0/3543126043 conn(0x7f86380fedc0 msgr2=0x7f86381011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:16.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.191+0000 7f863f800700 1 -- 172.21.15.6:0/3543126043 shutdown_connections 2024-05-09T04:52:16.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.191+0000 7f863f800700 1 -- 172.21.15.6:0/3543126043 wait complete. 2024-05-09T04:52:16.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.192+0000 7f863f800700 1 Processor -- start 2024-05-09T04:52:16.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.192+0000 7f863f800700 1 -- start start 2024-05-09T04:52:16.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.192+0000 7f863f800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8638105460 0x7f8638103150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:16.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.192+0000 7f863f800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8638105db0 0x7f8638103690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:16.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.192+0000 7f863f800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f863810e8d0 0x7f8638103bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:16.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.192+0000 7f863f800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8638114430 con 0x7f863810e8d0 2024-05-09T04:52:16.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.192+0000 7f863f800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f86381142b0 con 0x7f8638105db0 2024-05-09T04:52:16.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.192+0000 7f863f800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f86381145b0 con 0x7f8638105460 2024-05-09T04:52:16.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.193+0000 7f863ce00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8638105460 0x7f8638103150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:16.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.193+0000 7f863d800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f863810e8d0 0x7f8638103bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:16.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.193+0000 7f863ce00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8638105460 0x7f8638103150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:57112/0 (socket says 172.21.15.6:57112) 2024-05-09T04:52:16.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.193+0000 7f863ce00700 1 -- 172.21.15.6:0/2153604043 learned_addr learned my addr 172.21.15.6:0/2153604043 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:52:16.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.193+0000 7f8637e00700 1 --2- 172.21.15.6:0/2153604043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8638105db0 0x7f8638103690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:16.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.193+0000 7f863ce00700 1 -- 172.21.15.6:0/2153604043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8638105db0 msgr2=0x7f8638103690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:16.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.193+0000 7f863ce00700 1 --2- 172.21.15.6:0/2153604043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8638105db0 0x7f8638103690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:16.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.193+0000 7f863ce00700 1 -- 172.21.15.6:0/2153604043 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f863810e8d0 msgr2=0x7f8638103bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:16.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.193+0000 7f863ce00700 1 --2- 172.21.15.6:0/2153604043 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f863810e8d0 0x7f8638103bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:16.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.193+0000 7f863ce00700 1 -- 172.21.15.6:0/2153604043 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86381a90e0 con 0x7f8638105460 2024-05-09T04:52:16.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.194+0000 7f863ce00700 1 --2- 172.21.15.6:0/2153604043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8638105460 0x7f8638103150 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f862c00e9a0 tx=0x7f862c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:16.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.194+0000 7f8635600700 1 -- 172.21.15.6:0/2153604043 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862c00ca10 con 0x7f8638105460 2024-05-09T04:52:16.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.194+0000 7f8635600700 1 -- 172.21.15.6:0/2153604043 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f862c00cbb0 con 0x7f8638105460 2024-05-09T04:52:16.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.194+0000 7f863f800700 1 -- 172.21.15.6:0/2153604043 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86381a93d0 con 0x7f8638105460 2024-05-09T04:52:16.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.194+0000 7f8635600700 1 -- 172.21.15.6:0/2153604043 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862c018680 con 0x7f8638105460 2024-05-09T04:52:16.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.194+0000 7f863f800700 1 -- 172.21.15.6:0/2153604043 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86381a9910 con 0x7f8638105460 2024-05-09T04:52:16.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.195+0000 7f863f800700 1 -- 172.21.15.6:0/2153604043 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f86380ffd50 con 0x7f8638105460 2024-05-09T04:52:16.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.198+0000 7f8635600700 1 -- 172.21.15.6:0/2153604043 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f862c010430 con 0x7f8638105460 2024-05-09T04:52:16.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.199+0000 7f8635600700 1 --2- 172.21.15.6:0/2153604043 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f86200b2910 0x7f86200b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:16.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.199+0000 7f8635600700 1 -- 172.21.15.6:0/2153604043 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f862c0e2f00 con 0x7f8638105460 2024-05-09T04:52:16.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.200+0000 7f8637e00700 1 --2- 172.21.15.6:0/2153604043 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f86200b2910 0x7f86200b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:16.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.200+0000 7f8635600700 1 --2- 172.21.15.6:0/2153604043 >> [v2:172.21.15.186:6800/231435105,v1:172.21.15.186:6801/231435105] conn(0x7f86200bdb40 0x7f86200c0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:16.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.200+0000 7f8635600700 1 -- 172.21.15.6:0/2153604043 --> [v2:172.21.15.186:6800/231435105,v1:172.21.15.186:6801/231435105] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f862c024e70 con 0x7f86200bdb40 2024-05-09T04:52:16.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.200+0000 7f8635600700 1 -- 172.21.15.6:0/2153604043 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f862c0e32b0 con 0x7f8638105460 2024-05-09T04:52:16.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.200+0000 7f863d800700 1 --2- 172.21.15.6:0/2153604043 >> [v2:172.21.15.186:6800/231435105,v1:172.21.15.186:6801/231435105] conn(0x7f86200bdb40 0x7f86200c0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:16.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.201+0000 7f8637e00700 1 --2- 172.21.15.6:0/2153604043 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f86200b2910 0x7f86200b4e20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8638102320 tx=0x7f8624009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:16.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.201+0000 7f863d800700 1 --2- 172.21.15.6:0/2153604043 >> [v2:172.21.15.186:6800/231435105,v1:172.21.15.186:6801/231435105] conn(0x7f86200bdb40 0x7f86200c0030 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:16.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.202+0000 7f8635600700 1 -- 172.21.15.6:0/2153604043 <== osd.2 v2:172.21.15.186:6800/231435105 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f862c024e70 con 0x7f86200bdb40 2024-05-09T04:52:16.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.220+0000 7f863f800700 1 -- 172.21.15.6:0/2153604043 --> [v2:172.21.15.186:6800/231435105,v1:172.21.15.186:6801/231435105] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f863806bef0 con 0x7f86200bdb40 2024-05-09T04:52:16.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.221+0000 7f8635600700 1 -- 172.21.15.6:0/2153604043 <== osd.2 v2:172.21.15.186:6800/231435105 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f863806bef0 con 0x7f86200bdb40 2024-05-09T04:52:16.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.222+0000 7f863f800700 1 -- 172.21.15.6:0/2153604043 >> [v2:172.21.15.186:6800/231435105,v1:172.21.15.186:6801/231435105] conn(0x7f86200bdb40 msgr2=0x7f86200c0030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:16.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.222+0000 7f863f800700 1 --2- 172.21.15.6:0/2153604043 >> [v2:172.21.15.186:6800/231435105,v1:172.21.15.186:6801/231435105] conn(0x7f86200bdb40 0x7f86200c0030 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:16.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.222+0000 7f863f800700 1 -- 172.21.15.6:0/2153604043 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f86200b2910 msgr2=0x7f86200b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:16.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.222+0000 7f863f800700 1 --2- 172.21.15.6:0/2153604043 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f86200b2910 0x7f86200b4e20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8638102320 tx=0x7f8624009260 comp rx=0 tx=0).stop 2024-05-09T04:52:16.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.222+0000 7f863f800700 1 -- 172.21.15.6:0/2153604043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8638105460 msgr2=0x7f8638103150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:16.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.222+0000 7f863f800700 1 --2- 172.21.15.6:0/2153604043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8638105460 0x7f8638103150 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f862c00e9a0 tx=0x7f862c00ecb0 comp rx=0 tx=0).stop 2024-05-09T04:52:16.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.222+0000 7f863d800700 1 -- 172.21.15.6:0/2153604043 reap_dead start 2024-05-09T04:52:16.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.222+0000 7f863f800700 1 -- 172.21.15.6:0/2153604043 shutdown_connections 2024-05-09T04:52:16.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.222+0000 7f863f800700 1 -- 172.21.15.6:0/2153604043 >> 172.21.15.6:0/2153604043 conn(0x7f86380fedc0 msgr2=0x7f86380ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:16.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.222+0000 7f863f800700 1 -- 172.21.15.6:0/2153604043 shutdown_connections 2024-05-09T04:52:16.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:16.222+0000 7f863f800700 1 -- 172.21.15.6:0/2153604043 wait complete. 2024-05-09T04:52:17.101 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:16 smithi186 ceph-mon[34668]: pgmap v419: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:52:17.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:16 smithi169 ceph-mon[34075]: pgmap v419: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:52:17.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:16 smithi006 ceph-mon[29835]: pgmap v419: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:52:17.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.636+0000 7f416f000700 1 -- 172.21.15.6:0/1205947867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41681045e0 msgr2=0x7f41681049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:17.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.636+0000 7f416f000700 1 --2- 172.21.15.6:0/1205947867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41681045e0 0x7f41681049f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4158009920 tx=0x7f4158009c30 comp rx=0 tx=0).stop 2024-05-09T04:52:17.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.636+0000 7f416f000700 1 -- 172.21.15.6:0/1205947867 shutdown_connections 2024-05-09T04:52:17.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.636+0000 7f416f000700 1 --2- 172.21.15.6:0/1205947867 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4168109860 0x7f41681111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.636+0000 7f416f000700 1 --2- 172.21.15.6:0/1205947867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41681045e0 0x7f41681049f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.636+0000 7f416f000700 1 --2- 172.21.15.6:0/1205947867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4168108f10 0x7f4168109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.636+0000 7f416f000700 1 -- 172.21.15.6:0/1205947867 >> 172.21.15.6:0/1205947867 conn(0x7f41680ffe70 msgr2=0x7f41681022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:17.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.637+0000 7f416f000700 1 -- 172.21.15.6:0/1205947867 shutdown_connections 2024-05-09T04:52:17.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.637+0000 7f416f000700 1 -- 172.21.15.6:0/1205947867 wait complete. 2024-05-09T04:52:17.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.638+0000 7f416f000700 1 Processor -- start 2024-05-09T04:52:17.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.638+0000 7f416f000700 1 -- start start 2024-05-09T04:52:17.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.638+0000 7f416f000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4168108f10 0x7f4168075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:17.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.638+0000 7f416f000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4168109860 0x7f4168076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:17.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.638+0000 7f416f000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4168076540 0x7f41681a9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:17.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.639+0000 7f416f000700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4168114490 con 0x7f4168108f10 2024-05-09T04:52:17.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.639+0000 7f416f000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4168114310 con 0x7f4168109860 2024-05-09T04:52:17.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.639+0000 7f416f000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4168114610 con 0x7f4168076540 2024-05-09T04:52:17.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.639+0000 7f4167e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4168108f10 0x7f4168075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:17.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.639+0000 7f4167e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4168108f10 0x7f4168075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42326/0 (socket says 172.21.15.6:42326) 2024-05-09T04:52:17.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.639+0000 7f4167e00700 1 -- 172.21.15.6:0/2279159069 learned_addr learned my addr 172.21.15.6:0/2279159069 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:52:17.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.639+0000 7f416d000700 1 --2- 172.21.15.6:0/2279159069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4168076540 0x7f41681a9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:17.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.639+0000 7f4167400700 1 --2- 172.21.15.6:0/2279159069 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4168109860 0x7f4168076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:17.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.639+0000 7f4167e00700 1 -- 172.21.15.6:0/2279159069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4168076540 msgr2=0x7f41681a9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:17.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.639+0000 7f4167e00700 1 --2- 172.21.15.6:0/2279159069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4168076540 0x7f41681a9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.639+0000 7f4167e00700 1 -- 172.21.15.6:0/2279159069 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4168109860 msgr2=0x7f4168076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:17.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.639+0000 7f4167e00700 1 --2- 172.21.15.6:0/2279159069 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4168109860 0x7f4168076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.639+0000 7f4167e00700 1 -- 172.21.15.6:0/2279159069 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41681a9800 con 0x7f4168108f10 2024-05-09T04:52:17.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.639+0000 7f416d000700 1 --2- 172.21.15.6:0/2279159069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4168076540 0x7f41681a9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:17.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.640+0000 7f4167e00700 1 --2- 172.21.15.6:0/2279159069 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4168108f10 0x7f4168075ac0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4158000c00 tx=0x7f415800f9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:17.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.640+0000 7f4164c00700 1 -- 172.21.15.6:0/2279159069 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41580052d0 con 0x7f4168108f10 2024-05-09T04:52:17.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.640+0000 7f4164c00700 1 -- 172.21.15.6:0/2279159069 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f415800ba10 con 0x7f4168108f10 2024-05-09T04:52:17.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.640+0000 7f416f000700 1 -- 172.21.15.6:0/2279159069 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41681a9a90 con 0x7f4168108f10 2024-05-09T04:52:17.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.640+0000 7f4164c00700 1 -- 172.21.15.6:0/2279159069 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4158004bb0 con 0x7f4168108f10 2024-05-09T04:52:17.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.641+0000 7f416f000700 1 -- 172.21.15.6:0/2279159069 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41681a9e90 con 0x7f4168108f10 2024-05-09T04:52:17.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.642+0000 7f416f000700 1 -- 172.21.15.6:0/2279159069 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4168063ed0 con 0x7f4168108f10 2024-05-09T04:52:17.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.645+0000 7f4164c00700 1 -- 172.21.15.6:0/2279159069 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f4158005470 con 0x7f4168108f10 2024-05-09T04:52:17.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.646+0000 7f4164c00700 1 --2- 172.21.15.6:0/2279159069 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f414c0b2990 0x7f414c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:17.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.646+0000 7f4164c00700 1 -- 172.21.15.6:0/2279159069 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f41580e3d80 con 0x7f4168108f10 2024-05-09T04:52:17.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.646+0000 7f4167400700 1 --2- 172.21.15.6:0/2279159069 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f414c0b2990 0x7f414c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:17.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.646+0000 7f4164c00700 1 --2- 172.21.15.6:0/2279159069 >> [v2:172.21.15.186:6824/1138940441,v1:172.21.15.186:6825/1138940441] conn(0x7f414c0bdbc0 0x7f414c0c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:17.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.646+0000 7f4164c00700 1 -- 172.21.15.6:0/2279159069 --> [v2:172.21.15.186:6824/1138940441,v1:172.21.15.186:6825/1138940441] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f414c0c0780 con 0x7f414c0bdbc0 2024-05-09T04:52:17.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.646+0000 7f4164c00700 1 -- 172.21.15.6:0/2279159069 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f41580e4130 con 0x7f4168108f10 2024-05-09T04:52:17.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.647+0000 7f416d000700 1 --2- 172.21.15.6:0/2279159069 >> [v2:172.21.15.186:6824/1138940441,v1:172.21.15.186:6825/1138940441] conn(0x7f414c0bdbc0 0x7f414c0c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:17.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.647+0000 7f4167400700 1 --2- 172.21.15.6:0/2279159069 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f414c0b2990 0x7f414c0b4ea0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4154009680 tx=0x7f4154009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:17.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.647+0000 7f416d000700 1 --2- 172.21.15.6:0/2279159069 >> [v2:172.21.15.186:6824/1138940441,v1:172.21.15.186:6825/1138940441] conn(0x7f414c0bdbc0 0x7f414c0c00b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:17.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.648+0000 7f4164c00700 1 -- 172.21.15.6:0/2279159069 <== osd.11 v2:172.21.15.186:6824/1138940441 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f414c0c0780 con 0x7f414c0bdbc0 2024-05-09T04:52:17.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.667+0000 7f416f000700 1 -- 172.21.15.6:0/2279159069 --> [v2:172.21.15.186:6824/1138940441,v1:172.21.15.186:6825/1138940441] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4168064350 con 0x7f414c0bdbc0 2024-05-09T04:52:17.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.668+0000 7f4164c00700 1 -- 172.21.15.6:0/2279159069 <== osd.11 v2:172.21.15.186:6824/1138940441 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4168064350 con 0x7f414c0bdbc0 2024-05-09T04:52:17.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.669+0000 7f416f000700 1 -- 172.21.15.6:0/2279159069 >> [v2:172.21.15.186:6824/1138940441,v1:172.21.15.186:6825/1138940441] conn(0x7f414c0bdbc0 msgr2=0x7f414c0c00b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:17.669 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.669+0000 7f416f000700 1 --2- 172.21.15.6:0/2279159069 >> [v2:172.21.15.186:6824/1138940441,v1:172.21.15.186:6825/1138940441] conn(0x7f414c0bdbc0 0x7f414c0c00b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.669+0000 7f416f000700 1 -- 172.21.15.6:0/2279159069 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f414c0b2990 msgr2=0x7f414c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:17.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.669+0000 7f416f000700 1 --2- 172.21.15.6:0/2279159069 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f414c0b2990 0x7f414c0b4ea0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4154009680 tx=0x7f4154009260 comp rx=0 tx=0).stop 2024-05-09T04:52:17.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.669+0000 7f416f000700 1 -- 172.21.15.6:0/2279159069 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4168108f10 msgr2=0x7f4168075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:17.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.669+0000 7f416f000700 1 --2- 172.21.15.6:0/2279159069 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4168108f10 0x7f4168075ac0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4158000c00 tx=0x7f415800f9e0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.669+0000 7f416d000700 1 -- 172.21.15.6:0/2279159069 reap_dead start 2024-05-09T04:52:17.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.669+0000 7f416f000700 1 -- 172.21.15.6:0/2279159069 shutdown_connections 2024-05-09T04:52:17.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.669+0000 7f416f000700 1 -- 172.21.15.6:0/2279159069 >> 172.21.15.6:0/2279159069 conn(0x7f41680ffe70 msgr2=0x7f4168107450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:17.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.669+0000 7f416f000700 1 -- 172.21.15.6:0/2279159069 shutdown_connections 2024-05-09T04:52:17.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:17.669+0000 7f416f000700 1 -- 172.21.15.6:0/2279159069 wait complete. 2024-05-09T04:52:18.121 INFO:teuthology.orchestra.run.smithi006.stdout:77309411433 2024-05-09T04:52:18.122 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-09T04:52:18.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.409+0000 7febe7000700 1 -- 172.21.15.6:0/4280235745 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7febe0105df0 msgr2=0x7febe010e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:18.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.409+0000 7febe7000700 1 --2- 172.21.15.6:0/4280235745 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7febe0105df0 0x7febe010e340 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7febcc009990 tx=0x7febcc009ca0 comp rx=0 tx=0).stop 2024-05-09T04:52:18.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.409+0000 7febe7000700 1 -- 172.21.15.6:0/4280235745 shutdown_connections 2024-05-09T04:52:18.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.409+0000 7febe7000700 1 --2- 172.21.15.6:0/4280235745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febe010e910 0x7febe0110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:18.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.409+0000 7febe7000700 1 --2- 172.21.15.6:0/4280235745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febe01054a0 0x7febe01058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:18.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.409+0000 7febe7000700 1 --2- 172.21.15.6:0/4280235745 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7febe0105df0 0x7febe010e340 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:18.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.409+0000 7febe7000700 1 -- 172.21.15.6:0/4280235745 >> 172.21.15.6:0/4280235745 conn(0x7febe00fedc0 msgr2=0x7febe01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:18.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.409+0000 7febe7000700 1 -- 172.21.15.6:0/4280235745 shutdown_connections 2024-05-09T04:52:18.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.409+0000 7febe7000700 1 -- 172.21.15.6:0/4280235745 wait complete. 2024-05-09T04:52:18.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.411+0000 7febe7000700 1 Processor -- start 2024-05-09T04:52:18.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.411+0000 7febe7000700 1 -- start start 2024-05-09T04:52:18.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.412+0000 7febe7000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febe01054a0 0x7febe019d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:18.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.412+0000 7febe7000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febe0105df0 0x7febe019d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:18.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.412+0000 7febe7000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7febe010e910 0x7febe01a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:18.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.412+0000 7febe7000700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7febe01144a0 con 0x7febe010e910 2024-05-09T04:52:18.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.412+0000 7febe7000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7febe0114320 con 0x7febe01054a0 2024-05-09T04:52:18.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.412+0000 7febe7000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7febe0114620 con 0x7febe0105df0 2024-05-09T04:52:18.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.412+0000 7febe5000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7febe010e910 0x7febe01a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:18.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.412+0000 7febe5000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7febe010e910 0x7febe01a18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42340/0 (socket says 172.21.15.6:42340) 2024-05-09T04:52:18.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.412+0000 7febe5000700 1 -- 172.21.15.6:0/2419313211 learned_addr learned my addr 172.21.15.6:0/2419313211 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:52:18.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.412+0000 7febdf400700 1 --2- 172.21.15.6:0/2419313211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febe0105df0 0x7febe019d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:18.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.412+0000 7febdfe00700 1 --2- 172.21.15.6:0/2419313211 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febe01054a0 0x7febe019d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:18.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.413+0000 7febe5000700 1 -- 172.21.15.6:0/2419313211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febe0105df0 msgr2=0x7febe019d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:18.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.413+0000 7febe5000700 1 --2- 172.21.15.6:0/2419313211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febe0105df0 0x7febe019d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:18.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.413+0000 7febe5000700 1 -- 172.21.15.6:0/2419313211 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febe01054a0 msgr2=0x7febe019d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:18.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.413+0000 7febe5000700 1 --2- 172.21.15.6:0/2419313211 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febe01054a0 0x7febe019d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:18.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.413+0000 7febe5000700 1 -- 172.21.15.6:0/2419313211 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe01a1f60 con 0x7febe010e910 2024-05-09T04:52:18.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.413+0000 7febe5000700 1 --2- 172.21.15.6:0/2419313211 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7febe010e910 0x7febe01a18c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7febd800c370 tx=0x7febd800c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:18.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.413+0000 7febdcc00700 1 -- 172.21.15.6:0/2419313211 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd80092e0 con 0x7febe010e910 2024-05-09T04:52:18.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.414+0000 7febe7000700 1 -- 172.21.15.6:0/2419313211 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febe01a21f0 con 0x7febe010e910 2024-05-09T04:52:18.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.414+0000 7febe7000700 1 -- 172.21.15.6:0/2419313211 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febe01a9b70 con 0x7febe010e910 2024-05-09T04:52:18.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.414+0000 7febdcc00700 1 -- 172.21.15.6:0/2419313211 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febd8003a00 con 0x7febe010e910 2024-05-09T04:52:18.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.414+0000 7febdcc00700 1 -- 172.21.15.6:0/2419313211 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd80046f0 con 0x7febe010e910 2024-05-09T04:52:18.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.415+0000 7febe7000700 1 -- 172.21.15.6:0/2419313211 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7febe0103c60 con 0x7febe010e910 2024-05-09T04:52:18.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.418+0000 7febdcc00700 1 -- 172.21.15.6:0/2419313211 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7febd801a020 con 0x7febe010e910 2024-05-09T04:52:18.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.419+0000 7febdcc00700 1 --2- 172.21.15.6:0/2419313211 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7febc40b2990 0x7febc40b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:18.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.420+0000 7febdcc00700 1 -- 172.21.15.6:0/2419313211 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7febd80e2970 con 0x7febe010e910 2024-05-09T04:52:18.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.420+0000 7febdfe00700 1 --2- 172.21.15.6:0/2419313211 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7febc40b2990 0x7febc40b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:18.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.420+0000 7febdcc00700 1 --2- 172.21.15.6:0/2419313211 >> [v2:172.21.15.6:6810/2846613703,v1:172.21.15.6:6811/2846613703] conn(0x7febc40bdbc0 0x7febc40c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:18.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.420+0000 7febdcc00700 1 -- 172.21.15.6:0/2419313211 --> [v2:172.21.15.6:6810/2846613703,v1:172.21.15.6:6811/2846613703] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7febd800dea0 con 0x7febc40bdbc0 2024-05-09T04:52:18.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.420+0000 7febdcc00700 1 -- 172.21.15.6:0/2419313211 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7febd80e2d20 con 0x7febe010e910 2024-05-09T04:52:18.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.421+0000 7febdf400700 1 --2- 172.21.15.6:0/2419313211 >> [v2:172.21.15.6:6810/2846613703,v1:172.21.15.6:6811/2846613703] conn(0x7febc40bdbc0 0x7febc40c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:18.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.421+0000 7febdfe00700 1 --2- 172.21.15.6:0/2419313211 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7febc40b2990 0x7febc40b4ea0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7febd0009680 tx=0x7febd0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:18.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.421+0000 7febdf400700 1 --2- 172.21.15.6:0/2419313211 >> [v2:172.21.15.6:6810/2846613703,v1:172.21.15.6:6811/2846613703] conn(0x7febc40bdbc0 0x7febc40c00b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:18.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.422+0000 7febdcc00700 1 -- 172.21.15.6:0/2419313211 <== osd.3 v2:172.21.15.6:6810/2846613703 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7febd800dea0 con 0x7febc40bdbc0 2024-05-09T04:52:18.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.443+0000 7febe7000700 1 -- 172.21.15.6:0/2419313211 --> [v2:172.21.15.6:6810/2846613703,v1:172.21.15.6:6811/2846613703] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7febe006beb0 con 0x7febc40bdbc0 2024-05-09T04:52:18.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.443+0000 7febdcc00700 1 -- 172.21.15.6:0/2419313211 <== osd.3 v2:172.21.15.6:6810/2846613703 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7febe006beb0 con 0x7febc40bdbc0 2024-05-09T04:52:18.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.443+0000 7febe7000700 1 -- 172.21.15.6:0/2419313211 >> [v2:172.21.15.6:6810/2846613703,v1:172.21.15.6:6811/2846613703] conn(0x7febc40bdbc0 msgr2=0x7febc40c00b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:18.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.443+0000 7febe7000700 1 --2- 172.21.15.6:0/2419313211 >> [v2:172.21.15.6:6810/2846613703,v1:172.21.15.6:6811/2846613703] conn(0x7febc40bdbc0 0x7febc40c00b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:18.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.443+0000 7febe7000700 1 -- 172.21.15.6:0/2419313211 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7febc40b2990 msgr2=0x7febc40b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:18.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.443+0000 7febe7000700 1 --2- 172.21.15.6:0/2419313211 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7febc40b2990 0x7febc40b4ea0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7febd0009680 tx=0x7febd0009260 comp rx=0 tx=0).stop 2024-05-09T04:52:18.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.443+0000 7febe7000700 1 -- 172.21.15.6:0/2419313211 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7febe010e910 msgr2=0x7febe01a18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:18.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.443+0000 7febe7000700 1 --2- 172.21.15.6:0/2419313211 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7febe010e910 0x7febe01a18c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7febd800c370 tx=0x7febd800c680 comp rx=0 tx=0).stop 2024-05-09T04:52:18.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.443+0000 7febe5000700 1 -- 172.21.15.6:0/2419313211 reap_dead start 2024-05-09T04:52:18.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.444+0000 7febe7000700 1 -- 172.21.15.6:0/2419313211 shutdown_connections 2024-05-09T04:52:18.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.444+0000 7febe7000700 1 -- 172.21.15.6:0/2419313211 >> 172.21.15.6:0/2419313211 conn(0x7febe00fedc0 msgr2=0x7febe0103200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:18.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.444+0000 7febe7000700 1 -- 172.21.15.6:0/2419313211 shutdown_connections 2024-05-09T04:52:18.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:18.444+0000 7febe7000700 1 -- 172.21.15.6:0/2419313211 wait complete. 2024-05-09T04:52:18.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:18 smithi006 ceph-mon[29835]: pgmap v420: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:52:19.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:18 smithi186 ceph-mon[34668]: pgmap v420: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:52:19.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:18 smithi169 ceph-mon[34075]: pgmap v420: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:52:19.260 INFO:teuthology.orchestra.run.smithi006.stdout:158913790038 2024-05-09T04:52:19.261 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-09T04:52:20.798 INFO:teuthology.orchestra.run.smithi006.stdout:304942678066 2024-05-09T04:52:20.799 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-09T04:52:21.002 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:20 smithi006 ceph-mon[29835]: pgmap v421: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:52:21.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:20 smithi186 ceph-mon[34668]: pgmap v421: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:52:21.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:20 smithi169 ceph-mon[34075]: pgmap v421: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:52:22.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.901+0000 7f996b800700 1 -- 172.21.15.6:0/4155063217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9964106ed0 msgr2=0x7f9964113360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:22.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.901+0000 7f996b800700 1 --2- 172.21.15.6:0/4155063217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9964106ed0 0x7f9964113360 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f995c009f90 tx=0x7f995c00b330 comp rx=0 tx=0).stop 2024-05-09T04:52:22.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.901+0000 7f996b800700 1 -- 172.21.15.6:0/4155063217 shutdown_connections 2024-05-09T04:52:22.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.901+0000 7f996b800700 1 --2- 172.21.15.6:0/4155063217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9964106ed0 0x7f9964113360 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:22.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.901+0000 7f996b800700 1 --2- 172.21.15.6:0/4155063217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f996410c5c0 0x7f996410c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:22.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.901+0000 7f996b800700 1 --2- 172.21.15.6:0/4155063217 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9964106580 0x7f9964106990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:22.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.901+0000 7f996b800700 1 -- 172.21.15.6:0/4155063217 >> 172.21.15.6:0/4155063217 conn(0x7f9964102080 msgr2=0x7f99641044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:22.902 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:22 smithi006 ceph-mon[29835]: pgmap v422: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:52:22.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.901+0000 7f996b800700 1 -- 172.21.15.6:0/4155063217 shutdown_connections 2024-05-09T04:52:22.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.901+0000 7f996b800700 1 -- 172.21.15.6:0/4155063217 wait complete. 2024-05-09T04:52:22.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.903+0000 7f996b800700 1 Processor -- start 2024-05-09T04:52:22.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.903+0000 7f996b800700 1 -- start start 2024-05-09T04:52:22.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.904+0000 7f996b800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9964106580 0x7f996419d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:22.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.904+0000 7f996b800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9964106ed0 0x7f996419d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:22.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.904+0000 7f996b800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f996410c5c0 0x7f99641a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:22.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.904+0000 7f996b800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f99641166f0 con 0x7f9964106580 2024-05-09T04:52:22.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.904+0000 7f996b800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9964116570 con 0x7f996410c5c0 2024-05-09T04:52:22.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.904+0000 7f996b800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9964116870 con 0x7f9964106ed0 2024-05-09T04:52:22.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.904+0000 7f9968e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9964106580 0x7f996419d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:22.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.904+0000 7f9968e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9964106580 0x7f996419d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42386/0 (socket says 172.21.15.6:42386) 2024-05-09T04:52:22.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.904+0000 7f9968e00700 1 -- 172.21.15.6:0/3768302079 learned_addr learned my addr 172.21.15.6:0/3768302079 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:52:22.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.904+0000 7f9963e00700 1 --2- 172.21.15.6:0/3768302079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9964106ed0 0x7f996419d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:22.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.904+0000 7f9969800700 1 --2- 172.21.15.6:0/3768302079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f996410c5c0 0x7f99641a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:22.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.905+0000 7f9968e00700 1 -- 172.21.15.6:0/3768302079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9964106ed0 msgr2=0x7f996419d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:22.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.905+0000 7f9968e00700 1 --2- 172.21.15.6:0/3768302079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9964106ed0 0x7f996419d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:22.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.905+0000 7f9968e00700 1 -- 172.21.15.6:0/3768302079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f996410c5c0 msgr2=0x7f99641a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:22.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.905+0000 7f9968e00700 1 --2- 172.21.15.6:0/3768302079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f996410c5c0 0x7f99641a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:22.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.905+0000 7f9968e00700 1 -- 172.21.15.6:0/3768302079 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99641a1f10 con 0x7f9964106580 2024-05-09T04:52:22.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.905+0000 7f9968e00700 1 --2- 172.21.15.6:0/3768302079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9964106580 0x7f996419d040 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f995800ea30 tx=0x7f995800ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:22.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.905+0000 7f9961600700 1 -- 172.21.15.6:0/3768302079 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995800cbd0 con 0x7f9964106580 2024-05-09T04:52:22.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.905+0000 7f996b800700 1 -- 172.21.15.6:0/3768302079 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99641a2200 con 0x7f9964106580 2024-05-09T04:52:22.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.905+0000 7f9961600700 1 -- 172.21.15.6:0/3768302079 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99580044d0 con 0x7f9964106580 2024-05-09T04:52:22.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.906+0000 7f9961600700 1 -- 172.21.15.6:0/3768302079 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9958003dc0 con 0x7f9964106580 2024-05-09T04:52:22.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.906+0000 7f996b800700 1 -- 172.21.15.6:0/3768302079 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99641a9b20 con 0x7f9964106580 2024-05-09T04:52:22.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.907+0000 7f996b800700 1 -- 172.21.15.6:0/3768302079 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9964063f10 con 0x7f9964106580 2024-05-09T04:52:22.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.909+0000 7f9961600700 1 -- 172.21.15.6:0/3768302079 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f9958003fa0 con 0x7f9964106580 2024-05-09T04:52:22.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.911+0000 7f9961600700 1 --2- 172.21.15.6:0/3768302079 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f994c0b2910 0x7f994c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:22.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.911+0000 7f9961600700 1 -- 172.21.15.6:0/3768302079 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f99580e2af0 con 0x7f9964106580 2024-05-09T04:52:22.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.911+0000 7f9963e00700 1 --2- 172.21.15.6:0/3768302079 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f994c0b2910 0x7f994c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:22.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.912+0000 7f9961600700 1 --2- 172.21.15.6:0/3768302079 >> [v2:172.21.15.6:6802/1738750914,v1:172.21.15.6:6803/1738750914] conn(0x7f994c0bdb40 0x7f994c0c0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:22.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.912+0000 7f9961600700 1 -- 172.21.15.6:0/3768302079 --> [v2:172.21.15.6:6802/1738750914,v1:172.21.15.6:6803/1738750914] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9958016ec0 con 0x7f994c0bdb40 2024-05-09T04:52:22.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.912+0000 7f9961600700 1 -- 172.21.15.6:0/3768302079 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f99580e2ea0 con 0x7f9964106580 2024-05-09T04:52:22.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.912+0000 7f9969800700 1 --2- 172.21.15.6:0/3768302079 >> [v2:172.21.15.6:6802/1738750914,v1:172.21.15.6:6803/1738750914] conn(0x7f994c0bdb40 0x7f994c0c0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:22.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.912+0000 7f9963e00700 1 --2- 172.21.15.6:0/3768302079 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f994c0b2910 0x7f994c0b4e20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9964103c00 tx=0x7f9950009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:22.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.913+0000 7f9969800700 1 --2- 172.21.15.6:0/3768302079 >> [v2:172.21.15.6:6802/1738750914,v1:172.21.15.6:6803/1738750914] conn(0x7f994c0bdb40 0x7f994c0c0030 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:22.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.914+0000 7f9961600700 1 -- 172.21.15.6:0/3768302079 <== osd.0 v2:172.21.15.6:6802/1738750914 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9958016ec0 con 0x7f994c0bdb40 2024-05-09T04:52:22.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.935+0000 7f996b800700 1 -- 172.21.15.6:0/3768302079 --> [v2:172.21.15.6:6802/1738750914,v1:172.21.15.6:6803/1738750914] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9964064310 con 0x7f994c0bdb40 2024-05-09T04:52:22.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.936+0000 7f9961600700 1 -- 172.21.15.6:0/3768302079 <== osd.0 v2:172.21.15.6:6802/1738750914 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9964064310 con 0x7f994c0bdb40 2024-05-09T04:52:22.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.936+0000 7f996b800700 1 -- 172.21.15.6:0/3768302079 >> [v2:172.21.15.6:6802/1738750914,v1:172.21.15.6:6803/1738750914] conn(0x7f994c0bdb40 msgr2=0x7f994c0c0030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:22.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.936+0000 7f996b800700 1 --2- 172.21.15.6:0/3768302079 >> [v2:172.21.15.6:6802/1738750914,v1:172.21.15.6:6803/1738750914] conn(0x7f994c0bdb40 0x7f994c0c0030 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:22.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.937+0000 7f996b800700 1 -- 172.21.15.6:0/3768302079 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f994c0b2910 msgr2=0x7f994c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:22.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.937+0000 7f996b800700 1 --2- 172.21.15.6:0/3768302079 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f994c0b2910 0x7f994c0b4e20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9964103c00 tx=0x7f9950009310 comp rx=0 tx=0).stop 2024-05-09T04:52:22.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.937+0000 7f996b800700 1 -- 172.21.15.6:0/3768302079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9964106580 msgr2=0x7f996419d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:22.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.937+0000 7f996b800700 1 --2- 172.21.15.6:0/3768302079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9964106580 0x7f996419d040 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f995800ea30 tx=0x7f995800ed40 comp rx=0 tx=0).stop 2024-05-09T04:52:22.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.937+0000 7f9969800700 1 -- 172.21.15.6:0/3768302079 reap_dead start 2024-05-09T04:52:22.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.937+0000 7f996b800700 1 -- 172.21.15.6:0/3768302079 shutdown_connections 2024-05-09T04:52:22.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.937+0000 7f996b800700 1 -- 172.21.15.6:0/3768302079 >> 172.21.15.6:0/3768302079 conn(0x7f9964102080 msgr2=0x7f9964104790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:22.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.937+0000 7f996b800700 1 -- 172.21.15.6:0/3768302079 shutdown_connections 2024-05-09T04:52:22.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.937+0000 7f996b800700 1 -- 172.21.15.6:0/3768302079 wait complete. 2024-05-09T04:52:23.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:22 smithi186 ceph-mon[34668]: pgmap v422: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:52:23.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:22 smithi169 ceph-mon[34075]: pgmap v422: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:52:23.373 INFO:teuthology.orchestra.run.smithi006.stdout:111669149794 2024-05-09T04:52:23.374 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-09T04:52:23.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.672+0000 7f46dda00700 1 -- 172.21.15.6:0/217249290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46d81030c0 msgr2=0x7f46d81034d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:23.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.672+0000 7f46dda00700 1 --2- 172.21.15.6:0/217249290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46d81030c0 0x7f46d81034d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f46c8009990 tx=0x7f46c8009ca0 comp rx=0 tx=0).stop 2024-05-09T04:52:23.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.673+0000 7f46dda00700 1 -- 172.21.15.6:0/217249290 shutdown_connections 2024-05-09T04:52:23.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.673+0000 7f46dda00700 1 --2- 172.21.15.6:0/217249290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46d8110c70 0x7f46d8113090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:23.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.673+0000 7f46dda00700 1 --2- 172.21.15.6:0/217249290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46d81030c0 0x7f46d81034d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:23.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.673+0000 7f46dda00700 1 --2- 172.21.15.6:0/217249290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46d8103a10 0x7f46d8110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:23.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.673+0000 7f46dda00700 1 -- 172.21.15.6:0/217249290 >> 172.21.15.6:0/217249290 conn(0x7f46d80fedc0 msgr2=0x7f46d81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:23.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.673+0000 7f46dda00700 1 -- 172.21.15.6:0/217249290 shutdown_connections 2024-05-09T04:52:23.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.673+0000 7f46dda00700 1 -- 172.21.15.6:0/217249290 wait complete. 2024-05-09T04:52:23.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.674+0000 7f46dda00700 1 Processor -- start 2024-05-09T04:52:23.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.675+0000 7f46dda00700 1 -- start start 2024-05-09T04:52:23.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.675+0000 7f46dda00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46d81030c0 0x7f46d819d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:23.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.675+0000 7f46dda00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46d8103a10 0x7f46d819d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:23.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.675+0000 7f46dda00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46d8110c70 0x7f46d81a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:23.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.675+0000 7f46dda00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f46d81166b0 con 0x7f46d81030c0 2024-05-09T04:52:23.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.675+0000 7f46dda00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f46d8116530 con 0x7f46d8103a10 2024-05-09T04:52:23.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.675+0000 7f46dda00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f46d8116830 con 0x7f46d8110c70 2024-05-09T04:52:23.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.675+0000 7f46d6a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46d81030c0 0x7f46d819d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:23.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.676+0000 7f46d7400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46d8110c70 0x7f46d81a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:23.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.676+0000 7f46d7400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46d8110c70 0x7f46d81a1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:50372/0 (socket says 172.21.15.6:50372) 2024-05-09T04:52:23.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.676+0000 7f46d7400700 1 -- 172.21.15.6:0/4223336499 learned_addr learned my addr 172.21.15.6:0/4223336499 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:52:23.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.676+0000 7f46d6a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46d81030c0 0x7f46d819d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42376/0 (socket says 172.21.15.6:42376) 2024-05-09T04:52:23.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.676+0000 7f46d7400700 1 -- 172.21.15.6:0/4223336499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46d8103a10 msgr2=0x7f46d819d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:23.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.676+0000 7f46d7400700 1 --2- 172.21.15.6:0/4223336499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46d8103a10 0x7f46d819d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:23.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.676+0000 7f46d7400700 1 -- 172.21.15.6:0/4223336499 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46d81030c0 msgr2=0x7f46d819d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:23.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.676+0000 7f46d7400700 1 --2- 172.21.15.6:0/4223336499 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46d81030c0 0x7f46d819d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:23.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.676+0000 7f46d7400700 1 -- 172.21.15.6:0/4223336499 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d81a1ed0 con 0x7f46d8110c70 2024-05-09T04:52:23.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.676+0000 7f46d7400700 1 --2- 172.21.15.6:0/4223336499 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46d8110c70 0x7f46d81a1830 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f46d000ef30 tx=0x7f46d000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:23.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.677+0000 7f46cf400700 1 -- 172.21.15.6:0/4223336499 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d00092e0 con 0x7f46d8110c70 2024-05-09T04:52:23.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.677+0000 7f46cf400700 1 -- 172.21.15.6:0/4223336499 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46d0004bb0 con 0x7f46d8110c70 2024-05-09T04:52:23.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.677+0000 7f46dda00700 1 -- 172.21.15.6:0/4223336499 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d81a2160 con 0x7f46d8110c70 2024-05-09T04:52:23.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.677+0000 7f46cf400700 1 -- 172.21.15.6:0/4223336499 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d0004280 con 0x7f46d8110c70 2024-05-09T04:52:23.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.677+0000 7f46dda00700 1 -- 172.21.15.6:0/4223336499 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d81a9ae0 con 0x7f46d8110c70 2024-05-09T04:52:23.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.678+0000 7f46dda00700 1 -- 172.21.15.6:0/4223336499 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f46d8063f50 con 0x7f46d8110c70 2024-05-09T04:52:23.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.681+0000 7f46cf400700 1 -- 172.21.15.6:0/4223336499 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f46d001b030 con 0x7f46d8110c70 2024-05-09T04:52:23.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.682+0000 7f46cf400700 1 --2- 172.21.15.6:0/4223336499 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f46bc0b2990 0x7f46bc0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:23.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.682+0000 7f46cf400700 1 -- 172.21.15.6:0/4223336499 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f46d00e2bc0 con 0x7f46d8110c70 2024-05-09T04:52:23.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.683+0000 7f46d6a00700 1 --2- 172.21.15.6:0/4223336499 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f46bc0b2990 0x7f46bc0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:23.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.683+0000 7f46cf400700 1 --2- 172.21.15.6:0/4223336499 >> [v2:172.21.15.169:6816/1303584711,v1:172.21.15.169:6817/1303584711] conn(0x7f46bc0bdbc0 0x7f46bc0c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:23.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.683+0000 7f46cf400700 1 -- 172.21.15.6:0/4223336499 --> [v2:172.21.15.169:6816/1303584711,v1:172.21.15.169:6817/1303584711] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f46bc0c0780 con 0x7f46bc0bdbc0 2024-05-09T04:52:23.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.683+0000 7f46cf400700 1 -- 172.21.15.6:0/4223336499 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f46d00e2f70 con 0x7f46d8110c70 2024-05-09T04:52:23.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.683+0000 7f46d6000700 1 --2- 172.21.15.6:0/4223336499 >> [v2:172.21.15.169:6816/1303584711,v1:172.21.15.169:6817/1303584711] conn(0x7f46bc0bdbc0 0x7f46bc0c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:23.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.684+0000 7f46d6a00700 1 --2- 172.21.15.6:0/4223336499 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f46bc0b2990 0x7f46bc0b4ea0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f46c800b2e0 tx=0x7f46c8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:23.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.684+0000 7f46d6000700 1 --2- 172.21.15.6:0/4223336499 >> [v2:172.21.15.169:6816/1303584711,v1:172.21.15.169:6817/1303584711] conn(0x7f46bc0bdbc0 0x7f46bc0c00b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:23.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.685+0000 7f46cf400700 1 -- 172.21.15.6:0/4223336499 <== osd.7 v2:172.21.15.169:6816/1303584711 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f46bc0c0780 con 0x7f46bc0bdbc0 2024-05-09T04:52:23.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.703+0000 7f46dda00700 1 -- 172.21.15.6:0/4223336499 --> [v2:172.21.15.169:6816/1303584711,v1:172.21.15.169:6817/1303584711] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f46d80643d0 con 0x7f46bc0bdbc0 2024-05-09T04:52:23.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.704+0000 7f46cf400700 1 -- 172.21.15.6:0/4223336499 <== osd.7 v2:172.21.15.169:6816/1303584711 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f46d80643d0 con 0x7f46bc0bdbc0 2024-05-09T04:52:23.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.705+0000 7f46dda00700 1 -- 172.21.15.6:0/4223336499 >> [v2:172.21.15.169:6816/1303584711,v1:172.21.15.169:6817/1303584711] conn(0x7f46bc0bdbc0 msgr2=0x7f46bc0c00b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:23.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.705+0000 7f46dda00700 1 --2- 172.21.15.6:0/4223336499 >> [v2:172.21.15.169:6816/1303584711,v1:172.21.15.169:6817/1303584711] conn(0x7f46bc0bdbc0 0x7f46bc0c00b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:23.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.705+0000 7f46dda00700 1 -- 172.21.15.6:0/4223336499 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f46bc0b2990 msgr2=0x7f46bc0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:23.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.705+0000 7f46dda00700 1 --2- 172.21.15.6:0/4223336499 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f46bc0b2990 0x7f46bc0b4ea0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f46c800b2e0 tx=0x7f46c8005fd0 comp rx=0 tx=0).stop 2024-05-09T04:52:23.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.705+0000 7f46dda00700 1 -- 172.21.15.6:0/4223336499 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46d8110c70 msgr2=0x7f46d81a1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:23.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.705+0000 7f46dda00700 1 --2- 172.21.15.6:0/4223336499 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46d8110c70 0x7f46d81a1830 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f46d000ef30 tx=0x7f46d000c3a0 comp rx=0 tx=0).stop 2024-05-09T04:52:23.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.705+0000 7f46d7400700 1 -- 172.21.15.6:0/4223336499 reap_dead start 2024-05-09T04:52:23.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.705+0000 7f46dda00700 1 -- 172.21.15.6:0/4223336499 shutdown_connections 2024-05-09T04:52:23.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.705+0000 7f46dda00700 1 -- 172.21.15.6:0/4223336499 >> 172.21.15.6:0/4223336499 conn(0x7f46d80fedc0 msgr2=0x7f46d81126e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:23.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.705+0000 7f46dda00700 1 -- 172.21.15.6:0/4223336499 shutdown_connections 2024-05-09T04:52:23.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.705+0000 7f46dda00700 1 -- 172.21.15.6:0/4223336499 wait complete. 2024-05-09T04:52:24.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.245+0000 7f62bfe00700 1 -- 172.21.15.6:0/3385172755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62c0104400 msgr2=0x7f62c0104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:24.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.245+0000 7f62bfe00700 1 --2- 172.21.15.6:0/3385172755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62c0104400 0x7f62c0104810 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f62b0009920 tx=0x7f62b0009c30 comp rx=0 tx=0).stop 2024-05-09T04:52:24.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.245+0000 7f62bfe00700 1 -- 172.21.15.6:0/3385172755 shutdown_connections 2024-05-09T04:52:24.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.245+0000 7f62bfe00700 1 --2- 172.21.15.6:0/3385172755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62c0109680 0x7f62c0110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.245+0000 7f62bfe00700 1 --2- 172.21.15.6:0/3385172755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62c0104400 0x7f62c0104810 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.245+0000 7f62bfe00700 1 --2- 172.21.15.6:0/3385172755 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f62c0108d30 0x7f62c0109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.245+0000 7f62bfe00700 1 -- 172.21.15.6:0/3385172755 >> 172.21.15.6:0/3385172755 conn(0x7f62c0079d10 msgr2=0x7f62c007a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:24.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.245+0000 7f62bfe00700 1 -- 172.21.15.6:0/3385172755 shutdown_connections 2024-05-09T04:52:24.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.245+0000 7f62bfe00700 1 -- 172.21.15.6:0/3385172755 wait complete. 2024-05-09T04:52:24.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.246+0000 7f62bfe00700 1 Processor -- start 2024-05-09T04:52:24.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.247+0000 7f62bfe00700 1 -- start start 2024-05-09T04:52:24.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.247+0000 7f62bfe00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62c0104400 0x7f62c019cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:24.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.247+0000 7f62bfe00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62c0108d30 0x7f62c019d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:24.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.247+0000 7f62bfe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f62c0109680 0x7f62c01a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:24.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.247+0000 7f62bfe00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f62c0114300 con 0x7f62c0109680 2024-05-09T04:52:24.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.247+0000 7f62bfe00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f62c0114180 con 0x7f62c0108d30 2024-05-09T04:52:24.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.247+0000 7f62bfe00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f62c0114480 con 0x7f62c0104400 2024-05-09T04:52:24.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.247+0000 7f62bea00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62c0104400 0x7f62c019cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:24.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.248+0000 7f62b7e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62c0108d30 0x7f62c019d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:24.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.248+0000 7f62bea00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62c0104400 0x7f62c019cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:50364/0 (socket says 172.21.15.6:50364) 2024-05-09T04:52:24.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.248+0000 7f62bea00700 1 -- 172.21.15.6:0/1538484423 learned_addr learned my addr 172.21.15.6:0/1538484423 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:52:24.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.248+0000 7f62bf400700 1 --2- 172.21.15.6:0/1538484423 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f62c0109680 0x7f62c01a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:24.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.248+0000 7f62bf400700 1 -- 172.21.15.6:0/1538484423 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62c0104400 msgr2=0x7f62c019cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:24.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.248+0000 7f62bf400700 1 --2- 172.21.15.6:0/1538484423 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62c0104400 0x7f62c019cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.248+0000 7f62bf400700 1 -- 172.21.15.6:0/1538484423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62c0108d30 msgr2=0x7f62c019d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:24.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.248+0000 7f62bf400700 1 --2- 172.21.15.6:0/1538484423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62c0108d30 0x7f62c019d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.248+0000 7f62bf400700 1 -- 172.21.15.6:0/1538484423 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c01a1e00 con 0x7f62c0109680 2024-05-09T04:52:24.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.248+0000 7f62bea00700 1 --2- 172.21.15.6:0/1538484423 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62c0104400 0x7f62c019cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:24.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.248+0000 7f62b7e00700 1 --2- 172.21.15.6:0/1538484423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62c0108d30 0x7f62c019d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:24.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.249+0000 7f62bf400700 1 --2- 172.21.15.6:0/1538484423 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f62c0109680 0x7f62c01a1760 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f62b800efc0 tx=0x7f62b800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:24.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.249+0000 7f62b7400700 1 -- 172.21.15.6:0/1538484423 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b80092e0 con 0x7f62c0109680 2024-05-09T04:52:24.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.249+0000 7f62b7400700 1 -- 172.21.15.6:0/1538484423 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62b8004bb0 con 0x7f62c0109680 2024-05-09T04:52:24.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.249+0000 7f62bfe00700 1 -- 172.21.15.6:0/1538484423 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62c01a2090 con 0x7f62c0109680 2024-05-09T04:52:24.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.249+0000 7f62b7400700 1 -- 172.21.15.6:0/1538484423 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b8004020 con 0x7f62c0109680 2024-05-09T04:52:24.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.249+0000 7f62bfe00700 1 -- 172.21.15.6:0/1538484423 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62c01a2580 con 0x7f62c0109680 2024-05-09T04:52:24.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.250+0000 7f62bfe00700 1 -- 172.21.15.6:0/1538484423 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f62c01025e0 con 0x7f62c0109680 2024-05-09T04:52:24.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.253+0000 7f62b7400700 1 -- 172.21.15.6:0/1538484423 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f62b801b030 con 0x7f62c0109680 2024-05-09T04:52:24.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.255+0000 7f62b7400700 1 --2- 172.21.15.6:0/1538484423 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f62a40b2910 0x7f62a40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:24.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.255+0000 7f62b7400700 1 -- 172.21.15.6:0/1538484423 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f62b80e2b80 con 0x7f62c0109680 2024-05-09T04:52:24.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.255+0000 7f62bea00700 1 --2- 172.21.15.6:0/1538484423 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f62a40b2910 0x7f62a40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:24.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.255+0000 7f62b7400700 1 --2- 172.21.15.6:0/1538484423 >> [v2:172.21.15.6:6826/3590990973,v1:172.21.15.6:6827/3590990973] conn(0x7f62a40bdb40 0x7f62a40c0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:24.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.255+0000 7f62b7400700 1 -- 172.21.15.6:0/1538484423 --> [v2:172.21.15.6:6826/3590990973,v1:172.21.15.6:6827/3590990973] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f62b800be50 con 0x7f62a40bdb40 2024-05-09T04:52:24.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.255+0000 7f62b7400700 1 -- 172.21.15.6:0/1538484423 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f62b80e2f30 con 0x7f62c0109680 2024-05-09T04:52:24.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.255+0000 7f62b7e00700 1 --2- 172.21.15.6:0/1538484423 >> [v2:172.21.15.6:6826/3590990973,v1:172.21.15.6:6827/3590990973] conn(0x7f62a40bdb40 0x7f62a40c0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:24.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.256+0000 7f62bea00700 1 --2- 172.21.15.6:0/1538484423 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f62a40b2910 0x7f62a40b4e20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f62b0005f40 tx=0x7f62b0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:24.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.256+0000 7f62b7e00700 1 --2- 172.21.15.6:0/1538484423 >> [v2:172.21.15.6:6826/3590990973,v1:172.21.15.6:6827/3590990973] conn(0x7f62a40bdb40 0x7f62a40c0030 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:24.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.257+0000 7f62b7400700 1 -- 172.21.15.6:0/1538484423 <== osd.9 v2:172.21.15.6:6826/3590990973 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f62b800be50 con 0x7f62a40bdb40 2024-05-09T04:52:24.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.276+0000 7f62bfe00700 1 -- 172.21.15.6:0/1538484423 --> [v2:172.21.15.6:6826/3590990973,v1:172.21.15.6:6827/3590990973] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f62c01058b0 con 0x7f62a40bdb40 2024-05-09T04:52:24.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.277+0000 7f62b7400700 1 -- 172.21.15.6:0/1538484423 <== osd.9 v2:172.21.15.6:6826/3590990973 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f62c01058b0 con 0x7f62a40bdb40 2024-05-09T04:52:24.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.278+0000 7f62bfe00700 1 -- 172.21.15.6:0/1538484423 >> [v2:172.21.15.6:6826/3590990973,v1:172.21.15.6:6827/3590990973] conn(0x7f62a40bdb40 msgr2=0x7f62a40c0030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:24.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.278+0000 7f62bfe00700 1 --2- 172.21.15.6:0/1538484423 >> [v2:172.21.15.6:6826/3590990973,v1:172.21.15.6:6827/3590990973] conn(0x7f62a40bdb40 0x7f62a40c0030 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.278+0000 7f62bfe00700 1 -- 172.21.15.6:0/1538484423 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f62a40b2910 msgr2=0x7f62a40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:24.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.278+0000 7f62bfe00700 1 --2- 172.21.15.6:0/1538484423 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f62a40b2910 0x7f62a40b4e20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f62b0005f40 tx=0x7f62b0019040 comp rx=0 tx=0).stop 2024-05-09T04:52:24.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.278+0000 7f62bfe00700 1 -- 172.21.15.6:0/1538484423 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f62c0109680 msgr2=0x7f62c01a1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:24.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.278+0000 7f62bfe00700 1 --2- 172.21.15.6:0/1538484423 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f62c0109680 0x7f62c01a1760 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f62b800efc0 tx=0x7f62b800c3a0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.278+0000 7f62bf400700 1 -- 172.21.15.6:0/1538484423 reap_dead start 2024-05-09T04:52:24.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.279+0000 7f62bfe00700 1 -- 172.21.15.6:0/1538484423 shutdown_connections 2024-05-09T04:52:24.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.279+0000 7f62bfe00700 1 -- 172.21.15.6:0/1538484423 >> 172.21.15.6:0/1538484423 conn(0x7f62c0079d10 msgr2=0x7f62c0101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:24.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.279+0000 7f62bfe00700 1 -- 172.21.15.6:0/1538484423 shutdown_connections 2024-05-09T04:52:24.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:22.279+0000 7f62bfe00700 1 -- 172.21.15.6:0/1538484423 wait complete. 2024-05-09T04:52:25.055 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:24 smithi006 ceph-mon[29835]: pgmap v423: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:52:25.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:24 smithi186 ceph-mon[34668]: pgmap v423: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:52:25.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:24 smithi169 ceph-mon[34075]: pgmap v423: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:52:25.139 INFO:teuthology.orchestra.run.smithi006.stdout:34359738483 2024-05-09T04:52:25.140 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-09T04:52:25.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.431+0000 7f7fccc00700 1 -- 172.21.15.6:0/1459123615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fc81045f0 msgr2=0x7f7fc8104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:25.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.431+0000 7f7fccc00700 1 --2- 172.21.15.6:0/1459123615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fc81045f0 0x7f7fc8104a00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8009f90 tx=0x7f7fb800b330 comp rx=0 tx=0).stop 2024-05-09T04:52:25.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.431+0000 7f7fccc00700 1 -- 172.21.15.6:0/1459123615 shutdown_connections 2024-05-09T04:52:25.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.431+0000 7f7fccc00700 1 --2- 172.21.15.6:0/1459123615 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fc8109870 0x7f7fc81111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:25.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.431+0000 7f7fccc00700 1 --2- 172.21.15.6:0/1459123615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fc81045f0 0x7f7fc8104a00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:25.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.431+0000 7f7fccc00700 1 --2- 172.21.15.6:0/1459123615 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fc8108f20 0x7f7fc8109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:25.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.431+0000 7f7fccc00700 1 -- 172.21.15.6:0/1459123615 >> 172.21.15.6:0/1459123615 conn(0x7f7fc8100040 msgr2=0x7f7fc8102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:25.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.431+0000 7f7fccc00700 1 -- 172.21.15.6:0/1459123615 shutdown_connections 2024-05-09T04:52:25.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.431+0000 7f7fccc00700 1 -- 172.21.15.6:0/1459123615 wait complete. 2024-05-09T04:52:25.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.432+0000 7f7fccc00700 1 Processor -- start 2024-05-09T04:52:25.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.433+0000 7f7fccc00700 1 -- start start 2024-05-09T04:52:25.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.433+0000 7f7fccc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fc81045f0 0x7f7fc819ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:25.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.433+0000 7f7fccc00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fc8108f20 0x7f7fc819b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:25.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.433+0000 7f7fccc00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fc8109870 0x7f7fc819f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:25.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.433+0000 7f7fccc00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc81144a0 con 0x7f7fc8109870 2024-05-09T04:52:25.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.433+0000 7f7fccc00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc8114320 con 0x7f7fc8108f20 2024-05-09T04:52:25.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.433+0000 7f7fccc00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc8114620 con 0x7f7fc81045f0 2024-05-09T04:52:25.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.434+0000 7f7fbd400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fc8108f20 0x7f7fc819b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:25.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.434+0000 7f7fc6800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fc8109870 0x7f7fc819f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:25.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.434+0000 7f7fbd400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fc8108f20 0x7f7fc819b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:34372/0 (socket says 172.21.15.6:34372) 2024-05-09T04:52:25.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.434+0000 7f7fc5e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fc81045f0 0x7f7fc819ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:25.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.434+0000 7f7fbd400700 1 -- 172.21.15.6:0/4177326785 learned_addr learned my addr 172.21.15.6:0/4177326785 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:52:25.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.434+0000 7f7fbd400700 1 -- 172.21.15.6:0/4177326785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fc81045f0 msgr2=0x7f7fc819ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:25.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.434+0000 7f7fbd400700 1 --2- 172.21.15.6:0/4177326785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fc81045f0 0x7f7fc819ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:25.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.434+0000 7f7fbd400700 1 -- 172.21.15.6:0/4177326785 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fc8109870 msgr2=0x7f7fc819f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:25.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.434+0000 7f7fbd400700 1 --2- 172.21.15.6:0/4177326785 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fc8109870 0x7f7fc819f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:25.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.434+0000 7f7fbd400700 1 -- 172.21.15.6:0/4177326785 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc819fd00 con 0x7f7fc8108f20 2024-05-09T04:52:25.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.435+0000 7f7fbd400700 1 --2- 172.21.15.6:0/4177326785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fc8108f20 0x7f7fc819b3d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7fb000e970 tx=0x7f7fb000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:25.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.435+0000 7f7fbf400700 1 -- 172.21.15.6:0/4177326785 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb0004a30 con 0x7f7fc8108f20 2024-05-09T04:52:25.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.435+0000 7f7fccc00700 1 -- 172.21.15.6:0/4177326785 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fc819fff0 con 0x7f7fc8108f20 2024-05-09T04:52:25.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.435+0000 7f7fccc00700 1 -- 172.21.15.6:0/4177326785 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fc80759f0 con 0x7f7fc8108f20 2024-05-09T04:52:25.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.435+0000 7f7fbf400700 1 -- 172.21.15.6:0/4177326785 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fb000cc90 con 0x7f7fc8108f20 2024-05-09T04:52:25.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.435+0000 7f7fbf400700 1 -- 172.21.15.6:0/4177326785 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb00054c0 con 0x7f7fc8108f20 2024-05-09T04:52:25.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.436+0000 7f7fccc00700 1 -- 172.21.15.6:0/4177326785 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7fc8063f50 con 0x7f7fc8108f20 2024-05-09T04:52:25.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.439+0000 7f7fbf400700 1 -- 172.21.15.6:0/4177326785 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f7fb000c7e0 con 0x7f7fc8108f20 2024-05-09T04:52:25.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.441+0000 7f7fbf400700 1 --2- 172.21.15.6:0/4177326785 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f7fac0b2990 0x7f7fac0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:25.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.442+0000 7f7fbf400700 1 -- 172.21.15.6:0/4177326785 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f7fb00e2f50 con 0x7f7fc8108f20 2024-05-09T04:52:25.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.442+0000 7f7fbf400700 1 --2- 172.21.15.6:0/4177326785 >> [v2:172.21.15.186:6816/3475163771,v1:172.21.15.186:6817/3475163771] conn(0x7f7fac0bdbc0 0x7f7fac0c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:25.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.442+0000 7f7fbf400700 1 -- 172.21.15.6:0/4177326785 --> [v2:172.21.15.186:6816/3475163771,v1:172.21.15.186:6817/3475163771] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7fb0011e50 con 0x7f7fac0bdbc0 2024-05-09T04:52:25.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.442+0000 7f7fbf400700 1 -- 172.21.15.6:0/4177326785 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f7fb00e3300 con 0x7f7fc8108f20 2024-05-09T04:52:25.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.443+0000 7f7fc6800700 1 --2- 172.21.15.6:0/4177326785 >> [v2:172.21.15.186:6816/3475163771,v1:172.21.15.186:6817/3475163771] conn(0x7f7fac0bdbc0 0x7f7fac0c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:25.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.443+0000 7f7fc5e00700 1 --2- 172.21.15.6:0/4177326785 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f7fac0b2990 0x7f7fac0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:25.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.444+0000 7f7fc6800700 1 --2- 172.21.15.6:0/4177326785 >> [v2:172.21.15.186:6816/3475163771,v1:172.21.15.186:6817/3475163771] conn(0x7f7fac0bdbc0 0x7f7fac0c00b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:25.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.444+0000 7f7fc5e00700 1 --2- 172.21.15.6:0/4177326785 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f7fac0b2990 0x7f7fac0b4ea0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8009840 tx=0x7f7fb8009ae0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:25.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.445+0000 7f7fbf400700 1 -- 172.21.15.6:0/4177326785 <== osd.8 v2:172.21.15.186:6816/3475163771 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7fb0011e50 con 0x7f7fac0bdbc0 2024-05-09T04:52:25.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.474+0000 7f7fccc00700 1 -- 172.21.15.6:0/4177326785 --> [v2:172.21.15.186:6816/3475163771,v1:172.21.15.186:6817/3475163771] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7fc8064350 con 0x7f7fac0bdbc0 2024-05-09T04:52:25.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.475+0000 7f7fbf400700 1 -- 172.21.15.6:0/4177326785 <== osd.8 v2:172.21.15.186:6816/3475163771 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7fc8064350 con 0x7f7fac0bdbc0 2024-05-09T04:52:25.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.475+0000 7f7fccc00700 1 -- 172.21.15.6:0/4177326785 >> [v2:172.21.15.186:6816/3475163771,v1:172.21.15.186:6817/3475163771] conn(0x7f7fac0bdbc0 msgr2=0x7f7fac0c00b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:25.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.475+0000 7f7fccc00700 1 --2- 172.21.15.6:0/4177326785 >> [v2:172.21.15.186:6816/3475163771,v1:172.21.15.186:6817/3475163771] conn(0x7f7fac0bdbc0 0x7f7fac0c00b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:25.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.476+0000 7f7fccc00700 1 -- 172.21.15.6:0/4177326785 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f7fac0b2990 msgr2=0x7f7fac0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:25.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.476+0000 7f7fccc00700 1 --2- 172.21.15.6:0/4177326785 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f7fac0b2990 0x7f7fac0b4ea0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8009840 tx=0x7f7fb8009ae0 comp rx=0 tx=0).stop 2024-05-09T04:52:25.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.476+0000 7f7fccc00700 1 -- 172.21.15.6:0/4177326785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fc8108f20 msgr2=0x7f7fc819b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:25.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.476+0000 7f7fccc00700 1 --2- 172.21.15.6:0/4177326785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fc8108f20 0x7f7fc819b3d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7fb000e970 tx=0x7f7fb000ec80 comp rx=0 tx=0).stop 2024-05-09T04:52:25.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.476+0000 7f7fc6800700 1 -- 172.21.15.6:0/4177326785 reap_dead start 2024-05-09T04:52:25.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.476+0000 7f7fccc00700 1 -- 172.21.15.6:0/4177326785 shutdown_connections 2024-05-09T04:52:25.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.476+0000 7f7fccc00700 1 -- 172.21.15.6:0/4177326785 >> 172.21.15.6:0/4177326785 conn(0x7f7fc8100040 msgr2=0x7f7fc8102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:25.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.476+0000 7f7fccc00700 1 -- 172.21.15.6:0/4177326785 shutdown_connections 2024-05-09T04:52:25.476 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:25.476+0000 7f7fccc00700 1 -- 172.21.15.6:0/4177326785 wait complete. 2024-05-09T04:52:25.485 INFO:teuthology.orchestra.run.smithi006.stdout:137438953564 2024-05-09T04:52:25.486 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-09T04:52:26.297 INFO:teuthology.orchestra.run.smithi006.stdout:206158430282 2024-05-09T04:52:26.298 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-09T04:52:26.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:26 smithi006 ceph-mon[29835]: pgmap v424: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:52:27.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:26 smithi186 ceph-mon[34668]: pgmap v424: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:52:27.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:26 smithi169 ceph-mon[34075]: pgmap v424: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:52:27.287 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:27.346 INFO:teuthology.orchestra.run.smithi006.stdout:231928234054 2024-05-09T04:52:27.346 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-09T04:52:27.631 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:27.635 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:27.655 INFO:teuthology.orchestra.run.smithi006.stdout:257698037821 2024-05-09T04:52:27.655 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-09T04:52:28.407 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:28.407 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:28.407 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:28.827 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:28 smithi006 ceph-mon[29835]: pgmap v425: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:52:29.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:28 smithi186 ceph-mon[34668]: pgmap v425: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:52:29.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:28 smithi169 ceph-mon[34075]: pgmap v425: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:52:29.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.366+0000 7fd20ac00700 1 -- 172.21.15.6:0/162494588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd20410e8f0 msgr2=0x7fd204110dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:29.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.366+0000 7fd20ac00700 1 --2- 172.21.15.6:0/162494588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd20410e8f0 0x7fd204110dd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc00b0a0 tx=0x7fd1fc00b3b0 comp rx=0 tx=0).stop 2024-05-09T04:52:29.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.366+0000 7fd20ac00700 1 -- 172.21.15.6:0/162494588 shutdown_connections 2024-05-09T04:52:29.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.366+0000 7fd20ac00700 1 --2- 172.21.15.6:0/162494588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd20410e8f0 0x7fd204110dd0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:29.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.366+0000 7fd20ac00700 1 --2- 172.21.15.6:0/162494588 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2041054c0 0x7fd2041058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:29.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.366+0000 7fd20ac00700 1 --2- 172.21.15.6:0/162494588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd204105e10 0x7fd20410e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:29.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.366+0000 7fd20ac00700 1 -- 172.21.15.6:0/162494588 >> 172.21.15.6:0/162494588 conn(0x7fd2040fedc0 msgr2=0x7fd2041011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:29.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.366+0000 7fd20ac00700 1 -- 172.21.15.6:0/162494588 shutdown_connections 2024-05-09T04:52:29.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.366+0000 7fd20ac00700 1 -- 172.21.15.6:0/162494588 wait complete. 2024-05-09T04:52:29.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.368+0000 7fd20ac00700 1 Processor -- start 2024-05-09T04:52:29.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.368+0000 7fd20ac00700 1 -- start start 2024-05-09T04:52:29.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.369+0000 7fd20ac00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2041054c0 0x7fd20419cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:29.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.369+0000 7fd20ac00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd204105e10 0x7fd20419d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:29.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.369+0000 7fd20ac00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd20410e8f0 0x7fd2041a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:29.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.369+0000 7fd20ac00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd204114450 con 0x7fd2041054c0 2024-05-09T04:52:29.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.369+0000 7fd20ac00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd2041142d0 con 0x7fd20410e8f0 2024-05-09T04:52:29.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.369+0000 7fd20ac00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd2041145d0 con 0x7fd204105e10 2024-05-09T04:52:29.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.370+0000 7fd203e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2041054c0 0x7fd20419cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:29.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.370+0000 7fd203e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2041054c0 0x7fd20419cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60452/0 (socket says 172.21.15.6:60452) 2024-05-09T04:52:29.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.370+0000 7fd203400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd204105e10 0x7fd20419d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:29.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.370+0000 7fd203e00700 1 -- 172.21.15.6:0/3742339372 learned_addr learned my addr 172.21.15.6:0/3742339372 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:52:29.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.370+0000 7fd208c00700 1 --2- 172.21.15.6:0/3742339372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd20410e8f0 0x7fd2041a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:29.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.370+0000 7fd203e00700 1 -- 172.21.15.6:0/3742339372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd204105e10 msgr2=0x7fd20419d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:29.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.370+0000 7fd203e00700 1 --2- 172.21.15.6:0/3742339372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd204105e10 0x7fd20419d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:29.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.370+0000 7fd203e00700 1 -- 172.21.15.6:0/3742339372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd20410e8f0 msgr2=0x7fd2041a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:29.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.370+0000 7fd203e00700 1 --2- 172.21.15.6:0/3742339372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd20410e8f0 0x7fd2041a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:29.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.370+0000 7fd203e00700 1 -- 172.21.15.6:0/3742339372 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2041a1e50 con 0x7fd2041054c0 2024-05-09T04:52:29.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.370+0000 7fd208c00700 1 --2- 172.21.15.6:0/3742339372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd20410e8f0 0x7fd2041a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:29.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.371+0000 7fd203e00700 1 --2- 172.21.15.6:0/3742339372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2041054c0 0x7fd20419cf80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400e9a0 tx=0x7fd1f400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:29.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.371+0000 7fd200c00700 1 -- 172.21.15.6:0/3742339372 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f400ca10 con 0x7fd2041054c0 2024-05-09T04:52:29.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.371+0000 7fd20ac00700 1 -- 172.21.15.6:0/3742339372 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2041a2140 con 0x7fd2041054c0 2024-05-09T04:52:29.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.371+0000 7fd200c00700 1 -- 172.21.15.6:0/3742339372 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1f400cbb0 con 0x7fd2041054c0 2024-05-09T04:52:29.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.371+0000 7fd200c00700 1 -- 172.21.15.6:0/3742339372 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4018680 con 0x7fd2041054c0 2024-05-09T04:52:29.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.371+0000 7fd20ac00700 1 -- 172.21.15.6:0/3742339372 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2041a9a60 con 0x7fd2041054c0 2024-05-09T04:52:29.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.372+0000 7fd20ac00700 1 -- 172.21.15.6:0/3742339372 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd204103af0 con 0x7fd2041054c0 2024-05-09T04:52:29.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.375+0000 7fd200c00700 1 -- 172.21.15.6:0/3742339372 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fd1f4010430 con 0x7fd2041054c0 2024-05-09T04:52:29.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.376+0000 7fd200c00700 1 --2- 172.21.15.6:0/3742339372 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd1e80b2910 0x7fd1e80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:29.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.377+0000 7fd200c00700 1 -- 172.21.15.6:0/3742339372 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fd1f40e2f00 con 0x7fd2041054c0 2024-05-09T04:52:29.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.377+0000 7fd203400700 1 --2- 172.21.15.6:0/3742339372 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd1e80b2910 0x7fd1e80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:29.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.377+0000 7fd200c00700 1 --2- 172.21.15.6:0/3742339372 >> [v2:172.21.15.6:6818/3068252891,v1:172.21.15.6:6819/3068252891] conn(0x7fd1e80bdb40 0x7fd1e80c0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:29.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.377+0000 7fd200c00700 1 -- 172.21.15.6:0/3742339372 --> [v2:172.21.15.6:6818/3068252891,v1:172.21.15.6:6819/3068252891] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd1f4024e70 con 0x7fd1e80bdb40 2024-05-09T04:52:29.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.377+0000 7fd200c00700 1 -- 172.21.15.6:0/3742339372 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fd1f40e32b0 con 0x7fd2041054c0 2024-05-09T04:52:29.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.377+0000 7fd208c00700 1 --2- 172.21.15.6:0/3742339372 >> [v2:172.21.15.6:6818/3068252891,v1:172.21.15.6:6819/3068252891] conn(0x7fd1e80bdb40 0x7fd1e80c0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:29.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.378+0000 7fd203400700 1 --2- 172.21.15.6:0/3742339372 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd1e80b2910 0x7fd1e80b4e20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd204110710 tx=0x7fd1f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:29.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.378+0000 7fd208c00700 1 --2- 172.21.15.6:0/3742339372 >> [v2:172.21.15.6:6818/3068252891,v1:172.21.15.6:6819/3068252891] conn(0x7fd1e80bdb40 0x7fd1e80c0030 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:29.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.379+0000 7fd200c00700 1 -- 172.21.15.6:0/3742339372 <== osd.6 v2:172.21.15.6:6818/3068252891 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd1f4024e70 con 0x7fd1e80bdb40 2024-05-09T04:52:29.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.398+0000 7fd20ac00700 1 -- 172.21.15.6:0/3742339372 --> [v2:172.21.15.6:6818/3068252891,v1:172.21.15.6:6819/3068252891] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd20406be90 con 0x7fd1e80bdb40 2024-05-09T04:52:29.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.398+0000 7fd200c00700 1 -- 172.21.15.6:0/3742339372 <== osd.6 v2:172.21.15.6:6818/3068252891 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd20406be90 con 0x7fd1e80bdb40 2024-05-09T04:52:29.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.399+0000 7fd20ac00700 1 -- 172.21.15.6:0/3742339372 >> [v2:172.21.15.6:6818/3068252891,v1:172.21.15.6:6819/3068252891] conn(0x7fd1e80bdb40 msgr2=0x7fd1e80c0030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:29.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.399+0000 7fd20ac00700 1 --2- 172.21.15.6:0/3742339372 >> [v2:172.21.15.6:6818/3068252891,v1:172.21.15.6:6819/3068252891] conn(0x7fd1e80bdb40 0x7fd1e80c0030 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:29.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.399+0000 7fd20ac00700 1 -- 172.21.15.6:0/3742339372 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd1e80b2910 msgr2=0x7fd1e80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:29.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.399+0000 7fd20ac00700 1 --2- 172.21.15.6:0/3742339372 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd1e80b2910 0x7fd1e80b4e20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd204110710 tx=0x7fd1f0009260 comp rx=0 tx=0).stop 2024-05-09T04:52:29.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.399+0000 7fd20ac00700 1 -- 172.21.15.6:0/3742339372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2041054c0 msgr2=0x7fd20419cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:29.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.399+0000 7fd20ac00700 1 --2- 172.21.15.6:0/3742339372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2041054c0 0x7fd20419cf80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400e9a0 tx=0x7fd1f400ecb0 comp rx=0 tx=0).stop 2024-05-09T04:52:29.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.399+0000 7fd208c00700 1 -- 172.21.15.6:0/3742339372 reap_dead start 2024-05-09T04:52:29.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.399+0000 7fd20ac00700 1 -- 172.21.15.6:0/3742339372 shutdown_connections 2024-05-09T04:52:29.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.399+0000 7fd20ac00700 1 -- 172.21.15.6:0/3742339372 >> 172.21.15.6:0/3742339372 conn(0x7fd2040fedc0 msgr2=0x7fd204103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:29.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.400+0000 7fd20ac00700 1 -- 172.21.15.6:0/3742339372 shutdown_connections 2024-05-09T04:52:29.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:29.400+0000 7fd20ac00700 1 -- 172.21.15.6:0/3742339372 wait complete. 2024-05-09T04:52:30.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:30 smithi006 ceph-mon[29835]: pgmap v426: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-05-09T04:52:31.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:30 smithi186 ceph-mon[34668]: pgmap v426: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-05-09T04:52:31.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:30 smithi169 ceph-mon[34075]: pgmap v426: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-05-09T04:52:32.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:32 smithi006 ceph-mon[29835]: pgmap v427: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-09T04:52:33.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:32 smithi186 ceph-mon[34668]: pgmap v427: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-09T04:52:33.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:32 smithi169 ceph-mon[34075]: pgmap v427: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-09T04:52:34.674 INFO:teuthology.orchestra.run.smithi006.stdout:184683593810 2024-05-09T04:52:34.674 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-09T04:52:34.935 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:34 smithi006 ceph-mon[29835]: pgmap v428: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:52:35.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:34 smithi186 ceph-mon[34668]: pgmap v428: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:52:35.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:34 smithi169 ceph-mon[34075]: pgmap v428: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:52:35.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.930+0000 7fc3a0e00700 1 -- 172.21.15.6:0/3237503817 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc39c0764b0 msgr2=0x7fc39c114e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:35.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.930+0000 7fc3a0e00700 1 --2- 172.21.15.6:0/3237503817 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc39c0764b0 0x7fc39c114e30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc388009990 tx=0x7fc388009ca0 comp rx=0 tx=0).stop 2024-05-09T04:52:35.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.930+0000 7fc3a0e00700 1 -- 172.21.15.6:0/3237503817 shutdown_connections 2024-05-09T04:52:35.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.930+0000 7fc3a0e00700 1 --2- 172.21.15.6:0/3237503817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc39c115370 0x7fc39c117790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:35.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.930+0000 7fc3a0e00700 1 --2- 172.21.15.6:0/3237503817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc39c075b60 0x7fc39c075f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:35.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.930+0000 7fc3a0e00700 1 --2- 172.21.15.6:0/3237503817 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc39c0764b0 0x7fc39c114e30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:35.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.931+0000 7fc3a0e00700 1 -- 172.21.15.6:0/3237503817 >> 172.21.15.6:0/3237503817 conn(0x7fc39c100040 msgr2=0x7fc39c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:35.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.931+0000 7fc3a0e00700 1 -- 172.21.15.6:0/3237503817 shutdown_connections 2024-05-09T04:52:35.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.931+0000 7fc3a0e00700 1 -- 172.21.15.6:0/3237503817 wait complete. 2024-05-09T04:52:35.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.932+0000 7fc3a0e00700 1 Processor -- start 2024-05-09T04:52:35.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.932+0000 7fc3a0e00700 1 -- start start 2024-05-09T04:52:35.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.933+0000 7fc3a0e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc39c075b60 0x7fc39c1a1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:35.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.933+0000 7fc3a0e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc39c0764b0 0x7fc39c1a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:35.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.933+0000 7fc3a0e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc39c115370 0x7fc39c1a5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:35.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.933+0000 7fc3a0e00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc39c11aa70 con 0x7fc39c075b60 2024-05-09T04:52:35.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.933+0000 7fc3a0e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc39c11a8f0 con 0x7fc39c115370 2024-05-09T04:52:35.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.933+0000 7fc3a0e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc39c11abf0 con 0x7fc39c0764b0 2024-05-09T04:52:35.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.933+0000 7fc399e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc39c075b60 0x7fc39c1a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:35.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.933+0000 7fc399e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc39c075b60 0x7fc39c1a1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60470/0 (socket says 172.21.15.6:60470) 2024-05-09T04:52:35.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.933+0000 7fc399e00700 1 -- 172.21.15.6:0/1350570895 learned_addr learned my addr 172.21.15.6:0/1350570895 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:52:35.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.933+0000 7fc399400700 1 --2- 172.21.15.6:0/1350570895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc39c0764b0 0x7fc39c1a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:35.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.934+0000 7fc39a800700 1 --2- 172.21.15.6:0/1350570895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc39c115370 0x7fc39c1a5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:35.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.934+0000 7fc399e00700 1 -- 172.21.15.6:0/1350570895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc39c0764b0 msgr2=0x7fc39c1a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:35.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.934+0000 7fc399e00700 1 --2- 172.21.15.6:0/1350570895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc39c0764b0 0x7fc39c1a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:35.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.934+0000 7fc399e00700 1 -- 172.21.15.6:0/1350570895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc39c115370 msgr2=0x7fc39c1a5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:35.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.934+0000 7fc399e00700 1 --2- 172.21.15.6:0/1350570895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc39c115370 0x7fc39c1a5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:35.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.934+0000 7fc399e00700 1 -- 172.21.15.6:0/1350570895 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc39c1a62d0 con 0x7fc39c075b60 2024-05-09T04:52:35.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.934+0000 7fc39a800700 1 --2- 172.21.15.6:0/1350570895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc39c115370 0x7fc39c1a5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:35.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.934+0000 7fc399400700 1 --2- 172.21.15.6:0/1350570895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc39c0764b0 0x7fc39c1a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:35.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.935+0000 7fc399e00700 1 --2- 172.21.15.6:0/1350570895 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc39c075b60 0x7fc39c1a1400 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc38c00e9a0 tx=0x7fc38c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:35.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.935+0000 7fc392a00700 1 -- 172.21.15.6:0/1350570895 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38c00ca10 con 0x7fc39c075b60 2024-05-09T04:52:35.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.935+0000 7fc392a00700 1 -- 172.21.15.6:0/1350570895 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc38c00cbb0 con 0x7fc39c075b60 2024-05-09T04:52:35.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.935+0000 7fc3a0e00700 1 -- 172.21.15.6:0/1350570895 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc39c1a65c0 con 0x7fc39c075b60 2024-05-09T04:52:35.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.935+0000 7fc392a00700 1 -- 172.21.15.6:0/1350570895 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38c018680 con 0x7fc39c075b60 2024-05-09T04:52:35.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.935+0000 7fc3a0e00700 1 -- 172.21.15.6:0/1350570895 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc39c1adee0 con 0x7fc39c075b60 2024-05-09T04:52:35.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.936+0000 7fc3a0e00700 1 -- 172.21.15.6:0/1350570895 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc39c063ed0 con 0x7fc39c075b60 2024-05-09T04:52:35.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.939+0000 7fc392a00700 1 -- 172.21.15.6:0/1350570895 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fc38c010430 con 0x7fc39c075b60 2024-05-09T04:52:35.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.942+0000 7fc392a00700 1 --2- 172.21.15.6:0/1350570895 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fc3800b2990 0x7fc3800b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:35.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.942+0000 7fc392a00700 1 -- 172.21.15.6:0/1350570895 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fc38c0e2e60 con 0x7fc39c075b60 2024-05-09T04:52:35.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.942+0000 7fc399400700 1 --2- 172.21.15.6:0/1350570895 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fc3800b2990 0x7fc3800b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:35.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.942+0000 7fc392a00700 1 --2- 172.21.15.6:0/1350570895 >> [v2:172.21.15.169:6824/61007341,v1:172.21.15.169:6825/61007341] conn(0x7fc3800bdbc0 0x7fc3800c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:35.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.942+0000 7fc392a00700 1 -- 172.21.15.6:0/1350570895 --> [v2:172.21.15.169:6824/61007341,v1:172.21.15.169:6825/61007341] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc38c016ec0 con 0x7fc3800bdbc0 2024-05-09T04:52:35.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.942+0000 7fc392a00700 1 -- 172.21.15.6:0/1350570895 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fc38c0e3210 con 0x7fc39c075b60 2024-05-09T04:52:35.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.943+0000 7fc399400700 1 --2- 172.21.15.6:0/1350570895 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fc3800b2990 0x7fc3800b4ea0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc38800b2e0 tx=0x7fc388005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:35.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.943+0000 7fc39a800700 1 --2- 172.21.15.6:0/1350570895 >> [v2:172.21.15.169:6824/61007341,v1:172.21.15.169:6825/61007341] conn(0x7fc3800bdbc0 0x7fc3800c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:35.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.944+0000 7fc39a800700 1 --2- 172.21.15.6:0/1350570895 >> [v2:172.21.15.169:6824/61007341,v1:172.21.15.169:6825/61007341] conn(0x7fc3800bdbc0 0x7fc3800c00b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:35.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.945+0000 7fc392a00700 1 -- 172.21.15.6:0/1350570895 <== osd.10 v2:172.21.15.169:6824/61007341 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc38c016ec0 con 0x7fc3800bdbc0 2024-05-09T04:52:35.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.964+0000 7fc3a0e00700 1 -- 172.21.15.6:0/1350570895 --> [v2:172.21.15.169:6824/61007341,v1:172.21.15.169:6825/61007341] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc39c064350 con 0x7fc3800bdbc0 2024-05-09T04:52:35.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.965+0000 7fc392a00700 1 -- 172.21.15.6:0/1350570895 <== osd.10 v2:172.21.15.169:6824/61007341 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc39c064350 con 0x7fc3800bdbc0 2024-05-09T04:52:35.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.965+0000 7fc3a0e00700 1 -- 172.21.15.6:0/1350570895 >> [v2:172.21.15.169:6824/61007341,v1:172.21.15.169:6825/61007341] conn(0x7fc3800bdbc0 msgr2=0x7fc3800c00b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:35.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.965+0000 7fc3a0e00700 1 --2- 172.21.15.6:0/1350570895 >> [v2:172.21.15.169:6824/61007341,v1:172.21.15.169:6825/61007341] conn(0x7fc3800bdbc0 0x7fc3800c00b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:35.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.966+0000 7fc3a0e00700 1 -- 172.21.15.6:0/1350570895 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fc3800b2990 msgr2=0x7fc3800b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:35.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.966+0000 7fc3a0e00700 1 --2- 172.21.15.6:0/1350570895 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fc3800b2990 0x7fc3800b4ea0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc38800b2e0 tx=0x7fc388005960 comp rx=0 tx=0).stop 2024-05-09T04:52:35.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.966+0000 7fc3a0e00700 1 -- 172.21.15.6:0/1350570895 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc39c075b60 msgr2=0x7fc39c1a1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:35.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.966+0000 7fc3a0e00700 1 --2- 172.21.15.6:0/1350570895 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc39c075b60 0x7fc39c1a1400 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc38c00e9a0 tx=0x7fc38c00ecb0 comp rx=0 tx=0).stop 2024-05-09T04:52:35.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.966+0000 7fc39a800700 1 -- 172.21.15.6:0/1350570895 reap_dead start 2024-05-09T04:52:35.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.966+0000 7fc3a0e00700 1 -- 172.21.15.6:0/1350570895 shutdown_connections 2024-05-09T04:52:35.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.966+0000 7fc3a0e00700 1 -- 172.21.15.6:0/1350570895 >> 172.21.15.6:0/1350570895 conn(0x7fc39c100040 msgr2=0x7fc39c100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:35.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.966+0000 7fc3a0e00700 1 -- 172.21.15.6:0/1350570895 shutdown_connections 2024-05-09T04:52:35.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:35.966+0000 7fc3a0e00700 1 -- 172.21.15.6:0/1350570895 wait complete. 2024-05-09T04:52:36.903 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:36 smithi006 ceph-mon[29835]: pgmap v429: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:52:36.904 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:36 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/3459343097' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-09T04:52:37.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.032+0000 7f4e4c400700 1 -- 172.21.15.6:0/2271454019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e44107df0 msgr2=0x7f4e44108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:37.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.032+0000 7f4e4c400700 1 --2- 172.21.15.6:0/2271454019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e44107df0 0x7f4e44108200 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4e30009990 tx=0x7f4e30009ca0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.032+0000 7f4e4c400700 1 -- 172.21.15.6:0/2271454019 shutdown_connections 2024-05-09T04:52:37.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.032+0000 7f4e4c400700 1 --2- 172.21.15.6:0/2271454019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e44108740 0x7f4e441100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.032+0000 7f4e4c400700 1 --2- 172.21.15.6:0/2271454019 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e441034c0 0x7f4e441038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.032+0000 7f4e4c400700 1 --2- 172.21.15.6:0/2271454019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e44107df0 0x7f4e44108200 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.032+0000 7f4e4c400700 1 -- 172.21.15.6:0/2271454019 >> 172.21.15.6:0/2271454019 conn(0x7f4e440fedc0 msgr2=0x7f4e441011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:37.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.032+0000 7f4e4c400700 1 -- 172.21.15.6:0/2271454019 shutdown_connections 2024-05-09T04:52:37.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.033+0000 7f4e4c400700 1 -- 172.21.15.6:0/2271454019 wait complete. 2024-05-09T04:52:37.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.034+0000 7f4e4c400700 1 Processor -- start 2024-05-09T04:52:37.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.034+0000 7f4e4c400700 1 -- start start 2024-05-09T04:52:37.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.034+0000 7f4e4c400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e441034c0 0x7f4e4419ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:37.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.034+0000 7f4e4c400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e44107df0 0x7f4e4419b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:37.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.034+0000 7f4e4c400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e44108740 0x7f4e4419f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:37.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.034+0000 7f4e4c400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4e441133c0 con 0x7f4e44107df0 2024-05-09T04:52:37.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.034+0000 7f4e4c400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4e44113240 con 0x7f4e441034c0 2024-05-09T04:52:37.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.034+0000 7f4e4c400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4e44113540 con 0x7f4e44108740 2024-05-09T04:52:37.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.035+0000 7f4e49a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e441034c0 0x7f4e4419ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:37.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.035+0000 7f4e49000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e44107df0 0x7f4e4419b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:37.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.035+0000 7f4e49a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e441034c0 0x7f4e4419ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:35964/0 (socket says 172.21.15.6:35964) 2024-05-09T04:52:37.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.035+0000 7f4e49a00700 1 -- 172.21.15.6:0/3459343097 learned_addr learned my addr 172.21.15.6:0/3459343097 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:52:37.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.035+0000 7f4e4a400700 1 --2- 172.21.15.6:0/3459343097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e44108740 0x7f4e4419f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:37.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.035+0000 7f4e49a00700 1 -- 172.21.15.6:0/3459343097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e44108740 msgr2=0x7f4e4419f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:37.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.035+0000 7f4e49a00700 1 --2- 172.21.15.6:0/3459343097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e44108740 0x7f4e4419f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.035+0000 7f4e49a00700 1 -- 172.21.15.6:0/3459343097 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e44107df0 msgr2=0x7f4e4419b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:37.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.035+0000 7f4e49a00700 1 --2- 172.21.15.6:0/3459343097 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e44107df0 0x7f4e4419b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.035+0000 7f4e49a00700 1 -- 172.21.15.6:0/3459343097 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e4419fcf0 con 0x7f4e441034c0 2024-05-09T04:52:37.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.036+0000 7f4e49a00700 1 --2- 172.21.15.6:0/3459343097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e441034c0 0x7f4e4419ae20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4e3800e9a0 tx=0x7f4e3800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:37.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.036+0000 7f4e42000700 1 -- 172.21.15.6:0/3459343097 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3800cbd0 con 0x7f4e441034c0 2024-05-09T04:52:37.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.036+0000 7f4e42000700 1 -- 172.21.15.6:0/3459343097 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e3800cd70 con 0x7f4e441034c0 2024-05-09T04:52:37.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.036+0000 7f4e42000700 1 -- 172.21.15.6:0/3459343097 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e38018680 con 0x7f4e441034c0 2024-05-09T04:52:37.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.036+0000 7f4e4c400700 1 -- 172.21.15.6:0/3459343097 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e4419ffe0 con 0x7f4e441034c0 2024-05-09T04:52:37.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.036+0000 7f4e4c400700 1 -- 172.21.15.6:0/3459343097 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e441a7900 con 0x7f4e441034c0 2024-05-09T04:52:37.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.038+0000 7f4e4c400700 1 -- 172.21.15.6:0/3459343097 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e440512c0 con 0x7f4e441034c0 2024-05-09T04:52:37.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.042+0000 7f4e42000700 1 -- 172.21.15.6:0/3459343097 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f4e3804aae0 con 0x7f4e441034c0 2024-05-09T04:52:37.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.044+0000 7f4e42000700 1 --2- 172.21.15.6:0/3459343097 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4e2c0b2990 0x7f4e2c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:37.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.044+0000 7f4e42000700 1 -- 172.21.15.6:0/3459343097 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f4e380e29f0 con 0x7f4e441034c0 2024-05-09T04:52:37.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.044+0000 7f4e49000700 1 --2- 172.21.15.6:0/3459343097 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4e2c0b2990 0x7f4e2c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:37.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.045+0000 7f4e42000700 1 -- 172.21.15.6:0/3459343097 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4e380aa1d0 con 0x7f4e441034c0 2024-05-09T04:52:37.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.045+0000 7f4e49000700 1 --2- 172.21.15.6:0/3459343097 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4e2c0b2990 0x7f4e2c0b4ea0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4e3000b2e0 tx=0x7f4e30005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:37.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.291+0000 7f4e4c400700 1 -- 172.21.15.6:0/3459343097 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f4e4402e5f0 con 0x7f4e441034c0 2024-05-09T04:52:37.023 INFO:teuthology.orchestra.run.smithi006.stdout:111669149797 2024-05-09T04:52:37.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.292+0000 7f4e42000700 1 -- 172.21.15.6:0/3459343097 <== mon.2 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4e380a9ad0 con 0x7f4e441034c0 2024-05-09T04:52:37.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.295+0000 7f4e4c400700 1 -- 172.21.15.6:0/3459343097 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4e2c0b2990 msgr2=0x7f4e2c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:37.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.295+0000 7f4e4c400700 1 --2- 172.21.15.6:0/3459343097 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4e2c0b2990 0x7f4e2c0b4ea0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4e3000b2e0 tx=0x7f4e30005fb0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.295+0000 7f4e4c400700 1 -- 172.21.15.6:0/3459343097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e441034c0 msgr2=0x7f4e4419ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:37.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.295+0000 7f4e4c400700 1 --2- 172.21.15.6:0/3459343097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e441034c0 0x7f4e4419ae20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4e3800e9a0 tx=0x7f4e3800ecb0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.295+0000 7f4e4c400700 1 -- 172.21.15.6:0/3459343097 shutdown_connections 2024-05-09T04:52:37.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.295+0000 7f4e4c400700 1 --2- 172.21.15.6:0/3459343097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e44108740 0x7f4e4419f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.295+0000 7f4e4c400700 1 --2- 172.21.15.6:0/3459343097 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4e2c0b2990 0x7f4e2c0b4ea0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.295+0000 7f4e4c400700 1 --2- 172.21.15.6:0/3459343097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e441034c0 0x7f4e4419ae20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.295+0000 7f4e4c400700 1 --2- 172.21.15.6:0/3459343097 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e44107df0 0x7f4e4419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.296+0000 7f4e4c400700 1 -- 172.21.15.6:0/3459343097 >> 172.21.15.6:0/3459343097 conn(0x7f4e440fedc0 msgr2=0x7f4e4406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:37.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.296+0000 7f4e4c400700 1 -- 172.21.15.6:0/3459343097 shutdown_connections 2024-05-09T04:52:37.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:36.296+0000 7f4e4c400700 1 -- 172.21.15.6:0/3459343097 wait complete. 2024-05-09T04:52:37.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:36 smithi186 ceph-mon[34668]: pgmap v429: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:52:37.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:36 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/3459343097' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-09T04:52:37.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:36 smithi169 ceph-mon[34075]: pgmap v429: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:52:37.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:36 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/3459343097' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-09T04:52:37.275 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:37.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.592+0000 7fbb9be00700 1 -- 172.21.15.6:0/2599942201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb94107df0 msgr2=0x7fbb94108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:37.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.592+0000 7fbb9be00700 1 --2- 172.21.15.6:0/2599942201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb94107df0 0x7fbb94108200 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbb800098f0 tx=0x7fbb80009c00 comp rx=0 tx=0).stop 2024-05-09T04:52:37.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.592+0000 7fbb9be00700 1 -- 172.21.15.6:0/2599942201 shutdown_connections 2024-05-09T04:52:37.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.592+0000 7fbb9be00700 1 --2- 172.21.15.6:0/2599942201 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb94108740 0x7fbb941100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.592+0000 7fbb9be00700 1 --2- 172.21.15.6:0/2599942201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb941034c0 0x7fbb941038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.592+0000 7fbb9be00700 1 --2- 172.21.15.6:0/2599942201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb94107df0 0x7fbb94108200 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.592+0000 7fbb9be00700 1 -- 172.21.15.6:0/2599942201 >> 172.21.15.6:0/2599942201 conn(0x7fbb940fedc0 msgr2=0x7fbb941011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:37.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.592+0000 7fbb9be00700 1 -- 172.21.15.6:0/2599942201 shutdown_connections 2024-05-09T04:52:37.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.592+0000 7fbb9be00700 1 -- 172.21.15.6:0/2599942201 wait complete. 2024-05-09T04:52:37.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.594+0000 7fbb9be00700 1 Processor -- start 2024-05-09T04:52:37.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.594+0000 7fbb9be00700 1 -- start start 2024-05-09T04:52:37.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.594+0000 7fbb9be00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb941034c0 0x7fbb940ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:37.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.594+0000 7fbb9be00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb94107df0 0x7fbb940ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:37.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.594+0000 7fbb9be00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb94108740 0x7fbb940ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:37.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.594+0000 7fbb9be00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941133a0 con 0x7fbb94107df0 2024-05-09T04:52:37.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.594+0000 7fbb9be00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbb94113220 con 0x7fbb94108740 2024-05-09T04:52:37.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.594+0000 7fbb9be00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbb94113520 con 0x7fbb941034c0 2024-05-09T04:52:37.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.595+0000 7fbb99400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb941034c0 0x7fbb940ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:37.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.595+0000 7fbb98a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb94107df0 0x7fbb940ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:37.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.595+0000 7fbb99400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb941034c0 0x7fbb940ff2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:38958/0 (socket says 172.21.15.6:38958) 2024-05-09T04:52:37.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.595+0000 7fbb99400700 1 -- 172.21.15.6:0/4169204013 learned_addr learned my addr 172.21.15.6:0/4169204013 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:52:37.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.595+0000 7fbb99e00700 1 --2- 172.21.15.6:0/4169204013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb94108740 0x7fbb940ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:37.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.595+0000 7fbb98a00700 1 -- 172.21.15.6:0/4169204013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb941034c0 msgr2=0x7fbb940ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:37.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.595+0000 7fbb98a00700 1 --2- 172.21.15.6:0/4169204013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb941034c0 0x7fbb940ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.595+0000 7fbb98a00700 1 -- 172.21.15.6:0/4169204013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb94108740 msgr2=0x7fbb940ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:37.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.595+0000 7fbb98a00700 1 --2- 172.21.15.6:0/4169204013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb94108740 0x7fbb940ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.595+0000 7fbb98a00700 1 -- 172.21.15.6:0/4169204013 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb941a9240 con 0x7fbb94107df0 2024-05-09T04:52:37.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.595+0000 7fbb99e00700 1 --2- 172.21.15.6:0/4169204013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb94108740 0x7fbb940ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:37.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.596+0000 7fbb98a00700 1 --2- 172.21.15.6:0/4169204013 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb94107df0 0x7fbb940ff7e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbb80000c00 tx=0x7fbb8000fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:37.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.596+0000 7fbb92000700 1 -- 172.21.15.6:0/4169204013 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb800050c0 con 0x7fbb94107df0 2024-05-09T04:52:37.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.596+0000 7fbb92000700 1 -- 172.21.15.6:0/4169204013 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb8000b560 con 0x7fbb94107df0 2024-05-09T04:52:37.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.596+0000 7fbb92000700 1 -- 172.21.15.6:0/4169204013 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb800039b0 con 0x7fbb94107df0 2024-05-09T04:52:37.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.596+0000 7fbb9be00700 1 -- 172.21.15.6:0/4169204013 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb941a94d0 con 0x7fbb94107df0 2024-05-09T04:52:37.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.596+0000 7fbb9be00700 1 -- 172.21.15.6:0/4169204013 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb941a98d0 con 0x7fbb94107df0 2024-05-09T04:52:37.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.598+0000 7fbb9be00700 1 -- 172.21.15.6:0/4169204013 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb940512c0 con 0x7fbb94107df0 2024-05-09T04:52:37.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.600+0000 7fbb92000700 1 -- 172.21.15.6:0/4169204013 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fbb8000b9e0 con 0x7fbb94107df0 2024-05-09T04:52:37.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.602+0000 7fbb92000700 1 --2- 172.21.15.6:0/4169204013 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fbb7c0b2910 0x7fbb7c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:37.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.602+0000 7fbb92000700 1 -- 172.21.15.6:0/4169204013 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fbb800e3990 con 0x7fbb94107df0 2024-05-09T04:52:37.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.602+0000 7fbb99400700 1 --2- 172.21.15.6:0/4169204013 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fbb7c0b2910 0x7fbb7c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:37.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.603+0000 7fbb99400700 1 --2- 172.21.15.6:0/4169204013 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fbb7c0b2910 0x7fbb7c0b4e20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009fd0 tx=0x7fbb8800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:37.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.605+0000 7fbb92000700 1 -- 172.21.15.6:0/4169204013 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbb80013030 con 0x7fbb94107df0 2024-05-09T04:52:37.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.852+0000 7fbb9be00700 1 -- 172.21.15.6:0/4169204013 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fbb9402e850 con 0x7fbb94107df0 2024-05-09T04:52:37.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.853+0000 7fbb92000700 1 -- 172.21.15.6:0/4169204013 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbb800ab200 con 0x7fbb94107df0 2024-05-09T04:52:37.855 INFO:teuthology.orchestra.run.smithi006.stdout:206158430285 2024-05-09T04:52:37.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.857+0000 7fbb9be00700 1 -- 172.21.15.6:0/4169204013 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fbb7c0b2910 msgr2=0x7fbb7c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:37.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.857+0000 7fbb9be00700 1 --2- 172.21.15.6:0/4169204013 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fbb7c0b2910 0x7fbb7c0b4e20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009fd0 tx=0x7fbb8800b040 comp rx=0 tx=0).stop 2024-05-09T04:52:37.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.857+0000 7fbb9be00700 1 -- 172.21.15.6:0/4169204013 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb94107df0 msgr2=0x7fbb940ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:37.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.857+0000 7fbb9be00700 1 --2- 172.21.15.6:0/4169204013 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb94107df0 0x7fbb940ff7e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbb80000c00 tx=0x7fbb8000fa20 comp rx=0 tx=0).stop 2024-05-09T04:52:37.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.857+0000 7fbb9be00700 1 -- 172.21.15.6:0/4169204013 shutdown_connections 2024-05-09T04:52:37.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.857+0000 7fbb9be00700 1 --2- 172.21.15.6:0/4169204013 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fbb7c0b2910 0x7fbb7c0b4e20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.857+0000 7fbb9be00700 1 --2- 172.21.15.6:0/4169204013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb94108740 0x7fbb940ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.857+0000 7fbb9be00700 1 --2- 172.21.15.6:0/4169204013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb941034c0 0x7fbb940ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.857+0000 7fbb9be00700 1 --2- 172.21.15.6:0/4169204013 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb94107df0 0x7fbb940ff7e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.857+0000 7fbb9be00700 1 -- 172.21.15.6:0/4169204013 >> 172.21.15.6:0/4169204013 conn(0x7fbb940fedc0 msgr2=0x7fbb9406b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:37.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.857+0000 7fbb9be00700 1 -- 172.21.15.6:0/4169204013 shutdown_connections 2024-05-09T04:52:37.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:37.857+0000 7fbb9be00700 1 -- 172.21.15.6:0/4169204013 wait complete. 2024-05-09T04:52:38.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.073+0000 7f7e8ec00700 1 -- 172.21.15.6:0/3361986775 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e88108f00 msgr2=0x7f7e88109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:38.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.073+0000 7f7e8ec00700 1 --2- 172.21.15.6:0/3361986775 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e88108f00 0x7f7e88109310 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7e740099c0 tx=0x7f7e74009cd0 comp rx=0 tx=0).stop 2024-05-09T04:52:38.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.073+0000 7f7e8ec00700 1 -- 172.21.15.6:0/3361986775 shutdown_connections 2024-05-09T04:52:38.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.073+0000 7f7e8ec00700 1 --2- 172.21.15.6:0/3361986775 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e88109850 0x7f7e881111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:38.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.073+0000 7f7e8ec00700 1 --2- 172.21.15.6:0/3361986775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e881045d0 0x7f7e881049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:38.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.073+0000 7f7e8ec00700 1 --2- 172.21.15.6:0/3361986775 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e88108f00 0x7f7e88109310 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:38.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.073+0000 7f7e8ec00700 1 -- 172.21.15.6:0/3361986775 >> 172.21.15.6:0/3361986775 conn(0x7f7e880ffea0 msgr2=0x7f7e881022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:38.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.073+0000 7f7e8ec00700 1 -- 172.21.15.6:0/3361986775 shutdown_connections 2024-05-09T04:52:38.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.073+0000 7f7e8ec00700 1 -- 172.21.15.6:0/3361986775 wait complete. 2024-05-09T04:52:38.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.074+0000 7f7e8ec00700 1 Processor -- start 2024-05-09T04:52:38.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.074+0000 7f7e8ec00700 1 -- start start 2024-05-09T04:52:38.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.075+0000 7f7e8ec00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e881045d0 0x7f7e8819ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:38.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.075+0000 7f7e8ec00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e88108f00 0x7f7e8819b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:38.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.075+0000 7f7e8ec00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e88109850 0x7f7e8819f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:38.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.075+0000 7f7e8ec00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7e881144b0 con 0x7f7e88108f00 2024-05-09T04:52:38.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.075+0000 7f7e8ec00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7e88114330 con 0x7f7e88109850 2024-05-09T04:52:38.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.075+0000 7f7e8ec00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7e88114630 con 0x7f7e881045d0 2024-05-09T04:52:38.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.075+0000 7f7e87e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e881045d0 0x7f7e8819ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:38.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.075+0000 7f7e87e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e881045d0 0x7f7e8819ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:38980/0 (socket says 172.21.15.6:38980) 2024-05-09T04:52:38.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.075+0000 7f7e87e00700 1 -- 172.21.15.6:0/1778342037 learned_addr learned my addr 172.21.15.6:0/1778342037 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:52:38.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.075+0000 7f7e87400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e88108f00 0x7f7e8819b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:38.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.075+0000 7f7e8cc00700 1 --2- 172.21.15.6:0/1778342037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e88109850 0x7f7e8819f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:38.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.076+0000 7f7e87e00700 1 -- 172.21.15.6:0/1778342037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e88109850 msgr2=0x7f7e8819f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:38.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.076+0000 7f7e87e00700 1 --2- 172.21.15.6:0/1778342037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e88109850 0x7f7e8819f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:38.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.076+0000 7f7e87e00700 1 -- 172.21.15.6:0/1778342037 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e88108f00 msgr2=0x7f7e8819b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:38.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.076+0000 7f7e87e00700 1 --2- 172.21.15.6:0/1778342037 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e88108f00 0x7f7e8819b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:38.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.076+0000 7f7e87e00700 1 -- 172.21.15.6:0/1778342037 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8819fd30 con 0x7f7e881045d0 2024-05-09T04:52:38.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.076+0000 7f7e87e00700 1 --2- 172.21.15.6:0/1778342037 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e881045d0 0x7f7e8819ae60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800e9a0 tx=0x7f7e7800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:38.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.076+0000 7f7e84c00700 1 -- 172.21.15.6:0/1778342037 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7800ca10 con 0x7f7e881045d0 2024-05-09T04:52:38.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.077+0000 7f7e84c00700 1 -- 172.21.15.6:0/1778342037 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e7800cbb0 con 0x7f7e881045d0 2024-05-09T04:52:38.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.077+0000 7f7e8ec00700 1 -- 172.21.15.6:0/1778342037 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e881a0020 con 0x7f7e881045d0 2024-05-09T04:52:38.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.077+0000 7f7e84c00700 1 -- 172.21.15.6:0/1778342037 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78018680 con 0x7f7e881045d0 2024-05-09T04:52:38.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.077+0000 7f7e8ec00700 1 -- 172.21.15.6:0/1778342037 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e881a7940 con 0x7f7e881045d0 2024-05-09T04:52:38.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.078+0000 7f7e8ec00700 1 -- 172.21.15.6:0/1778342037 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e880512c0 con 0x7f7e881045d0 2024-05-09T04:52:38.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.080+0000 7f7e84c00700 1 -- 172.21.15.6:0/1778342037 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f7e78010430 con 0x7f7e881045d0 2024-05-09T04:52:38.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.082+0000 7f7e84c00700 1 --2- 172.21.15.6:0/1778342037 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f7e6c0b2990 0x7f7e6c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:38.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.082+0000 7f7e84c00700 1 -- 172.21.15.6:0/1778342037 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f7e780e2bb0 con 0x7f7e881045d0 2024-05-09T04:52:38.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.082+0000 7f7e87400700 1 --2- 172.21.15.6:0/1778342037 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f7e6c0b2990 0x7f7e6c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:38.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.083+0000 7f7e87400700 1 --2- 172.21.15.6:0/1778342037 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f7e6c0b2990 0x7f7e6c0b4ea0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7e74000c00 tx=0x7f7e7401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:38.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.084+0000 7f7e84c00700 1 -- 172.21.15.6:0/1778342037 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7e780aa390 con 0x7f7e881045d0 2024-05-09T04:52:38.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.339+0000 7f7e8ec00700 1 -- 172.21.15.6:0/1778342037 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f7e8802e5f0 con 0x7f7e881045d0 2024-05-09T04:52:38.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.344+0000 7f7e84c00700 1 -- 172.21.15.6:0/1778342037 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7e780b2410 con 0x7f7e881045d0 2024-05-09T04:52:38.345 INFO:teuthology.orchestra.run.smithi006.stdout:34359738486 2024-05-09T04:52:38.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.346+0000 7f7e8ec00700 1 -- 172.21.15.6:0/1778342037 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f7e6c0b2990 msgr2=0x7f7e6c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:38.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.347+0000 7f7e8ec00700 1 --2- 172.21.15.6:0/1778342037 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f7e6c0b2990 0x7f7e6c0b4ea0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7e74000c00 tx=0x7f7e7401a040 comp rx=0 tx=0).stop 2024-05-09T04:52:38.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.347+0000 7f7e8ec00700 1 -- 172.21.15.6:0/1778342037 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e881045d0 msgr2=0x7f7e8819ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:38.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.347+0000 7f7e8ec00700 1 --2- 172.21.15.6:0/1778342037 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e881045d0 0x7f7e8819ae60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800e9a0 tx=0x7f7e7800ecb0 comp rx=0 tx=0).stop 2024-05-09T04:52:38.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.347+0000 7f7e8ec00700 1 -- 172.21.15.6:0/1778342037 shutdown_connections 2024-05-09T04:52:38.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.347+0000 7f7e8ec00700 1 --2- 172.21.15.6:0/1778342037 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f7e6c0b2990 0x7f7e6c0b4ea0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:38.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.347+0000 7f7e8ec00700 1 --2- 172.21.15.6:0/1778342037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e88109850 0x7f7e8819f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:38.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.347+0000 7f7e8ec00700 1 --2- 172.21.15.6:0/1778342037 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e881045d0 0x7f7e8819ae60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:38.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.347+0000 7f7e8ec00700 1 --2- 172.21.15.6:0/1778342037 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e88108f00 0x7f7e8819b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:38.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.347+0000 7f7e8ec00700 1 -- 172.21.15.6:0/1778342037 >> 172.21.15.6:0/1778342037 conn(0x7f7e880ffea0 msgr2=0x7f7e8810a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:38.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.347+0000 7f7e8ec00700 1 -- 172.21.15.6:0/1778342037 shutdown_connections 2024-05-09T04:52:38.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:38.347+0000 7f7e8ec00700 1 -- 172.21.15.6:0/1778342037 wait complete. 2024-05-09T04:52:38.713 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:38.758 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430282 got 206158430285 for osd.7 2024-05-09T04:52:38.758 DEBUG:teuthology.parallel:result is None 2024-05-09T04:52:38.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:38 smithi006 ceph-mon[29835]: pgmap v430: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-05-09T04:52:38.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:38 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/4169204013' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-09T04:52:38.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:38 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1778342037' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-09T04:52:39.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:38 smithi186 ceph-mon[34668]: pgmap v430: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-05-09T04:52:39.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:38 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/4169204013' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-09T04:52:39.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:38 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/1778342037' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-09T04:52:39.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:38 smithi169 ceph-mon[34075]: pgmap v430: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-05-09T04:52:39.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:38 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/4169204013' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-09T04:52:39.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:38 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/1778342037' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-09T04:52:40.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.964+0000 7fcbad800700 1 -- 172.21.15.6:0/2170542759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcba8108f00 msgr2=0x7fcba8109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:40.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.964+0000 7fcbad800700 1 --2- 172.21.15.6:0/2170542759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcba8108f00 0x7fcba8109310 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcb940099c0 tx=0x7fcb94009cd0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.965+0000 7fcbad800700 1 -- 172.21.15.6:0/2170542759 shutdown_connections 2024-05-09T04:52:40.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.965+0000 7fcbad800700 1 --2- 172.21.15.6:0/2170542759 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcba8109850 0x7fcba81111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.965+0000 7fcbad800700 1 --2- 172.21.15.6:0/2170542759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcba81045d0 0x7fcba81049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.965+0000 7fcbad800700 1 --2- 172.21.15.6:0/2170542759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcba8108f00 0x7fcba8109310 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.965+0000 7fcbad800700 1 -- 172.21.15.6:0/2170542759 >> 172.21.15.6:0/2170542759 conn(0x7fcba80ffea0 msgr2=0x7fcba81022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:40.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.965+0000 7fcbad800700 1 -- 172.21.15.6:0/2170542759 shutdown_connections 2024-05-09T04:52:40.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.965+0000 7fcbad800700 1 -- 172.21.15.6:0/2170542759 wait complete. 2024-05-09T04:52:40.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.966+0000 7fcbad800700 1 Processor -- start 2024-05-09T04:52:40.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.966+0000 7fcbad800700 1 -- start start 2024-05-09T04:52:40.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.967+0000 7fcbad800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcba81045d0 0x7fcba819ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:40.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.967+0000 7fcbad800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcba8108f00 0x7fcba819b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:40.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.967+0000 7fcbad800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcba8109850 0x7fcba819f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:40.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.967+0000 7fcbad800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcba8114480 con 0x7fcba81045d0 2024-05-09T04:52:40.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.967+0000 7fcbad800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcba8114300 con 0x7fcba8109850 2024-05-09T04:52:40.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.967+0000 7fcbad800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcba8114600 con 0x7fcba8108f00 2024-05-09T04:52:40.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.967+0000 7fcba6000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcba8108f00 0x7fcba819b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:40.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.967+0000 7fcba6000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcba8108f00 0x7fcba819b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:38988/0 (socket says 172.21.15.6:38988) 2024-05-09T04:52:40.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.967+0000 7fcba6000700 1 -- 172.21.15.6:0/421418200 learned_addr learned my addr 172.21.15.6:0/421418200 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:52:40.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.967+0000 7fcba6a00700 1 --2- 172.21.15.6:0/421418200 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcba81045d0 0x7fcba819ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:40.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.967+0000 7fcba7400700 1 --2- 172.21.15.6:0/421418200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcba8109850 0x7fcba819f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:40.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.967+0000 7fcba6000700 1 -- 172.21.15.6:0/421418200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcba8109850 msgr2=0x7fcba819f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:40.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.967+0000 7fcba6000700 1 --2- 172.21.15.6:0/421418200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcba8109850 0x7fcba819f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.967+0000 7fcba6000700 1 -- 172.21.15.6:0/421418200 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcba81045d0 msgr2=0x7fcba819ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:40.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.967+0000 7fcba6000700 1 --2- 172.21.15.6:0/421418200 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcba81045d0 0x7fcba819ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.968+0000 7fcba6000700 1 -- 172.21.15.6:0/421418200 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba819fd20 con 0x7fcba8108f00 2024-05-09T04:52:40.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.968+0000 7fcba6a00700 1 --2- 172.21.15.6:0/421418200 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcba81045d0 0x7fcba819ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:52:40.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.968+0000 7fcba6000700 1 --2- 172.21.15.6:0/421418200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcba8108f00 0x7fcba819b3f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcb94005f50 tx=0x7fcb94004320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:40.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.968+0000 7fcb9f400700 1 -- 172.21.15.6:0/421418200 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb94026070 con 0x7fcba8108f00 2024-05-09T04:52:40.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.968+0000 7fcb9f400700 1 -- 172.21.15.6:0/421418200 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb94003c70 con 0x7fcba8108f00 2024-05-09T04:52:40.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.968+0000 7fcbad800700 1 -- 172.21.15.6:0/421418200 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcba819ffb0 con 0x7fcba8108f00 2024-05-09T04:52:40.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.968+0000 7fcb9f400700 1 -- 172.21.15.6:0/421418200 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9400fa60 con 0x7fcba8108f00 2024-05-09T04:52:40.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.968+0000 7fcbad800700 1 -- 172.21.15.6:0/421418200 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcba81a0490 con 0x7fcba8108f00 2024-05-09T04:52:40.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.969+0000 7fcbad800700 1 -- 172.21.15.6:0/421418200 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcba80512c0 con 0x7fcba8108f00 2024-05-09T04:52:40.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.972+0000 7fcb9f400700 1 -- 172.21.15.6:0/421418200 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fcb94029070 con 0x7fcba8108f00 2024-05-09T04:52:40.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.973+0000 7fcb9f400700 1 --2- 172.21.15.6:0/421418200 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fcb8c0b2910 0x7fcb8c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:40.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.974+0000 7fcb9f400700 1 -- 172.21.15.6:0/421418200 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fcb940e3770 con 0x7fcba8108f00 2024-05-09T04:52:40.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.974+0000 7fcba6a00700 1 --2- 172.21.15.6:0/421418200 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fcb8c0b2910 0x7fcb8c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:40.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.974+0000 7fcba6a00700 1 --2- 172.21.15.6:0/421418200 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fcb8c0b2910 0x7fcb8c0b4e20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcb98009cb0 tx=0x7fcb98009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:40.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:40.975+0000 7fcb9f400700 1 -- 172.21.15.6:0/421418200 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcb94030080 con 0x7fcba8108f00 2024-05-09T04:52:41.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:40 smithi186 ceph-mon[34668]: pgmap v431: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-05-09T04:52:41.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:40 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:52:41.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:40 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:52:41.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:40 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:52:41.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:40 smithi169 ceph-mon[34075]: pgmap v431: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-05-09T04:52:41.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:40 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:52:41.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:40 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:52:41.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:40 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:52:41.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:40 smithi006 ceph-mon[29835]: pgmap v431: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-05-09T04:52:41.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:40 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:52:41.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:40 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:52:41.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:40 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:52:41.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.208+0000 7fcbad800700 1 -- 172.21.15.6:0/421418200 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fcba802e5f0 con 0x7fcba8108f00 2024-05-09T04:52:41.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.208+0000 7fcb9f400700 1 -- 172.21.15.6:0/421418200 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcb94013070 con 0x7fcba8108f00 2024-05-09T04:52:41.209 INFO:teuthology.orchestra.run.smithi006.stdout:158913790042 2024-05-09T04:52:41.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.211+0000 7fcbad800700 1 -- 172.21.15.6:0/421418200 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fcb8c0b2910 msgr2=0x7fcb8c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:41.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.211+0000 7fcbad800700 1 --2- 172.21.15.6:0/421418200 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fcb8c0b2910 0x7fcb8c0b4e20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcb98009cb0 tx=0x7fcb98009260 comp rx=0 tx=0).stop 2024-05-09T04:52:41.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.211+0000 7fcbad800700 1 -- 172.21.15.6:0/421418200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcba8108f00 msgr2=0x7fcba819b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:41.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.211+0000 7fcbad800700 1 --2- 172.21.15.6:0/421418200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcba8108f00 0x7fcba819b3f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcb94005f50 tx=0x7fcb94004320 comp rx=0 tx=0).stop 2024-05-09T04:52:41.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.211+0000 7fcbad800700 1 -- 172.21.15.6:0/421418200 shutdown_connections 2024-05-09T04:52:41.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.211+0000 7fcbad800700 1 --2- 172.21.15.6:0/421418200 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fcb8c0b2910 0x7fcb8c0b4e20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:41.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.211+0000 7fcbad800700 1 --2- 172.21.15.6:0/421418200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcba8109850 0x7fcba819f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:41.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.211+0000 7fcbad800700 1 --2- 172.21.15.6:0/421418200 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcba81045d0 0x7fcba819ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:41.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.211+0000 7fcbad800700 1 --2- 172.21.15.6:0/421418200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcba8108f00 0x7fcba819b3f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:41.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.211+0000 7fcbad800700 1 -- 172.21.15.6:0/421418200 >> 172.21.15.6:0/421418200 conn(0x7fcba80ffea0 msgr2=0x7fcba81022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:41.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.212+0000 7fcbad800700 1 -- 172.21.15.6:0/421418200 shutdown_connections 2024-05-09T04:52:41.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.212+0000 7fcbad800700 1 -- 172.21.15.6:0/421418200 wait complete. 2024-05-09T04:52:41.919 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:41 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/421418200' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-09T04:52:42.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:41 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/421418200' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-09T04:52:42.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:41 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/421418200' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-09T04:52:42.353 INFO:teuthology.orchestra.run.smithi006.stdout:279172874300 2024-05-09T04:52:42.353 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-09T04:52:42.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:42 smithi006 ceph-mon[29835]: pgmap v432: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-05-09T04:52:42.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:42 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1664666341' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-09T04:52:42.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:42 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:52:42.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:42 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:52:42.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:42 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:52:43.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:42 smithi186 ceph-mon[34668]: pgmap v432: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-05-09T04:52:43.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:42 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/1664666341' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-09T04:52:43.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:42 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:52:43.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:42 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:52:43.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:42 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:52:43.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:42 smithi169 ceph-mon[34075]: pgmap v432: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-05-09T04:52:43.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:42 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/1664666341' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-09T04:52:43.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:42 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:52:43.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:42 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:52:43.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:42 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:52:43.418 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149794 got 111669149797 for osd.3 2024-05-09T04:52:43.418 DEBUG:teuthology.parallel:result is None 2024-05-09T04:52:43.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.668+0000 7fcfad800700 1 -- 172.21.15.6:0/670060597 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfa81034c0 msgr2=0x7fcfa81038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:43.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.668+0000 7fcfad800700 1 --2- 172.21.15.6:0/670060597 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfa81034c0 0x7fcfa81038d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcf980099c0 tx=0x7fcf98009cd0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.668+0000 7fcfad800700 1 -- 172.21.15.6:0/670060597 shutdown_connections 2024-05-09T04:52:43.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.668+0000 7fcfad800700 1 --2- 172.21.15.6:0/670060597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfa8108740 0x7fcfa81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.668+0000 7fcfad800700 1 --2- 172.21.15.6:0/670060597 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfa81034c0 0x7fcfa81038d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.668+0000 7fcfad800700 1 --2- 172.21.15.6:0/670060597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfa8107df0 0x7fcfa8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.668+0000 7fcfad800700 1 -- 172.21.15.6:0/670060597 >> 172.21.15.6:0/670060597 conn(0x7fcfa80fedc0 msgr2=0x7fcfa81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:43.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.668+0000 7fcfad800700 1 -- 172.21.15.6:0/670060597 shutdown_connections 2024-05-09T04:52:43.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.668+0000 7fcfad800700 1 -- 172.21.15.6:0/670060597 wait complete. 2024-05-09T04:52:43.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.670+0000 7fcfad800700 1 Processor -- start 2024-05-09T04:52:43.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.670+0000 7fcfad800700 1 -- start start 2024-05-09T04:52:43.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.671+0000 7fcfad800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfa81034c0 0x7fcfa819d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:43.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.671+0000 7fcfad800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfa8107df0 0x7fcfa819d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:43.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.671+0000 7fcfad800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfa8108740 0x7fcfa81a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:43.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.671+0000 7fcfad800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa8113370 con 0x7fcfa81034c0 2024-05-09T04:52:43.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.671+0000 7fcfad800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa81131f0 con 0x7fcfa8107df0 2024-05-09T04:52:43.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.671+0000 7fcfad800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa81134f0 con 0x7fcfa8108740 2024-05-09T04:52:43.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.671+0000 7fcfa6a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfa81034c0 0x7fcfa819d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:43.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.671+0000 7fcfa6a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfa81034c0 0x7fcfa819d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36534/0 (socket says 172.21.15.6:36534) 2024-05-09T04:52:43.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.671+0000 7fcfa6a00700 1 -- 172.21.15.6:0/1664666341 learned_addr learned my addr 172.21.15.6:0/1664666341 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:52:43.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.671+0000 7fcfa6000700 1 --2- 172.21.15.6:0/1664666341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfa8107df0 0x7fcfa819d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:43.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.671+0000 7fcfa7400700 1 --2- 172.21.15.6:0/1664666341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfa8108740 0x7fcfa81a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:43.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.672+0000 7fcfa6a00700 1 -- 172.21.15.6:0/1664666341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfa8108740 msgr2=0x7fcfa81a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:43.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.672+0000 7fcfa6a00700 1 --2- 172.21.15.6:0/1664666341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfa8108740 0x7fcfa81a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.672+0000 7fcfa6a00700 1 -- 172.21.15.6:0/1664666341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfa8107df0 msgr2=0x7fcfa819d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:43.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.672+0000 7fcfa6a00700 1 --2- 172.21.15.6:0/1664666341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfa8107df0 0x7fcfa819d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.672+0000 7fcfa6a00700 1 -- 172.21.15.6:0/1664666341 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa81a1f50 con 0x7fcfa81034c0 2024-05-09T04:52:43.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.672+0000 7fcfa7400700 1 --2- 172.21.15.6:0/1664666341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfa8108740 0x7fcfa81a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:52:43.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.672+0000 7fcfa6a00700 1 --2- 172.21.15.6:0/1664666341 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfa81034c0 0x7fcfa819d080 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcf98006010 tx=0x7fcf98004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:43.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.672+0000 7fcf9f400700 1 -- 172.21.15.6:0/1664666341 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf98026070 con 0x7fcfa81034c0 2024-05-09T04:52:43.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.672+0000 7fcf9f400700 1 -- 172.21.15.6:0/1664666341 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf98003c70 con 0x7fcfa81034c0 2024-05-09T04:52:43.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.672+0000 7fcfad800700 1 -- 172.21.15.6:0/1664666341 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfa81a21e0 con 0x7fcfa81034c0 2024-05-09T04:52:43.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.672+0000 7fcf9f400700 1 -- 172.21.15.6:0/1664666341 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9800fbe0 con 0x7fcfa81034c0 2024-05-09T04:52:43.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.672+0000 7fcfad800700 1 -- 172.21.15.6:0/1664666341 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfa81a26c0 con 0x7fcfa81034c0 2024-05-09T04:52:43.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.673+0000 7fcfad800700 1 -- 172.21.15.6:0/1664666341 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfa80512c0 con 0x7fcfa81034c0 2024-05-09T04:52:43.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.678+0000 7fcf9f400700 1 -- 172.21.15.6:0/1664666341 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fcf98029070 con 0x7fcfa81034c0 2024-05-09T04:52:43.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.679+0000 7fcf9f400700 1 --2- 172.21.15.6:0/1664666341 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fcf8c0b2910 0x7fcf8c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:43.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.680+0000 7fcf9f400700 1 -- 172.21.15.6:0/1664666341 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fcf980e3480 con 0x7fcfa81034c0 2024-05-09T04:52:43.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.680+0000 7fcfa6000700 1 --2- 172.21.15.6:0/1664666341 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fcf8c0b2910 0x7fcf8c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:43.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.680+0000 7fcf9f400700 1 -- 172.21.15.6:0/1664666341 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcf98013070 con 0x7fcfa81034c0 2024-05-09T04:52:43.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.681+0000 7fcfa6000700 1 --2- 172.21.15.6:0/1664666341 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fcf8c0b2910 0x7fcf8c0b4e20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcfa806c5b0 tx=0x7fcf94009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:43.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.918+0000 7fcfad800700 1 -- 172.21.15.6:0/1664666341 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fcfa802e5f0 con 0x7fcfa81034c0 2024-05-09T04:52:43.681 INFO:teuthology.orchestra.run.smithi006.stdout:55834574962 2024-05-09T04:52:43.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.918+0000 7fcf9f400700 1 -- 172.21.15.6:0/1664666341 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcf98003e20 con 0x7fcfa81034c0 2024-05-09T04:52:43.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.922+0000 7fcfad800700 1 -- 172.21.15.6:0/1664666341 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fcf8c0b2910 msgr2=0x7fcf8c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:43.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.922+0000 7fcfad800700 1 --2- 172.21.15.6:0/1664666341 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fcf8c0b2910 0x7fcf8c0b4e20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcfa806c5b0 tx=0x7fcf94009260 comp rx=0 tx=0).stop 2024-05-09T04:52:43.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.922+0000 7fcfad800700 1 -- 172.21.15.6:0/1664666341 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfa81034c0 msgr2=0x7fcfa819d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:43.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.922+0000 7fcfad800700 1 --2- 172.21.15.6:0/1664666341 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfa81034c0 0x7fcfa819d080 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcf98006010 tx=0x7fcf98004270 comp rx=0 tx=0).stop 2024-05-09T04:52:43.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.922+0000 7fcfad800700 1 -- 172.21.15.6:0/1664666341 shutdown_connections 2024-05-09T04:52:43.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.922+0000 7fcfad800700 1 --2- 172.21.15.6:0/1664666341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfa8108740 0x7fcfa81a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.922+0000 7fcfad800700 1 --2- 172.21.15.6:0/1664666341 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfa81034c0 0x7fcfa819d080 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.922+0000 7fcfad800700 1 --2- 172.21.15.6:0/1664666341 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fcf8c0b2910 0x7fcf8c0b4e20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.922+0000 7fcfad800700 1 --2- 172.21.15.6:0/1664666341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfa8107df0 0x7fcfa819d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.922+0000 7fcfad800700 1 -- 172.21.15.6:0/1664666341 >> 172.21.15.6:0/1664666341 conn(0x7fcfa80fedc0 msgr2=0x7fcfa8109780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:43.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.922+0000 7fcfad800700 1 -- 172.21.15.6:0/1664666341 shutdown_connections 2024-05-09T04:52:43.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:41.922+0000 7fcfad800700 1 -- 172.21.15.6:0/1664666341 wait complete. 2024-05-09T04:52:44.737 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574956 got 55834574962 for osd.1 2024-05-09T04:52:44.737 DEBUG:teuthology.parallel:result is None 2024-05-09T04:52:44.917 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:44.918 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:44.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:44 smithi006 ceph-mon[29835]: pgmap v433: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-05-09T04:52:45.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:44 smithi186 ceph-mon[34668]: pgmap v433: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-05-09T04:52:45.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:44 smithi169 ceph-mon[34075]: pgmap v433: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-05-09T04:52:46.400 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738483 got 34359738486 for osd.0 2024-05-09T04:52:46.401 DEBUG:teuthology.parallel:result is None 2024-05-09T04:52:46.761 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790038 got 158913790042 for osd.5 2024-05-09T04:52:46.761 DEBUG:teuthology.parallel:result is None 2024-05-09T04:52:46.976 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:46 smithi006 ceph-mon[29835]: pgmap v434: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:52:46.976 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:46 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1877995290' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-09T04:52:47.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:46 smithi186 ceph-mon[34668]: pgmap v434: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:52:47.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:46 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/1877995290' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-09T04:52:47.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:46 smithi169 ceph-mon[34075]: pgmap v434: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:52:47.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:46 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/1877995290' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-09T04:52:47.340 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:47.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.172+0000 7f2bfda00700 1 -- 172.21.15.6:0/1144289243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bf810b8a0 msgr2=0x7f2bf8113200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:47.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.172+0000 7f2bfda00700 1 --2- 172.21.15.6:0/1144289243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bf810b8a0 0x7f2bf8113200 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0009fc0 tx=0x7f2bf000b330 comp rx=0 tx=0).stop 2024-05-09T04:52:47.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.172+0000 7f2bfda00700 1 -- 172.21.15.6:0/1144289243 shutdown_connections 2024-05-09T04:52:47.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.172+0000 7f2bfda00700 1 --2- 172.21.15.6:0/1144289243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bf810b8a0 0x7f2bf8113200 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:47.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.172+0000 7f2bfda00700 1 --2- 172.21.15.6:0/1144289243 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bf8106620 0x7f2bf8106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:47.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.172+0000 7f2bfda00700 1 --2- 172.21.15.6:0/1144289243 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2bf810af50 0x7f2bf810b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:47.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.172+0000 7f2bfda00700 1 -- 172.21.15.6:0/1144289243 >> 172.21.15.6:0/1144289243 conn(0x7f2bf8102030 msgr2=0x7f2bf8104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:47.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.173+0000 7f2bfda00700 1 -- 172.21.15.6:0/1144289243 shutdown_connections 2024-05-09T04:52:47.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.173+0000 7f2bfda00700 1 -- 172.21.15.6:0/1144289243 wait complete. 2024-05-09T04:52:47.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.174+0000 7f2bfda00700 1 Processor -- start 2024-05-09T04:52:47.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.174+0000 7f2bfda00700 1 -- start start 2024-05-09T04:52:47.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.175+0000 7f2bfda00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2bf8106620 0x7f2bf819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:47.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.175+0000 7f2bfda00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bf810af50 0x7f2bf819d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:47.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.175+0000 7f2bfda00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bf810b8a0 0x7f2bf81a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:47.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.175+0000 7f2bfda00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf81164d0 con 0x7f2bf8106620 2024-05-09T04:52:47.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.175+0000 7f2bfda00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf8116350 con 0x7f2bf810b8a0 2024-05-09T04:52:47.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.175+0000 7f2bfda00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf8116650 con 0x7f2bf810af50 2024-05-09T04:52:47.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.175+0000 7f2bf7e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2bf8106620 0x7f2bf819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:47.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.175+0000 7f2bf7400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bf810af50 0x7f2bf819d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:47.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.175+0000 7f2bfd000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bf810b8a0 0x7f2bf81a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:47.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.175+0000 7f2bf7400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bf810af50 0x7f2bf819d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:39034/0 (socket says 172.21.15.6:39034) 2024-05-09T04:52:47.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.175+0000 7f2bf7400700 1 -- 172.21.15.6:0/1877995290 learned_addr learned my addr 172.21.15.6:0/1877995290 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:52:47.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.176+0000 7f2bf7400700 1 -- 172.21.15.6:0/1877995290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bf810b8a0 msgr2=0x7f2bf81a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:47.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.176+0000 7f2bf7400700 1 --2- 172.21.15.6:0/1877995290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bf810b8a0 0x7f2bf81a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:47.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.176+0000 7f2bf7400700 1 -- 172.21.15.6:0/1877995290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2bf8106620 msgr2=0x7f2bf819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:47.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.176+0000 7f2bf7400700 1 --2- 172.21.15.6:0/1877995290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2bf8106620 0x7f2bf819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:47.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.176+0000 7f2bf7400700 1 -- 172.21.15.6:0/1877995290 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf81a1c50 con 0x7f2bf810af50 2024-05-09T04:52:47.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.176+0000 7f2bfd000700 1 --2- 172.21.15.6:0/1877995290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bf810b8a0 0x7f2bf81a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:47.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.176+0000 7f2bf7400700 1 --2- 172.21.15.6:0/1877995290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bf810af50 0x7f2bf819d320 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2be400e970 tx=0x7f2be400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:47.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.176+0000 7f2bf4c00700 1 -- 172.21.15.6:0/1877995290 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be4004a30 con 0x7f2bf810af50 2024-05-09T04:52:47.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.176+0000 7f2bfda00700 1 -- 172.21.15.6:0/1877995290 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bf81a1f40 con 0x7f2bf810af50 2024-05-09T04:52:47.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.177+0000 7f2bf4c00700 1 -- 172.21.15.6:0/1877995290 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2be400cc90 con 0x7f2bf810af50 2024-05-09T04:52:47.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.177+0000 7f2bf4c00700 1 -- 172.21.15.6:0/1877995290 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be4005610 con 0x7f2bf810af50 2024-05-09T04:52:47.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.177+0000 7f2bfda00700 1 -- 172.21.15.6:0/1877995290 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bf81a9860 con 0x7f2bf810af50 2024-05-09T04:52:47.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.178+0000 7f2bfda00700 1 -- 172.21.15.6:0/1877995290 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bf80512c0 con 0x7f2bf810af50 2024-05-09T04:52:47.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.181+0000 7f2bf4c00700 1 -- 172.21.15.6:0/1877995290 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f2be400c7e0 con 0x7f2bf810af50 2024-05-09T04:52:47.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.183+0000 7f2bf4c00700 1 --2- 172.21.15.6:0/1877995290 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f2bdc0b2910 0x7f2bdc0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:47.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.183+0000 7f2bf4c00700 1 -- 172.21.15.6:0/1877995290 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f2be40e2ac0 con 0x7f2bf810af50 2024-05-09T04:52:47.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.183+0000 7f2bf7e00700 1 --2- 172.21.15.6:0/1877995290 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f2bdc0b2910 0x7f2bdc0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:47.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.184+0000 7f2bf7e00700 1 --2- 172.21.15.6:0/1877995290 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f2bdc0b2910 0x7f2bdc0b4e20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2be8009750 tx=0x7f2be8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:47.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.184+0000 7f2bf4c00700 1 -- 172.21.15.6:0/1877995290 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2be4054df0 con 0x7f2bf810af50 2024-05-09T04:52:47.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.431+0000 7f2bfda00700 1 -- 172.21.15.6:0/1877995290 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f2bf802e5f0 con 0x7f2bf810af50 2024-05-09T04:52:47.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.432+0000 7f2bf4c00700 1 -- 172.21.15.6:0/1877995290 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2be40aa2a0 con 0x7f2bf810af50 2024-05-09T04:52:47.786 INFO:teuthology.orchestra.run.smithi006.stdout:77309411439 2024-05-09T04:52:47.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.435+0000 7f2bfda00700 1 -- 172.21.15.6:0/1877995290 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f2bdc0b2910 msgr2=0x7f2bdc0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:47.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.435+0000 7f2bfda00700 1 --2- 172.21.15.6:0/1877995290 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f2bdc0b2910 0x7f2bdc0b4e20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2be8009750 tx=0x7f2be8009310 comp rx=0 tx=0).stop 2024-05-09T04:52:47.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.435+0000 7f2bfda00700 1 -- 172.21.15.6:0/1877995290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bf810af50 msgr2=0x7f2bf819d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:47.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.435+0000 7f2bfda00700 1 --2- 172.21.15.6:0/1877995290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bf810af50 0x7f2bf819d320 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2be400e970 tx=0x7f2be400ec80 comp rx=0 tx=0).stop 2024-05-09T04:52:47.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.435+0000 7f2bfda00700 1 -- 172.21.15.6:0/1877995290 shutdown_connections 2024-05-09T04:52:47.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.435+0000 7f2bfda00700 1 --2- 172.21.15.6:0/1877995290 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f2bdc0b2910 0x7f2bdc0b4e20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:47.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.435+0000 7f2bfda00700 1 --2- 172.21.15.6:0/1877995290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bf810b8a0 0x7f2bf81a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:47.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.435+0000 7f2bfda00700 1 --2- 172.21.15.6:0/1877995290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2bf8106620 0x7f2bf819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:47.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.435+0000 7f2bfda00700 1 --2- 172.21.15.6:0/1877995290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bf810af50 0x7f2bf819d320 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:47.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.435+0000 7f2bfda00700 1 -- 172.21.15.6:0/1877995290 >> 172.21.15.6:0/1877995290 conn(0x7f2bf8102030 msgr2=0x7f2bf810a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:47.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.435+0000 7f2bfda00700 1 -- 172.21.15.6:0/1877995290 shutdown_connections 2024-05-09T04:52:47.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:46.435+0000 7f2bfda00700 1 -- 172.21.15.6:0/1877995290 wait complete. 2024-05-09T04:52:48.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.438+0000 7fecec600700 1 -- 172.21.15.6:0/573346334 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fece410e8f0 msgr2=0x7fece4110dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:48.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.438+0000 7fecec600700 1 --2- 172.21.15.6:0/573346334 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fece410e8f0 0x7fece4110dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fecdc00b0a0 tx=0x7fecdc00b3b0 comp rx=0 tx=0).stop 2024-05-09T04:52:48.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.438+0000 7fecec600700 1 -- 172.21.15.6:0/573346334 shutdown_connections 2024-05-09T04:52:48.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.438+0000 7fecec600700 1 --2- 172.21.15.6:0/573346334 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fece410e8f0 0x7fece4110dd0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:48.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.438+0000 7fecec600700 1 --2- 172.21.15.6:0/573346334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fece4105480 0x7fece4105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:48.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.438+0000 7fecec600700 1 --2- 172.21.15.6:0/573346334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fece4105dd0 0x7fece410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:48.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.438+0000 7fecec600700 1 -- 172.21.15.6:0/573346334 >> 172.21.15.6:0/573346334 conn(0x7fece40fef30 msgr2=0x7fece4101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:48.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.438+0000 7fecec600700 1 -- 172.21.15.6:0/573346334 shutdown_connections 2024-05-09T04:52:48.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.438+0000 7fecec600700 1 -- 172.21.15.6:0/573346334 wait complete. 2024-05-09T04:52:48.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.439+0000 7fecec600700 1 Processor -- start 2024-05-09T04:52:48.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.439+0000 7fecec600700 1 -- start start 2024-05-09T04:52:48.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.440+0000 7fecec600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fece4105480 0x7fece419ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:48.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.440+0000 7fecec600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fece4105dd0 0x7fece419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:48.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.440+0000 7fecec600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fece410e8f0 0x7fece419f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:48.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.440+0000 7fecec600700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fece4114450 con 0x7fece4105dd0 2024-05-09T04:52:48.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.440+0000 7fecec600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fece41142d0 con 0x7fece410e8f0 2024-05-09T04:52:48.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.440+0000 7fecec600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fece41145d0 con 0x7fece4105480 2024-05-09T04:52:48.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.440+0000 7fece9200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fece4105dd0 0x7fece419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:48.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.440+0000 7fece9c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fece4105480 0x7fece419ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:48.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.440+0000 7fece9200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fece4105dd0 0x7fece419b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42394/0 (socket says 172.21.15.6:42394) 2024-05-09T04:52:48.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.440+0000 7fece9200700 1 -- 172.21.15.6:0/3979151104 learned_addr learned my addr 172.21.15.6:0/3979151104 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:52:48.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.440+0000 7fecea600700 1 --2- 172.21.15.6:0/3979151104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fece410e8f0 0x7fece419f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:48.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.441+0000 7fece9c00700 1 -- 172.21.15.6:0/3979151104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fece410e8f0 msgr2=0x7fece419f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:48.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.441+0000 7fece9c00700 1 --2- 172.21.15.6:0/3979151104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fece410e8f0 0x7fece419f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:48.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.441+0000 7fece9c00700 1 -- 172.21.15.6:0/3979151104 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fece4105dd0 msgr2=0x7fece419b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:48.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.441+0000 7fece9c00700 1 --2- 172.21.15.6:0/3979151104 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fece4105dd0 0x7fece419b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:48.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.441+0000 7fece9c00700 1 -- 172.21.15.6:0/3979151104 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece419fcd0 con 0x7fece4105480 2024-05-09T04:52:48.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.441+0000 7fece9c00700 1 --2- 172.21.15.6:0/3979151104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fece4105480 0x7fece419ae00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fecd800d750 tx=0x7fecd800da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:48.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.441+0000 7fece2000700 1 -- 172.21.15.6:0/3979151104 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd8014030 con 0x7fece4105480 2024-05-09T04:52:48.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.441+0000 7fece2000700 1 -- 172.21.15.6:0/3979151104 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecd80044d0 con 0x7fece4105480 2024-05-09T04:52:48.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.441+0000 7fece2000700 1 -- 172.21.15.6:0/3979151104 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd8005420 con 0x7fece4105480 2024-05-09T04:52:48.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.441+0000 7fecec600700 1 -- 172.21.15.6:0/3979151104 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fece419ffc0 con 0x7fece4105480 2024-05-09T04:52:48.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.442+0000 7fecec600700 1 -- 172.21.15.6:0/3979151104 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fece41a78e0 con 0x7fece4105480 2024-05-09T04:52:48.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.443+0000 7fecec600700 1 -- 172.21.15.6:0/3979151104 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fece40512c0 con 0x7fece4105480 2024-05-09T04:52:48.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.445+0000 7fece2000700 1 -- 172.21.15.6:0/3979151104 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fecd8004a50 con 0x7fece4105480 2024-05-09T04:52:48.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.447+0000 7fece2000700 1 --2- 172.21.15.6:0/3979151104 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7feccc0b2910 0x7feccc0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:48.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.447+0000 7fece2000700 1 -- 172.21.15.6:0/3979151104 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fecd80e2390 con 0x7fece4105480 2024-05-09T04:52:48.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.447+0000 7fece9200700 1 --2- 172.21.15.6:0/3979151104 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7feccc0b2910 0x7feccc0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:48.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.448+0000 7fece9200700 1 --2- 172.21.15.6:0/3979151104 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7feccc0b2910 0x7feccc0b4e20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fece41024c0 tx=0x7fecd0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:48.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.449+0000 7fece2000700 1 -- 172.21.15.6:0/3979151104 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fecd80a9b70 con 0x7fece4105480 2024-05-09T04:52:48.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.689+0000 7fecec600700 1 -- 172.21.15.6:0/3979151104 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fece402e5f0 con 0x7fece4105480 2024-05-09T04:52:48.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.690+0000 7fece2000700 1 -- 172.21.15.6:0/3979151104 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fecd80a9470 con 0x7fece4105480 2024-05-09T04:52:48.691 INFO:teuthology.orchestra.run.smithi006.stdout:304942678072 2024-05-09T04:52:48.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.693+0000 7fecec600700 1 -- 172.21.15.6:0/3979151104 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7feccc0b2910 msgr2=0x7feccc0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:48.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.693+0000 7fecec600700 1 --2- 172.21.15.6:0/3979151104 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7feccc0b2910 0x7feccc0b4e20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fece41024c0 tx=0x7fecd0009260 comp rx=0 tx=0).stop 2024-05-09T04:52:48.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.694+0000 7fecec600700 1 -- 172.21.15.6:0/3979151104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fece4105480 msgr2=0x7fece419ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:48.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.694+0000 7fecec600700 1 --2- 172.21.15.6:0/3979151104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fece4105480 0x7fece419ae00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fecd800d750 tx=0x7fecd800da60 comp rx=0 tx=0).stop 2024-05-09T04:52:48.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.694+0000 7fecec600700 1 -- 172.21.15.6:0/3979151104 shutdown_connections 2024-05-09T04:52:48.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.694+0000 7fecec600700 1 --2- 172.21.15.6:0/3979151104 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7feccc0b2910 0x7feccc0b4e20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:48.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.694+0000 7fecec600700 1 --2- 172.21.15.6:0/3979151104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fece410e8f0 0x7fece419f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:48.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.694+0000 7fecec600700 1 --2- 172.21.15.6:0/3979151104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fece4105480 0x7fece419ae00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:48.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.694+0000 7fecec600700 1 --2- 172.21.15.6:0/3979151104 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fece4105dd0 0x7fece419b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:48.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.694+0000 7fecec600700 1 -- 172.21.15.6:0/3979151104 >> 172.21.15.6:0/3979151104 conn(0x7fece40fef30 msgr2=0x7fece40ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:48.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.694+0000 7fecec600700 1 -- 172.21.15.6:0/3979151104 shutdown_connections 2024-05-09T04:52:48.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:48.694+0000 7fecec600700 1 -- 172.21.15.6:0/3979151104 wait complete. 2024-05-09T04:52:48.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:48 smithi006 ceph-mon[29835]: pgmap v435: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:52:48.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:48 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/3979151104' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-09T04:52:49.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:48 smithi186 ceph-mon[34668]: pgmap v435: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:52:49.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:48 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/3979151104' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-09T04:52:49.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:48 smithi169 ceph-mon[34075]: pgmap v435: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:52:49.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:48 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/3979151104' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-09T04:52:49.296 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411433 got 77309411439 for osd.2 2024-05-09T04:52:49.296 DEBUG:teuthology.parallel:result is None 2024-05-09T04:52:49.501 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:50.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.107+0000 7fe757600700 1 -- 172.21.15.6:0/2937871840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe750109870 msgr2=0x7fe7501111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:50.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.107+0000 7fe757600700 1 --2- 172.21.15.6:0/2937871840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe750109870 0x7fe7501111d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe74800b0d0 tx=0x7fe74800b3e0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.107+0000 7fe757600700 1 -- 172.21.15.6:0/2937871840 shutdown_connections 2024-05-09T04:52:50.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.107+0000 7fe757600700 1 --2- 172.21.15.6:0/2937871840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe750109870 0x7fe7501111d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.107+0000 7fe757600700 1 --2- 172.21.15.6:0/2937871840 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7501045f0 0x7fe750104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.107+0000 7fe757600700 1 --2- 172.21.15.6:0/2937871840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe750108f20 0x7fe750109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.107+0000 7fe757600700 1 -- 172.21.15.6:0/2937871840 >> 172.21.15.6:0/2937871840 conn(0x7fe750100040 msgr2=0x7fe750102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:50.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.107+0000 7fe757600700 1 -- 172.21.15.6:0/2937871840 shutdown_connections 2024-05-09T04:52:50.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.107+0000 7fe757600700 1 -- 172.21.15.6:0/2937871840 wait complete. 2024-05-09T04:52:50.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.109+0000 7fe757600700 1 Processor -- start 2024-05-09T04:52:50.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.109+0000 7fe757600700 1 -- start start 2024-05-09T04:52:50.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.110+0000 7fe757600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7501045f0 0x7fe750075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:50.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.110+0000 7fe757600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe750108f20 0x7fe750076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:50.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.110+0000 7fe757600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe750109870 0x7fe750076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:50.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.110+0000 7fe757600700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe7501144a0 con 0x7fe750108f20 2024-05-09T04:52:50.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.110+0000 7fe757600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe750114320 con 0x7fe7501045f0 2024-05-09T04:52:50.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.110+0000 7fe757600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe750114620 con 0x7fe750109870 2024-05-09T04:52:50.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.110+0000 7fe74fe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe750108f20 0x7fe750076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:50.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.111+0000 7fe74fe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe750108f20 0x7fe750076000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42414/0 (socket says 172.21.15.6:42414) 2024-05-09T04:52:50.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.111+0000 7fe74fe00700 1 -- 172.21.15.6:0/1545553572 learned_addr learned my addr 172.21.15.6:0/1545553572 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:52:50.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.111+0000 7fe755600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe750109870 0x7fe750076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:50.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.111+0000 7fe754c00700 1 --2- 172.21.15.6:0/1545553572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7501045f0 0x7fe750075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:50.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.111+0000 7fe755600700 1 -- 172.21.15.6:0/1545553572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7501045f0 msgr2=0x7fe750075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:50.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.111+0000 7fe755600700 1 --2- 172.21.15.6:0/1545553572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7501045f0 0x7fe750075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.111+0000 7fe755600700 1 -- 172.21.15.6:0/1545553572 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe750108f20 msgr2=0x7fe750076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:50.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.111+0000 7fe755600700 1 --2- 172.21.15.6:0/1545553572 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe750108f20 0x7fe750076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.111+0000 7fe755600700 1 -- 172.21.15.6:0/1545553572 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7501a9190 con 0x7fe750109870 2024-05-09T04:52:50.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.111+0000 7fe754c00700 1 --2- 172.21.15.6:0/1545553572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7501045f0 0x7fe750075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:50.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.112+0000 7fe755600700 1 --2- 172.21.15.6:0/1545553572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe750109870 0x7fe750076540 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe74800b0d0 tx=0x7fe748007a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:50.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.112+0000 7fe74d600700 1 -- 172.21.15.6:0/1545553572 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74801f070 con 0x7fe750109870 2024-05-09T04:52:50.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.112+0000 7fe757600700 1 -- 172.21.15.6:0/1545553572 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7501a9420 con 0x7fe750109870 2024-05-09T04:52:50.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.112+0000 7fe74d600700 1 -- 172.21.15.6:0/1545553572 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe748004c50 con 0x7fe750109870 2024-05-09T04:52:50.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.112+0000 7fe74d600700 1 -- 172.21.15.6:0/1545553572 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74801b430 con 0x7fe750109870 2024-05-09T04:52:50.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.112+0000 7fe757600700 1 -- 172.21.15.6:0/1545553572 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7501a98d0 con 0x7fe750109870 2024-05-09T04:52:50.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.114+0000 7fe757600700 1 -- 172.21.15.6:0/1545553572 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7500512c0 con 0x7fe750109870 2024-05-09T04:52:50.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.116+0000 7fe74d600700 1 -- 172.21.15.6:0/1545553572 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fe748022070 con 0x7fe750109870 2024-05-09T04:52:50.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.119+0000 7fe74d600700 1 --2- 172.21.15.6:0/1545553572 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe7380b2990 0x7fe7380b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:50.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.119+0000 7fe74d600700 1 -- 172.21.15.6:0/1545553572 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fe74800f020 con 0x7fe750109870 2024-05-09T04:52:50.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.119+0000 7fe754c00700 1 --2- 172.21.15.6:0/1545553572 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe7380b2990 0x7fe7380b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:50.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.120+0000 7fe74d600700 1 -- 172.21.15.6:0/1545553572 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe748029020 con 0x7fe750109870 2024-05-09T04:52:50.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.120+0000 7fe754c00700 1 --2- 172.21.15.6:0/1545553572 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe7380b2990 0x7fe7380b4ea0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe744009680 tx=0x7fe744009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:50.369 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.368+0000 7fe757600700 1 -- 172.21.15.6:0/1545553572 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fe75002e7b0 con 0x7fe750109870 2024-05-09T04:52:50.369 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.369+0000 7fe74d600700 1 -- 172.21.15.6:0/1545553572 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe7480ab290 con 0x7fe750109870 2024-05-09T04:52:50.370 INFO:teuthology.orchestra.run.smithi006.stdout:231928234059 2024-05-09T04:52:50.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.372+0000 7fe757600700 1 -- 172.21.15.6:0/1545553572 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe7380b2990 msgr2=0x7fe7380b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:50.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.372+0000 7fe757600700 1 --2- 172.21.15.6:0/1545553572 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe7380b2990 0x7fe7380b4ea0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe744009680 tx=0x7fe744009260 comp rx=0 tx=0).stop 2024-05-09T04:52:50.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.372+0000 7fe757600700 1 -- 172.21.15.6:0/1545553572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe750109870 msgr2=0x7fe750076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:50.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.372+0000 7fe757600700 1 --2- 172.21.15.6:0/1545553572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe750109870 0x7fe750076540 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe74800b0d0 tx=0x7fe748007a00 comp rx=0 tx=0).stop 2024-05-09T04:52:50.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.372+0000 7fe757600700 1 -- 172.21.15.6:0/1545553572 shutdown_connections 2024-05-09T04:52:50.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.372+0000 7fe757600700 1 --2- 172.21.15.6:0/1545553572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe750109870 0x7fe750076540 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.372+0000 7fe757600700 1 --2- 172.21.15.6:0/1545553572 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe7380b2990 0x7fe7380b4ea0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.372+0000 7fe757600700 1 --2- 172.21.15.6:0/1545553572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7501045f0 0x7fe750075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.372+0000 7fe757600700 1 --2- 172.21.15.6:0/1545553572 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe750108f20 0x7fe750076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.372+0000 7fe757600700 1 -- 172.21.15.6:0/1545553572 >> 172.21.15.6:0/1545553572 conn(0x7fe750100040 msgr2=0x7fe750108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:50.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.373+0000 7fe757600700 1 -- 172.21.15.6:0/1545553572 shutdown_connections 2024-05-09T04:52:50.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:50.373+0000 7fe757600700 1 -- 172.21.15.6:0/1545553572 wait complete. 2024-05-09T04:52:50.920 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:50 smithi006 ceph-mon[29835]: pgmap v436: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-09T04:52:50.920 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:50 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1545553572' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-09T04:52:51.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:50 smithi186 ceph-mon[34668]: pgmap v436: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-09T04:52:51.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:50 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/1545553572' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-09T04:52:51.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:50 smithi169 ceph-mon[34075]: pgmap v436: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-09T04:52:51.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:50 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/1545553572' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-09T04:52:51.512 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234054 got 231928234059 for osd.8 2024-05-09T04:52:51.513 DEBUG:teuthology.parallel:result is None 2024-05-09T04:52:52.658 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:52.701 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678066 got 304942678072 for osd.11 2024-05-09T04:52:52.701 DEBUG:teuthology.parallel:result is None 2024-05-09T04:52:52.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:52 smithi006 ceph-mon[29835]: pgmap v437: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-05-09T04:52:53.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:52 smithi186 ceph-mon[34668]: pgmap v437: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-05-09T04:52:53.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:52 smithi169 ceph-mon[34075]: pgmap v437: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-05-09T04:52:53.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.516+0000 7ff75d000700 1 -- 172.21.15.6:0/3834629025 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff758075d30 msgr2=0x7ff758076140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:53.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.516+0000 7ff75d000700 1 --2- 172.21.15.6:0/3834629025 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff758075d30 0x7ff758076140 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff7480098f0 tx=0x7ff748009c00 comp rx=0 tx=0).stop 2024-05-09T04:52:53.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.516+0000 7ff75d000700 1 -- 172.21.15.6:0/3834629025 shutdown_connections 2024-05-09T04:52:53.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.516+0000 7ff75d000700 1 --2- 172.21.15.6:0/3834629025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff758110ba0 0x7ff758113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:53.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.516+0000 7ff75d000700 1 --2- 172.21.15.6:0/3834629025 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff758075d30 0x7ff758076140 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:53.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.516+0000 7ff75d000700 1 --2- 172.21.15.6:0/3834629025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff758076680 0x7ff758110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:53.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.516+0000 7ff75d000700 1 -- 172.21.15.6:0/3834629025 >> 172.21.15.6:0/3834629025 conn(0x7ff758100040 msgr2=0x7ff758102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:53.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.516+0000 7ff75d000700 1 -- 172.21.15.6:0/3834629025 shutdown_connections 2024-05-09T04:52:53.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.517+0000 7ff75d000700 1 -- 172.21.15.6:0/3834629025 wait complete. 2024-05-09T04:52:53.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.518+0000 7ff75d000700 1 Processor -- start 2024-05-09T04:52:53.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.518+0000 7ff75d000700 1 -- start start 2024-05-09T04:52:53.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.519+0000 7ff75d000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff758075d30 0x7ff7581042c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:53.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.519+0000 7ff75d000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff758076680 0x7ff758104800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:53.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.519+0000 7ff75d000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff758110ba0 0x7ff758104d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:53.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.519+0000 7ff75d000700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff7581166e0 con 0x7ff758076680 2024-05-09T04:52:53.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.519+0000 7ff75d000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff758116560 con 0x7ff758110ba0 2024-05-09T04:52:53.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.519+0000 7ff75d000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff758116860 con 0x7ff758075d30 2024-05-09T04:52:53.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.519+0000 7ff755400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff758076680 0x7ff758104800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:53.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.519+0000 7ff755400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff758076680 0x7ff758104800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42430/0 (socket says 172.21.15.6:42430) 2024-05-09T04:52:53.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.519+0000 7ff755400700 1 -- 172.21.15.6:0/1287354803 learned_addr learned my addr 172.21.15.6:0/1287354803 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:52:53.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.519+0000 7ff756800700 1 --2- 172.21.15.6:0/1287354803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff758110ba0 0x7ff758104d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:53.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.519+0000 7ff755e00700 1 --2- 172.21.15.6:0/1287354803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff758075d30 0x7ff7581042c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:53.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.520+0000 7ff755400700 1 -- 172.21.15.6:0/1287354803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff758075d30 msgr2=0x7ff7581042c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:53.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.520+0000 7ff755400700 1 --2- 172.21.15.6:0/1287354803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff758075d30 0x7ff7581042c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:53.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.520+0000 7ff755400700 1 -- 172.21.15.6:0/1287354803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff758110ba0 msgr2=0x7ff758104d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:53.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.520+0000 7ff755400700 1 --2- 172.21.15.6:0/1287354803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff758110ba0 0x7ff758104d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:53.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.520+0000 7ff755400700 1 -- 172.21.15.6:0/1287354803 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7581ad650 con 0x7ff758076680 2024-05-09T04:52:53.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.520+0000 7ff756800700 1 --2- 172.21.15.6:0/1287354803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff758110ba0 0x7ff758104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:53.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.520+0000 7ff755400700 1 --2- 172.21.15.6:0/1287354803 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff758076680 0x7ff758104800 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff74400eb30 tx=0x7ff74400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:53.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.520+0000 7ff74ea00700 1 -- 172.21.15.6:0/1287354803 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74400cbd0 con 0x7ff758076680 2024-05-09T04:52:53.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.521+0000 7ff74ea00700 1 -- 172.21.15.6:0/1287354803 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7440044d0 con 0x7ff758076680 2024-05-09T04:52:53.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.521+0000 7ff75d000700 1 -- 172.21.15.6:0/1287354803 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7581ad8e0 con 0x7ff758076680 2024-05-09T04:52:53.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.521+0000 7ff74ea00700 1 -- 172.21.15.6:0/1287354803 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff744003e30 con 0x7ff758076680 2024-05-09T04:52:53.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.521+0000 7ff75d000700 1 -- 172.21.15.6:0/1287354803 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7581ade70 con 0x7ff758076680 2024-05-09T04:52:53.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.522+0000 7ff75d000700 1 -- 172.21.15.6:0/1287354803 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7580772a0 con 0x7ff758076680 2024-05-09T04:52:53.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.525+0000 7ff74ea00700 1 -- 172.21.15.6:0/1287354803 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7ff744004000 con 0x7ff758076680 2024-05-09T04:52:53.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.527+0000 7ff74ea00700 1 --2- 172.21.15.6:0/1287354803 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff73c0b2990 0x7ff73c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:53.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.527+0000 7ff74ea00700 1 -- 172.21.15.6:0/1287354803 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7ff7440e2a70 con 0x7ff758076680 2024-05-09T04:52:53.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.527+0000 7ff755e00700 1 --2- 172.21.15.6:0/1287354803 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff73c0b2990 0x7ff73c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:53.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.528+0000 7ff755e00700 1 --2- 172.21.15.6:0/1287354803 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff73c0b2990 0x7ff73c0b4ea0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff7480098c0 tx=0x7ff748019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:53.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.532+0000 7ff74ea00700 1 -- 172.21.15.6:0/1287354803 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff7440042f0 con 0x7ff758076680 2024-05-09T04:52:53.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.783+0000 7ff75d000700 1 -- 172.21.15.6:0/1287354803 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7ff758105820 con 0x7ff758076680 2024-05-09T04:52:53.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.785+0000 7ff74ea00700 1 -- 172.21.15.6:0/1287354803 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff758105820 con 0x7ff758076680 2024-05-09T04:52:53.786 INFO:teuthology.orchestra.run.smithi006.stdout:184683593814 2024-05-09T04:52:53.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.787+0000 7ff75d000700 1 -- 172.21.15.6:0/1287354803 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff73c0b2990 msgr2=0x7ff73c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:53.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.787+0000 7ff75d000700 1 --2- 172.21.15.6:0/1287354803 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff73c0b2990 0x7ff73c0b4ea0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff7480098c0 tx=0x7ff748019040 comp rx=0 tx=0).stop 2024-05-09T04:52:53.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.787+0000 7ff75d000700 1 -- 172.21.15.6:0/1287354803 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff758076680 msgr2=0x7ff758104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:53.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.787+0000 7ff75d000700 1 --2- 172.21.15.6:0/1287354803 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff758076680 0x7ff758104800 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff74400eb30 tx=0x7ff74400ee40 comp rx=0 tx=0).stop 2024-05-09T04:52:53.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.788+0000 7ff75d000700 1 -- 172.21.15.6:0/1287354803 shutdown_connections 2024-05-09T04:52:53.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.788+0000 7ff75d000700 1 --2- 172.21.15.6:0/1287354803 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff73c0b2990 0x7ff73c0b4ea0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:53.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.788+0000 7ff75d000700 1 --2- 172.21.15.6:0/1287354803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff758110ba0 0x7ff758104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:53.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.788+0000 7ff75d000700 1 --2- 172.21.15.6:0/1287354803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff758075d30 0x7ff7581042c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:53.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.788+0000 7ff75d000700 1 --2- 172.21.15.6:0/1287354803 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff758076680 0x7ff758104800 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:53.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.788+0000 7ff75d000700 1 -- 172.21.15.6:0/1287354803 >> 172.21.15.6:0/1287354803 conn(0x7ff758100040 msgr2=0x7ff758100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:53.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.788+0000 7ff75d000700 1 -- 172.21.15.6:0/1287354803 shutdown_connections 2024-05-09T04:52:53.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:53.788+0000 7ff75d000700 1 -- 172.21.15.6:0/1287354803 wait complete. 2024-05-09T04:52:54.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:53 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/1287354803' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-09T04:52:54.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:53 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1287354803' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-09T04:52:54.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:53 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/1287354803' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-09T04:52:54.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.759+0000 7f092ee00700 1 -- 172.21.15.6:0/1515668941 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0928106ec0 msgr2=0x7f0928113350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:54.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.759+0000 7f092ee00700 1 --2- 172.21.15.6:0/1515668941 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0928106ec0 0x7f0928113350 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f092000b0a0 tx=0x7f092000b3b0 comp rx=0 tx=0).stop 2024-05-09T04:52:54.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.759+0000 7f092ee00700 1 -- 172.21.15.6:0/1515668941 shutdown_connections 2024-05-09T04:52:54.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.759+0000 7f092ee00700 1 --2- 172.21.15.6:0/1515668941 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0928106ec0 0x7f0928113350 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:54.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.759+0000 7f092ee00700 1 --2- 172.21.15.6:0/1515668941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f092810c5b0 0x7f092810c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:54.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.759+0000 7f092ee00700 1 --2- 172.21.15.6:0/1515668941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0928106570 0x7f0928106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:54.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.759+0000 7f092ee00700 1 -- 172.21.15.6:0/1515668941 >> 172.21.15.6:0/1515668941 conn(0x7f0928102050 msgr2=0x7f09281044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:54.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.759+0000 7f092ee00700 1 -- 172.21.15.6:0/1515668941 shutdown_connections 2024-05-09T04:52:54.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.759+0000 7f092ee00700 1 -- 172.21.15.6:0/1515668941 wait complete. 2024-05-09T04:52:54.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.761+0000 7f092ee00700 1 Processor -- start 2024-05-09T04:52:54.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.761+0000 7f092ee00700 1 -- start start 2024-05-09T04:52:54.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.761+0000 7f092ee00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0928106570 0x7f092819d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:54.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.761+0000 7f092ee00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0928106ec0 0x7f092819d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:54.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.761+0000 7f092ee00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f092810c5b0 0x7f09281a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:54.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.761+0000 7f092ee00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0928116730 con 0x7f0928106ec0 2024-05-09T04:52:54.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.761+0000 7f092ee00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f09281165b0 con 0x7f092810c5b0 2024-05-09T04:52:54.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.761+0000 7f092ee00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f09281168b0 con 0x7f0928106570 2024-05-09T04:52:54.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.762+0000 7f0927e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0928106570 0x7f092819d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:54.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.762+0000 7f0927400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0928106ec0 0x7f092819d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:54.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.762+0000 7f0927e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0928106570 0x7f092819d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:46156/0 (socket says 172.21.15.6:46156) 2024-05-09T04:52:54.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.762+0000 7f0927e00700 1 -- 172.21.15.6:0/2038512999 learned_addr learned my addr 172.21.15.6:0/2038512999 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:52:54.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.762+0000 7f092ce00700 1 --2- 172.21.15.6:0/2038512999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f092810c5b0 0x7f09281a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:54.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.762+0000 7f0927e00700 1 -- 172.21.15.6:0/2038512999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f092810c5b0 msgr2=0x7f09281a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:54.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.762+0000 7f0927e00700 1 --2- 172.21.15.6:0/2038512999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f092810c5b0 0x7f09281a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:54.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.762+0000 7f0927e00700 1 -- 172.21.15.6:0/2038512999 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0928106ec0 msgr2=0x7f092819d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:54.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.762+0000 7f0927e00700 1 --2- 172.21.15.6:0/2038512999 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0928106ec0 0x7f092819d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:54.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.762+0000 7f0927e00700 1 -- 172.21.15.6:0/2038512999 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09281a1ed0 con 0x7f0928106570 2024-05-09T04:52:54.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.763+0000 7f0927e00700 1 --2- 172.21.15.6:0/2038512999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0928106570 0x7f092819d000 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f091800e970 tx=0x7f091800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:54.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.763+0000 7f0924c00700 1 -- 172.21.15.6:0/2038512999 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091800cbd0 con 0x7f0928106570 2024-05-09T04:52:54.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.763+0000 7f092ee00700 1 -- 172.21.15.6:0/2038512999 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09281a21c0 con 0x7f0928106570 2024-05-09T04:52:54.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.763+0000 7f092ee00700 1 -- 172.21.15.6:0/2038512999 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09281a9ae0 con 0x7f0928106570 2024-05-09T04:52:54.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.763+0000 7f0924c00700 1 -- 172.21.15.6:0/2038512999 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f091800cd70 con 0x7f0928106570 2024-05-09T04:52:54.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.763+0000 7f0924c00700 1 -- 172.21.15.6:0/2038512999 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09180055d0 con 0x7f0928106570 2024-05-09T04:52:54.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.765+0000 7f092ee00700 1 -- 172.21.15.6:0/2038512999 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0908005330 con 0x7f0928106570 2024-05-09T04:52:54.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.768+0000 7f0924c00700 1 -- 172.21.15.6:0/2038512999 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f0918047990 con 0x7f0928106570 2024-05-09T04:52:54.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.770+0000 7f0924c00700 1 --2- 172.21.15.6:0/2038512999 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f090c0b2990 0x7f090c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:54.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.770+0000 7f0924c00700 1 -- 172.21.15.6:0/2038512999 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f09180e2830 con 0x7f0928106570 2024-05-09T04:52:54.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.771+0000 7f0927400700 1 --2- 172.21.15.6:0/2038512999 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f090c0b2990 0x7f090c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:54.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.772+0000 7f0927400700 1 --2- 172.21.15.6:0/2038512999 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f090c0b2990 0x7f090c0b4ea0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f09281054b0 tx=0x7f0914009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:54.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:54.772+0000 7f0924c00700 1 -- 172.21.15.6:0/2038512999 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f09180aa010 con 0x7f0928106570 2024-05-09T04:52:55.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:55.020+0000 7f092ee00700 1 -- 172.21.15.6:0/2038512999 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f09080058f0 con 0x7f0928106570 2024-05-09T04:52:55.022 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:54 smithi006 ceph-mon[29835]: pgmap v438: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:52:55.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:55.021+0000 7f0924c00700 1 -- 172.21.15.6:0/2038512999 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f09180a9910 con 0x7f0928106570 2024-05-09T04:52:55.023 INFO:teuthology.orchestra.run.smithi006.stdout:137438953572 2024-05-09T04:52:55.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:55.025+0000 7f092ee00700 1 -- 172.21.15.6:0/2038512999 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f090c0b2990 msgr2=0x7f090c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:55.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:55.025+0000 7f092ee00700 1 --2- 172.21.15.6:0/2038512999 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f090c0b2990 0x7f090c0b4ea0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f09281054b0 tx=0x7f0914009260 comp rx=0 tx=0).stop 2024-05-09T04:52:55.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:55.025+0000 7f092ee00700 1 -- 172.21.15.6:0/2038512999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0928106570 msgr2=0x7f092819d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:55.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:55.025+0000 7f092ee00700 1 --2- 172.21.15.6:0/2038512999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0928106570 0x7f092819d000 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f091800e970 tx=0x7f091800ec80 comp rx=0 tx=0).stop 2024-05-09T04:52:55.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:55.025+0000 7f092ee00700 1 -- 172.21.15.6:0/2038512999 shutdown_connections 2024-05-09T04:52:55.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:55.025+0000 7f092ee00700 1 --2- 172.21.15.6:0/2038512999 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f090c0b2990 0x7f090c0b4ea0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:55.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:55.025+0000 7f092ee00700 1 --2- 172.21.15.6:0/2038512999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f092810c5b0 0x7f09281a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:55.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:55.025+0000 7f092ee00700 1 --2- 172.21.15.6:0/2038512999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0928106570 0x7f092819d000 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:55.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:55.025+0000 7f092ee00700 1 --2- 172.21.15.6:0/2038512999 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0928106ec0 0x7f092819d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:55.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:55.025+0000 7f092ee00700 1 -- 172.21.15.6:0/2038512999 >> 172.21.15.6:0/2038512999 conn(0x7f0928102050 msgr2=0x7f0928103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:55.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:55.026+0000 7f092ee00700 1 -- 172.21.15.6:0/2038512999 shutdown_connections 2024-05-09T04:52:55.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:55.026+0000 7f092ee00700 1 -- 172.21.15.6:0/2038512999 wait complete. 2024-05-09T04:52:55.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:54 smithi169 ceph-mon[34075]: pgmap v438: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:52:55.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:54 smithi186 ceph-mon[34668]: pgmap v438: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:52:55.600 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593810 got 184683593814 for osd.6 2024-05-09T04:52:55.600 DEBUG:teuthology.parallel:result is None 2024-05-09T04:52:56.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:55 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/2038512999' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-09T04:52:56.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:55 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/2038512999' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-09T04:52:56.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:55 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/2038512999' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-09T04:52:57.042 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:56 smithi006 ceph-mon[29835]: pgmap v439: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:52:57.043 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:56 smithi006 ceph-mon[29835]: mgrmap e18: x(active, since 12m), standbys: z, y 2024-05-09T04:52:57.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:56 smithi186 ceph-mon[34668]: pgmap v439: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:52:57.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:56 smithi186 ceph-mon[34668]: mgrmap e18: x(active, since 12m), standbys: z, y 2024-05-09T04:52:57.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:56 smithi169 ceph-mon[34075]: pgmap v439: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:52:57.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:56 smithi169 ceph-mon[34075]: mgrmap e18: x(active, since 12m), standbys: z, y 2024-05-09T04:52:58.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.030+0000 7fb133200700 1 -- 172.21.15.6:0/4254454853 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb13410b110 msgr2=0x7fb134117620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:58.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.030+0000 7fb133200700 1 --2- 172.21.15.6:0/4254454853 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb13410b110 0x7fb134117620 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00b0a0 tx=0x7fb12c00b3b0 comp rx=0 tx=0).stop 2024-05-09T04:52:58.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.031+0000 7fb133200700 1 -- 172.21.15.6:0/4254454853 shutdown_connections 2024-05-09T04:52:58.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.031+0000 7fb133200700 1 --2- 172.21.15.6:0/4254454853 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb13410b110 0x7fb134117620 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:58.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.031+0000 7fb133200700 1 --2- 172.21.15.6:0/4254454853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb13410f6c0 0x7fb13410fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:58.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.031+0000 7fb133200700 1 --2- 172.21.15.6:0/4254454853 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb13410a7c0 0x7fb13410abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:58.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.031+0000 7fb133200700 1 -- 172.21.15.6:0/4254454853 >> 172.21.15.6:0/4254454853 conn(0x7fb134076b00 msgr2=0x7fb134076f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:58.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.031+0000 7fb133200700 1 -- 172.21.15.6:0/4254454853 shutdown_connections 2024-05-09T04:52:58.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.031+0000 7fb133200700 1 -- 172.21.15.6:0/4254454853 wait complete. 2024-05-09T04:52:58.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.032+0000 7fb133200700 1 Processor -- start 2024-05-09T04:52:58.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.032+0000 7fb133200700 1 -- start start 2024-05-09T04:52:58.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.033+0000 7fb133200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb13410a7c0 0x7fb1341a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:58.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.033+0000 7fb133200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb13410b110 0x7fb1341a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:58.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.033+0000 7fb133200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb13410f6c0 0x7fb1341a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:58.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.033+0000 7fb133200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb13411a8b0 con 0x7fb13410a7c0 2024-05-09T04:52:58.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.033+0000 7fb133200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb13411a730 con 0x7fb13410f6c0 2024-05-09T04:52:58.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.033+0000 7fb133200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb13411aa30 con 0x7fb13410b110 2024-05-09T04:52:58.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.033+0000 7fb131e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb13410a7c0 0x7fb1341a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:58.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.033+0000 7fb131400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb13410b110 0x7fb1341a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:58.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.033+0000 7fb131e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb13410a7c0 0x7fb1341a1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56816/0 (socket says 172.21.15.6:56816) 2024-05-09T04:52:58.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.033+0000 7fb132800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb13410f6c0 0x7fb1341a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:58.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.033+0000 7fb131e00700 1 -- 172.21.15.6:0/2024790261 learned_addr learned my addr 172.21.15.6:0/2024790261 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:52:58.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.034+0000 7fb132800700 1 -- 172.21.15.6:0/2024790261 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb13410b110 msgr2=0x7fb1341a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:58.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.034+0000 7fb132800700 1 --2- 172.21.15.6:0/2024790261 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb13410b110 0x7fb1341a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:58.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.034+0000 7fb132800700 1 -- 172.21.15.6:0/2024790261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb13410a7c0 msgr2=0x7fb1341a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:58.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.034+0000 7fb132800700 1 --2- 172.21.15.6:0/2024790261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb13410a7c0 0x7fb1341a1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:58.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.034+0000 7fb132800700 1 -- 172.21.15.6:0/2024790261 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1341a6030 con 0x7fb13410f6c0 2024-05-09T04:52:58.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.034+0000 7fb131e00700 1 --2- 172.21.15.6:0/2024790261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb13410a7c0 0x7fb1341a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:58.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.034+0000 7fb132800700 1 --2- 172.21.15.6:0/2024790261 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb13410f6c0 0x7fb1341a5990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00b070 tx=0x7fb12c004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:58.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.034+0000 7fb12aa00700 1 -- 172.21.15.6:0/2024790261 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12c021070 con 0x7fb13410f6c0 2024-05-09T04:52:58.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.034+0000 7fb133200700 1 -- 172.21.15.6:0/2024790261 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1341a62c0 con 0x7fb13410f6c0 2024-05-09T04:52:58.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.035+0000 7fb12aa00700 1 -- 172.21.15.6:0/2024790261 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb12c007e30 con 0x7fb13410f6c0 2024-05-09T04:52:58.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.035+0000 7fb12aa00700 1 -- 172.21.15.6:0/2024790261 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12c0045d0 con 0x7fb13410f6c0 2024-05-09T04:52:58.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.035+0000 7fb133200700 1 -- 172.21.15.6:0/2024790261 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1341adc40 con 0x7fb13410f6c0 2024-05-09T04:52:58.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.036+0000 7fb133200700 1 -- 172.21.15.6:0/2024790261 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb13410beb0 con 0x7fb13410f6c0 2024-05-09T04:52:58.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.038+0000 7fb12aa00700 1 -- 172.21.15.6:0/2024790261 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fb12c003c00 con 0x7fb13410f6c0 2024-05-09T04:52:58.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.040+0000 7fb12aa00700 1 --2- 172.21.15.6:0/2024790261 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fb1180b2880 0x7fb1180b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:58.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.040+0000 7fb12aa00700 1 -- 172.21.15.6:0/2024790261 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fb12c00f020 con 0x7fb13410f6c0 2024-05-09T04:52:58.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.040+0000 7fb131e00700 1 --2- 172.21.15.6:0/2024790261 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fb1180b2880 0x7fb1180b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:58.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.041+0000 7fb131e00700 1 --2- 172.21.15.6:0/2024790261 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fb1180b2880 0x7fb1180b4d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb124011550 tx=0x7fb124009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:58.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.042+0000 7fb12aa00700 1 -- 172.21.15.6:0/2024790261 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb12c02a020 con 0x7fb13410f6c0 2024-05-09T04:52:58.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.287+0000 7fb133200700 1 -- 172.21.15.6:0/2024790261 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fb13402e5f0 con 0x7fb13410f6c0 2024-05-09T04:52:58.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.288+0000 7fb12aa00700 1 -- 172.21.15.6:0/2024790261 <== mon.2 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb12c0ab5f0 con 0x7fb13410f6c0 2024-05-09T04:52:58.288 INFO:teuthology.orchestra.run.smithi006.stdout:257698037828 2024-05-09T04:52:58.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.291+0000 7fb133200700 1 -- 172.21.15.6:0/2024790261 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fb1180b2880 msgr2=0x7fb1180b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:58.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.291+0000 7fb133200700 1 --2- 172.21.15.6:0/2024790261 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fb1180b2880 0x7fb1180b4d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb124011550 tx=0x7fb124009260 comp rx=0 tx=0).stop 2024-05-09T04:52:58.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.291+0000 7fb133200700 1 -- 172.21.15.6:0/2024790261 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb13410f6c0 msgr2=0x7fb1341a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:58.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.291+0000 7fb133200700 1 --2- 172.21.15.6:0/2024790261 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb13410f6c0 0x7fb1341a5990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00b070 tx=0x7fb12c004d00 comp rx=0 tx=0).stop 2024-05-09T04:52:58.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.291+0000 7fb133200700 1 -- 172.21.15.6:0/2024790261 shutdown_connections 2024-05-09T04:52:58.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.291+0000 7fb133200700 1 --2- 172.21.15.6:0/2024790261 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fb1180b2880 0x7fb1180b4d90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:58.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.291+0000 7fb133200700 1 --2- 172.21.15.6:0/2024790261 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb13410f6c0 0x7fb1341a5990 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:58.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.291+0000 7fb133200700 1 --2- 172.21.15.6:0/2024790261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb13410a7c0 0x7fb1341a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:58.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.291+0000 7fb133200700 1 --2- 172.21.15.6:0/2024790261 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb13410b110 0x7fb1341a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:58.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.291+0000 7fb133200700 1 -- 172.21.15.6:0/2024790261 >> 172.21.15.6:0/2024790261 conn(0x7fb134076b00 msgr2=0x7fb1341087c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:58.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.292+0000 7fb133200700 1 -- 172.21.15.6:0/2024790261 shutdown_connections 2024-05-09T04:52:58.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:58.292+0000 7fb133200700 1 -- 172.21.15.6:0/2024790261 wait complete. 2024-05-09T04:52:58.471 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953564 got 137438953572 for osd.4 2024-05-09T04:52:58.472 DEBUG:teuthology.parallel:result is None 2024-05-09T04:52:59.155 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:58 smithi006 ceph-mon[29835]: pgmap v440: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-09T04:52:59.155 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:58 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/2024790261' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-09T04:52:59.310 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037821 got 257698037828 for osd.9 2024-05-09T04:52:59.310 DEBUG:teuthology.parallel:result is None 2024-05-09T04:52:59.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:58 smithi186 ceph-mon[34668]: pgmap v440: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-09T04:52:59.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:58 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/2024790261' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-09T04:52:59.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:58 smithi169 ceph-mon[34075]: pgmap v440: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-09T04:52:59.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:58 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/2024790261' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-09T04:52:59.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.617+0000 7f60bfc00700 1 -- 172.21.15.6:0/1288546975 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60b8105dd0 msgr2=0x7f60b810e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:59.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.617+0000 7f60bfc00700 1 --2- 172.21.15.6:0/1288546975 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60b8105dd0 0x7f60b810e320 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f60a4009990 tx=0x7f60a4009ca0 comp rx=0 tx=0).stop 2024-05-09T04:52:59.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.618+0000 7f60bfc00700 1 -- 172.21.15.6:0/1288546975 shutdown_connections 2024-05-09T04:52:59.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.618+0000 7f60bfc00700 1 --2- 172.21.15.6:0/1288546975 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60b810e8f0 0x7f60b8110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:59.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.618+0000 7f60bfc00700 1 --2- 172.21.15.6:0/1288546975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60b8105480 0x7f60b8105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:59.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.618+0000 7f60bfc00700 1 --2- 172.21.15.6:0/1288546975 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60b8105dd0 0x7f60b810e320 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:59.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.618+0000 7f60bfc00700 1 -- 172.21.15.6:0/1288546975 >> 172.21.15.6:0/1288546975 conn(0x7f60b80fef30 msgr2=0x7f60b8101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:59.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.618+0000 7f60bfc00700 1 -- 172.21.15.6:0/1288546975 shutdown_connections 2024-05-09T04:52:59.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.618+0000 7f60bfc00700 1 -- 172.21.15.6:0/1288546975 wait complete. 2024-05-09T04:52:59.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.619+0000 7f60bfc00700 1 Processor -- start 2024-05-09T04:52:59.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.619+0000 7f60bfc00700 1 -- start start 2024-05-09T04:52:59.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.620+0000 7f60bfc00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60b8105480 0x7f60b819d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:59.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.620+0000 7f60bfc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60b8105dd0 0x7f60b819d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:59.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.620+0000 7f60bfc00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60b810e8f0 0x7f60b81a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:59.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.620+0000 7f60bfc00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f60b8114450 con 0x7f60b8105480 2024-05-09T04:52:59.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.620+0000 7f60bfc00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f60b81142d0 con 0x7f60b810e8f0 2024-05-09T04:52:59.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.620+0000 7f60bfc00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f60b81145d0 con 0x7f60b8105dd0 2024-05-09T04:52:59.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.620+0000 7f60bd200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60b8105480 0x7f60b819d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:59.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.620+0000 7f60b7e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60b8105dd0 0x7f60b819d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:59.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.620+0000 7f60b7e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60b8105dd0 0x7f60b819d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:60278/0 (socket says 172.21.15.6:60278) 2024-05-09T04:52:59.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.620+0000 7f60b7e00700 1 -- 172.21.15.6:0/1497010385 learned_addr learned my addr 172.21.15.6:0/1497010385 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:52:59.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.620+0000 7f60bd200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60b8105480 0x7f60b819d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56836/0 (socket says 172.21.15.6:56836) 2024-05-09T04:52:59.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.620+0000 7f60bdc00700 1 --2- 172.21.15.6:0/1497010385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60b810e8f0 0x7f60b81a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:59.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.621+0000 7f60b7e00700 1 -- 172.21.15.6:0/1497010385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60b810e8f0 msgr2=0x7f60b81a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:59.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.621+0000 7f60b7e00700 1 --2- 172.21.15.6:0/1497010385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60b810e8f0 0x7f60b81a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:59.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.621+0000 7f60b7e00700 1 -- 172.21.15.6:0/1497010385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60b8105480 msgr2=0x7f60b819d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:59.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.621+0000 7f60b7e00700 1 --2- 172.21.15.6:0/1497010385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60b8105480 0x7f60b819d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:59.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.621+0000 7f60b7e00700 1 -- 172.21.15.6:0/1497010385 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b81a1ed0 con 0x7f60b8105dd0 2024-05-09T04:52:59.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.621+0000 7f60bdc00700 1 --2- 172.21.15.6:0/1497010385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60b810e8f0 0x7f60b81a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:59.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.621+0000 7f60b7e00700 1 --2- 172.21.15.6:0/1497010385 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60b8105dd0 0x7f60b819d5a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f60a40057c0 tx=0x7f60a4009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:59.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.621+0000 7f60b5600700 1 -- 172.21.15.6:0/1497010385 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4026070 con 0x7f60b8105dd0 2024-05-09T04:52:59.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.622+0000 7f60b5600700 1 -- 172.21.15.6:0/1497010385 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60a4021410 con 0x7f60b8105dd0 2024-05-09T04:52:59.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.622+0000 7f60b5600700 1 -- 172.21.15.6:0/1497010385 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4017650 con 0x7f60b8105dd0 2024-05-09T04:52:59.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.622+0000 7f60bfc00700 1 -- 172.21.15.6:0/1497010385 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b81a2160 con 0x7f60b8105dd0 2024-05-09T04:52:59.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.622+0000 7f60bfc00700 1 -- 172.21.15.6:0/1497010385 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b81a2640 con 0x7f60b8105dd0 2024-05-09T04:52:59.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.623+0000 7f60bfc00700 1 -- 172.21.15.6:0/1497010385 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60b80512c0 con 0x7f60b8105dd0 2024-05-09T04:52:59.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.625+0000 7f60b5600700 1 -- 172.21.15.6:0/1497010385 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f60a4029070 con 0x7f60b8105dd0 2024-05-09T04:52:59.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.627+0000 7f60b5600700 1 --2- 172.21.15.6:0/1497010385 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f60a00b2880 0x7f60a00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:59.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.627+0000 7f60b5600700 1 -- 172.21.15.6:0/1497010385 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f60a40e3560 con 0x7f60b8105dd0 2024-05-09T04:52:59.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.627+0000 7f60bd200700 1 --2- 172.21.15.6:0/1497010385 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f60a00b2880 0x7f60a00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:59.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.628+0000 7f60bd200700 1 --2- 172.21.15.6:0/1497010385 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f60a00b2880 0x7f60a00b4d90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f60ac007be0 tx=0x7f60ac0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:59.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.629+0000 7f60b5600700 1 -- 172.21.15.6:0/1497010385 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f60a405edf0 con 0x7f60b8105dd0 2024-05-09T04:52:59.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.875+0000 7f60bfc00700 1 -- 172.21.15.6:0/1497010385 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f60b81a9b40 con 0x7f60b8105dd0 2024-05-09T04:52:59.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.876+0000 7f60b5600700 1 -- 172.21.15.6:0/1497010385 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f60a4013070 con 0x7f60b8105dd0 2024-05-09T04:52:59.877 INFO:teuthology.orchestra.run.smithi006.stdout:279172874304 2024-05-09T04:52:59.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.879+0000 7f60bfc00700 1 -- 172.21.15.6:0/1497010385 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f60a00b2880 msgr2=0x7f60a00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:59.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.879+0000 7f60bfc00700 1 --2- 172.21.15.6:0/1497010385 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f60a00b2880 0x7f60a00b4d90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f60ac007be0 tx=0x7f60ac0073d0 comp rx=0 tx=0).stop 2024-05-09T04:52:59.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.879+0000 7f60bfc00700 1 -- 172.21.15.6:0/1497010385 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60b8105dd0 msgr2=0x7f60b819d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:59.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.879+0000 7f60bfc00700 1 --2- 172.21.15.6:0/1497010385 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60b8105dd0 0x7f60b819d5a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f60a40057c0 tx=0x7f60a4009580 comp rx=0 tx=0).stop 2024-05-09T04:52:59.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.879+0000 7f60bfc00700 1 -- 172.21.15.6:0/1497010385 shutdown_connections 2024-05-09T04:52:59.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.879+0000 7f60bfc00700 1 --2- 172.21.15.6:0/1497010385 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f60a00b2880 0x7f60a00b4d90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:59.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.879+0000 7f60bfc00700 1 --2- 172.21.15.6:0/1497010385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60b810e8f0 0x7f60b81a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:59.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.879+0000 7f60bfc00700 1 --2- 172.21.15.6:0/1497010385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60b8105480 0x7f60b819d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:59.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.879+0000 7f60bfc00700 1 --2- 172.21.15.6:0/1497010385 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60b8105dd0 0x7f60b819d5a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:59.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.879+0000 7f60bfc00700 1 -- 172.21.15.6:0/1497010385 >> 172.21.15.6:0/1497010385 conn(0x7f60b80fef30 msgr2=0x7f60b8100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:59.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.880+0000 7f60bfc00700 1 -- 172.21.15.6:0/1497010385 shutdown_connections 2024-05-09T04:52:59.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:52:59.880+0000 7f60bfc00700 1 -- 172.21.15.6:0/1497010385 wait complete. 2024-05-09T04:53:00.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:52:59 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1497010385' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-09T04:53:00.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:52:59 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/1497010385' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-09T04:53:00.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:52:59 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/1497010385' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-09T04:53:00.734 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874300 got 279172874304 for osd.10 2024-05-09T04:53:00.734 DEBUG:teuthology.parallel:result is None 2024-05-09T04:53:00.735 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-09T04:53:00.735 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T04:53:00.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:00 smithi006 ceph-mon[29835]: pgmap v441: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T04:53:01.214 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:53:01.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:00 smithi186 ceph-mon[34668]: pgmap v441: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T04:53:01.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:00 smithi169 ceph-mon[34075]: pgmap v441: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T04:53:03.103 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:02 smithi006 ceph-mon[29835]: pgmap v442: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-09T04:53:03.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:02 smithi186 ceph-mon[34668]: pgmap v442: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-09T04:53:03.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:02 smithi169 ceph-mon[34075]: pgmap v442: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-09T04:53:03.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.486+0000 7f4e47e00700 1 -- 172.21.15.6:0/72910743 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e48106620 msgr2=0x7f4e48106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:03.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.486+0000 7f4e47e00700 1 --2- 172.21.15.6:0/72910743 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e48106620 0x7f4e48106a30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4e380099c0 tx=0x7f4e38009cd0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.486+0000 7f4e47e00700 1 -- 172.21.15.6:0/72910743 shutdown_connections 2024-05-09T04:53:03.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.486+0000 7f4e47e00700 1 --2- 172.21.15.6:0/72910743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e4810b8a0 0x7f4e48113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.486+0000 7f4e47e00700 1 --2- 172.21.15.6:0/72910743 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e48106620 0x7f4e48106a30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.486+0000 7f4e47e00700 1 --2- 172.21.15.6:0/72910743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e4810af50 0x7f4e4810b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.486+0000 7f4e47e00700 1 -- 172.21.15.6:0/72910743 >> 172.21.15.6:0/72910743 conn(0x7f4e48102030 msgr2=0x7f4e48104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:03.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.487+0000 7f4e47e00700 1 -- 172.21.15.6:0/72910743 shutdown_connections 2024-05-09T04:53:03.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.487+0000 7f4e47e00700 1 -- 172.21.15.6:0/72910743 wait complete. 2024-05-09T04:53:03.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.488+0000 7f4e47e00700 1 Processor -- start 2024-05-09T04:53:03.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.488+0000 7f4e47e00700 1 -- start start 2024-05-09T04:53:03.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.489+0000 7f4e47e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e48106620 0x7f4e4819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.489+0000 7f4e47e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e4810af50 0x7f4e4819d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.489+0000 7f4e47e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e4810b8a0 0x7f4e481a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.489+0000 7f4e47e00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4e48116520 con 0x7f4e4810af50 2024-05-09T04:53:03.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.489+0000 7f4e47e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4e481163a0 con 0x7f4e4810b8a0 2024-05-09T04:53:03.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.489+0000 7f4e47e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4e481166a0 con 0x7f4e48106620 2024-05-09T04:53:03.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.489+0000 7f4e46000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e4810af50 0x7f4e4819d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.490+0000 7f4e46000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e4810af50 0x7f4e4819d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56844/0 (socket says 172.21.15.6:56844) 2024-05-09T04:53:03.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.490+0000 7f4e46a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e48106620 0x7f4e4819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.490+0000 7f4e46000700 1 -- 172.21.15.6:0/1980758023 learned_addr learned my addr 172.21.15.6:0/1980758023 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:03.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.490+0000 7f4e47400700 1 --2- 172.21.15.6:0/1980758023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e4810b8a0 0x7f4e481a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.490+0000 7f4e46000700 1 -- 172.21.15.6:0/1980758023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e48106620 msgr2=0x7f4e4819ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:03.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.490+0000 7f4e46000700 1 --2- 172.21.15.6:0/1980758023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e48106620 0x7f4e4819ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.490+0000 7f4e46000700 1 -- 172.21.15.6:0/1980758023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e4810b8a0 msgr2=0x7f4e481a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:03.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.490+0000 7f4e46000700 1 --2- 172.21.15.6:0/1980758023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e4810b8a0 0x7f4e481a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.490+0000 7f4e46000700 1 -- 172.21.15.6:0/1980758023 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e481a1d60 con 0x7f4e4810af50 2024-05-09T04:53:03.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.490+0000 7f4e46a00700 1 --2- 172.21.15.6:0/1980758023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e48106620 0x7f4e4819ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:53:03.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.490+0000 7f4e46000700 1 --2- 172.21.15.6:0/1980758023 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e4810af50 0x7f4e4819d430 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4e3400e970 tx=0x7f4e3400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:03.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.491+0000 7f4e4ca00700 1 -- 172.21.15.6:0/1980758023 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e34004a30 con 0x7f4e4810af50 2024-05-09T04:53:03.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.491+0000 7f4e47e00700 1 -- 172.21.15.6:0/1980758023 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e481a2050 con 0x7f4e4810af50 2024-05-09T04:53:03.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.491+0000 7f4e4ca00700 1 -- 172.21.15.6:0/1980758023 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e3400cc70 con 0x7f4e4810af50 2024-05-09T04:53:03.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.491+0000 7f4e4ca00700 1 -- 172.21.15.6:0/1980758023 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e34005580 con 0x7f4e4810af50 2024-05-09T04:53:03.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.491+0000 7f4e47e00700 1 -- 172.21.15.6:0/1980758023 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e481a9970 con 0x7f4e4810af50 2024-05-09T04:53:03.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.492+0000 7f4e47e00700 1 -- 172.21.15.6:0/1980758023 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e480512c0 con 0x7f4e4810af50 2024-05-09T04:53:03.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.495+0000 7f4e4ca00700 1 -- 172.21.15.6:0/1980758023 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f4e3400c7c0 con 0x7f4e4810af50 2024-05-09T04:53:03.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.496+0000 7f4e4ca00700 1 --2- 172.21.15.6:0/1980758023 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4e2c0b2900 0x7f4e2c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.497+0000 7f4e4ca00700 1 -- 172.21.15.6:0/1980758023 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f4e340e2760 con 0x7f4e4810af50 2024-05-09T04:53:03.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.497+0000 7f4e46a00700 1 --2- 172.21.15.6:0/1980758023 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4e2c0b2900 0x7f4e2c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.498+0000 7f4e46a00700 1 --2- 172.21.15.6:0/1980758023 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4e2c0b2900 0x7f4e2c0b4e10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4e38000c00 tx=0x7f4e3801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:03.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.502+0000 7f4e4ca00700 1 -- 172.21.15.6:0/1980758023 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4e340aa0a0 con 0x7f4e4810af50 2024-05-09T04:53:03.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.734+0000 7f4e47e00700 1 -- 172.21.15.6:0/1980758023 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4e48063c00 con 0x7f4e2c0b2900 2024-05-09T04:53:03.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.751+0000 7f4e4ca00700 1 -- 172.21.15.6:0/1980758023 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340566 (secure 0 0 0) 0x7f4e3807b270 con 0x7f4e2c0b2900 2024-05-09T04:53:03.760 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:53:03.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.759+0000 7f4e47e00700 1 -- 172.21.15.6:0/1980758023 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4e2c0b2900 msgr2=0x7f4e2c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:03.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.759+0000 7f4e47e00700 1 --2- 172.21.15.6:0/1980758023 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4e2c0b2900 0x7f4e2c0b4e10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4e38000c00 tx=0x7f4e3801a040 comp rx=0 tx=0).stop 2024-05-09T04:53:03.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.759+0000 7f4e47e00700 1 -- 172.21.15.6:0/1980758023 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e4810af50 msgr2=0x7f4e4819d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:03.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.759+0000 7f4e47e00700 1 --2- 172.21.15.6:0/1980758023 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e4810af50 0x7f4e4819d430 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4e3400e970 tx=0x7f4e3400ec80 comp rx=0 tx=0).stop 2024-05-09T04:53:03.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.759+0000 7f4e47e00700 1 -- 172.21.15.6:0/1980758023 shutdown_connections 2024-05-09T04:53:03.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.759+0000 7f4e47e00700 1 --2- 172.21.15.6:0/1980758023 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4e2c0b2900 0x7f4e2c0b4e10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.759+0000 7f4e47e00700 1 --2- 172.21.15.6:0/1980758023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e4810b8a0 0x7f4e481a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.759+0000 7f4e47e00700 1 --2- 172.21.15.6:0/1980758023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e48106620 0x7f4e4819ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.759+0000 7f4e47e00700 1 --2- 172.21.15.6:0/1980758023 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e4810af50 0x7f4e4819d430 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.759+0000 7f4e47e00700 1 -- 172.21.15.6:0/1980758023 >> 172.21.15.6:0/1980758023 conn(0x7f4e48102030 msgr2=0x7f4e4810a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:03.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.759+0000 7f4e47e00700 1 -- 172.21.15.6:0/1980758023 shutdown_connections 2024-05-09T04:53:03.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:03.759+0000 7f4e47e00700 1 -- 172.21.15.6:0/1980758023 wait complete. 2024-05-09T04:53:03.764 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-05-09T04:53:04.601 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2024-05-09T04:53:03.639575+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662577,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1298,"num_read_kb":757,"num_write":384,"num_write_kb":2303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":818,"ondisk_log_size":818,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":367456,"kb_used_data":14144,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124574368,"statfs":{"total":1151940427776,"available":1151564152832,"internally_reserved":0,"allocated":14483456,"data_stored":8078009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19137,"internal_metadata":361674047},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":531,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":118,"num_read_kb":63,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006445"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.559481+0000","last_change":"2024-05-09T04:49:34.144921+0000","last_active":"2024-05-09T04:52:55.559481+0000","last_peered":"2024-05-09T04:52:55.559481+0000","last_clean":"2024-05-09T04:52:55.559481+0000","last_became_active":"2024-05-09T04:49:34.144275+0000","last_became_peered":"2024-05-09T04:49:34.144275+0000","last_unstale":"2024-05-09T04:52:55.559481+0000","last_undegraded":"2024-05-09T04:52:55.559481+0000","last_fullsized":"2024-05-09T04:52:55.559481+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:00:31.377738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.558665+0000","last_change":"2024-05-09T04:49:34.158057+0000","last_active":"2024-05-09T04:52:55.558665+0000","last_peered":"2024-05-09T04:52:55.558665+0000","last_clean":"2024-05-09T04:52:55.558665+0000","last_became_active":"2024-05-09T04:49:34.157129+0000","last_became_peered":"2024-05-09T04:49:34.157129+0000","last_unstale":"2024-05-09T04:52:55.558665+0000","last_undegraded":"2024-05-09T04:52:55.558665+0000","last_fullsized":"2024-05-09T04:52:55.558665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:49.360309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.554048+0000","last_change":"2024-05-09T04:49:34.145519+0000","last_active":"2024-05-09T04:52:55.554048+0000","last_peered":"2024-05-09T04:52:55.554048+0000","last_clean":"2024-05-09T04:52:55.554048+0000","last_became_active":"2024-05-09T04:49:34.144501+0000","last_became_peered":"2024-05-09T04:49:34.144501+0000","last_unstale":"2024-05-09T04:52:55.554048+0000","last_undegraded":"2024-05-09T04:52:55.554048+0000","last_fullsized":"2024-05-09T04:52:55.554048+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:05:53.727520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.521442+0000","last_change":"2024-05-09T04:49:34.134383+0000","last_active":"2024-05-09T04:52:55.521442+0000","last_peered":"2024-05-09T04:52:55.521442+0000","last_clean":"2024-05-09T04:52:55.521442+0000","last_became_active":"2024-05-09T04:49:34.134049+0000","last_became_peered":"2024-05-09T04:49:34.134049+0000","last_unstale":"2024-05-09T04:52:55.521442+0000","last_undegraded":"2024-05-09T04:52:55.521442+0000","last_fullsized":"2024-05-09T04:52:55.521442+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:36:02.548188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.535651+0000","last_change":"2024-05-09T04:49:34.157897+0000","last_active":"2024-05-09T04:52:55.535651+0000","last_peered":"2024-05-09T04:52:55.535651+0000","last_clean":"2024-05-09T04:52:55.535651+0000","last_became_active":"2024-05-09T04:49:34.157516+0000","last_became_peered":"2024-05-09T04:49:34.157516+0000","last_unstale":"2024-05-09T04:52:55.535651+0000","last_undegraded":"2024-05-09T04:52:55.535651+0000","last_fullsized":"2024-05-09T04:52:55.535651+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:33:25.444155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.535109+0000","last_change":"2024-05-09T04:49:34.146389+0000","last_active":"2024-05-09T04:52:55.535109+0000","last_peered":"2024-05-09T04:52:55.535109+0000","last_clean":"2024-05-09T04:52:55.535109+0000","last_became_active":"2024-05-09T04:49:34.145994+0000","last_became_peered":"2024-05-09T04:49:34.145994+0000","last_unstale":"2024-05-09T04:52:55.535109+0000","last_undegraded":"2024-05-09T04:52:55.535109+0000","last_fullsized":"2024-05-09T04:52:55.535109+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:59.094826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.537220+0000","last_change":"2024-05-09T04:49:34.146428+0000","last_active":"2024-05-09T04:52:55.537220+0000","last_peered":"2024-05-09T04:52:55.537220+0000","last_clean":"2024-05-09T04:52:55.537220+0000","last_became_active":"2024-05-09T04:49:34.146045+0000","last_became_peered":"2024-05-09T04:49:34.146045+0000","last_unstale":"2024-05-09T04:52:55.537220+0000","last_undegraded":"2024-05-09T04:52:55.537220+0000","last_fullsized":"2024-05-09T04:52:55.537220+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:13.129474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.559673+0000","last_change":"2024-05-09T04:49:34.135494+0000","last_active":"2024-05-09T04:52:55.559673+0000","last_peered":"2024-05-09T04:52:55.559673+0000","last_clean":"2024-05-09T04:52:55.559673+0000","last_became_active":"2024-05-09T04:49:34.135085+0000","last_became_peered":"2024-05-09T04:49:34.135085+0000","last_unstale":"2024-05-09T04:52:55.559673+0000","last_undegraded":"2024-05-09T04:52:55.559673+0000","last_fullsized":"2024-05-09T04:52:55.559673+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:57.023344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.541489+0000","last_change":"2024-05-09T04:49:34.131353+0000","last_active":"2024-05-09T04:52:55.541489+0000","last_peered":"2024-05-09T04:52:55.541489+0000","last_clean":"2024-05-09T04:52:55.541489+0000","last_became_active":"2024-05-09T04:49:34.130954+0000","last_became_peered":"2024-05-09T04:49:34.130954+0000","last_unstale":"2024-05-09T04:52:55.541489+0000","last_undegraded":"2024-05-09T04:52:55.541489+0000","last_fullsized":"2024-05-09T04:52:55.541489+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:04:46.947861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.526736+0000","last_change":"2024-05-09T04:49:34.134733+0000","last_active":"2024-05-09T04:52:55.526736+0000","last_peered":"2024-05-09T04:52:55.526736+0000","last_clean":"2024-05-09T04:52:55.526736+0000","last_became_active":"2024-05-09T04:49:34.134337+0000","last_became_peered":"2024-05-09T04:49:34.134337+0000","last_unstale":"2024-05-09T04:52:55.526736+0000","last_undegraded":"2024-05-09T04:52:55.526736+0000","last_fullsized":"2024-05-09T04:52:55.526736+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:05.452488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.540741+0000","last_change":"2024-05-09T04:49:34.139143+0000","last_active":"2024-05-09T04:52:55.540741+0000","last_peered":"2024-05-09T04:52:55.540741+0000","last_clean":"2024-05-09T04:52:55.540741+0000","last_became_active":"2024-05-09T04:49:34.138749+0000","last_became_peered":"2024-05-09T04:49:34.138749+0000","last_unstale":"2024-05-09T04:52:55.540741+0000","last_undegraded":"2024-05-09T04:52:55.540741+0000","last_fullsized":"2024-05-09T04:52:55.540741+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:11.552063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.524212+0000","last_change":"2024-05-09T04:49:34.134687+0000","last_active":"2024-05-09T04:52:55.524212+0000","last_peered":"2024-05-09T04:52:55.524212+0000","last_clean":"2024-05-09T04:52:55.524212+0000","last_became_active":"2024-05-09T04:49:34.134324+0000","last_became_peered":"2024-05-09T04:49:34.134324+0000","last_unstale":"2024-05-09T04:52:55.524212+0000","last_undegraded":"2024-05-09T04:52:55.524212+0000","last_fullsized":"2024-05-09T04:52:55.524212+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:11.667883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.542595+0000","last_change":"2024-05-09T04:49:34.153592+0000","last_active":"2024-05-09T04:52:55.542595+0000","last_peered":"2024-05-09T04:52:55.542595+0000","last_clean":"2024-05-09T04:52:55.542595+0000","last_became_active":"2024-05-09T04:49:34.153184+0000","last_became_peered":"2024-05-09T04:49:34.153184+0000","last_unstale":"2024-05-09T04:52:55.542595+0000","last_undegraded":"2024-05-09T04:52:55.542595+0000","last_fullsized":"2024-05-09T04:52:55.542595+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:36:56.784781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.550285+0000","last_change":"2024-05-09T04:49:34.144046+0000","last_active":"2024-05-09T04:52:55.550285+0000","last_peered":"2024-05-09T04:52:55.550285+0000","last_clean":"2024-05-09T04:52:55.550285+0000","last_became_active":"2024-05-09T04:49:34.141004+0000","last_became_peered":"2024-05-09T04:49:34.141004+0000","last_unstale":"2024-05-09T04:52:55.550285+0000","last_undegraded":"2024-05-09T04:52:55.550285+0000","last_fullsized":"2024-05-09T04:52:55.550285+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:09:12.274750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.554450+0000","last_change":"2024-05-09T04:49:34.147063+0000","last_active":"2024-05-09T04:52:55.554450+0000","last_peered":"2024-05-09T04:52:55.554450+0000","last_clean":"2024-05-09T04:52:55.554450+0000","last_became_active":"2024-05-09T04:49:34.144934+0000","last_became_peered":"2024-05-09T04:49:34.144934+0000","last_unstale":"2024-05-09T04:52:55.554450+0000","last_undegraded":"2024-05-09T04:52:55.554450+0000","last_fullsized":"2024-05-09T04:52:55.554450+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:47:59.125451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.540390+0000","last_change":"2024-05-09T04:49:34.141070+0000","last_active":"2024-05-09T04:52:55.540390+0000","last_peered":"2024-05-09T04:52:55.540390+0000","last_clean":"2024-05-09T04:52:55.540390+0000","last_became_active":"2024-05-09T04:49:34.140618+0000","last_became_peered":"2024-05-09T04:49:34.140618+0000","last_unstale":"2024-05-09T04:52:55.540390+0000","last_undegraded":"2024-05-09T04:52:55.540390+0000","last_fullsized":"2024-05-09T04:52:55.540390+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:02.435768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.559611+0000","last_change":"2024-05-09T04:49:34.144434+0000","last_active":"2024-05-09T04:52:55.559611+0000","last_peered":"2024-05-09T04:52:55.559611+0000","last_clean":"2024-05-09T04:52:55.559611+0000","last_became_active":"2024-05-09T04:49:34.144052+0000","last_became_peered":"2024-05-09T04:49:34.144052+0000","last_unstale":"2024-05-09T04:52:55.559611+0000","last_undegraded":"2024-05-09T04:52:55.559611+0000","last_fullsized":"2024-05-09T04:52:55.559611+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:05:26.141627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.546642+0000","last_change":"2024-05-09T04:49:34.148681+0000","last_active":"2024-05-09T04:52:55.546642+0000","last_peered":"2024-05-09T04:52:55.546642+0000","last_clean":"2024-05-09T04:52:55.546642+0000","last_became_active":"2024-05-09T04:49:34.145277+0000","last_became_peered":"2024-05-09T04:49:34.145277+0000","last_unstale":"2024-05-09T04:52:55.546642+0000","last_undegraded":"2024-05-09T04:52:55.546642+0000","last_fullsized":"2024-05-09T04:52:55.546642+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:09.546998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.548479+0000","last_change":"2024-05-09T04:49:34.153229+0000","last_active":"2024-05-09T04:52:55.548479+0000","last_peered":"2024-05-09T04:52:55.548479+0000","last_clean":"2024-05-09T04:52:55.548479+0000","last_became_active":"2024-05-09T04:49:34.152869+0000","last_became_peered":"2024-05-09T04:49:34.152869+0000","last_unstale":"2024-05-09T04:52:55.548479+0000","last_undegraded":"2024-05-09T04:52:55.548479+0000","last_fullsized":"2024-05-09T04:52:55.548479+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:54.350842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700390+0000","last_change":"2024-05-09T04:49:56.710411+0000","last_active":"2024-05-09T04:50:57.700390+0000","last_peered":"2024-05-09T04:50:57.700390+0000","last_clean":"2024-05-09T04:50:57.700390+0000","last_became_active":"2024-05-09T04:49:56.709903+0000","last_became_peered":"2024-05-09T04:49:56.709903+0000","last_unstale":"2024-05-09T04:50:57.700390+0000","last_undegraded":"2024-05-09T04:50:57.700390+0000","last_fullsized":"2024-05-09T04:50:57.700390+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:53:44.216648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,10,0],"acting":[2,3,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":175,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:53:01.052946+0000","last_change":"2024-05-09T04:49:30.061766+0000","last_active":"2024-05-09T04:53:01.052946+0000","last_peered":"2024-05-09T04:53:01.052946+0000","last_clean":"2024-05-09T04:53:01.052946+0000","last_became_active":"2024-05-09T04:49:30.060545+0000","last_became_peered":"2024-05-09T04:49:30.060545+0000","last_unstale":"2024-05-09T04:53:01.052946+0000","last_undegraded":"2024-05-09T04:53:01.052946+0000","last_fullsized":"2024-05-09T04:53:01.052946+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:15:03.857107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.552044+0000","last_change":"2024-05-09T04:49:34.140706+0000","last_active":"2024-05-09T04:52:55.552044+0000","last_peered":"2024-05-09T04:52:55.552044+0000","last_clean":"2024-05-09T04:52:55.552044+0000","last_became_active":"2024-05-09T04:49:34.139032+0000","last_became_peered":"2024-05-09T04:49:34.139032+0000","last_unstale":"2024-05-09T04:52:55.552044+0000","last_undegraded":"2024-05-09T04:52:55.552044+0000","last_fullsized":"2024-05-09T04:52:55.552044+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:19.878523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.201694+0000","last_change":"2024-05-09T04:49:47.674254+0000","last_active":"2024-05-09T04:50:58.201694+0000","last_peered":"2024-05-09T04:50:58.201694+0000","last_clean":"2024-05-09T04:50:58.201694+0000","last_became_active":"2024-05-09T04:49:47.673476+0000","last_became_peered":"2024-05-09T04:49:47.673476+0000","last_unstale":"2024-05-09T04:50:58.201694+0000","last_undegraded":"2024-05-09T04:50:58.201694+0000","last_fullsized":"2024-05-09T04:50:58.201694+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:41:31.709210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:00.747787+0000","last_change":"2024-05-09T04:49:30.154162+0000","last_active":"2024-05-09T04:51:00.747787+0000","last_peered":"2024-05-09T04:51:00.747787+0000","last_clean":"2024-05-09T04:51:00.747787+0000","last_became_active":"2024-05-09T04:49:30.152869+0000","last_became_peered":"2024-05-09T04:49:30.152869+0000","last_unstale":"2024-05-09T04:51:00.747787+0000","last_undegraded":"2024-05-09T04:51:00.747787+0000","last_fullsized":"2024-05-09T04:51:00.747787+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:43.433033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.563131+0000","last_change":"2024-05-09T04:49:34.157729+0000","last_active":"2024-05-09T04:52:55.563131+0000","last_peered":"2024-05-09T04:52:55.563131+0000","last_clean":"2024-05-09T04:52:55.563131+0000","last_became_active":"2024-05-09T04:49:34.155040+0000","last_became_peered":"2024-05-09T04:49:34.155040+0000","last_unstale":"2024-05-09T04:52:55.563131+0000","last_undegraded":"2024-05-09T04:52:55.563131+0000","last_fullsized":"2024-05-09T04:52:55.563131+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:43:30.104394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.202328+0000","last_change":"2024-05-09T04:49:47.679665+0000","last_active":"2024-05-09T04:50:58.202328+0000","last_peered":"2024-05-09T04:50:58.202328+0000","last_clean":"2024-05-09T04:50:58.202328+0000","last_became_active":"2024-05-09T04:49:47.679257+0000","last_became_peered":"2024-05-09T04:49:47.679257+0000","last_unstale":"2024-05-09T04:50:58.202328+0000","last_undegraded":"2024-05-09T04:50:58.202328+0000","last_fullsized":"2024-05-09T04:50:58.202328+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:53.111205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:04.762348+0000","last_change":"2024-05-09T04:49:30.047386+0000","last_active":"2024-05-09T04:51:04.762348+0000","last_peered":"2024-05-09T04:51:04.762348+0000","last_clean":"2024-05-09T04:51:04.762348+0000","last_became_active":"2024-05-09T04:49:30.046947+0000","last_became_peered":"2024-05-09T04:49:30.046947+0000","last_unstale":"2024-05-09T04:51:04.762348+0000","last_undegraded":"2024-05-09T04:51:04.762348+0000","last_fullsized":"2024-05-09T04:51:04.762348+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:46.861354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.545354+0000","last_change":"2024-05-09T04:49:34.153320+0000","last_active":"2024-05-09T04:52:55.545354+0000","last_peered":"2024-05-09T04:52:55.545354+0000","last_clean":"2024-05-09T04:52:55.545354+0000","last_became_active":"2024-05-09T04:49:34.152854+0000","last_became_peered":"2024-05-09T04:49:34.152854+0000","last_unstale":"2024-05-09T04:52:55.545354+0000","last_undegraded":"2024-05-09T04:52:55.545354+0000","last_fullsized":"2024-05-09T04:52:55.545354+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:40.801340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.202375+0000","last_change":"2024-05-09T04:49:47.668475+0000","last_active":"2024-05-09T04:50:58.202375+0000","last_peered":"2024-05-09T04:50:58.202375+0000","last_clean":"2024-05-09T04:50:58.202375+0000","last_became_active":"2024-05-09T04:49:47.667909+0000","last_became_peered":"2024-05-09T04:49:47.667909+0000","last_unstale":"2024-05-09T04:50:58.202375+0000","last_undegraded":"2024-05-09T04:50:58.202375+0000","last_fullsized":"2024-05-09T04:50:58.202375+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:39:27.687468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:03.783186+0000","last_change":"2024-05-09T04:49:30.048659+0000","last_active":"2024-05-09T04:51:03.783186+0000","last_peered":"2024-05-09T04:51:03.783186+0000","last_clean":"2024-05-09T04:51:03.783186+0000","last_became_active":"2024-05-09T04:49:30.048259+0000","last_became_peered":"2024-05-09T04:49:30.048259+0000","last_unstale":"2024-05-09T04:51:03.783186+0000","last_undegraded":"2024-05-09T04:51:03.783186+0000","last_fullsized":"2024-05-09T04:51:03.783186+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:37:46.012594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.539299+0000","last_change":"2024-05-09T04:49:34.131416+0000","last_active":"2024-05-09T04:52:55.539299+0000","last_peered":"2024-05-09T04:52:55.539299+0000","last_clean":"2024-05-09T04:52:55.539299+0000","last_became_active":"2024-05-09T04:49:34.130985+0000","last_became_peered":"2024-05-09T04:49:34.130985+0000","last_unstale":"2024-05-09T04:52:55.539299+0000","last_undegraded":"2024-05-09T04:52:55.539299+0000","last_fullsized":"2024-05-09T04:52:55.539299+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:02:43.228142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700686+0000","last_change":"2024-05-09T04:49:47.652247+0000","last_active":"2024-05-09T04:50:57.700686+0000","last_peered":"2024-05-09T04:50:57.700686+0000","last_clean":"2024-05-09T04:50:57.700686+0000","last_became_active":"2024-05-09T04:49:47.651747+0000","last_became_peered":"2024-05-09T04:49:47.651747+0000","last_unstale":"2024-05-09T04:50:57.700686+0000","last_undegraded":"2024-05-09T04:50:57.700686+0000","last_fullsized":"2024-05-09T04:50:57.700686+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:45.353027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":171,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:58.957560+0000","last_change":"2024-05-09T04:49:30.047421+0000","last_active":"2024-05-09T04:52:58.957560+0000","last_peered":"2024-05-09T04:52:58.957560+0000","last_clean":"2024-05-09T04:52:58.957560+0000","last_became_active":"2024-05-09T04:49:30.047007+0000","last_became_peered":"2024-05-09T04:49:30.047007+0000","last_unstale":"2024-05-09T04:52:58.957560+0000","last_undegraded":"2024-05-09T04:52:58.957560+0000","last_fullsized":"2024-05-09T04:52:58.957560+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:23.930326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.533380+0000","last_change":"2024-05-09T04:49:34.146605+0000","last_active":"2024-05-09T04:52:55.533380+0000","last_peered":"2024-05-09T04:52:55.533380+0000","last_clean":"2024-05-09T04:52:55.533380+0000","last_became_active":"2024-05-09T04:49:34.146205+0000","last_became_peered":"2024-05-09T04:49:34.146205+0000","last_unstale":"2024-05-09T04:52:55.533380+0000","last_undegraded":"2024-05-09T04:52:55.533380+0000","last_fullsized":"2024-05-09T04:52:55.533380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:13:52.234951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700734+0000","last_change":"2024-05-09T04:49:47.673652+0000","last_active":"2024-05-09T04:50:57.700734+0000","last_peered":"2024-05-09T04:50:57.700734+0000","last_clean":"2024-05-09T04:50:57.700734+0000","last_became_active":"2024-05-09T04:49:47.673255+0000","last_became_peered":"2024-05-09T04:49:47.673255+0000","last_unstale":"2024-05-09T04:50:57.700734+0000","last_undegraded":"2024-05-09T04:50:57.700734+0000","last_fullsized":"2024-05-09T04:50:57.700734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:12:50.672212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:01.759883+0000","last_change":"2024-05-09T04:49:30.056089+0000","last_active":"2024-05-09T04:51:01.759883+0000","last_peered":"2024-05-09T04:51:01.759883+0000","last_clean":"2024-05-09T04:51:01.759883+0000","last_became_active":"2024-05-09T04:49:30.055635+0000","last_became_peered":"2024-05-09T04:49:30.055635+0000","last_unstale":"2024-05-09T04:51:01.759883+0000","last_undegraded":"2024-05-09T04:51:01.759883+0000","last_fullsized":"2024-05-09T04:51:01.759883+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:35:06.529058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.557248+0000","last_change":"2024-05-09T04:49:34.147628+0000","last_active":"2024-05-09T04:52:55.557248+0000","last_peered":"2024-05-09T04:52:55.557248+0000","last_clean":"2024-05-09T04:52:55.557248+0000","last_became_active":"2024-05-09T04:49:34.147208+0000","last_became_peered":"2024-05-09T04:49:34.147208+0000","last_unstale":"2024-05-09T04:52:55.557248+0000","last_undegraded":"2024-05-09T04:52:55.557248+0000","last_fullsized":"2024-05-09T04:52:55.557248+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:09:22.520468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.816932+0000","last_change":"2024-05-09T04:49:56.758366+0000","last_active":"2024-05-09T04:50:57.816932+0000","last_peered":"2024-05-09T04:50:57.816932+0000","last_clean":"2024-05-09T04:50:57.816932+0000","last_became_active":"2024-05-09T04:49:56.757763+0000","last_became_peered":"2024-05-09T04:49:56.757763+0000","last_unstale":"2024-05-09T04:50:57.816932+0000","last_undegraded":"2024-05-09T04:50:57.816932+0000","last_fullsized":"2024-05-09T04:50:57.816932+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:58.711764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:02.769778+0000","last_change":"2024-05-09T04:49:30.058805+0000","last_active":"2024-05-09T04:51:02.769778+0000","last_peered":"2024-05-09T04:51:02.769778+0000","last_clean":"2024-05-09T04:51:02.769778+0000","last_became_active":"2024-05-09T04:49:30.057278+0000","last_became_peered":"2024-05-09T04:49:30.057278+0000","last_unstale":"2024-05-09T04:51:02.769778+0000","last_undegraded":"2024-05-09T04:51:02.769778+0000","last_fullsized":"2024-05-09T04:51:02.769778+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:34.727781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.557567+0000","last_change":"2024-05-09T04:49:34.148753+0000","last_active":"2024-05-09T04:52:55.557567+0000","last_peered":"2024-05-09T04:52:55.557567+0000","last_clean":"2024-05-09T04:52:55.557567+0000","last_became_active":"2024-05-09T04:49:34.148407+0000","last_became_peered":"2024-05-09T04:49:34.148407+0000","last_unstale":"2024-05-09T04:52:55.557567+0000","last_undegraded":"2024-05-09T04:52:55.557567+0000","last_fullsized":"2024-05-09T04:52:55.557567+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:39.602934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.204230+0000","last_change":"2024-05-09T04:49:47.672521+0000","last_active":"2024-05-09T04:50:58.204230+0000","last_peered":"2024-05-09T04:50:58.204230+0000","last_clean":"2024-05-09T04:50:58.204230+0000","last_became_active":"2024-05-09T04:49:47.671353+0000","last_became_peered":"2024-05-09T04:49:47.671353+0000","last_unstale":"2024-05-09T04:50:58.204230+0000","last_undegraded":"2024-05-09T04:50:58.204230+0000","last_fullsized":"2024-05-09T04:50:58.204230+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:56:34.832651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:03.778625+0000","last_change":"2024-05-09T04:49:30.048589+0000","last_active":"2024-05-09T04:51:03.778625+0000","last_peered":"2024-05-09T04:51:03.778625+0000","last_clean":"2024-05-09T04:51:03.778625+0000","last_became_active":"2024-05-09T04:49:30.048241+0000","last_became_peered":"2024-05-09T04:49:30.048241+0000","last_unstale":"2024-05-09T04:51:03.778625+0000","last_undegraded":"2024-05-09T04:51:03.778625+0000","last_fullsized":"2024-05-09T04:51:03.778625+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:38:36.204539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.562002+0000","last_change":"2024-05-09T04:49:34.151201+0000","last_active":"2024-05-09T04:52:55.562002+0000","last_peered":"2024-05-09T04:52:55.562002+0000","last_clean":"2024-05-09T04:52:55.562002+0000","last_became_active":"2024-05-09T04:49:34.150505+0000","last_became_peered":"2024-05-09T04:49:34.150505+0000","last_unstale":"2024-05-09T04:52:55.562002+0000","last_undegraded":"2024-05-09T04:52:55.562002+0000","last_fullsized":"2024-05-09T04:52:55.562002+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:40.011516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700453+0000","last_change":"2024-05-09T04:49:47.653827+0000","last_active":"2024-05-09T04:50:57.700453+0000","last_peered":"2024-05-09T04:50:57.700453+0000","last_clean":"2024-05-09T04:50:57.700453+0000","last_became_active":"2024-05-09T04:49:47.652423+0000","last_became_peered":"2024-05-09T04:49:47.652423+0000","last_unstale":"2024-05-09T04:50:57.700453+0000","last_undegraded":"2024-05-09T04:50:57.700453+0000","last_fullsized":"2024-05-09T04:50:57.700453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:07:07.602659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'5","reported_seq":282,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:59.958313+0000","last_change":"2024-05-09T04:49:30.062849+0000","last_active":"2024-05-09T04:52:59.958313+0000","last_peered":"2024-05-09T04:52:59.958313+0000","last_clean":"2024-05-09T04:52:59.958313+0000","last_became_active":"2024-05-09T04:49:30.062426+0000","last_became_peered":"2024-05-09T04:49:30.062426+0000","last_unstale":"2024-05-09T04:52:59.958313+0000","last_undegraded":"2024-05-09T04:52:59.958313+0000","last_fullsized":"2024-05-09T04:52:59.958313+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:25:11.996208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":234,"num_read_kb":234,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.543652+0000","last_change":"2024-05-09T04:49:34.151145+0000","last_active":"2024-05-09T04:52:55.543652+0000","last_peered":"2024-05-09T04:52:55.543652+0000","last_clean":"2024-05-09T04:52:55.543652+0000","last_became_active":"2024-05-09T04:49:34.150745+0000","last_became_peered":"2024-05-09T04:49:34.150745+0000","last_unstale":"2024-05-09T04:52:55.543652+0000","last_undegraded":"2024-05-09T04:52:55.543652+0000","last_fullsized":"2024-05-09T04:52:55.543652+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:40.647454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700544+0000","last_change":"2024-05-09T04:49:47.672796+0000","last_active":"2024-05-09T04:50:57.700544+0000","last_peered":"2024-05-09T04:50:57.700544+0000","last_clean":"2024-05-09T04:50:57.700544+0000","last_became_active":"2024-05-09T04:49:47.667807+0000","last_became_peered":"2024-05-09T04:49:47.667807+0000","last_unstale":"2024-05-09T04:50:57.700544+0000","last_undegraded":"2024-05-09T04:50:57.700544+0000","last_fullsized":"2024-05-09T04:50:57.700544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:58.542973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:02.767623+0000","last_change":"2024-05-09T04:49:30.047265+0000","last_active":"2024-05-09T04:51:02.767623+0000","last_peered":"2024-05-09T04:51:02.767623+0000","last_clean":"2024-05-09T04:51:02.767623+0000","last_became_active":"2024-05-09T04:49:30.046888+0000","last_became_peered":"2024-05-09T04:49:30.046888+0000","last_unstale":"2024-05-09T04:51:02.767623+0000","last_undegraded":"2024-05-09T04:51:02.767623+0000","last_fullsized":"2024-05-09T04:51:02.767623+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:38:41.878678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.554153+0000","last_change":"2024-05-09T04:49:34.144447+0000","last_active":"2024-05-09T04:52:55.554153+0000","last_peered":"2024-05-09T04:52:55.554153+0000","last_clean":"2024-05-09T04:52:55.554153+0000","last_became_active":"2024-05-09T04:49:34.144073+0000","last_became_peered":"2024-05-09T04:49:34.144073+0000","last_unstale":"2024-05-09T04:52:55.554153+0000","last_undegraded":"2024-05-09T04:52:55.554153+0000","last_fullsized":"2024-05-09T04:52:55.554153+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:53:42.619552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.202477+0000","last_change":"2024-05-09T04:49:47.673364+0000","last_active":"2024-05-09T04:50:58.202477+0000","last_peered":"2024-05-09T04:50:58.202477+0000","last_clean":"2024-05-09T04:50:58.202477+0000","last_became_active":"2024-05-09T04:49:47.672712+0000","last_became_peered":"2024-05-09T04:49:47.672712+0000","last_unstale":"2024-05-09T04:50:58.202477+0000","last_undegraded":"2024-05-09T04:50:58.202477+0000","last_fullsized":"2024-05-09T04:50:58.202477+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:25:57.165777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:03.779907+0000","last_change":"2024-05-09T04:49:30.160924+0000","last_active":"2024-05-09T04:51:03.779907+0000","last_peered":"2024-05-09T04:51:03.779907+0000","last_clean":"2024-05-09T04:51:03.779907+0000","last_became_active":"2024-05-09T04:49:30.160479+0000","last_became_peered":"2024-05-09T04:49:30.160479+0000","last_unstale":"2024-05-09T04:51:03.779907+0000","last_undegraded":"2024-05-09T04:51:03.779907+0000","last_fullsized":"2024-05-09T04:51:03.779907+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:44:57.171441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.545742+0000","last_change":"2024-05-09T04:49:34.136514+0000","last_active":"2024-05-09T04:52:55.545742+0000","last_peered":"2024-05-09T04:52:55.545742+0000","last_clean":"2024-05-09T04:52:55.545742+0000","last_became_active":"2024-05-09T04:49:34.136121+0000","last_became_peered":"2024-05-09T04:49:34.136121+0000","last_unstale":"2024-05-09T04:52:55.545742+0000","last_undegraded":"2024-05-09T04:52:55.545742+0000","last_fullsized":"2024-05-09T04:52:55.545742+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:43:27.781720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700659+0000","last_change":"2024-05-09T04:49:47.673371+0000","last_active":"2024-05-09T04:50:57.700659+0000","last_peered":"2024-05-09T04:50:57.700659+0000","last_clean":"2024-05-09T04:50:57.700659+0000","last_became_active":"2024-05-09T04:49:47.669727+0000","last_became_peered":"2024-05-09T04:49:47.669727+0000","last_unstale":"2024-05-09T04:50:57.700659+0000","last_undegraded":"2024-05-09T04:50:57.700659+0000","last_fullsized":"2024-05-09T04:50:57.700659+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:34:45.510934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'40","reported_seq":329,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:59.958301+0000","last_change":"2024-05-09T04:49:30.048359+0000","last_active":"2024-05-09T04:52:59.958301+0000","last_peered":"2024-05-09T04:52:59.958301+0000","last_clean":"2024-05-09T04:52:59.958301+0000","last_became_active":"2024-05-09T04:49:30.047949+0000","last_became_peered":"2024-05-09T04:49:30.047949+0000","last_unstale":"2024-05-09T04:52:59.958301+0000","last_undegraded":"2024-05-09T04:52:59.958301+0000","last_fullsized":"2024-05-09T04:52:59.958301+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:01:10.498997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64489,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":246,"num_read_kb":187,"num_write":42,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.521240+0000","last_change":"2024-05-09T04:49:34.132589+0000","last_active":"2024-05-09T04:52:55.521240+0000","last_peered":"2024-05-09T04:52:55.521240+0000","last_clean":"2024-05-09T04:52:55.521240+0000","last_became_active":"2024-05-09T04:49:34.132227+0000","last_became_peered":"2024-05-09T04:49:34.132227+0000","last_unstale":"2024-05-09T04:52:55.521240+0000","last_undegraded":"2024-05-09T04:52:55.521240+0000","last_fullsized":"2024-05-09T04:52:55.521240+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:50.003878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.033497+0000","last_change":"2024-05-09T04:49:47.668236+0000","last_active":"2024-05-09T04:50:58.033497+0000","last_peered":"2024-05-09T04:50:58.033497+0000","last_clean":"2024-05-09T04:50:58.033497+0000","last_became_active":"2024-05-09T04:49:47.667779+0000","last_became_peered":"2024-05-09T04:49:47.667779+0000","last_unstale":"2024-05-09T04:50:58.033497+0000","last_undegraded":"2024-05-09T04:50:58.033497+0000","last_fullsized":"2024-05-09T04:50:58.033497+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:16.177888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:03.776885+0000","last_change":"2024-05-09T04:49:30.046145+0000","last_active":"2024-05-09T04:51:03.776885+0000","last_peered":"2024-05-09T04:51:03.776885+0000","last_clean":"2024-05-09T04:51:03.776885+0000","last_became_active":"2024-05-09T04:49:30.045478+0000","last_became_peered":"2024-05-09T04:49:30.045478+0000","last_unstale":"2024-05-09T04:51:03.776885+0000","last_undegraded":"2024-05-09T04:51:03.776885+0000","last_fullsized":"2024-05-09T04:51:03.776885+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:52:08.313984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.550254+0000","last_change":"2024-05-09T04:49:34.154349+0000","last_active":"2024-05-09T04:52:55.550254+0000","last_peered":"2024-05-09T04:52:55.550254+0000","last_clean":"2024-05-09T04:52:55.550254+0000","last_became_active":"2024-05-09T04:49:34.154011+0000","last_became_peered":"2024-05-09T04:49:34.154011+0000","last_unstale":"2024-05-09T04:52:55.550254+0000","last_undegraded":"2024-05-09T04:52:55.550254+0000","last_fullsized":"2024-05-09T04:52:55.550254+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:50:32.345863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.816744+0000","last_change":"2024-05-09T04:49:47.670170+0000","last_active":"2024-05-09T04:50:57.816744+0000","last_peered":"2024-05-09T04:50:57.816744+0000","last_clean":"2024-05-09T04:50:57.816744+0000","last_became_active":"2024-05-09T04:49:47.669705+0000","last_became_peered":"2024-05-09T04:49:47.669705+0000","last_unstale":"2024-05-09T04:50:57.816744+0000","last_undegraded":"2024-05-09T04:50:57.816744+0000","last_fullsized":"2024-05-09T04:50:57.816744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:59:27.339811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:03.777501+0000","last_change":"2024-05-09T04:49:30.051159+0000","last_active":"2024-05-09T04:51:03.777501+0000","last_peered":"2024-05-09T04:51:03.777501+0000","last_clean":"2024-05-09T04:51:03.777501+0000","last_became_active":"2024-05-09T04:49:30.050108+0000","last_became_peered":"2024-05-09T04:49:30.050108+0000","last_unstale":"2024-05-09T04:51:03.777501+0000","last_undegraded":"2024-05-09T04:51:03.777501+0000","last_fullsized":"2024-05-09T04:51:03.777501+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:32:56.778567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.549097+0000","last_change":"2024-05-09T04:49:34.162751+0000","last_active":"2024-05-09T04:52:55.549097+0000","last_peered":"2024-05-09T04:52:55.549097+0000","last_clean":"2024-05-09T04:52:55.549097+0000","last_became_active":"2024-05-09T04:49:34.162336+0000","last_became_peered":"2024-05-09T04:49:34.162336+0000","last_unstale":"2024-05-09T04:52:55.549097+0000","last_undegraded":"2024-05-09T04:52:55.549097+0000","last_fullsized":"2024-05-09T04:52:55.549097+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:41.758074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.705898+0000","last_change":"2024-05-09T04:49:47.680955+0000","last_active":"2024-05-09T04:50:57.705898+0000","last_peered":"2024-05-09T04:50:57.705898+0000","last_clean":"2024-05-09T04:50:57.705898+0000","last_became_active":"2024-05-09T04:49:47.680041+0000","last_became_peered":"2024-05-09T04:49:47.680041+0000","last_unstale":"2024-05-09T04:50:57.705898+0000","last_undegraded":"2024-05-09T04:50:57.705898+0000","last_fullsized":"2024-05-09T04:50:57.705898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:46.297611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'7","reported_seq":171,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:53:00.057449+0000","last_change":"2024-05-09T04:49:30.045857+0000","last_active":"2024-05-09T04:53:00.057449+0000","last_peered":"2024-05-09T04:53:00.057449+0000","last_clean":"2024-05-09T04:53:00.057449+0000","last_became_active":"2024-05-09T04:49:30.045404+0000","last_became_peered":"2024-05-09T04:49:30.045404+0000","last_unstale":"2024-05-09T04:53:00.057449+0000","last_undegraded":"2024-05-09T04:53:00.057449+0000","last_fullsized":"2024-05-09T04:53:00.057449+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:44:19.473449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.538078+0000","last_change":"2024-05-09T04:49:34.131734+0000","last_active":"2024-05-09T04:52:55.538078+0000","last_peered":"2024-05-09T04:52:55.538078+0000","last_clean":"2024-05-09T04:52:55.538078+0000","last_became_active":"2024-05-09T04:49:34.131396+0000","last_became_peered":"2024-05-09T04:49:34.131396+0000","last_unstale":"2024-05-09T04:52:55.538078+0000","last_undegraded":"2024-05-09T04:52:55.538078+0000","last_fullsized":"2024-05-09T04:52:55.538078+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:21.080651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.088139+0000","last_change":"2024-05-09T04:49:56.710429+0000","last_active":"2024-05-09T04:50:58.088139+0000","last_peered":"2024-05-09T04:50:58.088139+0000","last_clean":"2024-05-09T04:50:58.088139+0000","last_became_active":"2024-05-09T04:49:56.708497+0000","last_became_peered":"2024-05-09T04:49:56.708497+0000","last_unstale":"2024-05-09T04:50:58.088139+0000","last_undegraded":"2024-05-09T04:50:58.088139+0000","last_fullsized":"2024-05-09T04:50:58.088139+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:36.281300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:03.777421+0000","last_change":"2024-05-09T04:49:56.718731+0000","last_active":"2024-05-09T04:51:03.777421+0000","last_peered":"2024-05-09T04:51:03.777421+0000","last_clean":"2024-05-09T04:51:03.777421+0000","last_became_active":"2024-05-09T04:49:56.709958+0000","last_became_peered":"2024-05-09T04:49:56.709958+0000","last_unstale":"2024-05-09T04:51:03.777421+0000","last_undegraded":"2024-05-09T04:51:03.777421+0000","last_fullsized":"2024-05-09T04:51:03.777421+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:11.003906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.538882+0000","last_change":"2024-05-09T04:49:34.131705+0000","last_active":"2024-05-09T04:52:55.538882+0000","last_peered":"2024-05-09T04:52:55.538882+0000","last_clean":"2024-05-09T04:52:55.538882+0000","last_became_active":"2024-05-09T04:49:34.131287+0000","last_became_peered":"2024-05-09T04:49:34.131287+0000","last_unstale":"2024-05-09T04:52:55.538882+0000","last_undegraded":"2024-05-09T04:52:55.538882+0000","last_fullsized":"2024-05-09T04:52:55.538882+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:10:24.914832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.032602+0000","last_change":"2024-05-09T04:49:47.663861+0000","last_active":"2024-05-09T04:50:58.032602+0000","last_peered":"2024-05-09T04:50:58.032602+0000","last_clean":"2024-05-09T04:50:58.032602+0000","last_became_active":"2024-05-09T04:49:47.663374+0000","last_became_peered":"2024-05-09T04:49:47.663374+0000","last_unstale":"2024-05-09T04:50:58.032602+0000","last_undegraded":"2024-05-09T04:50:58.032602+0000","last_fullsized":"2024-05-09T04:50:58.032602+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:08.063040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:03.776644+0000","last_change":"2024-05-09T04:49:30.044183+0000","last_active":"2024-05-09T04:51:03.776644+0000","last_peered":"2024-05-09T04:51:03.776644+0000","last_clean":"2024-05-09T04:51:03.776644+0000","last_became_active":"2024-05-09T04:49:30.043608+0000","last_became_peered":"2024-05-09T04:49:30.043608+0000","last_unstale":"2024-05-09T04:51:03.776644+0000","last_undegraded":"2024-05-09T04:51:03.776644+0000","last_fullsized":"2024-05-09T04:51:03.776644+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:48:57.788970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.524751+0000","last_change":"2024-05-09T04:49:34.133941+0000","last_active":"2024-05-09T04:52:55.524751+0000","last_peered":"2024-05-09T04:52:55.524751+0000","last_clean":"2024-05-09T04:52:55.524751+0000","last_became_active":"2024-05-09T04:49:34.133550+0000","last_became_peered":"2024-05-09T04:49:34.133550+0000","last_unstale":"2024-05-09T04:52:55.524751+0000","last_undegraded":"2024-05-09T04:52:55.524751+0000","last_fullsized":"2024-05-09T04:52:55.524751+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:45:14.703565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.201444+0000","last_change":"2024-05-09T04:49:47.901085+0000","last_active":"2024-05-09T04:50:58.201444+0000","last_peered":"2024-05-09T04:50:58.201444+0000","last_clean":"2024-05-09T04:50:58.201444+0000","last_became_active":"2024-05-09T04:49:47.900444+0000","last_became_peered":"2024-05-09T04:49:47.900444+0000","last_unstale":"2024-05-09T04:50:58.201444+0000","last_undegraded":"2024-05-09T04:50:58.201444+0000","last_fullsized":"2024-05-09T04:50:58.201444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:44.195683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:01.759239+0000","last_change":"2024-05-09T04:49:30.154060+0000","last_active":"2024-05-09T04:51:01.759239+0000","last_peered":"2024-05-09T04:51:01.759239+0000","last_clean":"2024-05-09T04:51:01.759239+0000","last_became_active":"2024-05-09T04:49:30.152556+0000","last_became_peered":"2024-05-09T04:49:30.152556+0000","last_unstale":"2024-05-09T04:51:01.759239+0000","last_undegraded":"2024-05-09T04:51:01.759239+0000","last_fullsized":"2024-05-09T04:51:01.759239+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:27.056251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.531335+0000","last_change":"2024-05-09T04:49:34.153128+0000","last_active":"2024-05-09T04:52:55.531335+0000","last_peered":"2024-05-09T04:52:55.531335+0000","last_clean":"2024-05-09T04:52:55.531335+0000","last_became_active":"2024-05-09T04:49:34.152734+0000","last_became_peered":"2024-05-09T04:49:34.152734+0000","last_unstale":"2024-05-09T04:52:55.531335+0000","last_undegraded":"2024-05-09T04:52:55.531335+0000","last_fullsized":"2024-05-09T04:52:55.531335+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:54:06.920877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.087826+0000","last_change":"2024-05-09T04:49:47.670263+0000","last_active":"2024-05-09T04:50:58.087826+0000","last_peered":"2024-05-09T04:50:58.087826+0000","last_clean":"2024-05-09T04:50:58.087826+0000","last_became_active":"2024-05-09T04:49:47.665743+0000","last_became_peered":"2024-05-09T04:49:47.665743+0000","last_unstale":"2024-05-09T04:50:58.087826+0000","last_undegraded":"2024-05-09T04:50:58.087826+0000","last_fullsized":"2024-05-09T04:50:58.087826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:27.980226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":165,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:58.047011+0000","last_change":"2024-05-09T04:49:30.056119+0000","last_active":"2024-05-09T04:52:58.047011+0000","last_peered":"2024-05-09T04:52:58.047011+0000","last_clean":"2024-05-09T04:52:58.047011+0000","last_became_active":"2024-05-09T04:49:30.055660+0000","last_became_peered":"2024-05-09T04:49:30.055660+0000","last_unstale":"2024-05-09T04:52:58.047011+0000","last_undegraded":"2024-05-09T04:52:58.047011+0000","last_fullsized":"2024-05-09T04:52:58.047011+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:21.513879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.524713+0000","last_change":"2024-05-09T04:49:56.764303+0000","last_active":"2024-05-09T04:52:55.524713+0000","last_peered":"2024-05-09T04:52:55.524713+0000","last_clean":"2024-05-09T04:52:55.524713+0000","last_became_active":"2024-05-09T04:49:56.763854+0000","last_became_peered":"2024-05-09T04:49:56.763854+0000","last_unstale":"2024-05-09T04:52:55.524713+0000","last_undegraded":"2024-05-09T04:52:55.524713+0000","last_fullsized":"2024-05-09T04:52:55.524713+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:06.086602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.088298+0000","last_change":"2024-05-09T04:49:47.900866+0000","last_active":"2024-05-09T04:50:58.088298+0000","last_peered":"2024-05-09T04:50:58.088298+0000","last_clean":"2024-05-09T04:50:58.088298+0000","last_became_active":"2024-05-09T04:49:47.900332+0000","last_became_peered":"2024-05-09T04:49:47.900332+0000","last_unstale":"2024-05-09T04:50:58.088298+0000","last_undegraded":"2024-05-09T04:50:58.088298+0000","last_fullsized":"2024-05-09T04:50:58.088298+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:06:41.448262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.527044+0000","last_change":"2024-05-09T04:49:56.702559+0000","last_active":"2024-05-09T04:52:55.527044+0000","last_peered":"2024-05-09T04:52:55.527044+0000","last_clean":"2024-05-09T04:52:55.527044+0000","last_became_active":"2024-05-09T04:49:56.702135+0000","last_became_peered":"2024-05-09T04:49:56.702135+0000","last_unstale":"2024-05-09T04:52:55.527044+0000","last_undegraded":"2024-05-09T04:52:55.527044+0000","last_fullsized":"2024-05-09T04:52:55.527044+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:43:05.848239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":169,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:53:01.058363+0000","last_change":"2024-05-09T04:49:30.158050+0000","last_active":"2024-05-09T04:53:01.058363+0000","last_peered":"2024-05-09T04:53:01.058363+0000","last_clean":"2024-05-09T04:53:01.058363+0000","last_became_active":"2024-05-09T04:49:30.156719+0000","last_became_peered":"2024-05-09T04:49:30.156719+0000","last_unstale":"2024-05-09T04:53:01.058363+0000","last_undegraded":"2024-05-09T04:53:01.058363+0000","last_fullsized":"2024-05-09T04:53:01.058363+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:47.963111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:15.645536+0000","last_change":"2024-05-09T04:49:47.673594+0000","last_active":"2024-05-09T04:51:15.645536+0000","last_peered":"2024-05-09T04:51:15.645536+0000","last_clean":"2024-05-09T04:51:15.645536+0000","last_became_active":"2024-05-09T04:49:47.672671+0000","last_became_peered":"2024-05-09T04:49:47.672671+0000","last_unstale":"2024-05-09T04:51:15.645536+0000","last_undegraded":"2024-05-09T04:51:15.645536+0000","last_fullsized":"2024-05-09T04:51:15.645536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:58:56.078180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.547961+0000","last_change":"2024-05-09T04:50:58.035054+0000","last_active":"2024-05-09T04:52:55.547961+0000","last_peered":"2024-05-09T04:52:55.547961+0000","last_clean":"2024-05-09T04:52:55.547961+0000","last_became_active":"2024-05-09T04:50:58.034297+0000","last_became_peered":"2024-05-09T04:50:58.034297+0000","last_unstale":"2024-05-09T04:52:55.547961+0000","last_undegraded":"2024-05-09T04:52:55.547961+0000","last_fullsized":"2024-05-09T04:52:55.547961+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:10:56.827950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:03.777300+0000","last_change":"2024-05-09T04:49:30.060224+0000","last_active":"2024-05-09T04:51:03.777300+0000","last_peered":"2024-05-09T04:51:03.777300+0000","last_clean":"2024-05-09T04:51:03.777300+0000","last_became_active":"2024-05-09T04:49:30.059830+0000","last_became_peered":"2024-05-09T04:49:30.059830+0000","last_unstale":"2024-05-09T04:51:03.777300+0000","last_undegraded":"2024-05-09T04:51:03.777300+0000","last_fullsized":"2024-05-09T04:51:03.777300+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:02.925226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":117,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203150+0000","last_change":"2024-05-09T04:47:53.050380+0000","last_active":"2024-05-09T04:50:58.203150+0000","last_peered":"2024-05-09T04:50:58.203150+0000","last_clean":"2024-05-09T04:50:58.203150+0000","last_became_active":"2024-05-09T04:47:53.044058+0000","last_became_peered":"2024-05-09T04:47:53.044058+0000","last_unstale":"2024-05-09T04:50:58.203150+0000","last_undegraded":"2024-05-09T04:50:58.203150+0000","last_fullsized":"2024-05-09T04:50:58.203150+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:43:48.067159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:43:48.067159+0000","last_clean_scrub_stamp":"2024-05-09T04:43:48.067159+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:17:18.714607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.033328+0000","last_change":"2024-05-09T04:49:47.901114+0000","last_active":"2024-05-09T04:50:58.033328+0000","last_peered":"2024-05-09T04:50:58.033328+0000","last_clean":"2024-05-09T04:50:58.033328+0000","last_became_active":"2024-05-09T04:49:47.900450+0000","last_became_peered":"2024-05-09T04:49:47.900450+0000","last_unstale":"2024-05-09T04:50:58.033328+0000","last_undegraded":"2024-05-09T04:50:58.033328+0000","last_fullsized":"2024-05-09T04:50:58.033328+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:25:44.517789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.537021+0000","last_change":"2024-05-09T04:49:34.147027+0000","last_active":"2024-05-09T04:52:55.537021+0000","last_peered":"2024-05-09T04:52:55.537021+0000","last_clean":"2024-05-09T04:52:55.537021+0000","last_became_active":"2024-05-09T04:49:34.146672+0000","last_became_peered":"2024-05-09T04:49:34.146672+0000","last_unstale":"2024-05-09T04:52:55.537021+0000","last_undegraded":"2024-05-09T04:52:55.537021+0000","last_fullsized":"2024-05-09T04:52:55.537021+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:15:44.847304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:02.768703+0000","last_change":"2024-05-09T04:49:30.043877+0000","last_active":"2024-05-09T04:51:02.768703+0000","last_peered":"2024-05-09T04:51:02.768703+0000","last_clean":"2024-05-09T04:51:02.768703+0000","last_became_active":"2024-05-09T04:49:30.043513+0000","last_became_peered":"2024-05-09T04:49:30.043513+0000","last_unstale":"2024-05-09T04:51:02.768703+0000","last_undegraded":"2024-05-09T04:51:02.768703+0000","last_fullsized":"2024-05-09T04:51:02.768703+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:16.501692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.816615+0000","last_change":"2024-05-09T04:49:47.668605+0000","last_active":"2024-05-09T04:50:57.816615+0000","last_peered":"2024-05-09T04:50:57.816615+0000","last_clean":"2024-05-09T04:50:57.816615+0000","last_became_active":"2024-05-09T04:49:47.668095+0000","last_became_peered":"2024-05-09T04:49:47.668095+0000","last_unstale":"2024-05-09T04:50:57.816615+0000","last_undegraded":"2024-05-09T04:50:57.816615+0000","last_fullsized":"2024-05-09T04:50:57.816615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:39:30.508618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:03.777515+0000","last_change":"2024-05-09T04:49:30.159554+0000","last_active":"2024-05-09T04:51:03.777515+0000","last_peered":"2024-05-09T04:51:03.777515+0000","last_clean":"2024-05-09T04:51:03.777515+0000","last_became_active":"2024-05-09T04:49:30.159266+0000","last_became_peered":"2024-05-09T04:49:30.159266+0000","last_unstale":"2024-05-09T04:51:03.777515+0000","last_undegraded":"2024-05-09T04:51:03.777515+0000","last_fullsized":"2024-05-09T04:51:03.777515+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:42:04.252862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.541970+0000","last_change":"2024-05-09T04:49:34.150688+0000","last_active":"2024-05-09T04:52:55.541970+0000","last_peered":"2024-05-09T04:52:55.541970+0000","last_clean":"2024-05-09T04:52:55.541970+0000","last_became_active":"2024-05-09T04:49:34.150319+0000","last_became_peered":"2024-05-09T04:49:34.150319+0000","last_unstale":"2024-05-09T04:52:55.541970+0000","last_undegraded":"2024-05-09T04:52:55.541970+0000","last_fullsized":"2024-05-09T04:52:55.541970+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:35:04.832761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700647+0000","last_change":"2024-05-09T04:49:47.902323+0000","last_active":"2024-05-09T04:50:57.700647+0000","last_peered":"2024-05-09T04:50:57.700647+0000","last_clean":"2024-05-09T04:50:57.700647+0000","last_became_active":"2024-05-09T04:49:47.901389+0000","last_became_peered":"2024-05-09T04:49:47.901389+0000","last_unstale":"2024-05-09T04:50:57.700647+0000","last_undegraded":"2024-05-09T04:50:57.700647+0000","last_fullsized":"2024-05-09T04:50:57.700647+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:32.615944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'16","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:42.839922+0000","last_change":"2024-05-09T04:49:30.061172+0000","last_active":"2024-05-09T04:51:42.839922+0000","last_peered":"2024-05-09T04:51:42.839922+0000","last_clean":"2024-05-09T04:51:42.839922+0000","last_became_active":"2024-05-09T04:49:30.060680+0000","last_became_peered":"2024-05-09T04:49:30.060680+0000","last_unstale":"2024-05-09T04:51:42.839922+0000","last_undegraded":"2024-05-09T04:51:42.839922+0000","last_fullsized":"2024-05-09T04:51:42.839922+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:45:44.649124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.551472+0000","last_change":"2024-05-09T04:49:34.130319+0000","last_active":"2024-05-09T04:52:55.551472+0000","last_peered":"2024-05-09T04:52:55.551472+0000","last_clean":"2024-05-09T04:52:55.551472+0000","last_became_active":"2024-05-09T04:49:34.129610+0000","last_became_peered":"2024-05-09T04:49:34.129610+0000","last_unstale":"2024-05-09T04:52:55.551472+0000","last_undegraded":"2024-05-09T04:52:55.551472+0000","last_fullsized":"2024-05-09T04:52:55.551472+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:45.505221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700269+0000","last_change":"2024-05-09T04:49:47.647513+0000","last_active":"2024-05-09T04:50:57.700269+0000","last_peered":"2024-05-09T04:50:57.700269+0000","last_clean":"2024-05-09T04:50:57.700269+0000","last_became_active":"2024-05-09T04:49:47.647069+0000","last_became_peered":"2024-05-09T04:49:47.647069+0000","last_unstale":"2024-05-09T04:50:57.700269+0000","last_undegraded":"2024-05-09T04:50:57.700269+0000","last_fullsized":"2024-05-09T04:50:57.700269+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:05:27.658919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:03.775689+0000","last_change":"2024-05-09T04:49:30.350850+0000","last_active":"2024-05-09T04:51:03.775689+0000","last_peered":"2024-05-09T04:51:03.775689+0000","last_clean":"2024-05-09T04:51:03.775689+0000","last_became_active":"2024-05-09T04:49:30.350431+0000","last_became_peered":"2024-05-09T04:49:30.350431+0000","last_unstale":"2024-05-09T04:51:03.775689+0000","last_undegraded":"2024-05-09T04:51:03.775689+0000","last_fullsized":"2024-05-09T04:51:03.775689+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:53:05.388338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.562337+0000","last_change":"2024-05-09T04:49:34.153630+0000","last_active":"2024-05-09T04:52:55.562337+0000","last_peered":"2024-05-09T04:52:55.562337+0000","last_clean":"2024-05-09T04:52:55.562337+0000","last_became_active":"2024-05-09T04:49:34.152045+0000","last_became_peered":"2024-05-09T04:49:34.152045+0000","last_unstale":"2024-05-09T04:52:55.562337+0000","last_undegraded":"2024-05-09T04:52:55.562337+0000","last_fullsized":"2024-05-09T04:52:55.562337+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:32:35.991731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.202819+0000","last_change":"2024-05-09T04:49:47.901256+0000","last_active":"2024-05-09T04:50:58.202819+0000","last_peered":"2024-05-09T04:50:58.202819+0000","last_clean":"2024-05-09T04:50:58.202819+0000","last_became_active":"2024-05-09T04:49:47.900413+0000","last_became_peered":"2024-05-09T04:49:47.900413+0000","last_unstale":"2024-05-09T04:50:58.202819+0000","last_undegraded":"2024-05-09T04:50:58.202819+0000","last_fullsized":"2024-05-09T04:50:58.202819+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:46:11.845220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:59.742698+0000","last_change":"2024-05-09T04:49:30.152273+0000","last_active":"2024-05-09T04:50:59.742698+0000","last_peered":"2024-05-09T04:50:59.742698+0000","last_clean":"2024-05-09T04:50:59.742698+0000","last_became_active":"2024-05-09T04:49:30.151973+0000","last_became_peered":"2024-05-09T04:49:30.151973+0000","last_unstale":"2024-05-09T04:50:59.742698+0000","last_undegraded":"2024-05-09T04:50:59.742698+0000","last_fullsized":"2024-05-09T04:50:59.742698+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:14:52.912588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.508227+0000","last_change":"2024-05-09T04:49:34.126829+0000","last_active":"2024-05-09T04:52:55.508227+0000","last_peered":"2024-05-09T04:52:55.508227+0000","last_clean":"2024-05-09T04:52:55.508227+0000","last_became_active":"2024-05-09T04:49:34.126430+0000","last_became_peered":"2024-05-09T04:49:34.126430+0000","last_unstale":"2024-05-09T04:52:55.508227+0000","last_undegraded":"2024-05-09T04:52:55.508227+0000","last_fullsized":"2024-05-09T04:52:55.508227+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:38.247148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.706236+0000","last_change":"2024-05-09T04:49:47.675292+0000","last_active":"2024-05-09T04:50:57.706236+0000","last_peered":"2024-05-09T04:50:57.706236+0000","last_clean":"2024-05-09T04:50:57.706236+0000","last_became_active":"2024-05-09T04:49:47.674802+0000","last_became_peered":"2024-05-09T04:49:47.674802+0000","last_unstale":"2024-05-09T04:50:57.706236+0000","last_undegraded":"2024-05-09T04:50:57.706236+0000","last_fullsized":"2024-05-09T04:50:57.706236+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:47:21.988054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:02.769886+0000","last_change":"2024-05-09T04:49:30.057854+0000","last_active":"2024-05-09T04:51:02.769886+0000","last_peered":"2024-05-09T04:51:02.769886+0000","last_clean":"2024-05-09T04:51:02.769886+0000","last_became_active":"2024-05-09T04:49:30.056625+0000","last_became_peered":"2024-05-09T04:49:30.056625+0000","last_unstale":"2024-05-09T04:51:02.769886+0000","last_undegraded":"2024-05-09T04:51:02.769886+0000","last_fullsized":"2024-05-09T04:51:02.769886+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:51.272533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.559278+0000","last_change":"2024-05-09T04:49:34.144414+0000","last_active":"2024-05-09T04:52:55.559278+0000","last_peered":"2024-05-09T04:52:55.559278+0000","last_clean":"2024-05-09T04:52:55.559278+0000","last_became_active":"2024-05-09T04:49:34.144064+0000","last_became_peered":"2024-05-09T04:49:34.144064+0000","last_unstale":"2024-05-09T04:52:55.559278+0000","last_undegraded":"2024-05-09T04:52:55.559278+0000","last_fullsized":"2024-05-09T04:52:55.559278+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:44:40.991576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:23.898074+0000","last_change":"2024-05-09T04:49:47.672956+0000","last_active":"2024-05-09T04:51:23.898074+0000","last_peered":"2024-05-09T04:51:23.898074+0000","last_clean":"2024-05-09T04:51:23.898074+0000","last_became_active":"2024-05-09T04:49:47.672513+0000","last_became_peered":"2024-05-09T04:49:47.672513+0000","last_unstale":"2024-05-09T04:51:23.898074+0000","last_undegraded":"2024-05-09T04:51:23.898074+0000","last_fullsized":"2024-05-09T04:51:23.898074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:32.250039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:02.754199+0000","last_change":"2024-05-09T04:49:56.704058+0000","last_active":"2024-05-09T04:51:02.754199+0000","last_peered":"2024-05-09T04:51:02.754199+0000","last_clean":"2024-05-09T04:51:02.754199+0000","last_became_active":"2024-05-09T04:49:56.703589+0000","last_became_peered":"2024-05-09T04:49:56.703589+0000","last_unstale":"2024-05-09T04:51:02.754199+0000","last_undegraded":"2024-05-09T04:51:02.754199+0000","last_fullsized":"2024-05-09T04:51:02.754199+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:54.596200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,1],"acting":[11,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.555562+0000","last_change":"2024-05-09T04:49:34.159321+0000","last_active":"2024-05-09T04:52:55.555562+0000","last_peered":"2024-05-09T04:52:55.555562+0000","last_clean":"2024-05-09T04:52:55.555562+0000","last_became_active":"2024-05-09T04:49:34.157918+0000","last_became_peered":"2024-05-09T04:49:34.157918+0000","last_unstale":"2024-05-09T04:52:55.555562+0000","last_undegraded":"2024-05-09T04:52:55.555562+0000","last_fullsized":"2024-05-09T04:52:55.555562+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:33:59.659905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.202864+0000","last_change":"2024-05-09T04:49:47.900692+0000","last_active":"2024-05-09T04:50:58.202864+0000","last_peered":"2024-05-09T04:50:58.202864+0000","last_clean":"2024-05-09T04:50:58.202864+0000","last_became_active":"2024-05-09T04:49:47.900095+0000","last_became_peered":"2024-05-09T04:49:47.900095+0000","last_unstale":"2024-05-09T04:50:58.202864+0000","last_undegraded":"2024-05-09T04:50:58.202864+0000","last_fullsized":"2024-05-09T04:50:58.202864+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:08.632408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:03.775992+0000","last_change":"2024-05-09T04:49:30.048768+0000","last_active":"2024-05-09T04:51:03.775992+0000","last_peered":"2024-05-09T04:51:03.775992+0000","last_clean":"2024-05-09T04:51:03.775992+0000","last_became_active":"2024-05-09T04:49:30.048235+0000","last_became_peered":"2024-05-09T04:49:30.048235+0000","last_unstale":"2024-05-09T04:51:03.775992+0000","last_undegraded":"2024-05-09T04:51:03.775992+0000","last_fullsized":"2024-05-09T04:51:03.775992+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:51:16.988726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.539908+0000","last_change":"2024-05-09T04:49:34.142110+0000","last_active":"2024-05-09T04:52:55.539908+0000","last_peered":"2024-05-09T04:52:55.539908+0000","last_clean":"2024-05-09T04:52:55.539908+0000","last_became_active":"2024-05-09T04:49:34.141709+0000","last_became_peered":"2024-05-09T04:49:34.141709+0000","last_unstale":"2024-05-09T04:52:55.539908+0000","last_undegraded":"2024-05-09T04:52:55.539908+0000","last_fullsized":"2024-05-09T04:52:55.539908+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:16.365915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.952118+0000","last_change":"2024-05-09T04:49:47.667784+0000","last_active":"2024-05-09T04:50:57.952118+0000","last_peered":"2024-05-09T04:50:57.952118+0000","last_clean":"2024-05-09T04:50:57.952118+0000","last_became_active":"2024-05-09T04:49:47.667316+0000","last_became_peered":"2024-05-09T04:49:47.667316+0000","last_unstale":"2024-05-09T04:50:57.952118+0000","last_undegraded":"2024-05-09T04:50:57.952118+0000","last_fullsized":"2024-05-09T04:50:57.952118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:04:20.233287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":169,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:53:00.960608+0000","last_change":"2024-05-09T04:49:30.351231+0000","last_active":"2024-05-09T04:53:00.960608+0000","last_peered":"2024-05-09T04:53:00.960608+0000","last_clean":"2024-05-09T04:53:00.960608+0000","last_became_active":"2024-05-09T04:49:30.350817+0000","last_became_peered":"2024-05-09T04:49:30.350817+0000","last_unstale":"2024-05-09T04:53:00.960608+0000","last_undegraded":"2024-05-09T04:53:00.960608+0000","last_fullsized":"2024-05-09T04:53:00.960608+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:51.234300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.540250+0000","last_change":"2024-05-09T04:49:34.129625+0000","last_active":"2024-05-09T04:52:55.540250+0000","last_peered":"2024-05-09T04:52:55.540250+0000","last_clean":"2024-05-09T04:52:55.540250+0000","last_became_active":"2024-05-09T04:49:34.129215+0000","last_became_peered":"2024-05-09T04:49:34.129215+0000","last_unstale":"2024-05-09T04:52:55.540250+0000","last_undegraded":"2024-05-09T04:52:55.540250+0000","last_fullsized":"2024-05-09T04:52:55.540250+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:26.247116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203944+0000","last_change":"2024-05-09T04:49:47.682819+0000","last_active":"2024-05-09T04:50:58.203944+0000","last_peered":"2024-05-09T04:50:58.203944+0000","last_clean":"2024-05-09T04:50:58.203944+0000","last_became_active":"2024-05-09T04:49:47.682331+0000","last_became_peered":"2024-05-09T04:49:47.682331+0000","last_unstale":"2024-05-09T04:50:58.203944+0000","last_undegraded":"2024-05-09T04:50:58.203944+0000","last_fullsized":"2024-05-09T04:50:58.203944+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:03.852975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:03.776136+0000","last_change":"2024-05-09T04:49:30.050247+0000","last_active":"2024-05-09T04:51:03.776136+0000","last_peered":"2024-05-09T04:51:03.776136+0000","last_clean":"2024-05-09T04:51:03.776136+0000","last_became_active":"2024-05-09T04:49:30.049835+0000","last_became_peered":"2024-05-09T04:49:30.049835+0000","last_unstale":"2024-05-09T04:51:03.776136+0000","last_undegraded":"2024-05-09T04:51:03.776136+0000","last_fullsized":"2024-05-09T04:51:03.776136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:52.591186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.559642+0000","last_change":"2024-05-09T04:49:34.145406+0000","last_active":"2024-05-09T04:52:55.559642+0000","last_peered":"2024-05-09T04:52:55.559642+0000","last_clean":"2024-05-09T04:52:55.559642+0000","last_became_active":"2024-05-09T04:49:34.144946+0000","last_became_peered":"2024-05-09T04:49:34.144946+0000","last_unstale":"2024-05-09T04:52:55.559642+0000","last_undegraded":"2024-05-09T04:52:55.559642+0000","last_fullsized":"2024-05-09T04:52:55.559642+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:43:04.704215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.201371+0000","last_change":"2024-05-09T04:49:56.758844+0000","last_active":"2024-05-09T04:50:58.201371+0000","last_peered":"2024-05-09T04:50:58.201371+0000","last_clean":"2024-05-09T04:50:58.201371+0000","last_became_active":"2024-05-09T04:49:56.758335+0000","last_became_peered":"2024-05-09T04:49:56.758335+0000","last_unstale":"2024-05-09T04:50:58.201371+0000","last_undegraded":"2024-05-09T04:50:58.201371+0000","last_fullsized":"2024-05-09T04:50:58.201371+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:39:33.344225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:02.770832+0000","last_change":"2024-05-09T04:49:56.766262+0000","last_active":"2024-05-09T04:51:02.770832+0000","last_peered":"2024-05-09T04:51:02.770832+0000","last_clean":"2024-05-09T04:51:02.770832+0000","last_became_active":"2024-05-09T04:49:56.758765+0000","last_became_peered":"2024-05-09T04:49:56.758765+0000","last_unstale":"2024-05-09T04:51:02.770832+0000","last_undegraded":"2024-05-09T04:51:02.770832+0000","last_fullsized":"2024-05-09T04:51:02.770832+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:44:18.271729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.540798+0000","last_change":"2024-05-09T04:49:34.134957+0000","last_active":"2024-05-09T04:52:55.540798+0000","last_peered":"2024-05-09T04:52:55.540798+0000","last_clean":"2024-05-09T04:52:55.540798+0000","last_became_active":"2024-05-09T04:49:34.134544+0000","last_became_peered":"2024-05-09T04:49:34.134544+0000","last_unstale":"2024-05-09T04:52:55.540798+0000","last_undegraded":"2024-05-09T04:52:55.540798+0000","last_fullsized":"2024-05-09T04:52:55.540798+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:21:54.855905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.520962+0000","last_change":"2024-05-09T04:49:34.138112+0000","last_active":"2024-05-09T04:52:55.520962+0000","last_peered":"2024-05-09T04:52:55.520962+0000","last_clean":"2024-05-09T04:52:55.520962+0000","last_became_active":"2024-05-09T04:49:34.137716+0000","last_became_peered":"2024-05-09T04:49:34.137716+0000","last_unstale":"2024-05-09T04:52:55.520962+0000","last_undegraded":"2024-05-09T04:52:55.520962+0000","last_fullsized":"2024-05-09T04:52:55.520962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:17.028262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.534385+0000","last_change":"2024-05-09T04:50:58.204287+0000","last_active":"2024-05-09T04:52:55.534385+0000","last_peered":"2024-05-09T04:52:55.534385+0000","last_clean":"2024-05-09T04:52:55.534385+0000","last_became_active":"2024-05-09T04:50:58.203894+0000","last_became_peered":"2024-05-09T04:50:58.203894+0000","last_unstale":"2024-05-09T04:52:55.534385+0000","last_undegraded":"2024-05-09T04:52:55.534385+0000","last_fullsized":"2024-05-09T04:52:55.534385+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:02.214347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.558541+0000","last_change":"2024-05-09T04:49:34.147742+0000","last_active":"2024-05-09T04:52:55.558541+0000","last_peered":"2024-05-09T04:52:55.558541+0000","last_clean":"2024-05-09T04:52:55.558541+0000","last_became_active":"2024-05-09T04:49:34.147301+0000","last_became_peered":"2024-05-09T04:49:34.147301+0000","last_unstale":"2024-05-09T04:52:55.558541+0000","last_undegraded":"2024-05-09T04:52:55.558541+0000","last_fullsized":"2024-05-09T04:52:55.558541+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:36:55.371212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.550147+0000","last_change":"2024-05-09T04:49:34.163599+0000","last_active":"2024-05-09T04:52:55.550147+0000","last_peered":"2024-05-09T04:52:55.550147+0000","last_clean":"2024-05-09T04:52:55.550147+0000","last_became_active":"2024-05-09T04:49:34.162898+0000","last_became_peered":"2024-05-09T04:49:34.162898+0000","last_unstale":"2024-05-09T04:52:55.550147+0000","last_undegraded":"2024-05-09T04:52:55.550147+0000","last_fullsized":"2024-05-09T04:52:55.550147+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:55:57.213901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.552305+0000","last_change":"2024-05-09T04:49:34.151648+0000","last_active":"2024-05-09T04:52:55.552305+0000","last_peered":"2024-05-09T04:52:55.552305+0000","last_clean":"2024-05-09T04:52:55.552305+0000","last_became_active":"2024-05-09T04:49:34.148701+0000","last_became_peered":"2024-05-09T04:49:34.148701+0000","last_unstale":"2024-05-09T04:52:55.552305+0000","last_undegraded":"2024-05-09T04:52:55.552305+0000","last_fullsized":"2024-05-09T04:52:55.552305+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:54:30.122252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.516835+0000","last_change":"2024-05-09T04:49:34.136679+0000","last_active":"2024-05-09T04:52:55.516835+0000","last_peered":"2024-05-09T04:52:55.516835+0000","last_clean":"2024-05-09T04:52:55.516835+0000","last_became_active":"2024-05-09T04:49:34.136306+0000","last_became_peered":"2024-05-09T04:49:34.136306+0000","last_unstale":"2024-05-09T04:52:55.516835+0000","last_undegraded":"2024-05-09T04:52:55.516835+0000","last_fullsized":"2024-05-09T04:52:55.516835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:51:39.476564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.538881+0000","last_change":"2024-05-09T04:49:34.130769+0000","last_active":"2024-05-09T04:52:55.538881+0000","last_peered":"2024-05-09T04:52:55.538881+0000","last_clean":"2024-05-09T04:52:55.538881+0000","last_became_active":"2024-05-09T04:49:34.130362+0000","last_became_peered":"2024-05-09T04:49:34.130362+0000","last_unstale":"2024-05-09T04:52:55.538881+0000","last_undegraded":"2024-05-09T04:52:55.538881+0000","last_fullsized":"2024-05-09T04:52:55.538881+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:54.942259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.552404+0000","last_change":"2024-05-09T04:49:34.138054+0000","last_active":"2024-05-09T04:52:55.552404+0000","last_peered":"2024-05-09T04:52:55.552404+0000","last_clean":"2024-05-09T04:52:55.552404+0000","last_became_active":"2024-05-09T04:49:34.136111+0000","last_became_peered":"2024-05-09T04:49:34.136111+0000","last_unstale":"2024-05-09T04:52:55.552404+0000","last_undegraded":"2024-05-09T04:52:55.552404+0000","last_fullsized":"2024-05-09T04:52:55.552404+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:15:27.870874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.562236+0000","last_change":"2024-05-09T04:49:34.146370+0000","last_active":"2024-05-09T04:52:55.562236+0000","last_peered":"2024-05-09T04:52:55.562236+0000","last_clean":"2024-05-09T04:52:55.562236+0000","last_became_active":"2024-05-09T04:49:34.145948+0000","last_became_peered":"2024-05-09T04:49:34.145948+0000","last_unstale":"2024-05-09T04:52:55.562236+0000","last_undegraded":"2024-05-09T04:52:55.562236+0000","last_fullsized":"2024-05-09T04:52:55.562236+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:14.826168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.555257+0000","last_change":"2024-05-09T04:49:34.138028+0000","last_active":"2024-05-09T04:52:55.555257+0000","last_peered":"2024-05-09T04:52:55.555257+0000","last_clean":"2024-05-09T04:52:55.555257+0000","last_became_active":"2024-05-09T04:49:34.137558+0000","last_became_peered":"2024-05-09T04:49:34.137558+0000","last_unstale":"2024-05-09T04:52:55.555257+0000","last_undegraded":"2024-05-09T04:52:55.555257+0000","last_fullsized":"2024-05-09T04:52:55.555257+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:50:16.258945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.552311+0000","last_change":"2024-05-09T04:49:34.137946+0000","last_active":"2024-05-09T04:52:55.552311+0000","last_peered":"2024-05-09T04:52:55.552311+0000","last_clean":"2024-05-09T04:52:55.552311+0000","last_became_active":"2024-05-09T04:49:34.137498+0000","last_became_peered":"2024-05-09T04:49:34.137498+0000","last_unstale":"2024-05-09T04:52:55.552311+0000","last_undegraded":"2024-05-09T04:52:55.552311+0000","last_fullsized":"2024-05-09T04:52:55.552311+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:36:54.473689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.551855+0000","last_change":"2024-05-09T04:49:34.162103+0000","last_active":"2024-05-09T04:52:55.551855+0000","last_peered":"2024-05-09T04:52:55.551855+0000","last_clean":"2024-05-09T04:52:55.551855+0000","last_became_active":"2024-05-09T04:49:34.161681+0000","last_became_peered":"2024-05-09T04:49:34.161681+0000","last_unstale":"2024-05-09T04:52:55.551855+0000","last_undegraded":"2024-05-09T04:52:55.551855+0000","last_fullsized":"2024-05-09T04:52:55.551855+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:36:10.460401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.528201+0000","last_change":"2024-05-09T04:49:34.136538+0000","last_active":"2024-05-09T04:52:55.528201+0000","last_peered":"2024-05-09T04:52:55.528201+0000","last_clean":"2024-05-09T04:52:55.528201+0000","last_became_active":"2024-05-09T04:49:34.134903+0000","last_became_peered":"2024-05-09T04:49:34.134903+0000","last_unstale":"2024-05-09T04:52:55.528201+0000","last_undegraded":"2024-05-09T04:52:55.528201+0000","last_fullsized":"2024-05-09T04:52:55.528201+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:44:48.673600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71975,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1222,"num_read_kb":693,"num_write":259,"num_write_kb":151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":215925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":224,"ondisk_log_size":224,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678075,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":510251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874305,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37016,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708136,"statfs":{"total":95995035648,"available":95957131264,"internally_reserved":0,"allocated":1720320,"data_stored":1161976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037830,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":514165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234062,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":510335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430290,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1642496,"data_stored":1107617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593817,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":505573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":571092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738491,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":507029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411443,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1114112,"data_stored":571844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149803,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":505281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953574,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":510179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790047,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1650688,"data_stored":1102667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T04:53:04.607 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T04:53:05.088 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:53:05.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:04 smithi006 ceph-mon[29835]: pgmap v443: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-09T04:53:05.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:04 smithi006 ceph-mon[29835]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:53:05.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:04 smithi186 ceph-mon[34668]: pgmap v443: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-09T04:53:05.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:04 smithi186 ceph-mon[34668]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:53:05.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:04 smithi169 ceph-mon[34075]: pgmap v443: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-09T04:53:05.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:04 smithi169 ceph-mon[34075]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:53:07.108 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:06 smithi006 ceph-mon[29835]: pgmap v444: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-09T04:53:07.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:06 smithi186 ceph-mon[34668]: pgmap v444: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-09T04:53:07.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.358+0000 7f4b0cc00700 1 -- 172.21.15.6:0/2531800230 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b0810e920 msgr2=0x7f4b08110e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:07.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.358+0000 7f4b0cc00700 1 --2- 172.21.15.6:0/2531800230 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b0810e920 0x7f4b08110e00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000b0a0 tx=0x7f4b0000b3b0 comp rx=0 tx=0).stop 2024-05-09T04:53:07.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.358+0000 7f4b0cc00700 1 -- 172.21.15.6:0/2531800230 shutdown_connections 2024-05-09T04:53:07.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.358+0000 7f4b0cc00700 1 --2- 172.21.15.6:0/2531800230 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b0810e920 0x7f4b08110e00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:07.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.358+0000 7f4b0cc00700 1 --2- 172.21.15.6:0/2531800230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b081054f0 0x7f4b08105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:07.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.358+0000 7f4b0cc00700 1 --2- 172.21.15.6:0/2531800230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b08105e40 0x7f4b0810e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:07.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.358+0000 7f4b0cc00700 1 -- 172.21.15.6:0/2531800230 >> 172.21.15.6:0/2531800230 conn(0x7f4b08078e00 msgr2=0x7f4b08079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:07.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.358+0000 7f4b0cc00700 1 -- 172.21.15.6:0/2531800230 shutdown_connections 2024-05-09T04:53:07.359 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.358+0000 7f4b0cc00700 1 -- 172.21.15.6:0/2531800230 wait complete. 2024-05-09T04:53:07.360 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.360+0000 7f4b0cc00700 1 Processor -- start 2024-05-09T04:53:07.360 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.360+0000 7f4b0cc00700 1 -- start start 2024-05-09T04:53:07.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.360+0000 7f4b0cc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b081054f0 0x7f4b0819cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:07.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.360+0000 7f4b0cc00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b08105e40 0x7f4b0819d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:07.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.360+0000 7f4b0cc00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b0810e920 0x7f4b081a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:07.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.360+0000 7f4b0cc00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4b081144d0 con 0x7f4b0810e920 2024-05-09T04:53:07.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.360+0000 7f4b0cc00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4b08114350 con 0x7f4b08105e40 2024-05-09T04:53:07.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.360+0000 7f4b0cc00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4b08114650 con 0x7f4b081054f0 2024-05-09T04:53:07.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.361+0000 7f4b06800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b0810e920 0x7f4b081a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:07.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.361+0000 7f4b05e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b081054f0 0x7f4b0819cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:07.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.361+0000 7f4b06800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b0810e920 0x7f4b081a1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54962/0 (socket says 172.21.15.6:54962) 2024-05-09T04:53:07.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.361+0000 7f4b06800700 1 -- 172.21.15.6:0/2696795503 learned_addr learned my addr 172.21.15.6:0/2696795503 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:07.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.361+0000 7f4b05400700 1 --2- 172.21.15.6:0/2696795503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b08105e40 0x7f4b0819d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:07.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.361+0000 7f4b06800700 1 -- 172.21.15.6:0/2696795503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b081054f0 msgr2=0x7f4b0819cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:07.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.361+0000 7f4b06800700 1 --2- 172.21.15.6:0/2696795503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b081054f0 0x7f4b0819cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:07.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.361+0000 7f4b06800700 1 -- 172.21.15.6:0/2696795503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b08105e40 msgr2=0x7f4b0819d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:07.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.361+0000 7f4b06800700 1 --2- 172.21.15.6:0/2696795503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b08105e40 0x7f4b0819d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:07.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.361+0000 7f4b06800700 1 -- 172.21.15.6:0/2696795503 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b081a1eb0 con 0x7f4b0810e920 2024-05-09T04:53:07.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.361+0000 7f4b05e00700 1 --2- 172.21.15.6:0/2696795503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b081054f0 0x7f4b0819cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:53:07.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.362+0000 7f4b06800700 1 --2- 172.21.15.6:0/2696795503 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b0810e920 0x7f4b081a1810 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000b070 tx=0x7f4b00004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:07.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.362+0000 7f4afea00700 1 -- 172.21.15.6:0/2696795503 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00021070 con 0x7f4b0810e920 2024-05-09T04:53:07.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.362+0000 7f4afea00700 1 -- 172.21.15.6:0/2696795503 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b00007e30 con 0x7f4b0810e920 2024-05-09T04:53:07.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.362+0000 7f4b0cc00700 1 -- 172.21.15.6:0/2696795503 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b081a2140 con 0x7f4b0810e920 2024-05-09T04:53:07.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.362+0000 7f4b0cc00700 1 -- 172.21.15.6:0/2696795503 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b081a9ac0 con 0x7f4b0810e920 2024-05-09T04:53:07.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.362+0000 7f4afea00700 1 -- 172.21.15.6:0/2696795503 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b000046d0 con 0x7f4b0810e920 2024-05-09T04:53:07.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.363+0000 7f4b0cc00700 1 -- 172.21.15.6:0/2696795503 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b080512c0 con 0x7f4b0810e920 2024-05-09T04:53:07.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:06 smithi169 ceph-mon[34075]: pgmap v444: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-09T04:53:07.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.366+0000 7f4afea00700 1 -- 172.21.15.6:0/2696795503 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f4b00007980 con 0x7f4b0810e920 2024-05-09T04:53:07.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.368+0000 7f4afea00700 1 --2- 172.21.15.6:0/2696795503 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4aec0b2880 0x7f4aec0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:07.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.368+0000 7f4afea00700 1 -- 172.21.15.6:0/2696795503 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f4b0000f020 con 0x7f4b0810e920 2024-05-09T04:53:07.369 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.368+0000 7f4b05e00700 1 --2- 172.21.15.6:0/2696795503 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4aec0b2880 0x7f4aec0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:07.369 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.369+0000 7f4b05e00700 1 --2- 172.21.15.6:0/2696795503 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4aec0b2880 0x7f4aec0b4d90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4af8011490 tx=0x7f4af8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:07.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.371+0000 7f4afea00700 1 -- 172.21.15.6:0/2696795503 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4b0002a020 con 0x7f4b0810e920 2024-05-09T04:53:07.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.605+0000 7f4b0cc00700 1 -- 172.21.15.6:0/2696795503 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4b08102560 con 0x7f4aec0b2880 2024-05-09T04:53:07.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.616+0000 7f4afea00700 1 -- 172.21.15.6:0/2696795503 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340530 (secure 0 0 0) 0x7f4b08102560 con 0x7f4aec0b2880 2024-05-09T04:53:07.623 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:53:07.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.623+0000 7f4b0cc00700 1 -- 172.21.15.6:0/2696795503 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4aec0b2880 msgr2=0x7f4aec0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:07.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.623+0000 7f4b0cc00700 1 --2- 172.21.15.6:0/2696795503 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4aec0b2880 0x7f4aec0b4d90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4af8011490 tx=0x7f4af8009260 comp rx=0 tx=0).stop 2024-05-09T04:53:07.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.623+0000 7f4b0cc00700 1 -- 172.21.15.6:0/2696795503 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b0810e920 msgr2=0x7f4b081a1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:07.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.623+0000 7f4b0cc00700 1 --2- 172.21.15.6:0/2696795503 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b0810e920 0x7f4b081a1810 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000b070 tx=0x7f4b00004cf0 comp rx=0 tx=0).stop 2024-05-09T04:53:07.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.623+0000 7f4b0cc00700 1 -- 172.21.15.6:0/2696795503 shutdown_connections 2024-05-09T04:53:07.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.623+0000 7f4b0cc00700 1 --2- 172.21.15.6:0/2696795503 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b0810e920 0x7f4b081a1810 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:07.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.623+0000 7f4b0cc00700 1 --2- 172.21.15.6:0/2696795503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b081054f0 0x7f4b0819cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:07.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.623+0000 7f4b0cc00700 1 --2- 172.21.15.6:0/2696795503 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4aec0b2880 0x7f4aec0b4d90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:07.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.623+0000 7f4b0cc00700 1 --2- 172.21.15.6:0/2696795503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b08105e40 0x7f4b0819d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:07.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.623+0000 7f4b0cc00700 1 -- 172.21.15.6:0/2696795503 >> 172.21.15.6:0/2696795503 conn(0x7f4b08078e00 msgr2=0x7f4b08101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:07.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.623+0000 7f4b0cc00700 1 -- 172.21.15.6:0/2696795503 shutdown_connections 2024-05-09T04:53:07.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:07.624+0000 7f4b0cc00700 1 -- 172.21.15.6:0/2696795503 wait complete. 2024-05-09T04:53:07.626 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-05-09T04:53:08.493 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":444,"stamp":"2024-05-09T04:53:05.640992+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662577,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1323,"num_read_kb":772,"num_write":384,"num_write_kb":2303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":818,"ondisk_log_size":818,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":367456,"kb_used_data":14144,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124574368,"statfs":{"total":1151940427776,"available":1151564152832,"internally_reserved":0,"allocated":14483456,"data_stored":8078009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19137,"internal_metadata":361674047},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":531,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":68,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006583"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.559481+0000","last_change":"2024-05-09T04:49:34.144921+0000","last_active":"2024-05-09T04:52:55.559481+0000","last_peered":"2024-05-09T04:52:55.559481+0000","last_clean":"2024-05-09T04:52:55.559481+0000","last_became_active":"2024-05-09T04:49:34.144275+0000","last_became_peered":"2024-05-09T04:49:34.144275+0000","last_unstale":"2024-05-09T04:52:55.559481+0000","last_undegraded":"2024-05-09T04:52:55.559481+0000","last_fullsized":"2024-05-09T04:52:55.559481+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:00:31.377738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.558665+0000","last_change":"2024-05-09T04:49:34.158057+0000","last_active":"2024-05-09T04:52:55.558665+0000","last_peered":"2024-05-09T04:52:55.558665+0000","last_clean":"2024-05-09T04:52:55.558665+0000","last_became_active":"2024-05-09T04:49:34.157129+0000","last_became_peered":"2024-05-09T04:49:34.157129+0000","last_unstale":"2024-05-09T04:52:55.558665+0000","last_undegraded":"2024-05-09T04:52:55.558665+0000","last_fullsized":"2024-05-09T04:52:55.558665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:49.360309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.554048+0000","last_change":"2024-05-09T04:49:34.145519+0000","last_active":"2024-05-09T04:52:55.554048+0000","last_peered":"2024-05-09T04:52:55.554048+0000","last_clean":"2024-05-09T04:52:55.554048+0000","last_became_active":"2024-05-09T04:49:34.144501+0000","last_became_peered":"2024-05-09T04:49:34.144501+0000","last_unstale":"2024-05-09T04:52:55.554048+0000","last_undegraded":"2024-05-09T04:52:55.554048+0000","last_fullsized":"2024-05-09T04:52:55.554048+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:05:53.727520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.521442+0000","last_change":"2024-05-09T04:49:34.134383+0000","last_active":"2024-05-09T04:52:55.521442+0000","last_peered":"2024-05-09T04:52:55.521442+0000","last_clean":"2024-05-09T04:52:55.521442+0000","last_became_active":"2024-05-09T04:49:34.134049+0000","last_became_peered":"2024-05-09T04:49:34.134049+0000","last_unstale":"2024-05-09T04:52:55.521442+0000","last_undegraded":"2024-05-09T04:52:55.521442+0000","last_fullsized":"2024-05-09T04:52:55.521442+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:36:02.548188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.535651+0000","last_change":"2024-05-09T04:49:34.157897+0000","last_active":"2024-05-09T04:52:55.535651+0000","last_peered":"2024-05-09T04:52:55.535651+0000","last_clean":"2024-05-09T04:52:55.535651+0000","last_became_active":"2024-05-09T04:49:34.157516+0000","last_became_peered":"2024-05-09T04:49:34.157516+0000","last_unstale":"2024-05-09T04:52:55.535651+0000","last_undegraded":"2024-05-09T04:52:55.535651+0000","last_fullsized":"2024-05-09T04:52:55.535651+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:33:25.444155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.535109+0000","last_change":"2024-05-09T04:49:34.146389+0000","last_active":"2024-05-09T04:52:55.535109+0000","last_peered":"2024-05-09T04:52:55.535109+0000","last_clean":"2024-05-09T04:52:55.535109+0000","last_became_active":"2024-05-09T04:49:34.145994+0000","last_became_peered":"2024-05-09T04:49:34.145994+0000","last_unstale":"2024-05-09T04:52:55.535109+0000","last_undegraded":"2024-05-09T04:52:55.535109+0000","last_fullsized":"2024-05-09T04:52:55.535109+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:59.094826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.537220+0000","last_change":"2024-05-09T04:49:34.146428+0000","last_active":"2024-05-09T04:52:55.537220+0000","last_peered":"2024-05-09T04:52:55.537220+0000","last_clean":"2024-05-09T04:52:55.537220+0000","last_became_active":"2024-05-09T04:49:34.146045+0000","last_became_peered":"2024-05-09T04:49:34.146045+0000","last_unstale":"2024-05-09T04:52:55.537220+0000","last_undegraded":"2024-05-09T04:52:55.537220+0000","last_fullsized":"2024-05-09T04:52:55.537220+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:13.129474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.559673+0000","last_change":"2024-05-09T04:49:34.135494+0000","last_active":"2024-05-09T04:52:55.559673+0000","last_peered":"2024-05-09T04:52:55.559673+0000","last_clean":"2024-05-09T04:52:55.559673+0000","last_became_active":"2024-05-09T04:49:34.135085+0000","last_became_peered":"2024-05-09T04:49:34.135085+0000","last_unstale":"2024-05-09T04:52:55.559673+0000","last_undegraded":"2024-05-09T04:52:55.559673+0000","last_fullsized":"2024-05-09T04:52:55.559673+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:57.023344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.541489+0000","last_change":"2024-05-09T04:49:34.131353+0000","last_active":"2024-05-09T04:52:55.541489+0000","last_peered":"2024-05-09T04:52:55.541489+0000","last_clean":"2024-05-09T04:52:55.541489+0000","last_became_active":"2024-05-09T04:49:34.130954+0000","last_became_peered":"2024-05-09T04:49:34.130954+0000","last_unstale":"2024-05-09T04:52:55.541489+0000","last_undegraded":"2024-05-09T04:52:55.541489+0000","last_fullsized":"2024-05-09T04:52:55.541489+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:04:46.947861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.526736+0000","last_change":"2024-05-09T04:49:34.134733+0000","last_active":"2024-05-09T04:52:55.526736+0000","last_peered":"2024-05-09T04:52:55.526736+0000","last_clean":"2024-05-09T04:52:55.526736+0000","last_became_active":"2024-05-09T04:49:34.134337+0000","last_became_peered":"2024-05-09T04:49:34.134337+0000","last_unstale":"2024-05-09T04:52:55.526736+0000","last_undegraded":"2024-05-09T04:52:55.526736+0000","last_fullsized":"2024-05-09T04:52:55.526736+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:05.452488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.540741+0000","last_change":"2024-05-09T04:49:34.139143+0000","last_active":"2024-05-09T04:52:55.540741+0000","last_peered":"2024-05-09T04:52:55.540741+0000","last_clean":"2024-05-09T04:52:55.540741+0000","last_became_active":"2024-05-09T04:49:34.138749+0000","last_became_peered":"2024-05-09T04:49:34.138749+0000","last_unstale":"2024-05-09T04:52:55.540741+0000","last_undegraded":"2024-05-09T04:52:55.540741+0000","last_fullsized":"2024-05-09T04:52:55.540741+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:11.552063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.524212+0000","last_change":"2024-05-09T04:49:34.134687+0000","last_active":"2024-05-09T04:52:55.524212+0000","last_peered":"2024-05-09T04:52:55.524212+0000","last_clean":"2024-05-09T04:52:55.524212+0000","last_became_active":"2024-05-09T04:49:34.134324+0000","last_became_peered":"2024-05-09T04:49:34.134324+0000","last_unstale":"2024-05-09T04:52:55.524212+0000","last_undegraded":"2024-05-09T04:52:55.524212+0000","last_fullsized":"2024-05-09T04:52:55.524212+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:11.667883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.542595+0000","last_change":"2024-05-09T04:49:34.153592+0000","last_active":"2024-05-09T04:52:55.542595+0000","last_peered":"2024-05-09T04:52:55.542595+0000","last_clean":"2024-05-09T04:52:55.542595+0000","last_became_active":"2024-05-09T04:49:34.153184+0000","last_became_peered":"2024-05-09T04:49:34.153184+0000","last_unstale":"2024-05-09T04:52:55.542595+0000","last_undegraded":"2024-05-09T04:52:55.542595+0000","last_fullsized":"2024-05-09T04:52:55.542595+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:36:56.784781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.550285+0000","last_change":"2024-05-09T04:49:34.144046+0000","last_active":"2024-05-09T04:52:55.550285+0000","last_peered":"2024-05-09T04:52:55.550285+0000","last_clean":"2024-05-09T04:52:55.550285+0000","last_became_active":"2024-05-09T04:49:34.141004+0000","last_became_peered":"2024-05-09T04:49:34.141004+0000","last_unstale":"2024-05-09T04:52:55.550285+0000","last_undegraded":"2024-05-09T04:52:55.550285+0000","last_fullsized":"2024-05-09T04:52:55.550285+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:09:12.274750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.554450+0000","last_change":"2024-05-09T04:49:34.147063+0000","last_active":"2024-05-09T04:52:55.554450+0000","last_peered":"2024-05-09T04:52:55.554450+0000","last_clean":"2024-05-09T04:52:55.554450+0000","last_became_active":"2024-05-09T04:49:34.144934+0000","last_became_peered":"2024-05-09T04:49:34.144934+0000","last_unstale":"2024-05-09T04:52:55.554450+0000","last_undegraded":"2024-05-09T04:52:55.554450+0000","last_fullsized":"2024-05-09T04:52:55.554450+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:47:59.125451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.540390+0000","last_change":"2024-05-09T04:49:34.141070+0000","last_active":"2024-05-09T04:52:55.540390+0000","last_peered":"2024-05-09T04:52:55.540390+0000","last_clean":"2024-05-09T04:52:55.540390+0000","last_became_active":"2024-05-09T04:49:34.140618+0000","last_became_peered":"2024-05-09T04:49:34.140618+0000","last_unstale":"2024-05-09T04:52:55.540390+0000","last_undegraded":"2024-05-09T04:52:55.540390+0000","last_fullsized":"2024-05-09T04:52:55.540390+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:02.435768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.559611+0000","last_change":"2024-05-09T04:49:34.144434+0000","last_active":"2024-05-09T04:52:55.559611+0000","last_peered":"2024-05-09T04:52:55.559611+0000","last_clean":"2024-05-09T04:52:55.559611+0000","last_became_active":"2024-05-09T04:49:34.144052+0000","last_became_peered":"2024-05-09T04:49:34.144052+0000","last_unstale":"2024-05-09T04:52:55.559611+0000","last_undegraded":"2024-05-09T04:52:55.559611+0000","last_fullsized":"2024-05-09T04:52:55.559611+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:05:26.141627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.546642+0000","last_change":"2024-05-09T04:49:34.148681+0000","last_active":"2024-05-09T04:52:55.546642+0000","last_peered":"2024-05-09T04:52:55.546642+0000","last_clean":"2024-05-09T04:52:55.546642+0000","last_became_active":"2024-05-09T04:49:34.145277+0000","last_became_peered":"2024-05-09T04:49:34.145277+0000","last_unstale":"2024-05-09T04:52:55.546642+0000","last_undegraded":"2024-05-09T04:52:55.546642+0000","last_fullsized":"2024-05-09T04:52:55.546642+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:09.546998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.548479+0000","last_change":"2024-05-09T04:49:34.153229+0000","last_active":"2024-05-09T04:52:55.548479+0000","last_peered":"2024-05-09T04:52:55.548479+0000","last_clean":"2024-05-09T04:52:55.548479+0000","last_became_active":"2024-05-09T04:49:34.152869+0000","last_became_peered":"2024-05-09T04:49:34.152869+0000","last_unstale":"2024-05-09T04:52:55.548479+0000","last_undegraded":"2024-05-09T04:52:55.548479+0000","last_fullsized":"2024-05-09T04:52:55.548479+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:54.350842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700390+0000","last_change":"2024-05-09T04:49:56.710411+0000","last_active":"2024-05-09T04:50:57.700390+0000","last_peered":"2024-05-09T04:50:57.700390+0000","last_clean":"2024-05-09T04:50:57.700390+0000","last_became_active":"2024-05-09T04:49:56.709903+0000","last_became_peered":"2024-05-09T04:49:56.709903+0000","last_unstale":"2024-05-09T04:50:57.700390+0000","last_undegraded":"2024-05-09T04:50:57.700390+0000","last_fullsized":"2024-05-09T04:50:57.700390+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:53:44.216648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,10,0],"acting":[2,3,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":175,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:53:01.052946+0000","last_change":"2024-05-09T04:49:30.061766+0000","last_active":"2024-05-09T04:53:01.052946+0000","last_peered":"2024-05-09T04:53:01.052946+0000","last_clean":"2024-05-09T04:53:01.052946+0000","last_became_active":"2024-05-09T04:49:30.060545+0000","last_became_peered":"2024-05-09T04:49:30.060545+0000","last_unstale":"2024-05-09T04:53:01.052946+0000","last_undegraded":"2024-05-09T04:53:01.052946+0000","last_fullsized":"2024-05-09T04:53:01.052946+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:15:03.857107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.552044+0000","last_change":"2024-05-09T04:49:34.140706+0000","last_active":"2024-05-09T04:52:55.552044+0000","last_peered":"2024-05-09T04:52:55.552044+0000","last_clean":"2024-05-09T04:52:55.552044+0000","last_became_active":"2024-05-09T04:49:34.139032+0000","last_became_peered":"2024-05-09T04:49:34.139032+0000","last_unstale":"2024-05-09T04:52:55.552044+0000","last_undegraded":"2024-05-09T04:52:55.552044+0000","last_fullsized":"2024-05-09T04:52:55.552044+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:19.878523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.201694+0000","last_change":"2024-05-09T04:49:47.674254+0000","last_active":"2024-05-09T04:50:58.201694+0000","last_peered":"2024-05-09T04:50:58.201694+0000","last_clean":"2024-05-09T04:50:58.201694+0000","last_became_active":"2024-05-09T04:49:47.673476+0000","last_became_peered":"2024-05-09T04:49:47.673476+0000","last_unstale":"2024-05-09T04:50:58.201694+0000","last_undegraded":"2024-05-09T04:50:58.201694+0000","last_fullsized":"2024-05-09T04:50:58.201694+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:41:31.709210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:00.747787+0000","last_change":"2024-05-09T04:49:30.154162+0000","last_active":"2024-05-09T04:51:00.747787+0000","last_peered":"2024-05-09T04:51:00.747787+0000","last_clean":"2024-05-09T04:51:00.747787+0000","last_became_active":"2024-05-09T04:49:30.152869+0000","last_became_peered":"2024-05-09T04:49:30.152869+0000","last_unstale":"2024-05-09T04:51:00.747787+0000","last_undegraded":"2024-05-09T04:51:00.747787+0000","last_fullsized":"2024-05-09T04:51:00.747787+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:43.433033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.563131+0000","last_change":"2024-05-09T04:49:34.157729+0000","last_active":"2024-05-09T04:52:55.563131+0000","last_peered":"2024-05-09T04:52:55.563131+0000","last_clean":"2024-05-09T04:52:55.563131+0000","last_became_active":"2024-05-09T04:49:34.155040+0000","last_became_peered":"2024-05-09T04:49:34.155040+0000","last_unstale":"2024-05-09T04:52:55.563131+0000","last_undegraded":"2024-05-09T04:52:55.563131+0000","last_fullsized":"2024-05-09T04:52:55.563131+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:43:30.104394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.202328+0000","last_change":"2024-05-09T04:49:47.679665+0000","last_active":"2024-05-09T04:50:58.202328+0000","last_peered":"2024-05-09T04:50:58.202328+0000","last_clean":"2024-05-09T04:50:58.202328+0000","last_became_active":"2024-05-09T04:49:47.679257+0000","last_became_peered":"2024-05-09T04:49:47.679257+0000","last_unstale":"2024-05-09T04:50:58.202328+0000","last_undegraded":"2024-05-09T04:50:58.202328+0000","last_fullsized":"2024-05-09T04:50:58.202328+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:53.111205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:04.762348+0000","last_change":"2024-05-09T04:49:30.047386+0000","last_active":"2024-05-09T04:51:04.762348+0000","last_peered":"2024-05-09T04:51:04.762348+0000","last_clean":"2024-05-09T04:51:04.762348+0000","last_became_active":"2024-05-09T04:49:30.046947+0000","last_became_peered":"2024-05-09T04:49:30.046947+0000","last_unstale":"2024-05-09T04:51:04.762348+0000","last_undegraded":"2024-05-09T04:51:04.762348+0000","last_fullsized":"2024-05-09T04:51:04.762348+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:46.861354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.545354+0000","last_change":"2024-05-09T04:49:34.153320+0000","last_active":"2024-05-09T04:52:55.545354+0000","last_peered":"2024-05-09T04:52:55.545354+0000","last_clean":"2024-05-09T04:52:55.545354+0000","last_became_active":"2024-05-09T04:49:34.152854+0000","last_became_peered":"2024-05-09T04:49:34.152854+0000","last_unstale":"2024-05-09T04:52:55.545354+0000","last_undegraded":"2024-05-09T04:52:55.545354+0000","last_fullsized":"2024-05-09T04:52:55.545354+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:40.801340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.202375+0000","last_change":"2024-05-09T04:49:47.668475+0000","last_active":"2024-05-09T04:50:58.202375+0000","last_peered":"2024-05-09T04:50:58.202375+0000","last_clean":"2024-05-09T04:50:58.202375+0000","last_became_active":"2024-05-09T04:49:47.667909+0000","last_became_peered":"2024-05-09T04:49:47.667909+0000","last_unstale":"2024-05-09T04:50:58.202375+0000","last_undegraded":"2024-05-09T04:50:58.202375+0000","last_fullsized":"2024-05-09T04:50:58.202375+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:39:27.687468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:03.783186+0000","last_change":"2024-05-09T04:49:30.048659+0000","last_active":"2024-05-09T04:51:03.783186+0000","last_peered":"2024-05-09T04:51:03.783186+0000","last_clean":"2024-05-09T04:51:03.783186+0000","last_became_active":"2024-05-09T04:49:30.048259+0000","last_became_peered":"2024-05-09T04:49:30.048259+0000","last_unstale":"2024-05-09T04:51:03.783186+0000","last_undegraded":"2024-05-09T04:51:03.783186+0000","last_fullsized":"2024-05-09T04:51:03.783186+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:37:46.012594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.539299+0000","last_change":"2024-05-09T04:49:34.131416+0000","last_active":"2024-05-09T04:52:55.539299+0000","last_peered":"2024-05-09T04:52:55.539299+0000","last_clean":"2024-05-09T04:52:55.539299+0000","last_became_active":"2024-05-09T04:49:34.130985+0000","last_became_peered":"2024-05-09T04:49:34.130985+0000","last_unstale":"2024-05-09T04:52:55.539299+0000","last_undegraded":"2024-05-09T04:52:55.539299+0000","last_fullsized":"2024-05-09T04:52:55.539299+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:02:43.228142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700686+0000","last_change":"2024-05-09T04:49:47.652247+0000","last_active":"2024-05-09T04:50:57.700686+0000","last_peered":"2024-05-09T04:50:57.700686+0000","last_clean":"2024-05-09T04:50:57.700686+0000","last_became_active":"2024-05-09T04:49:47.651747+0000","last_became_peered":"2024-05-09T04:49:47.651747+0000","last_unstale":"2024-05-09T04:50:57.700686+0000","last_undegraded":"2024-05-09T04:50:57.700686+0000","last_fullsized":"2024-05-09T04:50:57.700686+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:45.353027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":171,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:58.957560+0000","last_change":"2024-05-09T04:49:30.047421+0000","last_active":"2024-05-09T04:52:58.957560+0000","last_peered":"2024-05-09T04:52:58.957560+0000","last_clean":"2024-05-09T04:52:58.957560+0000","last_became_active":"2024-05-09T04:49:30.047007+0000","last_became_peered":"2024-05-09T04:49:30.047007+0000","last_unstale":"2024-05-09T04:52:58.957560+0000","last_undegraded":"2024-05-09T04:52:58.957560+0000","last_fullsized":"2024-05-09T04:52:58.957560+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:23.930326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.533380+0000","last_change":"2024-05-09T04:49:34.146605+0000","last_active":"2024-05-09T04:52:55.533380+0000","last_peered":"2024-05-09T04:52:55.533380+0000","last_clean":"2024-05-09T04:52:55.533380+0000","last_became_active":"2024-05-09T04:49:34.146205+0000","last_became_peered":"2024-05-09T04:49:34.146205+0000","last_unstale":"2024-05-09T04:52:55.533380+0000","last_undegraded":"2024-05-09T04:52:55.533380+0000","last_fullsized":"2024-05-09T04:52:55.533380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:13:52.234951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700734+0000","last_change":"2024-05-09T04:49:47.673652+0000","last_active":"2024-05-09T04:50:57.700734+0000","last_peered":"2024-05-09T04:50:57.700734+0000","last_clean":"2024-05-09T04:50:57.700734+0000","last_became_active":"2024-05-09T04:49:47.673255+0000","last_became_peered":"2024-05-09T04:49:47.673255+0000","last_unstale":"2024-05-09T04:50:57.700734+0000","last_undegraded":"2024-05-09T04:50:57.700734+0000","last_fullsized":"2024-05-09T04:50:57.700734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:12:50.672212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:01.759883+0000","last_change":"2024-05-09T04:49:30.056089+0000","last_active":"2024-05-09T04:51:01.759883+0000","last_peered":"2024-05-09T04:51:01.759883+0000","last_clean":"2024-05-09T04:51:01.759883+0000","last_became_active":"2024-05-09T04:49:30.055635+0000","last_became_peered":"2024-05-09T04:49:30.055635+0000","last_unstale":"2024-05-09T04:51:01.759883+0000","last_undegraded":"2024-05-09T04:51:01.759883+0000","last_fullsized":"2024-05-09T04:51:01.759883+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:35:06.529058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.557248+0000","last_change":"2024-05-09T04:49:34.147628+0000","last_active":"2024-05-09T04:52:55.557248+0000","last_peered":"2024-05-09T04:52:55.557248+0000","last_clean":"2024-05-09T04:52:55.557248+0000","last_became_active":"2024-05-09T04:49:34.147208+0000","last_became_peered":"2024-05-09T04:49:34.147208+0000","last_unstale":"2024-05-09T04:52:55.557248+0000","last_undegraded":"2024-05-09T04:52:55.557248+0000","last_fullsized":"2024-05-09T04:52:55.557248+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:09:22.520468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.816932+0000","last_change":"2024-05-09T04:49:56.758366+0000","last_active":"2024-05-09T04:50:57.816932+0000","last_peered":"2024-05-09T04:50:57.816932+0000","last_clean":"2024-05-09T04:50:57.816932+0000","last_became_active":"2024-05-09T04:49:56.757763+0000","last_became_peered":"2024-05-09T04:49:56.757763+0000","last_unstale":"2024-05-09T04:50:57.816932+0000","last_undegraded":"2024-05-09T04:50:57.816932+0000","last_fullsized":"2024-05-09T04:50:57.816932+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:58.711764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:02.769778+0000","last_change":"2024-05-09T04:49:30.058805+0000","last_active":"2024-05-09T04:51:02.769778+0000","last_peered":"2024-05-09T04:51:02.769778+0000","last_clean":"2024-05-09T04:51:02.769778+0000","last_became_active":"2024-05-09T04:49:30.057278+0000","last_became_peered":"2024-05-09T04:49:30.057278+0000","last_unstale":"2024-05-09T04:51:02.769778+0000","last_undegraded":"2024-05-09T04:51:02.769778+0000","last_fullsized":"2024-05-09T04:51:02.769778+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:34.727781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.557567+0000","last_change":"2024-05-09T04:49:34.148753+0000","last_active":"2024-05-09T04:52:55.557567+0000","last_peered":"2024-05-09T04:52:55.557567+0000","last_clean":"2024-05-09T04:52:55.557567+0000","last_became_active":"2024-05-09T04:49:34.148407+0000","last_became_peered":"2024-05-09T04:49:34.148407+0000","last_unstale":"2024-05-09T04:52:55.557567+0000","last_undegraded":"2024-05-09T04:52:55.557567+0000","last_fullsized":"2024-05-09T04:52:55.557567+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:39.602934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.204230+0000","last_change":"2024-05-09T04:49:47.672521+0000","last_active":"2024-05-09T04:50:58.204230+0000","last_peered":"2024-05-09T04:50:58.204230+0000","last_clean":"2024-05-09T04:50:58.204230+0000","last_became_active":"2024-05-09T04:49:47.671353+0000","last_became_peered":"2024-05-09T04:49:47.671353+0000","last_unstale":"2024-05-09T04:50:58.204230+0000","last_undegraded":"2024-05-09T04:50:58.204230+0000","last_fullsized":"2024-05-09T04:50:58.204230+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:56:34.832651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:03.778625+0000","last_change":"2024-05-09T04:49:30.048589+0000","last_active":"2024-05-09T04:51:03.778625+0000","last_peered":"2024-05-09T04:51:03.778625+0000","last_clean":"2024-05-09T04:51:03.778625+0000","last_became_active":"2024-05-09T04:49:30.048241+0000","last_became_peered":"2024-05-09T04:49:30.048241+0000","last_unstale":"2024-05-09T04:51:03.778625+0000","last_undegraded":"2024-05-09T04:51:03.778625+0000","last_fullsized":"2024-05-09T04:51:03.778625+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:38:36.204539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.562002+0000","last_change":"2024-05-09T04:49:34.151201+0000","last_active":"2024-05-09T04:52:55.562002+0000","last_peered":"2024-05-09T04:52:55.562002+0000","last_clean":"2024-05-09T04:52:55.562002+0000","last_became_active":"2024-05-09T04:49:34.150505+0000","last_became_peered":"2024-05-09T04:49:34.150505+0000","last_unstale":"2024-05-09T04:52:55.562002+0000","last_undegraded":"2024-05-09T04:52:55.562002+0000","last_fullsized":"2024-05-09T04:52:55.562002+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:40.011516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700453+0000","last_change":"2024-05-09T04:49:47.653827+0000","last_active":"2024-05-09T04:50:57.700453+0000","last_peered":"2024-05-09T04:50:57.700453+0000","last_clean":"2024-05-09T04:50:57.700453+0000","last_became_active":"2024-05-09T04:49:47.652423+0000","last_became_peered":"2024-05-09T04:49:47.652423+0000","last_unstale":"2024-05-09T04:50:57.700453+0000","last_undegraded":"2024-05-09T04:50:57.700453+0000","last_fullsized":"2024-05-09T04:50:57.700453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:07:07.602659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'5","reported_seq":292,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:53:04.966694+0000","last_change":"2024-05-09T04:49:30.062849+0000","last_active":"2024-05-09T04:53:04.966694+0000","last_peered":"2024-05-09T04:53:04.966694+0000","last_clean":"2024-05-09T04:53:04.966694+0000","last_became_active":"2024-05-09T04:49:30.062426+0000","last_became_peered":"2024-05-09T04:49:30.062426+0000","last_unstale":"2024-05-09T04:53:04.966694+0000","last_undegraded":"2024-05-09T04:53:04.966694+0000","last_fullsized":"2024-05-09T04:53:04.966694+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:25:11.996208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":244,"num_read_kb":244,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.543652+0000","last_change":"2024-05-09T04:49:34.151145+0000","last_active":"2024-05-09T04:52:55.543652+0000","last_peered":"2024-05-09T04:52:55.543652+0000","last_clean":"2024-05-09T04:52:55.543652+0000","last_became_active":"2024-05-09T04:49:34.150745+0000","last_became_peered":"2024-05-09T04:49:34.150745+0000","last_unstale":"2024-05-09T04:52:55.543652+0000","last_undegraded":"2024-05-09T04:52:55.543652+0000","last_fullsized":"2024-05-09T04:52:55.543652+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:40.647454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700544+0000","last_change":"2024-05-09T04:49:47.672796+0000","last_active":"2024-05-09T04:50:57.700544+0000","last_peered":"2024-05-09T04:50:57.700544+0000","last_clean":"2024-05-09T04:50:57.700544+0000","last_became_active":"2024-05-09T04:49:47.667807+0000","last_became_peered":"2024-05-09T04:49:47.667807+0000","last_unstale":"2024-05-09T04:50:57.700544+0000","last_undegraded":"2024-05-09T04:50:57.700544+0000","last_fullsized":"2024-05-09T04:50:57.700544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:58.542973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:02.767623+0000","last_change":"2024-05-09T04:49:30.047265+0000","last_active":"2024-05-09T04:51:02.767623+0000","last_peered":"2024-05-09T04:51:02.767623+0000","last_clean":"2024-05-09T04:51:02.767623+0000","last_became_active":"2024-05-09T04:49:30.046888+0000","last_became_peered":"2024-05-09T04:49:30.046888+0000","last_unstale":"2024-05-09T04:51:02.767623+0000","last_undegraded":"2024-05-09T04:51:02.767623+0000","last_fullsized":"2024-05-09T04:51:02.767623+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:38:41.878678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.554153+0000","last_change":"2024-05-09T04:49:34.144447+0000","last_active":"2024-05-09T04:52:55.554153+0000","last_peered":"2024-05-09T04:52:55.554153+0000","last_clean":"2024-05-09T04:52:55.554153+0000","last_became_active":"2024-05-09T04:49:34.144073+0000","last_became_peered":"2024-05-09T04:49:34.144073+0000","last_unstale":"2024-05-09T04:52:55.554153+0000","last_undegraded":"2024-05-09T04:52:55.554153+0000","last_fullsized":"2024-05-09T04:52:55.554153+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:53:42.619552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.202477+0000","last_change":"2024-05-09T04:49:47.673364+0000","last_active":"2024-05-09T04:50:58.202477+0000","last_peered":"2024-05-09T04:50:58.202477+0000","last_clean":"2024-05-09T04:50:58.202477+0000","last_became_active":"2024-05-09T04:49:47.672712+0000","last_became_peered":"2024-05-09T04:49:47.672712+0000","last_unstale":"2024-05-09T04:50:58.202477+0000","last_undegraded":"2024-05-09T04:50:58.202477+0000","last_fullsized":"2024-05-09T04:50:58.202477+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:25:57.165777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:03.779907+0000","last_change":"2024-05-09T04:49:30.160924+0000","last_active":"2024-05-09T04:51:03.779907+0000","last_peered":"2024-05-09T04:51:03.779907+0000","last_clean":"2024-05-09T04:51:03.779907+0000","last_became_active":"2024-05-09T04:49:30.160479+0000","last_became_peered":"2024-05-09T04:49:30.160479+0000","last_unstale":"2024-05-09T04:51:03.779907+0000","last_undegraded":"2024-05-09T04:51:03.779907+0000","last_fullsized":"2024-05-09T04:51:03.779907+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:44:57.171441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.545742+0000","last_change":"2024-05-09T04:49:34.136514+0000","last_active":"2024-05-09T04:52:55.545742+0000","last_peered":"2024-05-09T04:52:55.545742+0000","last_clean":"2024-05-09T04:52:55.545742+0000","last_became_active":"2024-05-09T04:49:34.136121+0000","last_became_peered":"2024-05-09T04:49:34.136121+0000","last_unstale":"2024-05-09T04:52:55.545742+0000","last_undegraded":"2024-05-09T04:52:55.545742+0000","last_fullsized":"2024-05-09T04:52:55.545742+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:43:27.781720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700659+0000","last_change":"2024-05-09T04:49:47.673371+0000","last_active":"2024-05-09T04:50:57.700659+0000","last_peered":"2024-05-09T04:50:57.700659+0000","last_clean":"2024-05-09T04:50:57.700659+0000","last_became_active":"2024-05-09T04:49:47.669727+0000","last_became_peered":"2024-05-09T04:49:47.669727+0000","last_unstale":"2024-05-09T04:50:57.700659+0000","last_undegraded":"2024-05-09T04:50:57.700659+0000","last_fullsized":"2024-05-09T04:50:57.700659+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:34:45.510934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'40","reported_seq":339,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:53:04.966669+0000","last_change":"2024-05-09T04:49:30.048359+0000","last_active":"2024-05-09T04:53:04.966669+0000","last_peered":"2024-05-09T04:53:04.966669+0000","last_clean":"2024-05-09T04:53:04.966669+0000","last_became_active":"2024-05-09T04:49:30.047949+0000","last_became_peered":"2024-05-09T04:49:30.047949+0000","last_unstale":"2024-05-09T04:53:04.966669+0000","last_undegraded":"2024-05-09T04:53:04.966669+0000","last_fullsized":"2024-05-09T04:53:04.966669+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:01:10.498997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64489,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":256,"num_read_kb":192,"num_write":42,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.521240+0000","last_change":"2024-05-09T04:49:34.132589+0000","last_active":"2024-05-09T04:52:55.521240+0000","last_peered":"2024-05-09T04:52:55.521240+0000","last_clean":"2024-05-09T04:52:55.521240+0000","last_became_active":"2024-05-09T04:49:34.132227+0000","last_became_peered":"2024-05-09T04:49:34.132227+0000","last_unstale":"2024-05-09T04:52:55.521240+0000","last_undegraded":"2024-05-09T04:52:55.521240+0000","last_fullsized":"2024-05-09T04:52:55.521240+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:50.003878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.033497+0000","last_change":"2024-05-09T04:49:47.668236+0000","last_active":"2024-05-09T04:50:58.033497+0000","last_peered":"2024-05-09T04:50:58.033497+0000","last_clean":"2024-05-09T04:50:58.033497+0000","last_became_active":"2024-05-09T04:49:47.667779+0000","last_became_peered":"2024-05-09T04:49:47.667779+0000","last_unstale":"2024-05-09T04:50:58.033497+0000","last_undegraded":"2024-05-09T04:50:58.033497+0000","last_fullsized":"2024-05-09T04:50:58.033497+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:16.177888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:03.776885+0000","last_change":"2024-05-09T04:49:30.046145+0000","last_active":"2024-05-09T04:51:03.776885+0000","last_peered":"2024-05-09T04:51:03.776885+0000","last_clean":"2024-05-09T04:51:03.776885+0000","last_became_active":"2024-05-09T04:49:30.045478+0000","last_became_peered":"2024-05-09T04:49:30.045478+0000","last_unstale":"2024-05-09T04:51:03.776885+0000","last_undegraded":"2024-05-09T04:51:03.776885+0000","last_fullsized":"2024-05-09T04:51:03.776885+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:52:08.313984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.550254+0000","last_change":"2024-05-09T04:49:34.154349+0000","last_active":"2024-05-09T04:52:55.550254+0000","last_peered":"2024-05-09T04:52:55.550254+0000","last_clean":"2024-05-09T04:52:55.550254+0000","last_became_active":"2024-05-09T04:49:34.154011+0000","last_became_peered":"2024-05-09T04:49:34.154011+0000","last_unstale":"2024-05-09T04:52:55.550254+0000","last_undegraded":"2024-05-09T04:52:55.550254+0000","last_fullsized":"2024-05-09T04:52:55.550254+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:50:32.345863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.816744+0000","last_change":"2024-05-09T04:49:47.670170+0000","last_active":"2024-05-09T04:50:57.816744+0000","last_peered":"2024-05-09T04:50:57.816744+0000","last_clean":"2024-05-09T04:50:57.816744+0000","last_became_active":"2024-05-09T04:49:47.669705+0000","last_became_peered":"2024-05-09T04:49:47.669705+0000","last_unstale":"2024-05-09T04:50:57.816744+0000","last_undegraded":"2024-05-09T04:50:57.816744+0000","last_fullsized":"2024-05-09T04:50:57.816744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:59:27.339811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:03.777501+0000","last_change":"2024-05-09T04:49:30.051159+0000","last_active":"2024-05-09T04:51:03.777501+0000","last_peered":"2024-05-09T04:51:03.777501+0000","last_clean":"2024-05-09T04:51:03.777501+0000","last_became_active":"2024-05-09T04:49:30.050108+0000","last_became_peered":"2024-05-09T04:49:30.050108+0000","last_unstale":"2024-05-09T04:51:03.777501+0000","last_undegraded":"2024-05-09T04:51:03.777501+0000","last_fullsized":"2024-05-09T04:51:03.777501+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:32:56.778567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.549097+0000","last_change":"2024-05-09T04:49:34.162751+0000","last_active":"2024-05-09T04:52:55.549097+0000","last_peered":"2024-05-09T04:52:55.549097+0000","last_clean":"2024-05-09T04:52:55.549097+0000","last_became_active":"2024-05-09T04:49:34.162336+0000","last_became_peered":"2024-05-09T04:49:34.162336+0000","last_unstale":"2024-05-09T04:52:55.549097+0000","last_undegraded":"2024-05-09T04:52:55.549097+0000","last_fullsized":"2024-05-09T04:52:55.549097+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:41.758074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.705898+0000","last_change":"2024-05-09T04:49:47.680955+0000","last_active":"2024-05-09T04:50:57.705898+0000","last_peered":"2024-05-09T04:50:57.705898+0000","last_clean":"2024-05-09T04:50:57.705898+0000","last_became_active":"2024-05-09T04:49:47.680041+0000","last_became_peered":"2024-05-09T04:49:47.680041+0000","last_unstale":"2024-05-09T04:50:57.705898+0000","last_undegraded":"2024-05-09T04:50:57.705898+0000","last_fullsized":"2024-05-09T04:50:57.705898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:46.297611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'7","reported_seq":171,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:53:00.057449+0000","last_change":"2024-05-09T04:49:30.045857+0000","last_active":"2024-05-09T04:53:00.057449+0000","last_peered":"2024-05-09T04:53:00.057449+0000","last_clean":"2024-05-09T04:53:00.057449+0000","last_became_active":"2024-05-09T04:49:30.045404+0000","last_became_peered":"2024-05-09T04:49:30.045404+0000","last_unstale":"2024-05-09T04:53:00.057449+0000","last_undegraded":"2024-05-09T04:53:00.057449+0000","last_fullsized":"2024-05-09T04:53:00.057449+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:44:19.473449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.538078+0000","last_change":"2024-05-09T04:49:34.131734+0000","last_active":"2024-05-09T04:52:55.538078+0000","last_peered":"2024-05-09T04:52:55.538078+0000","last_clean":"2024-05-09T04:52:55.538078+0000","last_became_active":"2024-05-09T04:49:34.131396+0000","last_became_peered":"2024-05-09T04:49:34.131396+0000","last_unstale":"2024-05-09T04:52:55.538078+0000","last_undegraded":"2024-05-09T04:52:55.538078+0000","last_fullsized":"2024-05-09T04:52:55.538078+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:21.080651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.088139+0000","last_change":"2024-05-09T04:49:56.710429+0000","last_active":"2024-05-09T04:50:58.088139+0000","last_peered":"2024-05-09T04:50:58.088139+0000","last_clean":"2024-05-09T04:50:58.088139+0000","last_became_active":"2024-05-09T04:49:56.708497+0000","last_became_peered":"2024-05-09T04:49:56.708497+0000","last_unstale":"2024-05-09T04:50:58.088139+0000","last_undegraded":"2024-05-09T04:50:58.088139+0000","last_fullsized":"2024-05-09T04:50:58.088139+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:36.281300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:03.777421+0000","last_change":"2024-05-09T04:49:56.718731+0000","last_active":"2024-05-09T04:51:03.777421+0000","last_peered":"2024-05-09T04:51:03.777421+0000","last_clean":"2024-05-09T04:51:03.777421+0000","last_became_active":"2024-05-09T04:49:56.709958+0000","last_became_peered":"2024-05-09T04:49:56.709958+0000","last_unstale":"2024-05-09T04:51:03.777421+0000","last_undegraded":"2024-05-09T04:51:03.777421+0000","last_fullsized":"2024-05-09T04:51:03.777421+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:11.003906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.538882+0000","last_change":"2024-05-09T04:49:34.131705+0000","last_active":"2024-05-09T04:52:55.538882+0000","last_peered":"2024-05-09T04:52:55.538882+0000","last_clean":"2024-05-09T04:52:55.538882+0000","last_became_active":"2024-05-09T04:49:34.131287+0000","last_became_peered":"2024-05-09T04:49:34.131287+0000","last_unstale":"2024-05-09T04:52:55.538882+0000","last_undegraded":"2024-05-09T04:52:55.538882+0000","last_fullsized":"2024-05-09T04:52:55.538882+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:10:24.914832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.032602+0000","last_change":"2024-05-09T04:49:47.663861+0000","last_active":"2024-05-09T04:50:58.032602+0000","last_peered":"2024-05-09T04:50:58.032602+0000","last_clean":"2024-05-09T04:50:58.032602+0000","last_became_active":"2024-05-09T04:49:47.663374+0000","last_became_peered":"2024-05-09T04:49:47.663374+0000","last_unstale":"2024-05-09T04:50:58.032602+0000","last_undegraded":"2024-05-09T04:50:58.032602+0000","last_fullsized":"2024-05-09T04:50:58.032602+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:08.063040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:03.776644+0000","last_change":"2024-05-09T04:49:30.044183+0000","last_active":"2024-05-09T04:51:03.776644+0000","last_peered":"2024-05-09T04:51:03.776644+0000","last_clean":"2024-05-09T04:51:03.776644+0000","last_became_active":"2024-05-09T04:49:30.043608+0000","last_became_peered":"2024-05-09T04:49:30.043608+0000","last_unstale":"2024-05-09T04:51:03.776644+0000","last_undegraded":"2024-05-09T04:51:03.776644+0000","last_fullsized":"2024-05-09T04:51:03.776644+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:48:57.788970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.524751+0000","last_change":"2024-05-09T04:49:34.133941+0000","last_active":"2024-05-09T04:52:55.524751+0000","last_peered":"2024-05-09T04:52:55.524751+0000","last_clean":"2024-05-09T04:52:55.524751+0000","last_became_active":"2024-05-09T04:49:34.133550+0000","last_became_peered":"2024-05-09T04:49:34.133550+0000","last_unstale":"2024-05-09T04:52:55.524751+0000","last_undegraded":"2024-05-09T04:52:55.524751+0000","last_fullsized":"2024-05-09T04:52:55.524751+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:45:14.703565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.201444+0000","last_change":"2024-05-09T04:49:47.901085+0000","last_active":"2024-05-09T04:50:58.201444+0000","last_peered":"2024-05-09T04:50:58.201444+0000","last_clean":"2024-05-09T04:50:58.201444+0000","last_became_active":"2024-05-09T04:49:47.900444+0000","last_became_peered":"2024-05-09T04:49:47.900444+0000","last_unstale":"2024-05-09T04:50:58.201444+0000","last_undegraded":"2024-05-09T04:50:58.201444+0000","last_fullsized":"2024-05-09T04:50:58.201444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:44.195683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:01.759239+0000","last_change":"2024-05-09T04:49:30.154060+0000","last_active":"2024-05-09T04:51:01.759239+0000","last_peered":"2024-05-09T04:51:01.759239+0000","last_clean":"2024-05-09T04:51:01.759239+0000","last_became_active":"2024-05-09T04:49:30.152556+0000","last_became_peered":"2024-05-09T04:49:30.152556+0000","last_unstale":"2024-05-09T04:51:01.759239+0000","last_undegraded":"2024-05-09T04:51:01.759239+0000","last_fullsized":"2024-05-09T04:51:01.759239+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:27.056251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.531335+0000","last_change":"2024-05-09T04:49:34.153128+0000","last_active":"2024-05-09T04:52:55.531335+0000","last_peered":"2024-05-09T04:52:55.531335+0000","last_clean":"2024-05-09T04:52:55.531335+0000","last_became_active":"2024-05-09T04:49:34.152734+0000","last_became_peered":"2024-05-09T04:49:34.152734+0000","last_unstale":"2024-05-09T04:52:55.531335+0000","last_undegraded":"2024-05-09T04:52:55.531335+0000","last_fullsized":"2024-05-09T04:52:55.531335+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:54:06.920877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.087826+0000","last_change":"2024-05-09T04:49:47.670263+0000","last_active":"2024-05-09T04:50:58.087826+0000","last_peered":"2024-05-09T04:50:58.087826+0000","last_clean":"2024-05-09T04:50:58.087826+0000","last_became_active":"2024-05-09T04:49:47.665743+0000","last_became_peered":"2024-05-09T04:49:47.665743+0000","last_unstale":"2024-05-09T04:50:58.087826+0000","last_undegraded":"2024-05-09T04:50:58.087826+0000","last_fullsized":"2024-05-09T04:50:58.087826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:27.980226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":170,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:53:03.059729+0000","last_change":"2024-05-09T04:49:30.056119+0000","last_active":"2024-05-09T04:53:03.059729+0000","last_peered":"2024-05-09T04:53:03.059729+0000","last_clean":"2024-05-09T04:53:03.059729+0000","last_became_active":"2024-05-09T04:49:30.055660+0000","last_became_peered":"2024-05-09T04:49:30.055660+0000","last_unstale":"2024-05-09T04:53:03.059729+0000","last_undegraded":"2024-05-09T04:53:03.059729+0000","last_fullsized":"2024-05-09T04:53:03.059729+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:21.513879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.524713+0000","last_change":"2024-05-09T04:49:56.764303+0000","last_active":"2024-05-09T04:52:55.524713+0000","last_peered":"2024-05-09T04:52:55.524713+0000","last_clean":"2024-05-09T04:52:55.524713+0000","last_became_active":"2024-05-09T04:49:56.763854+0000","last_became_peered":"2024-05-09T04:49:56.763854+0000","last_unstale":"2024-05-09T04:52:55.524713+0000","last_undegraded":"2024-05-09T04:52:55.524713+0000","last_fullsized":"2024-05-09T04:52:55.524713+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:06.086602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.088298+0000","last_change":"2024-05-09T04:49:47.900866+0000","last_active":"2024-05-09T04:50:58.088298+0000","last_peered":"2024-05-09T04:50:58.088298+0000","last_clean":"2024-05-09T04:50:58.088298+0000","last_became_active":"2024-05-09T04:49:47.900332+0000","last_became_peered":"2024-05-09T04:49:47.900332+0000","last_unstale":"2024-05-09T04:50:58.088298+0000","last_undegraded":"2024-05-09T04:50:58.088298+0000","last_fullsized":"2024-05-09T04:50:58.088298+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:06:41.448262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.527044+0000","last_change":"2024-05-09T04:49:56.702559+0000","last_active":"2024-05-09T04:52:55.527044+0000","last_peered":"2024-05-09T04:52:55.527044+0000","last_clean":"2024-05-09T04:52:55.527044+0000","last_became_active":"2024-05-09T04:49:56.702135+0000","last_became_peered":"2024-05-09T04:49:56.702135+0000","last_unstale":"2024-05-09T04:52:55.527044+0000","last_undegraded":"2024-05-09T04:52:55.527044+0000","last_fullsized":"2024-05-09T04:52:55.527044+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:43:05.848239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":169,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:53:01.058363+0000","last_change":"2024-05-09T04:49:30.158050+0000","last_active":"2024-05-09T04:53:01.058363+0000","last_peered":"2024-05-09T04:53:01.058363+0000","last_clean":"2024-05-09T04:53:01.058363+0000","last_became_active":"2024-05-09T04:49:30.156719+0000","last_became_peered":"2024-05-09T04:49:30.156719+0000","last_unstale":"2024-05-09T04:53:01.058363+0000","last_undegraded":"2024-05-09T04:53:01.058363+0000","last_fullsized":"2024-05-09T04:53:01.058363+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:47.963111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:15.645536+0000","last_change":"2024-05-09T04:49:47.673594+0000","last_active":"2024-05-09T04:51:15.645536+0000","last_peered":"2024-05-09T04:51:15.645536+0000","last_clean":"2024-05-09T04:51:15.645536+0000","last_became_active":"2024-05-09T04:49:47.672671+0000","last_became_peered":"2024-05-09T04:49:47.672671+0000","last_unstale":"2024-05-09T04:51:15.645536+0000","last_undegraded":"2024-05-09T04:51:15.645536+0000","last_fullsized":"2024-05-09T04:51:15.645536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:58:56.078180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.547961+0000","last_change":"2024-05-09T04:50:58.035054+0000","last_active":"2024-05-09T04:52:55.547961+0000","last_peered":"2024-05-09T04:52:55.547961+0000","last_clean":"2024-05-09T04:52:55.547961+0000","last_became_active":"2024-05-09T04:50:58.034297+0000","last_became_peered":"2024-05-09T04:50:58.034297+0000","last_unstale":"2024-05-09T04:52:55.547961+0000","last_undegraded":"2024-05-09T04:52:55.547961+0000","last_fullsized":"2024-05-09T04:52:55.547961+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:10:56.827950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:03.777300+0000","last_change":"2024-05-09T04:49:30.060224+0000","last_active":"2024-05-09T04:51:03.777300+0000","last_peered":"2024-05-09T04:51:03.777300+0000","last_clean":"2024-05-09T04:51:03.777300+0000","last_became_active":"2024-05-09T04:49:30.059830+0000","last_became_peered":"2024-05-09T04:49:30.059830+0000","last_unstale":"2024-05-09T04:51:03.777300+0000","last_undegraded":"2024-05-09T04:51:03.777300+0000","last_fullsized":"2024-05-09T04:51:03.777300+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:02.925226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":117,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203150+0000","last_change":"2024-05-09T04:47:53.050380+0000","last_active":"2024-05-09T04:50:58.203150+0000","last_peered":"2024-05-09T04:50:58.203150+0000","last_clean":"2024-05-09T04:50:58.203150+0000","last_became_active":"2024-05-09T04:47:53.044058+0000","last_became_peered":"2024-05-09T04:47:53.044058+0000","last_unstale":"2024-05-09T04:50:58.203150+0000","last_undegraded":"2024-05-09T04:50:58.203150+0000","last_fullsized":"2024-05-09T04:50:58.203150+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:43:48.067159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:43:48.067159+0000","last_clean_scrub_stamp":"2024-05-09T04:43:48.067159+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:17:18.714607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.033328+0000","last_change":"2024-05-09T04:49:47.901114+0000","last_active":"2024-05-09T04:50:58.033328+0000","last_peered":"2024-05-09T04:50:58.033328+0000","last_clean":"2024-05-09T04:50:58.033328+0000","last_became_active":"2024-05-09T04:49:47.900450+0000","last_became_peered":"2024-05-09T04:49:47.900450+0000","last_unstale":"2024-05-09T04:50:58.033328+0000","last_undegraded":"2024-05-09T04:50:58.033328+0000","last_fullsized":"2024-05-09T04:50:58.033328+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:25:44.517789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.537021+0000","last_change":"2024-05-09T04:49:34.147027+0000","last_active":"2024-05-09T04:52:55.537021+0000","last_peered":"2024-05-09T04:52:55.537021+0000","last_clean":"2024-05-09T04:52:55.537021+0000","last_became_active":"2024-05-09T04:49:34.146672+0000","last_became_peered":"2024-05-09T04:49:34.146672+0000","last_unstale":"2024-05-09T04:52:55.537021+0000","last_undegraded":"2024-05-09T04:52:55.537021+0000","last_fullsized":"2024-05-09T04:52:55.537021+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:15:44.847304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:02.768703+0000","last_change":"2024-05-09T04:49:30.043877+0000","last_active":"2024-05-09T04:51:02.768703+0000","last_peered":"2024-05-09T04:51:02.768703+0000","last_clean":"2024-05-09T04:51:02.768703+0000","last_became_active":"2024-05-09T04:49:30.043513+0000","last_became_peered":"2024-05-09T04:49:30.043513+0000","last_unstale":"2024-05-09T04:51:02.768703+0000","last_undegraded":"2024-05-09T04:51:02.768703+0000","last_fullsized":"2024-05-09T04:51:02.768703+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:16.501692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.816615+0000","last_change":"2024-05-09T04:49:47.668605+0000","last_active":"2024-05-09T04:50:57.816615+0000","last_peered":"2024-05-09T04:50:57.816615+0000","last_clean":"2024-05-09T04:50:57.816615+0000","last_became_active":"2024-05-09T04:49:47.668095+0000","last_became_peered":"2024-05-09T04:49:47.668095+0000","last_unstale":"2024-05-09T04:50:57.816615+0000","last_undegraded":"2024-05-09T04:50:57.816615+0000","last_fullsized":"2024-05-09T04:50:57.816615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:39:30.508618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:03.777515+0000","last_change":"2024-05-09T04:49:30.159554+0000","last_active":"2024-05-09T04:51:03.777515+0000","last_peered":"2024-05-09T04:51:03.777515+0000","last_clean":"2024-05-09T04:51:03.777515+0000","last_became_active":"2024-05-09T04:49:30.159266+0000","last_became_peered":"2024-05-09T04:49:30.159266+0000","last_unstale":"2024-05-09T04:51:03.777515+0000","last_undegraded":"2024-05-09T04:51:03.777515+0000","last_fullsized":"2024-05-09T04:51:03.777515+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:42:04.252862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.541970+0000","last_change":"2024-05-09T04:49:34.150688+0000","last_active":"2024-05-09T04:52:55.541970+0000","last_peered":"2024-05-09T04:52:55.541970+0000","last_clean":"2024-05-09T04:52:55.541970+0000","last_became_active":"2024-05-09T04:49:34.150319+0000","last_became_peered":"2024-05-09T04:49:34.150319+0000","last_unstale":"2024-05-09T04:52:55.541970+0000","last_undegraded":"2024-05-09T04:52:55.541970+0000","last_fullsized":"2024-05-09T04:52:55.541970+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:35:04.832761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700647+0000","last_change":"2024-05-09T04:49:47.902323+0000","last_active":"2024-05-09T04:50:57.700647+0000","last_peered":"2024-05-09T04:50:57.700647+0000","last_clean":"2024-05-09T04:50:57.700647+0000","last_became_active":"2024-05-09T04:49:47.901389+0000","last_became_peered":"2024-05-09T04:49:47.901389+0000","last_unstale":"2024-05-09T04:50:57.700647+0000","last_undegraded":"2024-05-09T04:50:57.700647+0000","last_fullsized":"2024-05-09T04:50:57.700647+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:32.615944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'16","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:42.839922+0000","last_change":"2024-05-09T04:49:30.061172+0000","last_active":"2024-05-09T04:51:42.839922+0000","last_peered":"2024-05-09T04:51:42.839922+0000","last_clean":"2024-05-09T04:51:42.839922+0000","last_became_active":"2024-05-09T04:49:30.060680+0000","last_became_peered":"2024-05-09T04:49:30.060680+0000","last_unstale":"2024-05-09T04:51:42.839922+0000","last_undegraded":"2024-05-09T04:51:42.839922+0000","last_fullsized":"2024-05-09T04:51:42.839922+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:45:44.649124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.551472+0000","last_change":"2024-05-09T04:49:34.130319+0000","last_active":"2024-05-09T04:52:55.551472+0000","last_peered":"2024-05-09T04:52:55.551472+0000","last_clean":"2024-05-09T04:52:55.551472+0000","last_became_active":"2024-05-09T04:49:34.129610+0000","last_became_peered":"2024-05-09T04:49:34.129610+0000","last_unstale":"2024-05-09T04:52:55.551472+0000","last_undegraded":"2024-05-09T04:52:55.551472+0000","last_fullsized":"2024-05-09T04:52:55.551472+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:45.505221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.700269+0000","last_change":"2024-05-09T04:49:47.647513+0000","last_active":"2024-05-09T04:50:57.700269+0000","last_peered":"2024-05-09T04:50:57.700269+0000","last_clean":"2024-05-09T04:50:57.700269+0000","last_became_active":"2024-05-09T04:49:47.647069+0000","last_became_peered":"2024-05-09T04:49:47.647069+0000","last_unstale":"2024-05-09T04:50:57.700269+0000","last_undegraded":"2024-05-09T04:50:57.700269+0000","last_fullsized":"2024-05-09T04:50:57.700269+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:05:27.658919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:03.775689+0000","last_change":"2024-05-09T04:49:30.350850+0000","last_active":"2024-05-09T04:51:03.775689+0000","last_peered":"2024-05-09T04:51:03.775689+0000","last_clean":"2024-05-09T04:51:03.775689+0000","last_became_active":"2024-05-09T04:49:30.350431+0000","last_became_peered":"2024-05-09T04:49:30.350431+0000","last_unstale":"2024-05-09T04:51:03.775689+0000","last_undegraded":"2024-05-09T04:51:03.775689+0000","last_fullsized":"2024-05-09T04:51:03.775689+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:53:05.388338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.562337+0000","last_change":"2024-05-09T04:49:34.153630+0000","last_active":"2024-05-09T04:52:55.562337+0000","last_peered":"2024-05-09T04:52:55.562337+0000","last_clean":"2024-05-09T04:52:55.562337+0000","last_became_active":"2024-05-09T04:49:34.152045+0000","last_became_peered":"2024-05-09T04:49:34.152045+0000","last_unstale":"2024-05-09T04:52:55.562337+0000","last_undegraded":"2024-05-09T04:52:55.562337+0000","last_fullsized":"2024-05-09T04:52:55.562337+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:32:35.991731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.202819+0000","last_change":"2024-05-09T04:49:47.901256+0000","last_active":"2024-05-09T04:50:58.202819+0000","last_peered":"2024-05-09T04:50:58.202819+0000","last_clean":"2024-05-09T04:50:58.202819+0000","last_became_active":"2024-05-09T04:49:47.900413+0000","last_became_peered":"2024-05-09T04:49:47.900413+0000","last_unstale":"2024-05-09T04:50:58.202819+0000","last_undegraded":"2024-05-09T04:50:58.202819+0000","last_fullsized":"2024-05-09T04:50:58.202819+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:46:11.845220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:59.742698+0000","last_change":"2024-05-09T04:49:30.152273+0000","last_active":"2024-05-09T04:50:59.742698+0000","last_peered":"2024-05-09T04:50:59.742698+0000","last_clean":"2024-05-09T04:50:59.742698+0000","last_became_active":"2024-05-09T04:49:30.151973+0000","last_became_peered":"2024-05-09T04:49:30.151973+0000","last_unstale":"2024-05-09T04:50:59.742698+0000","last_undegraded":"2024-05-09T04:50:59.742698+0000","last_fullsized":"2024-05-09T04:50:59.742698+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:14:52.912588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.508227+0000","last_change":"2024-05-09T04:49:34.126829+0000","last_active":"2024-05-09T04:52:55.508227+0000","last_peered":"2024-05-09T04:52:55.508227+0000","last_clean":"2024-05-09T04:52:55.508227+0000","last_became_active":"2024-05-09T04:49:34.126430+0000","last_became_peered":"2024-05-09T04:49:34.126430+0000","last_unstale":"2024-05-09T04:52:55.508227+0000","last_undegraded":"2024-05-09T04:52:55.508227+0000","last_fullsized":"2024-05-09T04:52:55.508227+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:38.247148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.706236+0000","last_change":"2024-05-09T04:49:47.675292+0000","last_active":"2024-05-09T04:50:57.706236+0000","last_peered":"2024-05-09T04:50:57.706236+0000","last_clean":"2024-05-09T04:50:57.706236+0000","last_became_active":"2024-05-09T04:49:47.674802+0000","last_became_peered":"2024-05-09T04:49:47.674802+0000","last_unstale":"2024-05-09T04:50:57.706236+0000","last_undegraded":"2024-05-09T04:50:57.706236+0000","last_fullsized":"2024-05-09T04:50:57.706236+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:47:21.988054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:02.769886+0000","last_change":"2024-05-09T04:49:30.057854+0000","last_active":"2024-05-09T04:51:02.769886+0000","last_peered":"2024-05-09T04:51:02.769886+0000","last_clean":"2024-05-09T04:51:02.769886+0000","last_became_active":"2024-05-09T04:49:30.056625+0000","last_became_peered":"2024-05-09T04:49:30.056625+0000","last_unstale":"2024-05-09T04:51:02.769886+0000","last_undegraded":"2024-05-09T04:51:02.769886+0000","last_fullsized":"2024-05-09T04:51:02.769886+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:51.272533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.559278+0000","last_change":"2024-05-09T04:49:34.144414+0000","last_active":"2024-05-09T04:52:55.559278+0000","last_peered":"2024-05-09T04:52:55.559278+0000","last_clean":"2024-05-09T04:52:55.559278+0000","last_became_active":"2024-05-09T04:49:34.144064+0000","last_became_peered":"2024-05-09T04:49:34.144064+0000","last_unstale":"2024-05-09T04:52:55.559278+0000","last_undegraded":"2024-05-09T04:52:55.559278+0000","last_fullsized":"2024-05-09T04:52:55.559278+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:44:40.991576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:23.898074+0000","last_change":"2024-05-09T04:49:47.672956+0000","last_active":"2024-05-09T04:51:23.898074+0000","last_peered":"2024-05-09T04:51:23.898074+0000","last_clean":"2024-05-09T04:51:23.898074+0000","last_became_active":"2024-05-09T04:49:47.672513+0000","last_became_peered":"2024-05-09T04:49:47.672513+0000","last_unstale":"2024-05-09T04:51:23.898074+0000","last_undegraded":"2024-05-09T04:51:23.898074+0000","last_fullsized":"2024-05-09T04:51:23.898074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:32.250039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:02.754199+0000","last_change":"2024-05-09T04:49:56.704058+0000","last_active":"2024-05-09T04:51:02.754199+0000","last_peered":"2024-05-09T04:51:02.754199+0000","last_clean":"2024-05-09T04:51:02.754199+0000","last_became_active":"2024-05-09T04:49:56.703589+0000","last_became_peered":"2024-05-09T04:49:56.703589+0000","last_unstale":"2024-05-09T04:51:02.754199+0000","last_undegraded":"2024-05-09T04:51:02.754199+0000","last_fullsized":"2024-05-09T04:51:02.754199+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:54.596200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,1],"acting":[11,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.555562+0000","last_change":"2024-05-09T04:49:34.159321+0000","last_active":"2024-05-09T04:52:55.555562+0000","last_peered":"2024-05-09T04:52:55.555562+0000","last_clean":"2024-05-09T04:52:55.555562+0000","last_became_active":"2024-05-09T04:49:34.157918+0000","last_became_peered":"2024-05-09T04:49:34.157918+0000","last_unstale":"2024-05-09T04:52:55.555562+0000","last_undegraded":"2024-05-09T04:52:55.555562+0000","last_fullsized":"2024-05-09T04:52:55.555562+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:33:59.659905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.202864+0000","last_change":"2024-05-09T04:49:47.900692+0000","last_active":"2024-05-09T04:50:58.202864+0000","last_peered":"2024-05-09T04:50:58.202864+0000","last_clean":"2024-05-09T04:50:58.202864+0000","last_became_active":"2024-05-09T04:49:47.900095+0000","last_became_peered":"2024-05-09T04:49:47.900095+0000","last_unstale":"2024-05-09T04:50:58.202864+0000","last_undegraded":"2024-05-09T04:50:58.202864+0000","last_fullsized":"2024-05-09T04:50:58.202864+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:08.632408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:03.775992+0000","last_change":"2024-05-09T04:49:30.048768+0000","last_active":"2024-05-09T04:51:03.775992+0000","last_peered":"2024-05-09T04:51:03.775992+0000","last_clean":"2024-05-09T04:51:03.775992+0000","last_became_active":"2024-05-09T04:49:30.048235+0000","last_became_peered":"2024-05-09T04:49:30.048235+0000","last_unstale":"2024-05-09T04:51:03.775992+0000","last_undegraded":"2024-05-09T04:51:03.775992+0000","last_fullsized":"2024-05-09T04:51:03.775992+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:51:16.988726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.539908+0000","last_change":"2024-05-09T04:49:34.142110+0000","last_active":"2024-05-09T04:52:55.539908+0000","last_peered":"2024-05-09T04:52:55.539908+0000","last_clean":"2024-05-09T04:52:55.539908+0000","last_became_active":"2024-05-09T04:49:34.141709+0000","last_became_peered":"2024-05-09T04:49:34.141709+0000","last_unstale":"2024-05-09T04:52:55.539908+0000","last_undegraded":"2024-05-09T04:52:55.539908+0000","last_fullsized":"2024-05-09T04:52:55.539908+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:16.365915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:57.952118+0000","last_change":"2024-05-09T04:49:47.667784+0000","last_active":"2024-05-09T04:50:57.952118+0000","last_peered":"2024-05-09T04:50:57.952118+0000","last_clean":"2024-05-09T04:50:57.952118+0000","last_became_active":"2024-05-09T04:49:47.667316+0000","last_became_peered":"2024-05-09T04:49:47.667316+0000","last_unstale":"2024-05-09T04:50:57.952118+0000","last_undegraded":"2024-05-09T04:50:57.952118+0000","last_fullsized":"2024-05-09T04:50:57.952118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:04:20.233287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":169,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:53:00.960608+0000","last_change":"2024-05-09T04:49:30.351231+0000","last_active":"2024-05-09T04:53:00.960608+0000","last_peered":"2024-05-09T04:53:00.960608+0000","last_clean":"2024-05-09T04:53:00.960608+0000","last_became_active":"2024-05-09T04:49:30.350817+0000","last_became_peered":"2024-05-09T04:49:30.350817+0000","last_unstale":"2024-05-09T04:53:00.960608+0000","last_undegraded":"2024-05-09T04:53:00.960608+0000","last_fullsized":"2024-05-09T04:53:00.960608+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:51.234300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.540250+0000","last_change":"2024-05-09T04:49:34.129625+0000","last_active":"2024-05-09T04:52:55.540250+0000","last_peered":"2024-05-09T04:52:55.540250+0000","last_clean":"2024-05-09T04:52:55.540250+0000","last_became_active":"2024-05-09T04:49:34.129215+0000","last_became_peered":"2024-05-09T04:49:34.129215+0000","last_unstale":"2024-05-09T04:52:55.540250+0000","last_undegraded":"2024-05-09T04:52:55.540250+0000","last_fullsized":"2024-05-09T04:52:55.540250+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:26.247116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.203944+0000","last_change":"2024-05-09T04:49:47.682819+0000","last_active":"2024-05-09T04:50:58.203944+0000","last_peered":"2024-05-09T04:50:58.203944+0000","last_clean":"2024-05-09T04:50:58.203944+0000","last_became_active":"2024-05-09T04:49:47.682331+0000","last_became_peered":"2024-05-09T04:49:47.682331+0000","last_unstale":"2024-05-09T04:50:58.203944+0000","last_undegraded":"2024-05-09T04:50:58.203944+0000","last_fullsized":"2024-05-09T04:50:58.203944+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:03.852975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:03.776136+0000","last_change":"2024-05-09T04:49:30.050247+0000","last_active":"2024-05-09T04:51:03.776136+0000","last_peered":"2024-05-09T04:51:03.776136+0000","last_clean":"2024-05-09T04:51:03.776136+0000","last_became_active":"2024-05-09T04:49:30.049835+0000","last_became_peered":"2024-05-09T04:49:30.049835+0000","last_unstale":"2024-05-09T04:51:03.776136+0000","last_undegraded":"2024-05-09T04:51:03.776136+0000","last_fullsized":"2024-05-09T04:51:03.776136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:52.591186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.559642+0000","last_change":"2024-05-09T04:49:34.145406+0000","last_active":"2024-05-09T04:52:55.559642+0000","last_peered":"2024-05-09T04:52:55.559642+0000","last_clean":"2024-05-09T04:52:55.559642+0000","last_became_active":"2024-05-09T04:49:34.144946+0000","last_became_peered":"2024-05-09T04:49:34.144946+0000","last_unstale":"2024-05-09T04:52:55.559642+0000","last_undegraded":"2024-05-09T04:52:55.559642+0000","last_fullsized":"2024-05-09T04:52:55.559642+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:43:04.704215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:50:58.201371+0000","last_change":"2024-05-09T04:49:56.758844+0000","last_active":"2024-05-09T04:50:58.201371+0000","last_peered":"2024-05-09T04:50:58.201371+0000","last_clean":"2024-05-09T04:50:58.201371+0000","last_became_active":"2024-05-09T04:49:56.758335+0000","last_became_peered":"2024-05-09T04:49:56.758335+0000","last_unstale":"2024-05-09T04:50:58.201371+0000","last_undegraded":"2024-05-09T04:50:58.201371+0000","last_fullsized":"2024-05-09T04:50:58.201371+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:39:33.344225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:02.770832+0000","last_change":"2024-05-09T04:49:56.766262+0000","last_active":"2024-05-09T04:51:02.770832+0000","last_peered":"2024-05-09T04:51:02.770832+0000","last_clean":"2024-05-09T04:51:02.770832+0000","last_became_active":"2024-05-09T04:49:56.758765+0000","last_became_peered":"2024-05-09T04:49:56.758765+0000","last_unstale":"2024-05-09T04:51:02.770832+0000","last_undegraded":"2024-05-09T04:51:02.770832+0000","last_fullsized":"2024-05-09T04:51:02.770832+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:44:18.271729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.540798+0000","last_change":"2024-05-09T04:49:34.134957+0000","last_active":"2024-05-09T04:52:55.540798+0000","last_peered":"2024-05-09T04:52:55.540798+0000","last_clean":"2024-05-09T04:52:55.540798+0000","last_became_active":"2024-05-09T04:49:34.134544+0000","last_became_peered":"2024-05-09T04:49:34.134544+0000","last_unstale":"2024-05-09T04:52:55.540798+0000","last_undegraded":"2024-05-09T04:52:55.540798+0000","last_fullsized":"2024-05-09T04:52:55.540798+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:21:54.855905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.520962+0000","last_change":"2024-05-09T04:49:34.138112+0000","last_active":"2024-05-09T04:52:55.520962+0000","last_peered":"2024-05-09T04:52:55.520962+0000","last_clean":"2024-05-09T04:52:55.520962+0000","last_became_active":"2024-05-09T04:49:34.137716+0000","last_became_peered":"2024-05-09T04:49:34.137716+0000","last_unstale":"2024-05-09T04:52:55.520962+0000","last_undegraded":"2024-05-09T04:52:55.520962+0000","last_fullsized":"2024-05-09T04:52:55.520962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:17.028262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.534385+0000","last_change":"2024-05-09T04:50:58.204287+0000","last_active":"2024-05-09T04:52:55.534385+0000","last_peered":"2024-05-09T04:52:55.534385+0000","last_clean":"2024-05-09T04:52:55.534385+0000","last_became_active":"2024-05-09T04:50:58.203894+0000","last_became_peered":"2024-05-09T04:50:58.203894+0000","last_unstale":"2024-05-09T04:52:55.534385+0000","last_undegraded":"2024-05-09T04:52:55.534385+0000","last_fullsized":"2024-05-09T04:52:55.534385+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:02.214347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.558541+0000","last_change":"2024-05-09T04:49:34.147742+0000","last_active":"2024-05-09T04:52:55.558541+0000","last_peered":"2024-05-09T04:52:55.558541+0000","last_clean":"2024-05-09T04:52:55.558541+0000","last_became_active":"2024-05-09T04:49:34.147301+0000","last_became_peered":"2024-05-09T04:49:34.147301+0000","last_unstale":"2024-05-09T04:52:55.558541+0000","last_undegraded":"2024-05-09T04:52:55.558541+0000","last_fullsized":"2024-05-09T04:52:55.558541+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:36:55.371212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.550147+0000","last_change":"2024-05-09T04:49:34.163599+0000","last_active":"2024-05-09T04:52:55.550147+0000","last_peered":"2024-05-09T04:52:55.550147+0000","last_clean":"2024-05-09T04:52:55.550147+0000","last_became_active":"2024-05-09T04:49:34.162898+0000","last_became_peered":"2024-05-09T04:49:34.162898+0000","last_unstale":"2024-05-09T04:52:55.550147+0000","last_undegraded":"2024-05-09T04:52:55.550147+0000","last_fullsized":"2024-05-09T04:52:55.550147+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:55:57.213901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.552305+0000","last_change":"2024-05-09T04:49:34.151648+0000","last_active":"2024-05-09T04:52:55.552305+0000","last_peered":"2024-05-09T04:52:55.552305+0000","last_clean":"2024-05-09T04:52:55.552305+0000","last_became_active":"2024-05-09T04:49:34.148701+0000","last_became_peered":"2024-05-09T04:49:34.148701+0000","last_unstale":"2024-05-09T04:52:55.552305+0000","last_undegraded":"2024-05-09T04:52:55.552305+0000","last_fullsized":"2024-05-09T04:52:55.552305+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:54:30.122252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.516835+0000","last_change":"2024-05-09T04:49:34.136679+0000","last_active":"2024-05-09T04:52:55.516835+0000","last_peered":"2024-05-09T04:52:55.516835+0000","last_clean":"2024-05-09T04:52:55.516835+0000","last_became_active":"2024-05-09T04:49:34.136306+0000","last_became_peered":"2024-05-09T04:49:34.136306+0000","last_unstale":"2024-05-09T04:52:55.516835+0000","last_undegraded":"2024-05-09T04:52:55.516835+0000","last_fullsized":"2024-05-09T04:52:55.516835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:51:39.476564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.538881+0000","last_change":"2024-05-09T04:49:34.130769+0000","last_active":"2024-05-09T04:52:55.538881+0000","last_peered":"2024-05-09T04:52:55.538881+0000","last_clean":"2024-05-09T04:52:55.538881+0000","last_became_active":"2024-05-09T04:49:34.130362+0000","last_became_peered":"2024-05-09T04:49:34.130362+0000","last_unstale":"2024-05-09T04:52:55.538881+0000","last_undegraded":"2024-05-09T04:52:55.538881+0000","last_fullsized":"2024-05-09T04:52:55.538881+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:54.942259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.552404+0000","last_change":"2024-05-09T04:49:34.138054+0000","last_active":"2024-05-09T04:52:55.552404+0000","last_peered":"2024-05-09T04:52:55.552404+0000","last_clean":"2024-05-09T04:52:55.552404+0000","last_became_active":"2024-05-09T04:49:34.136111+0000","last_became_peered":"2024-05-09T04:49:34.136111+0000","last_unstale":"2024-05-09T04:52:55.552404+0000","last_undegraded":"2024-05-09T04:52:55.552404+0000","last_fullsized":"2024-05-09T04:52:55.552404+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:15:27.870874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.562236+0000","last_change":"2024-05-09T04:49:34.146370+0000","last_active":"2024-05-09T04:52:55.562236+0000","last_peered":"2024-05-09T04:52:55.562236+0000","last_clean":"2024-05-09T04:52:55.562236+0000","last_became_active":"2024-05-09T04:49:34.145948+0000","last_became_peered":"2024-05-09T04:49:34.145948+0000","last_unstale":"2024-05-09T04:52:55.562236+0000","last_undegraded":"2024-05-09T04:52:55.562236+0000","last_fullsized":"2024-05-09T04:52:55.562236+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:14.826168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.555257+0000","last_change":"2024-05-09T04:49:34.138028+0000","last_active":"2024-05-09T04:52:55.555257+0000","last_peered":"2024-05-09T04:52:55.555257+0000","last_clean":"2024-05-09T04:52:55.555257+0000","last_became_active":"2024-05-09T04:49:34.137558+0000","last_became_peered":"2024-05-09T04:49:34.137558+0000","last_unstale":"2024-05-09T04:52:55.555257+0000","last_undegraded":"2024-05-09T04:52:55.555257+0000","last_fullsized":"2024-05-09T04:52:55.555257+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:50:16.258945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.552311+0000","last_change":"2024-05-09T04:49:34.137946+0000","last_active":"2024-05-09T04:52:55.552311+0000","last_peered":"2024-05-09T04:52:55.552311+0000","last_clean":"2024-05-09T04:52:55.552311+0000","last_became_active":"2024-05-09T04:49:34.137498+0000","last_became_peered":"2024-05-09T04:49:34.137498+0000","last_unstale":"2024-05-09T04:52:55.552311+0000","last_undegraded":"2024-05-09T04:52:55.552311+0000","last_fullsized":"2024-05-09T04:52:55.552311+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:36:54.473689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.551855+0000","last_change":"2024-05-09T04:49:34.162103+0000","last_active":"2024-05-09T04:52:55.551855+0000","last_peered":"2024-05-09T04:52:55.551855+0000","last_clean":"2024-05-09T04:52:55.551855+0000","last_became_active":"2024-05-09T04:49:34.161681+0000","last_became_peered":"2024-05-09T04:49:34.161681+0000","last_unstale":"2024-05-09T04:52:55.551855+0000","last_undegraded":"2024-05-09T04:52:55.551855+0000","last_fullsized":"2024-05-09T04:52:55.551855+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:36:10.460401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.528201+0000","last_change":"2024-05-09T04:49:34.136538+0000","last_active":"2024-05-09T04:52:55.528201+0000","last_peered":"2024-05-09T04:52:55.528201+0000","last_clean":"2024-05-09T04:52:55.528201+0000","last_became_active":"2024-05-09T04:49:34.134903+0000","last_became_peered":"2024-05-09T04:49:34.134903+0000","last_unstale":"2024-05-09T04:52:55.528201+0000","last_undegraded":"2024-05-09T04:52:55.528201+0000","last_fullsized":"2024-05-09T04:52:55.528201+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:44:48.673600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71975,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1247,"num_read_kb":708,"num_write":259,"num_write_kb":151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":215925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":224,"ondisk_log_size":224,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678075,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":510251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874306,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37016,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708136,"statfs":{"total":95995035648,"available":95957131264,"internally_reserved":0,"allocated":1720320,"data_stored":1161976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037830,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":514165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234062,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":510335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430291,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1642496,"data_stored":1107617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593817,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":505573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":571092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738491,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":507029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411443,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1114112,"data_stored":571844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149803,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":505281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953574,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":510179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790047,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1650688,"data_stored":1102667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T04:53:08.498 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-09T04:53:08.498 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-09T04:53:08.498 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-09T04:53:08.498 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph health --format=json 2024-05-09T04:53:09.012 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:53:09.170 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:08 smithi006 ceph-mon[29835]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:53:09.170 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:08 smithi006 ceph-mon[29835]: pgmap v445: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-09T04:53:09.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:08 smithi186 ceph-mon[34668]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:53:09.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:08 smithi186 ceph-mon[34668]: pgmap v445: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-09T04:53:09.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:08 smithi169 ceph-mon[34075]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:53:09.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:08 smithi169 ceph-mon[34075]: pgmap v445: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-09T04:53:11.019 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:10 smithi006 ceph-mon[29835]: pgmap v446: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-09T04:53:11.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:10 smithi186 ceph-mon[34668]: pgmap v446: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-09T04:53:11.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:10 smithi169 ceph-mon[34075]: pgmap v446: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-09T04:53:11.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.380+0000 7fa2e2e00700 1 -- 172.21.15.6:0/2960359915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2dc10b110 msgr2=0x7fa2dc117620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:11.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.380+0000 7fa2e2e00700 1 --2- 172.21.15.6:0/2960359915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2dc10b110 0x7fa2dc117620 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa2d400b0a0 tx=0x7fa2d400b3b0 comp rx=0 tx=0).stop 2024-05-09T04:53:11.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.380+0000 7fa2e2e00700 1 -- 172.21.15.6:0/2960359915 shutdown_connections 2024-05-09T04:53:11.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.380+0000 7fa2e2e00700 1 --2- 172.21.15.6:0/2960359915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2dc10b110 0x7fa2dc117620 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:11.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.380+0000 7fa2e2e00700 1 --2- 172.21.15.6:0/2960359915 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2dc10f6c0 0x7fa2dc10fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:11.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.380+0000 7fa2e2e00700 1 --2- 172.21.15.6:0/2960359915 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2dc10a7c0 0x7fa2dc10abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:11.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.380+0000 7fa2e2e00700 1 -- 172.21.15.6:0/2960359915 >> 172.21.15.6:0/2960359915 conn(0x7fa2dc076b00 msgr2=0x7fa2dc076f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:11.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.380+0000 7fa2e2e00700 1 -- 172.21.15.6:0/2960359915 shutdown_connections 2024-05-09T04:53:11.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.380+0000 7fa2e2e00700 1 -- 172.21.15.6:0/2960359915 wait complete. 2024-05-09T04:53:11.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.381+0000 7fa2e2e00700 1 Processor -- start 2024-05-09T04:53:11.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.381+0000 7fa2e2e00700 1 -- start start 2024-05-09T04:53:11.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.382+0000 7fa2e2e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2dc10a7c0 0x7fa2dc1a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:11.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.382+0000 7fa2e2e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2dc10b110 0x7fa2dc1a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:11.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.382+0000 7fa2e2e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2dc10f6c0 0x7fa2dc1a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:11.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.382+0000 7fa2e2e00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc11a8b0 con 0x7fa2dc10b110 2024-05-09T04:53:11.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.382+0000 7fa2e2e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc11a730 con 0x7fa2dc10a7c0 2024-05-09T04:53:11.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.382+0000 7fa2e2e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc11aa30 con 0x7fa2dc10f6c0 2024-05-09T04:53:11.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.382+0000 7fa2e1000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2dc10b110 0x7fa2dc1a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:11.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.382+0000 7fa2e1a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2dc10a7c0 0x7fa2dc1a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:11.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.382+0000 7fa2e1a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2dc10a7c0 0x7fa2dc1a1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:58734/0 (socket says 172.21.15.6:58734) 2024-05-09T04:53:11.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.382+0000 7fa2e1000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2dc10b110 0x7fa2dc1a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54978/0 (socket says 172.21.15.6:54978) 2024-05-09T04:53:11.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.382+0000 7fa2e1a00700 1 -- 172.21.15.6:0/3931576469 learned_addr learned my addr 172.21.15.6:0/3931576469 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:11.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.382+0000 7fa2e2400700 1 --2- 172.21.15.6:0/3931576469 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2dc10f6c0 0x7fa2dc1a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:11.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.383+0000 7fa2e1000700 1 -- 172.21.15.6:0/3931576469 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2dc10f6c0 msgr2=0x7fa2dc1a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:11.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.383+0000 7fa2e1000700 1 --2- 172.21.15.6:0/3931576469 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2dc10f6c0 0x7fa2dc1a5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:11.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.383+0000 7fa2e1000700 1 -- 172.21.15.6:0/3931576469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2dc10a7c0 msgr2=0x7fa2dc1a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:11.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.383+0000 7fa2e1000700 1 --2- 172.21.15.6:0/3931576469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2dc10a7c0 0x7fa2dc1a1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:11.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.383+0000 7fa2e1000700 1 -- 172.21.15.6:0/3931576469 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2dc1a6030 con 0x7fa2dc10b110 2024-05-09T04:53:11.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.383+0000 7fa2e1a00700 1 --2- 172.21.15.6:0/3931576469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2dc10a7c0 0x7fa2dc1a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:53:11.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.383+0000 7fa2e1000700 1 --2- 172.21.15.6:0/3931576469 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2dc10b110 0x7fa2dc1a1700 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800e9f0 tx=0x7fa2c800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:11.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.384+0000 7fa2da000700 1 -- 172.21.15.6:0/3931576469 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c800c9c0 con 0x7fa2dc10b110 2024-05-09T04:53:11.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.384+0000 7fa2e2e00700 1 -- 172.21.15.6:0/3931576469 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2dc1a6320 con 0x7fa2dc10b110 2024-05-09T04:53:11.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.384+0000 7fa2da000700 1 -- 172.21.15.6:0/3931576469 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2c800cb60 con 0x7fa2dc10b110 2024-05-09T04:53:11.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.384+0000 7fa2da000700 1 -- 172.21.15.6:0/3931576469 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8018680 con 0x7fa2dc10b110 2024-05-09T04:53:11.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.384+0000 7fa2e2e00700 1 -- 172.21.15.6:0/3931576469 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2dc1adc40 con 0x7fa2dc10b110 2024-05-09T04:53:11.386 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.385+0000 7fa2e2e00700 1 -- 172.21.15.6:0/3931576469 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2dc0512c0 con 0x7fa2dc10b110 2024-05-09T04:53:11.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.388+0000 7fa2da000700 1 -- 172.21.15.6:0/3931576469 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fa2c8010430 con 0x7fa2dc10b110 2024-05-09T04:53:11.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.389+0000 7fa2da000700 1 --2- 172.21.15.6:0/3931576469 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa2c40b2880 0x7fa2c40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:11.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.390+0000 7fa2da000700 1 -- 172.21.15.6:0/3931576469 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fa2c80e2ac0 con 0x7fa2dc10b110 2024-05-09T04:53:11.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.390+0000 7fa2e1a00700 1 --2- 172.21.15.6:0/3931576469 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa2c40b2880 0x7fa2c40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:11.391 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.390+0000 7fa2e1a00700 1 --2- 172.21.15.6:0/3931576469 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa2c40b2880 0x7fa2c40b4d90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009680 tx=0x7fa2d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:11.392 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.392+0000 7fa2da000700 1 -- 172.21.15.6:0/3931576469 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa2c80aa400 con 0x7fa2dc10b110 2024-05-09T04:53:11.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.686+0000 7fa2e2e00700 1 -- 172.21.15.6:0/3931576469 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa2dc02e5f0 con 0x7fa2dc10b110 2024-05-09T04:53:11.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.687+0000 7fa2da000700 1 -- 172.21.15.6:0/3931576469 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fa2c80a9d00 con 0x7fa2dc10b110 2024-05-09T04:53:11.688 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:53:11.688 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-09T04:53:11.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.690+0000 7fa2e2e00700 1 -- 172.21.15.6:0/3931576469 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa2c40b2880 msgr2=0x7fa2c40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:11.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.690+0000 7fa2e2e00700 1 --2- 172.21.15.6:0/3931576469 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa2c40b2880 0x7fa2c40b4d90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009680 tx=0x7fa2d0009260 comp rx=0 tx=0).stop 2024-05-09T04:53:11.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.690+0000 7fa2e2e00700 1 -- 172.21.15.6:0/3931576469 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2dc10b110 msgr2=0x7fa2dc1a1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:11.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.690+0000 7fa2e2e00700 1 --2- 172.21.15.6:0/3931576469 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2dc10b110 0x7fa2dc1a1700 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800e9f0 tx=0x7fa2c800ed00 comp rx=0 tx=0).stop 2024-05-09T04:53:11.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.691+0000 7fa2e2e00700 1 -- 172.21.15.6:0/3931576469 shutdown_connections 2024-05-09T04:53:11.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.691+0000 7fa2e2e00700 1 --2- 172.21.15.6:0/3931576469 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2dc10f6c0 0x7fa2dc1a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:11.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.691+0000 7fa2e2e00700 1 --2- 172.21.15.6:0/3931576469 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa2c40b2880 0x7fa2c40b4d90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:11.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.691+0000 7fa2e2e00700 1 --2- 172.21.15.6:0/3931576469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2dc10a7c0 0x7fa2dc1a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:11.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.691+0000 7fa2e2e00700 1 --2- 172.21.15.6:0/3931576469 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2dc10b110 0x7fa2dc1a1700 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:11.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.691+0000 7fa2e2e00700 1 -- 172.21.15.6:0/3931576469 >> 172.21.15.6:0/3931576469 conn(0x7fa2dc076b00 msgr2=0x7fa2dc1087c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:11.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.691+0000 7fa2e2e00700 1 -- 172.21.15.6:0/3931576469 shutdown_connections 2024-05-09T04:53:11.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:11.691+0000 7fa2e2e00700 1 -- 172.21.15.6:0/3931576469 wait complete. 2024-05-09T04:53:11.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:11 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/3931576469' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-09T04:53:12.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:11 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/3931576469' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-09T04:53:12.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:11 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/3931576469' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-09T04:53:12.511 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-09T04:53:12.512 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:53:13.006 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:53:13.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:12 smithi006 ceph-mon[29835]: pgmap v447: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 341 B/s wr, 11 op/s 2024-05-09T04:53:13.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:12 smithi186 ceph-mon[34668]: pgmap v447: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 341 B/s wr, 11 op/s 2024-05-09T04:53:13.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:12 smithi169 ceph-mon[34075]: pgmap v447: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 341 B/s wr, 11 op/s 2024-05-09T04:53:15.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:14 smithi006 ceph-mon[29835]: pgmap v448: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:53:15.201 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.200+0000 7f5933e00700 1 -- 172.21.15.6:0/1826327369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5934106620 msgr2=0x7f5934106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:15.201 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.200+0000 7f5933e00700 1 --2- 172.21.15.6:0/1826327369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5934106620 0x7f5934106a30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f59240099c0 tx=0x7f5924009cd0 comp rx=0 tx=0).stop 2024-05-09T04:53:15.201 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.200+0000 7f5933e00700 1 -- 172.21.15.6:0/1826327369 shutdown_connections 2024-05-09T04:53:15.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.200+0000 7f5933e00700 1 --2- 172.21.15.6:0/1826327369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f593410b8a0 0x7f5934113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:15.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.200+0000 7f5933e00700 1 --2- 172.21.15.6:0/1826327369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5934106620 0x7f5934106a30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:15.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.200+0000 7f5933e00700 1 --2- 172.21.15.6:0/1826327369 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f593410af50 0x7f593410b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:15.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.200+0000 7f5933e00700 1 -- 172.21.15.6:0/1826327369 >> 172.21.15.6:0/1826327369 conn(0x7f5934102030 msgr2=0x7f5934104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:15.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.201+0000 7f5933e00700 1 -- 172.21.15.6:0/1826327369 shutdown_connections 2024-05-09T04:53:15.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.201+0000 7f5933e00700 1 -- 172.21.15.6:0/1826327369 wait complete. 2024-05-09T04:53:15.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.202+0000 7f5933e00700 1 Processor -- start 2024-05-09T04:53:15.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.202+0000 7f5933e00700 1 -- start start 2024-05-09T04:53:15.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.203+0000 7f5933e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5934106620 0x7f593419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:15.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.203+0000 7f5933e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f593410af50 0x7f593419d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:15.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.203+0000 7f5933e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f593410b8a0 0x7f59341a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:15.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.203+0000 7f5933e00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5934116520 con 0x7f5934106620 2024-05-09T04:53:15.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.203+0000 7f5933e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f59341163a0 con 0x7f593410af50 2024-05-09T04:53:15.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.203+0000 7f5933e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f59341166a0 con 0x7f593410b8a0 2024-05-09T04:53:15.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.203+0000 7f5932a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5934106620 0x7f593419ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:15.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.203+0000 7f5933400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f593410b8a0 0x7f59341a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:15.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.203+0000 7f5932a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5934106620 0x7f593419ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54990/0 (socket says 172.21.15.6:54990) 2024-05-09T04:53:15.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.203+0000 7f5932000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f593410af50 0x7f593419d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:15.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.203+0000 7f5932a00700 1 -- 172.21.15.6:0/2762532503 learned_addr learned my addr 172.21.15.6:0/2762532503 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:15.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.204+0000 7f5932a00700 1 -- 172.21.15.6:0/2762532503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f593410b8a0 msgr2=0x7f59341a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:15.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.204+0000 7f5932a00700 1 --2- 172.21.15.6:0/2762532503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f593410b8a0 0x7f59341a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:15.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.204+0000 7f5932a00700 1 -- 172.21.15.6:0/2762532503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f593410af50 msgr2=0x7f593419d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:15.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.204+0000 7f5932a00700 1 --2- 172.21.15.6:0/2762532503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f593410af50 0x7f593419d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:15.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.204+0000 7f5932a00700 1 -- 172.21.15.6:0/2762532503 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59341a1d80 con 0x7f5934106620 2024-05-09T04:53:15.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.204+0000 7f5933400700 1 --2- 172.21.15.6:0/2762532503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f593410b8a0 0x7f59341a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:53:15.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.204+0000 7f5932a00700 1 --2- 172.21.15.6:0/2762532503 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5934106620 0x7f593419ceb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5924005910 tx=0x7f5924004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:15.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.204+0000 7f592b400700 1 -- 172.21.15.6:0/2762532503 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5924026070 con 0x7f5934106620 2024-05-09T04:53:15.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.205+0000 7f5933e00700 1 -- 172.21.15.6:0/2762532503 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59341a2010 con 0x7f5934106620 2024-05-09T04:53:15.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.205+0000 7f592b400700 1 -- 172.21.15.6:0/2762532503 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5924003c70 con 0x7f5934106620 2024-05-09T04:53:15.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.205+0000 7f592b400700 1 -- 172.21.15.6:0/2762532503 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592400fa40 con 0x7f5934106620 2024-05-09T04:53:15.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.205+0000 7f5933e00700 1 -- 172.21.15.6:0/2762532503 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59341a9990 con 0x7f5934106620 2024-05-09T04:53:15.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.206+0000 7f5933e00700 1 -- 172.21.15.6:0/2762532503 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59340512c0 con 0x7f5934106620 2024-05-09T04:53:15.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.209+0000 7f592b400700 1 -- 172.21.15.6:0/2762532503 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f5924029070 con 0x7f5934106620 2024-05-09T04:53:15.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.210+0000 7f592b400700 1 --2- 172.21.15.6:0/2762532503 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f59180b2880 0x7f59180b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:15.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.210+0000 7f592b400700 1 -- 172.21.15.6:0/2762532503 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f59240e33e0 con 0x7f5934106620 2024-05-09T04:53:15.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.210+0000 7f5932000700 1 --2- 172.21.15.6:0/2762532503 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f59180b2880 0x7f59180b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:15.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.211+0000 7f5932000700 1 --2- 172.21.15.6:0/2762532503 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f59180b2880 0x7f59180b4d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f59341095b0 tx=0x7f5920009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:15.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.213+0000 7f592b400700 1 -- 172.21.15.6:0/2762532503 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5924013070 con 0x7f5934106620 2024-05-09T04:53:15.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:14 smithi186 ceph-mon[34668]: pgmap v448: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:53:15.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:14 smithi169 ceph-mon[34075]: pgmap v448: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:53:15.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.501+0000 7f5933e00700 1 -- 172.21.15.6:0/2762532503 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f593419e210 con 0x7f5934106620 2024-05-09T04:53:15.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.503+0000 7f592b400700 1 -- 172.21.15.6:0/2762532503 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11927 (secure 0 0 0) 0x7f59240aae10 con 0x7f5934106620 2024-05-09T04:53:15.504 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:53:15.504 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/162094599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":162094599},{"type":"v1","addr":"172.21.15.6:6835","nonce":162094599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/589294363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":589294363},{"type":"v1","addr":"172.21.15.6:6841","nonce":589294363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:49:36.920376+0000","modified":"2024-05-09T04:51:07.589004+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24554,"mds_1":24530,"mds_2":24512,"mds_3":24511,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24442":{"gid":24442,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6833/2977145206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2977145206},{"type":"v1","addr":"172.21.15.186:6833","nonce":2977145206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6833/133345948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":133345948},{"type":"v1","addr":"172.21.15.169:6833","nonce":133345948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.6:6837/1281521088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":1281521088},{"type":"v1","addr":"172.21.15.6:6837","nonce":1281521088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2268322007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2268322007},{"type":"v1","addr":"172.21.15.186:6835","nonce":2268322007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/3811005052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3811005052},{"type":"v1","addr":"172.21.15.169:6835","nonce":3811005052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.186:6837/1647607118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1647607118},{"type":"v1","addr":"172.21.15.186:6837","nonce":1647607118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":32,"addr":"172.21.15.186:6839/3954883105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":3954883105},{"type":"v1","addr":"172.21.15.186:6839","nonce":3954883105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.169:6837/1845715294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1845715294},{"type":"v1","addr":"172.21.15.169:6837","nonce":1845715294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.6:6839/2780306492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":2780306492},{"type":"v1","addr":"172.21.15.6:6839","nonce":2780306492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.169:6839/614388771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":614388771},{"type":"v1","addr":"172.21.15.169:6839","nonce":614388771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24493,24511,24530,24554,24512]},"id":1}]} 2024-05-09T04:53:15.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.506+0000 7f5933e00700 1 -- 172.21.15.6:0/2762532503 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f59180b2880 msgr2=0x7f59180b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:15.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.506+0000 7f5933e00700 1 --2- 172.21.15.6:0/2762532503 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f59180b2880 0x7f59180b4d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f59341095b0 tx=0x7f5920009260 comp rx=0 tx=0).stop 2024-05-09T04:53:15.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.506+0000 7f5933e00700 1 -- 172.21.15.6:0/2762532503 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5934106620 msgr2=0x7f593419ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:15.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.506+0000 7f5933e00700 1 --2- 172.21.15.6:0/2762532503 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5934106620 0x7f593419ceb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5924005910 tx=0x7f5924004270 comp rx=0 tx=0).stop 2024-05-09T04:53:15.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.506+0000 7f5933e00700 1 -- 172.21.15.6:0/2762532503 shutdown_connections 2024-05-09T04:53:15.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.506+0000 7f5933e00700 1 --2- 172.21.15.6:0/2762532503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f593410b8a0 0x7f59341a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:15.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.506+0000 7f5933e00700 1 --2- 172.21.15.6:0/2762532503 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5934106620 0x7f593419ceb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:15.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.506+0000 7f5933e00700 1 --2- 172.21.15.6:0/2762532503 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f59180b2880 0x7f59180b4d90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:15.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.506+0000 7f5933e00700 1 --2- 172.21.15.6:0/2762532503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f593410af50 0x7f593419d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:15.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.506+0000 7f5933e00700 1 -- 172.21.15.6:0/2762532503 >> 172.21.15.6:0/2762532503 conn(0x7f5934102030 msgr2=0x7f593410a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:15.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.506+0000 7f5933e00700 1 -- 172.21.15.6:0/2762532503 shutdown_connections 2024-05-09T04:53:15.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:15.506+0000 7f5933e00700 1 -- 172.21.15.6:0/2762532503 wait complete. 2024-05-09T04:53:15.508 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 29 2024-05-09T04:53:16.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:15 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/2762532503' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:53:16.360 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:15 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/2762532503' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:53:16.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:15 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/2762532503' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:53:16.442 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:53:16.913 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:53:17.281 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:16 smithi006 ceph-mon[29835]: pgmap v449: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:53:17.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:16 smithi186 ceph-mon[34668]: pgmap v449: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:53:17.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:16 smithi169 ceph-mon[34075]: pgmap v449: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:53:19.064 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:18 smithi006 ceph-mon[29835]: pgmap v450: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:53:19.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.317+0000 7f6228e00700 1 -- 172.21.15.6:0/3996551607 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f622410b2e0 msgr2=0x7f6224117810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:19.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.317+0000 7f6228e00700 1 --2- 172.21.15.6:0/3996551607 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f622410b2e0 0x7f6224117810 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f621c009f90 tx=0x7f621c00b330 comp rx=0 tx=0).stop 2024-05-09T04:53:19.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.318+0000 7f6228e00700 1 -- 172.21.15.6:0/3996551607 shutdown_connections 2024-05-09T04:53:19.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.318+0000 7f6228e00700 1 --2- 172.21.15.6:0/3996551607 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f622410b2e0 0x7f6224117810 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:19.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.318+0000 7f6228e00700 1 --2- 172.21.15.6:0/3996551607 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6224110970 0x7f6224110d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:19.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.318+0000 7f6228e00700 1 --2- 172.21.15.6:0/3996551607 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f622410a990 0x7f622410ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:19.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.318+0000 7f6228e00700 1 -- 172.21.15.6:0/3996551607 >> 172.21.15.6:0/3996551607 conn(0x7f6224077c80 msgr2=0x7f62240780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:19.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.318+0000 7f6228e00700 1 -- 172.21.15.6:0/3996551607 shutdown_connections 2024-05-09T04:53:19.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.318+0000 7f6228e00700 1 -- 172.21.15.6:0/3996551607 wait complete. 2024-05-09T04:53:19.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.319+0000 7f6228e00700 1 Processor -- start 2024-05-09T04:53:19.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.320+0000 7f6228e00700 1 -- start start 2024-05-09T04:53:19.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.320+0000 7f6228e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f622410a990 0x7f62241a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:19.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.320+0000 7f6228e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f622410b2e0 0x7f62241a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:19.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.320+0000 7f6228e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6224110970 0x7f62241a5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:19.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.320+0000 7f6228e00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f622411aae0 con 0x7f622410a990 2024-05-09T04:53:19.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.320+0000 7f6228e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f622411a960 con 0x7f622410b2e0 2024-05-09T04:53:19.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.320+0000 7f6228e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f622411ac60 con 0x7f6224110970 2024-05-09T04:53:19.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.321+0000 7f6221e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f622410a990 0x7f62241a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:19.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.321+0000 7f6221400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f622410b2e0 0x7f62241a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:19.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.321+0000 7f6221400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f622410b2e0 0x7f62241a1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:52536/0 (socket says 172.21.15.6:52536) 2024-05-09T04:53:19.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.321+0000 7f6221400700 1 -- 172.21.15.6:0/879387780 learned_addr learned my addr 172.21.15.6:0/879387780 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:19.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.321+0000 7f6222800700 1 --2- 172.21.15.6:0/879387780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6224110970 0x7f62241a5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:19.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.321+0000 7f6221400700 1 -- 172.21.15.6:0/879387780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6224110970 msgr2=0x7f62241a5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:19.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.321+0000 7f6221400700 1 --2- 172.21.15.6:0/879387780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6224110970 0x7f62241a5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:19.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.321+0000 7f6221400700 1 -- 172.21.15.6:0/879387780 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f622410a990 msgr2=0x7f62241a13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:19.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.321+0000 7f6221400700 1 --2- 172.21.15.6:0/879387780 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f622410a990 0x7f62241a13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:19.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.321+0000 7f6221400700 1 -- 172.21.15.6:0/879387780 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62241a62c0 con 0x7f622410b2e0 2024-05-09T04:53:19.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.322+0000 7f6221400700 1 --2- 172.21.15.6:0/879387780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f622410b2e0 0x7f62241a1990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f621000e9f0 tx=0x7f621000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:19.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.322+0000 7f621aa00700 1 -- 172.21.15.6:0/879387780 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621000c9c0 con 0x7f622410b2e0 2024-05-09T04:53:19.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.322+0000 7f6228e00700 1 -- 172.21.15.6:0/879387780 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62241a65b0 con 0x7f622410b2e0 2024-05-09T04:53:19.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.322+0000 7f621aa00700 1 -- 172.21.15.6:0/879387780 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f621000cb60 con 0x7f622410b2e0 2024-05-09T04:53:19.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.322+0000 7f621aa00700 1 -- 172.21.15.6:0/879387780 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6210018680 con 0x7f622410b2e0 2024-05-09T04:53:19.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.322+0000 7f6228e00700 1 -- 172.21.15.6:0/879387780 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62241aded0 con 0x7f622410b2e0 2024-05-09T04:53:19.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.323+0000 7f6228e00700 1 -- 172.21.15.6:0/879387780 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f622410c080 con 0x7f622410b2e0 2024-05-09T04:53:19.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.327+0000 7f621aa00700 1 -- 172.21.15.6:0/879387780 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f6210010430 con 0x7f622410b2e0 2024-05-09T04:53:19.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.328+0000 7f621aa00700 1 --2- 172.21.15.6:0/879387780 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f62080b2880 0x7f62080b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:19.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.328+0000 7f621aa00700 1 -- 172.21.15.6:0/879387780 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f62100e2b90 con 0x7f622410b2e0 2024-05-09T04:53:19.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.329+0000 7f6221e00700 1 --2- 172.21.15.6:0/879387780 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f62080b2880 0x7f62080b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:19.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.329+0000 7f6221e00700 1 --2- 172.21.15.6:0/879387780 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f62080b2880 0x7f62080b4d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f622407d0f0 tx=0x7f6214009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:19.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.330+0000 7f621aa00700 1 -- 172.21.15.6:0/879387780 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f62100aa4d0 con 0x7f622410b2e0 2024-05-09T04:53:19.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:18 smithi186 ceph-mon[34668]: pgmap v450: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:53:19.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:18 smithi169 ceph-mon[34075]: pgmap v450: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:53:19.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.618+0000 7f6228e00700 1 -- 172.21.15.6:0/879387780 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62241a2750 con 0x7f622410b2e0 2024-05-09T04:53:19.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.619+0000 7f621aa00700 1 -- 172.21.15.6:0/879387780 <== mon.2 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11927 (secure 0 0 0) 0x7f62100a9dd0 con 0x7f622410b2e0 2024-05-09T04:53:19.620 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:53:19.621 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/162094599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":162094599},{"type":"v1","addr":"172.21.15.6:6835","nonce":162094599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/589294363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":589294363},{"type":"v1","addr":"172.21.15.6:6841","nonce":589294363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:49:36.920376+0000","modified":"2024-05-09T04:51:07.589004+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24554,"mds_1":24530,"mds_2":24512,"mds_3":24511,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24442":{"gid":24442,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6833/2977145206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2977145206},{"type":"v1","addr":"172.21.15.186:6833","nonce":2977145206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6833/133345948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":133345948},{"type":"v1","addr":"172.21.15.169:6833","nonce":133345948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.6:6837/1281521088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":1281521088},{"type":"v1","addr":"172.21.15.6:6837","nonce":1281521088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2268322007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2268322007},{"type":"v1","addr":"172.21.15.186:6835","nonce":2268322007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/3811005052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3811005052},{"type":"v1","addr":"172.21.15.169:6835","nonce":3811005052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.186:6837/1647607118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1647607118},{"type":"v1","addr":"172.21.15.186:6837","nonce":1647607118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":32,"addr":"172.21.15.186:6839/3954883105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":3954883105},{"type":"v1","addr":"172.21.15.186:6839","nonce":3954883105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.169:6837/1845715294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1845715294},{"type":"v1","addr":"172.21.15.169:6837","nonce":1845715294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.6:6839/2780306492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":2780306492},{"type":"v1","addr":"172.21.15.6:6839","nonce":2780306492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.169:6839/614388771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":614388771},{"type":"v1","addr":"172.21.15.169:6839","nonce":614388771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24493,24511,24530,24554,24512]},"id":1}]} 2024-05-09T04:53:19.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.622+0000 7f6228e00700 1 -- 172.21.15.6:0/879387780 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f62080b2880 msgr2=0x7f62080b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:19.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.622+0000 7f6228e00700 1 --2- 172.21.15.6:0/879387780 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f62080b2880 0x7f62080b4d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f622407d0f0 tx=0x7f6214009260 comp rx=0 tx=0).stop 2024-05-09T04:53:19.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.622+0000 7f6228e00700 1 -- 172.21.15.6:0/879387780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f622410b2e0 msgr2=0x7f62241a1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:19.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.622+0000 7f6228e00700 1 --2- 172.21.15.6:0/879387780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f622410b2e0 0x7f62241a1990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f621000e9f0 tx=0x7f621000ed00 comp rx=0 tx=0).stop 2024-05-09T04:53:19.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.622+0000 7f6228e00700 1 -- 172.21.15.6:0/879387780 shutdown_connections 2024-05-09T04:53:19.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.622+0000 7f6228e00700 1 --2- 172.21.15.6:0/879387780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6224110970 0x7f62241a5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:19.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.622+0000 7f6228e00700 1 --2- 172.21.15.6:0/879387780 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f622410a990 0x7f62241a13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:19.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.622+0000 7f6228e00700 1 --2- 172.21.15.6:0/879387780 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f62080b2880 0x7f62080b4d90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:19.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.622+0000 7f6228e00700 1 --2- 172.21.15.6:0/879387780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f622410b2e0 0x7f62241a1990 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:19.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.622+0000 7f6228e00700 1 -- 172.21.15.6:0/879387780 >> 172.21.15.6:0/879387780 conn(0x7f6224077c80 msgr2=0x7f622407b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:19.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.623+0000 7f6228e00700 1 -- 172.21.15.6:0/879387780 shutdown_connections 2024-05-09T04:53:19.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:19.623+0000 7f6228e00700 1 -- 172.21.15.6:0/879387780 wait complete. 2024-05-09T04:53:19.625 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 29 2024-05-09T04:53:20.268 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:19 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/879387780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:53:20.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:19 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/879387780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:53:20.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:19 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/879387780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:53:20.572 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T04:53:21.028 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:53:21.328 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:20 smithi006 ceph-mon[29835]: pgmap v451: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:53:21.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:20 smithi186 ceph-mon[34668]: pgmap v451: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:53:21.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:20 smithi169 ceph-mon[34075]: pgmap v451: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:53:22.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:21 smithi186 ceph-mon[34668]: pgmap v452: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:53:22.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:21 smithi169 ceph-mon[34075]: pgmap v452: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:53:22.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:21 smithi006 ceph-mon[29835]: pgmap v452: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:53:23.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.526+0000 7f6e97400700 1 -- 172.21.15.6:0/2539666596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e9010e8f0 msgr2=0x7f6e90110dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:23.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.526+0000 7f6e97400700 1 --2- 172.21.15.6:0/2539666596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e9010e8f0 0x7f6e90110dd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6e88009f90 tx=0x7f6e8800b330 comp rx=0 tx=0).stop 2024-05-09T04:53:23.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.526+0000 7f6e97400700 1 -- 172.21.15.6:0/2539666596 shutdown_connections 2024-05-09T04:53:23.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.526+0000 7f6e97400700 1 --2- 172.21.15.6:0/2539666596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e9010e8f0 0x7f6e90110dd0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:23.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.526+0000 7f6e97400700 1 --2- 172.21.15.6:0/2539666596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e90105480 0x7f6e90105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:23.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.526+0000 7f6e97400700 1 --2- 172.21.15.6:0/2539666596 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e90105dd0 0x7f6e9010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:23.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.526+0000 7f6e97400700 1 -- 172.21.15.6:0/2539666596 >> 172.21.15.6:0/2539666596 conn(0x7f6e900fef30 msgr2=0x7f6e90101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:23.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.526+0000 7f6e97400700 1 -- 172.21.15.6:0/2539666596 shutdown_connections 2024-05-09T04:53:23.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.526+0000 7f6e97400700 1 -- 172.21.15.6:0/2539666596 wait complete. 2024-05-09T04:53:23.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.528+0000 7f6e97400700 1 Processor -- start 2024-05-09T04:53:23.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.528+0000 7f6e97400700 1 -- start start 2024-05-09T04:53:23.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.529+0000 7f6e97400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e90105480 0x7f6e9019d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:23.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.529+0000 7f6e97400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e90105dd0 0x7f6e9019d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:23.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.529+0000 7f6e97400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e9010e8f0 0x7f6e901a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:23.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.529+0000 7f6e97400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6e901144a0 con 0x7f6e90105dd0 2024-05-09T04:53:23.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.529+0000 7f6e97400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6e90114320 con 0x7f6e9010e8f0 2024-05-09T04:53:23.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.529+0000 7f6e97400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6e90114620 con 0x7f6e90105480 2024-05-09T04:53:23.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.529+0000 7f6e94a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e90105480 0x7f6e9019d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:23.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.529+0000 7f6e8fe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e90105dd0 0x7f6e9019d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:23.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.529+0000 7f6e94a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e90105480 0x7f6e9019d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:60522/0 (socket says 172.21.15.6:60522) 2024-05-09T04:53:23.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.529+0000 7f6e94a00700 1 -- 172.21.15.6:0/583529978 learned_addr learned my addr 172.21.15.6:0/583529978 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:23.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.529+0000 7f6e95400700 1 --2- 172.21.15.6:0/583529978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e9010e8f0 0x7f6e901a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:23.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.530+0000 7f6e94a00700 1 -- 172.21.15.6:0/583529978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e9010e8f0 msgr2=0x7f6e901a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:23.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.530+0000 7f6e94a00700 1 --2- 172.21.15.6:0/583529978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e9010e8f0 0x7f6e901a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:23.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.530+0000 7f6e94a00700 1 -- 172.21.15.6:0/583529978 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e90105dd0 msgr2=0x7f6e9019d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:23.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.530+0000 7f6e94a00700 1 --2- 172.21.15.6:0/583529978 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e90105dd0 0x7f6e9019d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:23.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.530+0000 7f6e94a00700 1 -- 172.21.15.6:0/583529978 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e901a1f40 con 0x7f6e90105480 2024-05-09T04:53:23.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.530+0000 7f6e95400700 1 --2- 172.21.15.6:0/583529978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e9010e8f0 0x7f6e901a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:53:23.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.530+0000 7f6e94a00700 1 --2- 172.21.15.6:0/583529978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e90105480 0x7f6e9019d070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400e9a0 tx=0x7f6e8400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:23.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.531+0000 7f6e8d600700 1 -- 172.21.15.6:0/583529978 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e8400ca10 con 0x7f6e90105480 2024-05-09T04:53:23.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.531+0000 7f6e8d600700 1 -- 172.21.15.6:0/583529978 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e8400cbb0 con 0x7f6e90105480 2024-05-09T04:53:23.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.531+0000 7f6e97400700 1 -- 172.21.15.6:0/583529978 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e901a2230 con 0x7f6e90105480 2024-05-09T04:53:23.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.531+0000 7f6e8d600700 1 -- 172.21.15.6:0/583529978 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e84018680 con 0x7f6e90105480 2024-05-09T04:53:23.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.531+0000 7f6e97400700 1 -- 172.21.15.6:0/583529978 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e901a9b50 con 0x7f6e90105480 2024-05-09T04:53:23.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.532+0000 7f6e97400700 1 -- 172.21.15.6:0/583529978 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e900512c0 con 0x7f6e90105480 2024-05-09T04:53:23.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.535+0000 7f6e8d600700 1 -- 172.21.15.6:0/583529978 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f6e84010430 con 0x7f6e90105480 2024-05-09T04:53:23.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.536+0000 7f6e8d600700 1 --2- 172.21.15.6:0/583529978 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f6e780b2880 0x7f6e780b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:23.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.537+0000 7f6e8d600700 1 -- 172.21.15.6:0/583529978 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f6e840e2aa0 con 0x7f6e90105480 2024-05-09T04:53:23.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.537+0000 7f6e8fe00700 1 --2- 172.21.15.6:0/583529978 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f6e780b2880 0x7f6e780b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:23.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.537+0000 7f6e8fe00700 1 --2- 172.21.15.6:0/583529978 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f6e780b2880 0x7f6e780b4d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c009750 tx=0x7f6e7c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:23.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.538+0000 7f6e8d600700 1 -- 172.21.15.6:0/583529978 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6e840aa3e0 con 0x7f6e90105480 2024-05-09T04:53:23.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.788+0000 7f6e97400700 1 -- 172.21.15.6:0/583529978 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6e901a9d20 con 0x7f6e90105480 2024-05-09T04:53:23.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.790+0000 7f6e8d600700 1 -- 172.21.15.6:0/583529978 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24659 (secure 0 0 0) 0x7f6e840a9ce0 con 0x7f6e90105480 2024-05-09T04:53:23.791 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:53:23.792 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":89,"fsid":"f74d813c-0dbd-11ef-bc97-c7b262605968","created":"2024-05-09T04:39:35.287470+0000","modified":"2024-05-09T04:50:57.686719+0000","last_up_change":"2024-05-09T04:48:16.919842+0000","last_in_change":"2024-05-09T04:47:56.452645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:43:47.538306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:49:28.805447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T04:49:32.970287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T04:49:45.637000+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1b732fd3-e648-4b0d-aece-083bdaaa5d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6803","nonce":1738750914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6805","nonce":1738750914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6809","nonce":1738750914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6807","nonce":1738750914}]},"public_addr":"172.21.15.6:6803/1738750914","cluster_addr":"172.21.15.6:6805/1738750914","heartbeat_back_addr":"172.21.15.6:6809/1738750914","heartbeat_front_addr":"172.21.15.6:6807/1738750914","state":["exists","up"]},{"osd":1,"uuid":"9770c07d-4243-43bf-a0cd-9d2ac558556c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6801","nonce":2552207737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6803","nonce":2552207737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6807","nonce":2552207737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6805","nonce":2552207737}]},"public_addr":"172.21.15.169:6801/2552207737","cluster_addr":"172.21.15.169:6803/2552207737","heartbeat_back_addr":"172.21.15.169:6807/2552207737","heartbeat_front_addr":"172.21.15.169:6805/2552207737","state":["exists","up"]},{"osd":2,"uuid":"3f50ed89-b2a8-415e-b78d-8510d98d9368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6801","nonce":231435105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6803","nonce":231435105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6807","nonce":231435105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6805","nonce":231435105}]},"public_addr":"172.21.15.186:6801/231435105","cluster_addr":"172.21.15.186:6803/231435105","heartbeat_back_addr":"172.21.15.186:6807/231435105","heartbeat_front_addr":"172.21.15.186:6805/231435105","state":["exists","up"]},{"osd":3,"uuid":"afcbc36b-5669-4044-8dc5-2ebfe7970774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6811","nonce":2846613703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6813","nonce":2846613703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6817","nonce":2846613703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6815","nonce":2846613703}]},"public_addr":"172.21.15.6:6811/2846613703","cluster_addr":"172.21.15.6:6813/2846613703","heartbeat_back_addr":"172.21.15.6:6817/2846613703","heartbeat_front_addr":"172.21.15.6:6815/2846613703","state":["exists","up"]},{"osd":4,"uuid":"bb84cc5e-0297-4e23-be50-7ee2f7e59665","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6809","nonce":1928722453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6811","nonce":1928722453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6815","nonce":1928722453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6813","nonce":1928722453}]},"public_addr":"172.21.15.169:6809/1928722453","cluster_addr":"172.21.15.169:6811/1928722453","heartbeat_back_addr":"172.21.15.169:6815/1928722453","heartbeat_front_addr":"172.21.15.169:6813/1928722453","state":["exists","up"]},{"osd":5,"uuid":"830dc7f3-352e-4b01-8d4c-a9b68dcb40d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6809","nonce":3644445439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6811","nonce":3644445439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6815","nonce":3644445439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6813","nonce":3644445439}]},"public_addr":"172.21.15.186:6809/3644445439","cluster_addr":"172.21.15.186:6811/3644445439","heartbeat_back_addr":"172.21.15.186:6815/3644445439","heartbeat_front_addr":"172.21.15.186:6813/3644445439","state":["exists","up"]},{"osd":6,"uuid":"2e8478cb-c8c9-47ce-a3a5-48ffc959ab24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6819","nonce":3068252891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6821","nonce":3068252891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6825","nonce":3068252891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6823","nonce":3068252891}]},"public_addr":"172.21.15.6:6819/3068252891","cluster_addr":"172.21.15.6:6821/3068252891","heartbeat_back_addr":"172.21.15.6:6825/3068252891","heartbeat_front_addr":"172.21.15.6:6823/3068252891","state":["exists","up"]},{"osd":7,"uuid":"0f432ee2-42a4-4efa-9233-8e5a2fd65d10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6817","nonce":1303584711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6819","nonce":1303584711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6823","nonce":1303584711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6821","nonce":1303584711}]},"public_addr":"172.21.15.169:6817/1303584711","cluster_addr":"172.21.15.169:6819/1303584711","heartbeat_back_addr":"172.21.15.169:6823/1303584711","heartbeat_front_addr":"172.21.15.169:6821/1303584711","state":["exists","up"]},{"osd":8,"uuid":"cf5dca81-131b-48e7-9417-d01d4be47316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6817","nonce":3475163771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6819","nonce":3475163771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6823","nonce":3475163771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6821","nonce":3475163771}]},"public_addr":"172.21.15.186:6817/3475163771","cluster_addr":"172.21.15.186:6819/3475163771","heartbeat_back_addr":"172.21.15.186:6823/3475163771","heartbeat_front_addr":"172.21.15.186:6821/3475163771","state":["exists","up"]},{"osd":9,"uuid":"bca7bedd-1a57-4465-9033-064ca468add2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6827","nonce":3590990973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6829","nonce":3590990973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6833","nonce":3590990973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6831","nonce":3590990973}]},"public_addr":"172.21.15.6:6827/3590990973","cluster_addr":"172.21.15.6:6829/3590990973","heartbeat_back_addr":"172.21.15.6:6833/3590990973","heartbeat_front_addr":"172.21.15.6:6831/3590990973","state":["exists","up"]},{"osd":10,"uuid":"97b0e058-64f8-42ee-9b99-3ef8e39c8a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6825","nonce":61007341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6827","nonce":61007341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6831","nonce":61007341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6829","nonce":61007341}]},"public_addr":"172.21.15.169:6825/61007341","cluster_addr":"172.21.15.169:6827/61007341","heartbeat_back_addr":"172.21.15.169:6831/61007341","heartbeat_front_addr":"172.21.15.169:6829/61007341","state":["exists","up"]},{"osd":11,"uuid":"7f2220b0-24cf-48b2-95ae-1d66e00e57fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6825","nonce":1138940441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6827","nonce":1138940441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6831","nonce":1138940441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6829","nonce":1138940441}]},"public_addr":"172.21.15.186:6825/1138940441","cluster_addr":"172.21.15.186:6827/1138940441","heartbeat_back_addr":"172.21.15.186:6831/1138940441","heartbeat_front_addr":"172.21.15.186:6829/1138940441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:53.766295+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:20.195243+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:44.153812+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:20.472086+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:48.415371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:13.795415+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:50.236398+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:18.066623+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:43.781009+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:47:20.084719+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:47:48.219901+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:48:14.965536+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/3742411767":"2024-05-10T04:40:55.357057+0000","172.21.15.6:0/2410771702":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6801/937713898":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6800/937713898":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6801/3021503778":"2024-05-10T04:40:19.639151+0000","172.21.15.6:6800/3021503778":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/2559510850":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/3810474743":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/4133970555":"2024-05-10T04:40:55.357057+0000","172.21.15.6:0/664817743":"2024-05-10T04:40:19.639151+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:53:23.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.793+0000 7f6e97400700 1 -- 172.21.15.6:0/583529978 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f6e780b2880 msgr2=0x7f6e780b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:23.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.793+0000 7f6e97400700 1 --2- 172.21.15.6:0/583529978 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f6e780b2880 0x7f6e780b4d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c009750 tx=0x7f6e7c009310 comp rx=0 tx=0).stop 2024-05-09T04:53:23.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.794+0000 7f6e97400700 1 -- 172.21.15.6:0/583529978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e90105480 msgr2=0x7f6e9019d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:23.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.794+0000 7f6e97400700 1 --2- 172.21.15.6:0/583529978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e90105480 0x7f6e9019d070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400e9a0 tx=0x7f6e8400ecb0 comp rx=0 tx=0).stop 2024-05-09T04:53:23.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.794+0000 7f6e97400700 1 -- 172.21.15.6:0/583529978 shutdown_connections 2024-05-09T04:53:23.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.794+0000 7f6e97400700 1 --2- 172.21.15.6:0/583529978 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f6e780b2880 0x7f6e780b4d90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:23.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.794+0000 7f6e97400700 1 --2- 172.21.15.6:0/583529978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e9010e8f0 0x7f6e901a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:23.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.794+0000 7f6e97400700 1 --2- 172.21.15.6:0/583529978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e90105480 0x7f6e9019d070 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:23.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.794+0000 7f6e97400700 1 --2- 172.21.15.6:0/583529978 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6e90105dd0 0x7f6e9019d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:23.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.794+0000 7f6e97400700 1 -- 172.21.15.6:0/583529978 >> 172.21.15.6:0/583529978 conn(0x7f6e900fef30 msgr2=0x7f6e90103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:23.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.794+0000 7f6e97400700 1 -- 172.21.15.6:0/583529978 shutdown_connections 2024-05-09T04:53:23.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:23.794+0000 7f6e97400700 1 -- 172.21.15.6:0/583529978 wait complete. 2024-05-09T04:53:24.682 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:53:24.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:24 smithi006 ceph-mon[29835]: pgmap v453: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:53:24.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:24 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/583529978' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:53:25.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:24 smithi186 ceph-mon[34668]: pgmap v453: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:53:25.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:24 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/583529978' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:53:25.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:24 smithi169 ceph-mon[34075]: pgmap v453: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:53:25.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:24 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/583529978' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:53:25.162 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:53:27.060 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:26 smithi006 ceph-mon[29835]: pgmap v454: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:53:27.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:26 smithi186 ceph-mon[34668]: pgmap v454: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:53:27.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:26 smithi169 ceph-mon[34075]: pgmap v454: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:53:27.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.656+0000 7f039e800700 1 -- 172.21.15.6:0/436564045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0398110970 msgr2=0x7f0398110d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:27.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.656+0000 7f039e800700 1 --2- 172.21.15.6:0/436564045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0398110970 0x7f0398110d80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0388009990 tx=0x7f0388009ca0 comp rx=0 tx=0).stop 2024-05-09T04:53:27.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.656+0000 7f039e800700 1 -- 172.21.15.6:0/436564045 shutdown_connections 2024-05-09T04:53:27.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.656+0000 7f039e800700 1 --2- 172.21.15.6:0/436564045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f039810b2e0 0x7f0398117810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:27.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.656+0000 7f039e800700 1 --2- 172.21.15.6:0/436564045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0398110970 0x7f0398110d80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:27.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.656+0000 7f039e800700 1 --2- 172.21.15.6:0/436564045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f039810a990 0x7f039810ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:27.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.656+0000 7f039e800700 1 -- 172.21.15.6:0/436564045 >> 172.21.15.6:0/436564045 conn(0x7f0398077c80 msgr2=0x7f03980780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:27.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.656+0000 7f039e800700 1 -- 172.21.15.6:0/436564045 shutdown_connections 2024-05-09T04:53:27.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.656+0000 7f039e800700 1 -- 172.21.15.6:0/436564045 wait complete. 2024-05-09T04:53:27.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.657+0000 7f039e800700 1 Processor -- start 2024-05-09T04:53:27.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.658+0000 7f039e800700 1 -- start start 2024-05-09T04:53:27.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.658+0000 7f039e800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f039810a990 0x7f03981a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:27.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.658+0000 7f039e800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f039810b2e0 0x7f03981a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:27.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.658+0000 7f039e800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0398110970 0x7f03981a5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:27.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.658+0000 7f039e800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f039811aa90 con 0x7f039810b2e0 2024-05-09T04:53:27.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.658+0000 7f039e800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f039811a910 con 0x7f0398110970 2024-05-09T04:53:27.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.658+0000 7f039e800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f039811ac10 con 0x7f039810a990 2024-05-09T04:53:27.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.659+0000 7f0397400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f039810a990 0x7f03981a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:27.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.659+0000 7f0396a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f039810b2e0 0x7f03981a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:27.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.659+0000 7f0397400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f039810a990 0x7f03981a1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:55064/0 (socket says 172.21.15.6:55064) 2024-05-09T04:53:27.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.659+0000 7f0397400700 1 -- 172.21.15.6:0/697800684 learned_addr learned my addr 172.21.15.6:0/697800684 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:27.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.659+0000 7f0397e00700 1 --2- 172.21.15.6:0/697800684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0398110970 0x7f03981a5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:27.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.659+0000 7f0397400700 1 -- 172.21.15.6:0/697800684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0398110970 msgr2=0x7f03981a5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:27.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.659+0000 7f0397400700 1 --2- 172.21.15.6:0/697800684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0398110970 0x7f03981a5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:27.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.659+0000 7f0397400700 1 -- 172.21.15.6:0/697800684 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f039810b2e0 msgr2=0x7f03981a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:27.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.659+0000 7f0397400700 1 --2- 172.21.15.6:0/697800684 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f039810b2e0 0x7f03981a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:27.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.659+0000 7f0397400700 1 -- 172.21.15.6:0/697800684 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03981a6350 con 0x7f039810a990 2024-05-09T04:53:27.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.660+0000 7f0397400700 1 --2- 172.21.15.6:0/697800684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f039810a990 0x7f03981a1480 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0388000c00 tx=0x7f038800bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:27.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.660+0000 7f038fe00700 1 -- 172.21.15.6:0/697800684 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0388027070 con 0x7f039810a990 2024-05-09T04:53:27.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.660+0000 7f039e800700 1 -- 172.21.15.6:0/697800684 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03981a65e0 con 0x7f039810a990 2024-05-09T04:53:27.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.660+0000 7f039e800700 1 -- 172.21.15.6:0/697800684 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03981adf60 con 0x7f039810a990 2024-05-09T04:53:27.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.660+0000 7f038fe00700 1 -- 172.21.15.6:0/697800684 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0388022470 con 0x7f039810a990 2024-05-09T04:53:27.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.660+0000 7f038fe00700 1 -- 172.21.15.6:0/697800684 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0388017650 con 0x7f039810a990 2024-05-09T04:53:27.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.662+0000 7f039e800700 1 -- 172.21.15.6:0/697800684 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0378005330 con 0x7f039810a990 2024-05-09T04:53:27.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.664+0000 7f038fe00700 1 -- 172.21.15.6:0/697800684 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f038802a070 con 0x7f039810a990 2024-05-09T04:53:27.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.665+0000 7f038fe00700 1 --2- 172.21.15.6:0/697800684 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f037c0b2900 0x7f037c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:27.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.666+0000 7f038fe00700 1 -- 172.21.15.6:0/697800684 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f03880e3330 con 0x7f039810a990 2024-05-09T04:53:27.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.666+0000 7f0396a00700 1 --2- 172.21.15.6:0/697800684 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f037c0b2900 0x7f037c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:27.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.666+0000 7f0396a00700 1 --2- 172.21.15.6:0/697800684 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f037c0b2900 0x7f037c0b4e10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f039807aff0 tx=0x7f0384009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:27.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.670+0000 7f038fe00700 1 -- 172.21.15.6:0/697800684 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0388013070 con 0x7f039810a990 2024-05-09T04:53:27.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.962+0000 7f039e800700 1 -- 172.21.15.6:0/697800684 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0378005620 con 0x7f039810a990 2024-05-09T04:53:27.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.963+0000 7f038fe00700 1 -- 172.21.15.6:0/697800684 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11927 (secure 0 0 0) 0x7f03880aad00 con 0x7f039810a990 2024-05-09T04:53:27.964 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:53:27.965 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/162094599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":162094599},{"type":"v1","addr":"172.21.15.6:6835","nonce":162094599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/589294363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":589294363},{"type":"v1","addr":"172.21.15.6:6841","nonce":589294363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:49:36.920376+0000","modified":"2024-05-09T04:51:07.589004+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24554,"mds_1":24530,"mds_2":24512,"mds_3":24511,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24442":{"gid":24442,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6833/2977145206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2977145206},{"type":"v1","addr":"172.21.15.186:6833","nonce":2977145206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6833/133345948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":133345948},{"type":"v1","addr":"172.21.15.169:6833","nonce":133345948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.6:6837/1281521088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":1281521088},{"type":"v1","addr":"172.21.15.6:6837","nonce":1281521088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2268322007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2268322007},{"type":"v1","addr":"172.21.15.186:6835","nonce":2268322007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/3811005052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3811005052},{"type":"v1","addr":"172.21.15.169:6835","nonce":3811005052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.186:6837/1647607118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1647607118},{"type":"v1","addr":"172.21.15.186:6837","nonce":1647607118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":32,"addr":"172.21.15.186:6839/3954883105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":3954883105},{"type":"v1","addr":"172.21.15.186:6839","nonce":3954883105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.169:6837/1845715294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1845715294},{"type":"v1","addr":"172.21.15.169:6837","nonce":1845715294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.6:6839/2780306492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":2780306492},{"type":"v1","addr":"172.21.15.6:6839","nonce":2780306492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.169:6839/614388771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":614388771},{"type":"v1","addr":"172.21.15.169:6839","nonce":614388771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24493,24511,24530,24554,24512]},"id":1}]} 2024-05-09T04:53:27.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.966+0000 7f039e800700 1 -- 172.21.15.6:0/697800684 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f037c0b2900 msgr2=0x7f037c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:27.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.966+0000 7f039e800700 1 --2- 172.21.15.6:0/697800684 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f037c0b2900 0x7f037c0b4e10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f039807aff0 tx=0x7f0384009260 comp rx=0 tx=0).stop 2024-05-09T04:53:27.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.966+0000 7f039e800700 1 -- 172.21.15.6:0/697800684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f039810a990 msgr2=0x7f03981a1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:27.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.966+0000 7f039e800700 1 --2- 172.21.15.6:0/697800684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f039810a990 0x7f03981a1480 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0388000c00 tx=0x7f038800bfd0 comp rx=0 tx=0).stop 2024-05-09T04:53:27.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.966+0000 7f039e800700 1 -- 172.21.15.6:0/697800684 shutdown_connections 2024-05-09T04:53:27.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.966+0000 7f039e800700 1 --2- 172.21.15.6:0/697800684 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f037c0b2900 0x7f037c0b4e10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:27.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.966+0000 7f039e800700 1 --2- 172.21.15.6:0/697800684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0398110970 0x7f03981a5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:27.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.966+0000 7f039e800700 1 --2- 172.21.15.6:0/697800684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f039810a990 0x7f03981a1480 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:27.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.966+0000 7f039e800700 1 --2- 172.21.15.6:0/697800684 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f039810b2e0 0x7f03981a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:27.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.966+0000 7f039e800700 1 -- 172.21.15.6:0/697800684 >> 172.21.15.6:0/697800684 conn(0x7f0398077c80 msgr2=0x7f039807bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:27.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.967+0000 7f039e800700 1 -- 172.21.15.6:0/697800684 shutdown_connections 2024-05-09T04:53:27.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:27.967+0000 7f039e800700 1 -- 172.21.15.6:0/697800684 wait complete. 2024-05-09T04:53:27.969 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 29 2024-05-09T04:53:28.821 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-09T04:49:36.920376+0000', 'modified': '2024-05-09T04:51:07.589004+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24554, 'mds_1': 24530, 'mds_2': 24512, 'mds_3': 24511, 'mds_4': 24493}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24442': {'gid': 24442, 'name': 'l', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6833/2977145206', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6832', 'nonce': 2977145206}, {'type': 'v1', 'addr': '172.21.15.186:6833', 'nonce': 2977145206}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24461': {'gid': 24461, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.169:6833/133345948', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6832', 'nonce': 133345948}, {'type': 'v1', 'addr': '172.21.15.169:6833', 'nonce': 133345948}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'd', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.6:6837/1281521088', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.6:6836', 'nonce': 1281521088}, {'type': 'v1', 'addr': '172.21.15.6:6837', 'nonce': 1281521088}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6835/2268322007', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6834', 'nonce': 2268322007}, {'type': 'v1', 'addr': '172.21.15.186:6835', 'nonce': 2268322007}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24491': {'gid': 24491, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.169:6835/3811005052', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6834', 'nonce': 3811005052}, {'type': 'v1', 'addr': '172.21.15.169:6835', 'nonce': 3811005052}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'c', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 33, 'addr': '172.21.15.186:6837/1647607118', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6836', 'nonce': 1647607118}, {'type': 'v1', 'addr': '172.21.15.186:6837', 'nonce': 1647607118}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'i', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 32, 'addr': '172.21.15.186:6839/3954883105', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6838', 'nonce': 3954883105}, {'type': 'v1', 'addr': '172.21.15.186:6839', 'nonce': 3954883105}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 32, 'addr': '172.21.15.169:6837/1845715294', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6836', 'nonce': 1845715294}, {'type': 'v1', 'addr': '172.21.15.169:6837', 'nonce': 1845715294}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24530': {'gid': 24530, 'name': 'g', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 30, 'addr': '172.21.15.6:6839/2780306492', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.6:6838', 'nonce': 2780306492}, {'type': 'v1', 'addr': '172.21.15.6:6839', 'nonce': 2780306492}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24554': {'gid': 24554, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.169:6839/614388771', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6838', 'nonce': 614388771}, {'type': 'v1', 'addr': '172.21.15.169:6839', 'nonce': 614388771}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24554, 'qdb_cluster': [24493, 24511, 24530, 24554, 24512]} 2024-05-09T04:53:28.821 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-09T04:53:28.822 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.c status 2024-05-09T04:53:29.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:28 smithi186 ceph-mon[34668]: pgmap v455: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:53:29.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:28 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/697800684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:53:29.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:28 smithi169 ceph-mon[34075]: pgmap v455: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:53:29.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:28 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/697800684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:53:29.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:28 smithi006 ceph-mon[29835]: pgmap v455: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:53:29.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:28 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/697800684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:53:29.292 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:53:30.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:30 smithi006 ceph-mon[29835]: pgmap v456: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:53:31.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:30 smithi186 ceph-mon[34668]: pgmap v456: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:53:31.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:30 smithi169 ceph-mon[34075]: pgmap v456: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:53:31.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.624+0000 7f07c3200700 1 -- 172.21.15.6:0/1820854446 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07c4102f30 msgr2=0x7f07c4103340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:31.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.624+0000 7f07c3200700 1 --2- 172.21.15.6:0/1820854446 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07c4102f30 0x7f07c4103340 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f07b4009990 tx=0x7f07b4009ca0 comp rx=0 tx=0).stop 2024-05-09T04:53:31.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.624+0000 7f07c3200700 1 -- 172.21.15.6:0/1820854446 shutdown_connections 2024-05-09T04:53:31.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.624+0000 7f07c3200700 1 --2- 172.21.15.6:0/1820854446 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07c4110ae0 0x7f07c4112f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:31.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.624+0000 7f07c3200700 1 --2- 172.21.15.6:0/1820854446 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07c4102f30 0x7f07c4103340 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:31.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.624+0000 7f07c3200700 1 --2- 172.21.15.6:0/1820854446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07c4103880 0x7f07c41105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:31.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.624+0000 7f07c3200700 1 -- 172.21.15.6:0/1820854446 >> 172.21.15.6:0/1820854446 conn(0x7f07c4078ac0 msgr2=0x7f07c4078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:31.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.624+0000 7f07c3200700 1 -- 172.21.15.6:0/1820854446 shutdown_connections 2024-05-09T04:53:31.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.625+0000 7f07c3200700 1 -- 172.21.15.6:0/1820854446 wait complete. 2024-05-09T04:53:31.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.626+0000 7f07c3200700 1 Processor -- start 2024-05-09T04:53:31.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.626+0000 7f07c3200700 1 -- start start 2024-05-09T04:53:31.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.627+0000 7f07c3200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07c4102f30 0x7f07c419cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:31.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.627+0000 7f07c3200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07c4103880 0x7f07c419d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:31.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.627+0000 7f07c3200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07c4110ae0 0x7f07c41a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:31.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.627+0000 7f07c3200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f07c41164b0 con 0x7f07c4103880 2024-05-09T04:53:31.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.627+0000 7f07c3200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f07c4116330 con 0x7f07c4102f30 2024-05-09T04:53:31.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.627+0000 7f07c3200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f07c4116630 con 0x7f07c4110ae0 2024-05-09T04:53:31.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.627+0000 7f07c1400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07c4103880 0x7f07c419d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:31.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.627+0000 7f07c1400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07c4103880 0x7f07c419d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41378/0 (socket says 172.21.15.6:41378) 2024-05-09T04:53:31.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.627+0000 7f07c1400700 1 -- 172.21.15.6:0/157352077 learned_addr learned my addr 172.21.15.6:0/157352077 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:31.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.627+0000 7f07c1e00700 1 --2- 172.21.15.6:0/157352077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07c4102f30 0x7f07c419cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:31.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.627+0000 7f07c2800700 1 --2- 172.21.15.6:0/157352077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07c4110ae0 0x7f07c41a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:31.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.628+0000 7f07c1400700 1 -- 172.21.15.6:0/157352077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07c4110ae0 msgr2=0x7f07c41a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:31.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.628+0000 7f07c1400700 1 --2- 172.21.15.6:0/157352077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07c4110ae0 0x7f07c41a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:31.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.628+0000 7f07c1400700 1 -- 172.21.15.6:0/157352077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07c4102f30 msgr2=0x7f07c419cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:31.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.628+0000 7f07c1400700 1 --2- 172.21.15.6:0/157352077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07c4102f30 0x7f07c419cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:31.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.628+0000 7f07c1400700 1 -- 172.21.15.6:0/157352077 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c41a1d70 con 0x7f07c4103880 2024-05-09T04:53:31.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.628+0000 7f07c1400700 1 --2- 172.21.15.6:0/157352077 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07c4103880 0x7f07c419d440 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f07b000d950 tx=0x7f07b000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:31.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.628+0000 7f07baa00700 1 -- 172.21.15.6:0/157352077 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b0016070 con 0x7f07c4103880 2024-05-09T04:53:31.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.628+0000 7f07baa00700 1 -- 172.21.15.6:0/157352077 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07b0005020 con 0x7f07c4103880 2024-05-09T04:53:31.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.628+0000 7f07baa00700 1 -- 172.21.15.6:0/157352077 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b000f9d0 con 0x7f07c4103880 2024-05-09T04:53:31.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.628+0000 7f07c3200700 1 -- 172.21.15.6:0/157352077 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07c41a2060 con 0x7f07c4103880 2024-05-09T04:53:31.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.629+0000 7f07c3200700 1 -- 172.21.15.6:0/157352077 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07c41a9980 con 0x7f07c4103880 2024-05-09T04:53:31.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.636+0000 7f07baa00700 1 -- 172.21.15.6:0/157352077 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f07b00051d0 con 0x7f07c4103880 2024-05-09T04:53:31.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.636+0000 7f07d759cb80 1 Processor -- start 2024-05-09T04:53:31.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.637+0000 7f07d759cb80 1 -- start start 2024-05-09T04:53:31.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.637+0000 7f07d759cb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55811d84b450 0x55811d9ce100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:31.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.637+0000 7f07d759cb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55811d9ce640 0x55811d9c8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:31.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.637+0000 7f07d759cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55811d9c8b60 0x55811d9c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:31.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.637+0000 7f07d759cb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55811d8eb260 con 0x55811d9ce640 2024-05-09T04:53:31.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.637+0000 7f07d759cb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55811d889090 con 0x55811d84b450 2024-05-09T04:53:31.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.637+0000 7f07d759cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55811d8e7220 con 0x55811d9c8b60 2024-05-09T04:53:31.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.638+0000 7f07c1400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55811d9c8b60 0x55811d9c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:31.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.638+0000 7f07c1400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55811d9c8b60 0x55811d9c8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:55094/0 (socket says 172.21.15.6:55094) 2024-05-09T04:53:31.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.638+0000 7f07c1400700 1 -- 172.21.15.6:0/646879163 learned_addr learned my addr 172.21.15.6:0/646879163 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:31.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.638+0000 7f07c1e00700 1 --2- 172.21.15.6:0/646879163 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55811d9ce640 0x55811d9c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:31.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.638+0000 7f07c2800700 1 --2- 172.21.15.6:0/646879163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55811d84b450 0x55811d9ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:31.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.638+0000 7f07c1400700 1 -- 172.21.15.6:0/646879163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55811d84b450 msgr2=0x55811d9ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:31.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.638+0000 7f07c1400700 1 --2- 172.21.15.6:0/646879163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55811d84b450 0x55811d9ce100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:31.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.638+0000 7f07c1400700 1 -- 172.21.15.6:0/646879163 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55811d9ce640 msgr2=0x55811d9c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:31.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.638+0000 7f07c1400700 1 --2- 172.21.15.6:0/646879163 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55811d9ce640 0x55811d9c8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:31.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.638+0000 7f07c1400700 1 -- 172.21.15.6:0/646879163 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55811d766440 con 0x55811d9c8b60 2024-05-09T04:53:31.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.638+0000 7f07baa00700 1 --2- 172.21.15.6:0/157352077 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f07a80b2880 0x7f07a80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:31.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.639+0000 7f07baa00700 1 -- 172.21.15.6:0/157352077 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f07b00e1d10 con 0x7f07c4103880 2024-05-09T04:53:31.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.639+0000 7f07c2800700 1 --2- 172.21.15.6:0/157352077 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f07a80b2880 0x7f07a80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:31.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.639+0000 7f07c1400700 1 --2- 172.21.15.6:0/646879163 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55811d9c8b60 0x55811d9c8f70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f07b00e5e40 tx=0x7f07b00ea670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23a0874f71a57353 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:31.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.639+0000 7f07af400700 1 -- 172.21.15.6:0/646879163 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b001f020 con 0x55811d9c8b60 2024-05-09T04:53:31.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.639+0000 7f07af400700 1 -- 172.21.15.6:0/646879163 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07b00a8a60 con 0x55811d9c8b60 2024-05-09T04:53:31.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.639+0000 7f07af400700 1 -- 172.21.15.6:0/646879163 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b00a8d20 con 0x55811d9c8b60 2024-05-09T04:53:31.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.640+0000 7f07d759cb80 1 -- 172.21.15.6:0/646879163 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55811d9c8b60 msgr2=0x55811d9c8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:31.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.640+0000 7f07d759cb80 1 --2- 172.21.15.6:0/646879163 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55811d9c8b60 0x55811d9c8f70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f07b00e5e40 tx=0x7f07b00ea670 comp rx=0 tx=0).stop 2024-05-09T04:53:31.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.640+0000 7f07c2800700 1 --2- 172.21.15.6:0/157352077 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f07a80b2880 0x7f07a80b4d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x55811d9cdc10 tx=0x7f07bc00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:31.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.640+0000 7f07d759cb80 1 -- 172.21.15.6:0/646879163 shutdown_connections 2024-05-09T04:53:31.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.640+0000 7f07d759cb80 1 --2- 172.21.15.6:0/646879163 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55811d9c8b60 0x55811d9c8f70 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:31.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.640+0000 7f07d759cb80 1 --2- 172.21.15.6:0/646879163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55811d84b450 0x55811d9ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:31.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.640+0000 7f07d759cb80 1 --2- 172.21.15.6:0/646879163 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55811d9ce640 0x55811d9c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:31.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.640+0000 7f07d759cb80 1 -- 172.21.15.6:0/646879163 >> 172.21.15.6:0/646879163 conn(0x55811d88fe20 msgr2=0x55811d9c3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:31.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.640+0000 7f07d759cb80 1 -- 172.21.15.6:0/646879163 shutdown_connections 2024-05-09T04:53:31.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.640+0000 7f07d759cb80 1 -- 172.21.15.6:0/646879163 wait complete. 2024-05-09T04:53:31.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.641+0000 7f07d759cb80 1 -- start start 2024-05-09T04:53:31.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.641+0000 7f07d759cb80 1 Processor -- start 2024-05-09T04:53:31.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.642+0000 7f07d759cb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55811d84b450 0x55811d9d1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:31.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.642+0000 7f07d759cb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55811d9c8b60 0x55811d9d2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:31.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.642+0000 7f07d759cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55811d9ce640 0x55811d9d27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:31.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.642+0000 7f07d759cb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55811d9d50d0 con 0x55811d9c8b60 2024-05-09T04:53:31.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.642+0000 7f07d759cb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55811d850670 con 0x55811d84b450 2024-05-09T04:53:31.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.642+0000 7f07d759cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55811d9d5250 con 0x55811d9ce640 2024-05-09T04:53:31.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.642+0000 7f07d759cb80 10 client.0 _ll_register_callbacks cb 0x55811d882540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:31.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.642+0000 7f07c1400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55811d9c8b60 0x55811d9d2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:31.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.642+0000 7f07c1e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55811d84b450 0x55811d9d1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:31.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.642+0000 7f07c1400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55811d9c8b60 0x55811d9d2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41390/0 (socket says 172.21.15.6:41390) 2024-05-09T04:53:31.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.642+0000 7f07c1400700 1 -- 172.21.15.6:0/3037407084 learned_addr learned my addr 172.21.15.6:0/3037407084 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:31.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.642+0000 7f07c2800700 1 --2- 172.21.15.6:0/3037407084 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55811d9ce640 0x55811d9d27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:31.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.643+0000 7f07c1e00700 1 -- 172.21.15.6:0/3037407084 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55811d9ce640 msgr2=0x55811d9d27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:31.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.643+0000 7f07c1e00700 1 --2- 172.21.15.6:0/3037407084 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55811d9ce640 0x55811d9d27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:31.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.643+0000 7f07c1e00700 1 -- 172.21.15.6:0/3037407084 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55811d9c8b60 msgr2=0x55811d9d2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:31.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.643+0000 7f07c1e00700 1 --2- 172.21.15.6:0/3037407084 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55811d9c8b60 0x55811d9d2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:31.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.643+0000 7f07c1e00700 1 -- 172.21.15.6:0/3037407084 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55811d9de5a0 con 0x55811d84b450 2024-05-09T04:53:31.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.643+0000 7f07c1400700 1 --2- 172.21.15.6:0/3037407084 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55811d9c8b60 0x55811d9d2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:53:31.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.643+0000 7f07c1e00700 1 --2- 172.21.15.6:0/3037407084 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55811d84b450 0x55811d9d1d50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f07b4006010 tx=0x7f07b4005610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f276c2905be7fa1d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:31.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.643+0000 7f07a3e00700 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-05-09T04:53:31.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.644+0000 7f07a3e00700 1 -- 172.21.15.6:0/3037407084 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b4003680 con 0x55811d84b450 2024-05-09T04:53:31.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.644+0000 7f07d759cb80 1 -- 172.21.15.6:0/3037407084 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07b00a8a60 con 0x55811d84b450 2024-05-09T04:53:31.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.644+0000 7f07a3e00700 1 -- 172.21.15.6:0/3037407084 <== mon.2 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f07b4003820 con 0x55811d84b450 2024-05-09T04:53:31.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.644+0000 7f07a3e00700 1 -- 172.21.15.6:0/3037407084 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f07b4031050 con 0x55811d84b450 2024-05-09T04:53:31.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.644+0000 7f07d759cb80 10 client.24992 fetch_fsmap learned FSMap version 29 2024-05-09T04:53:31.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.644+0000 7f07d759cb80 1 -- 172.21.15.6:0/3037407084 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55811d9dfe90 con 0x55811d84b450 2024-05-09T04:53:31.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.645+0000 7f07a3e00700 1 -- 172.21.15.6:0/3037407084 <== mon.2 v2:172.21.15.169:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07b00a8a60 con 0x55811d84b450 2024-05-09T04:53:31.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.645+0000 7f07a3e00700 1 -- 172.21.15.6:0/3037407084 <== mon.2 v2:172.21.15.169:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f07b4023970 con 0x55811d84b450 2024-05-09T04:53:31.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.645+0000 7f07d759cb80 10 client.24992 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T04:53:31.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.645+0000 7f07d759cb80 10 client.24992 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T04:53:31.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.645+0000 7f07d759cb80 20 client.24992 populate_metadata read hostname 'smithi006' 2024-05-09T04:53:31.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.645+0000 7f07d759cb80 1 --2- 172.21.15.6:0/3037407084 >> [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] conn(0x55811d9e24c0 0x55811d9e3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:31.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.645+0000 7f07d759cb80 4 client.24992 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:53:31.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.645+0000 7f07d759cb80 1 -- 172.21.15.6:0/3037407084 --> [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55811d994820 con 0x55811d9e24c0 2024-05-09T04:53:31.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.646+0000 7f07c1400700 1 --2- 172.21.15.6:0/3037407084 >> [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] conn(0x55811d9e24c0 0x55811d9e3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:31.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.646+0000 7f07c1400700 1 --2- 172.21.15.6:0/3037407084 >> [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] conn(0x55811d9e24c0 0x55811d9e3930 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ac3a313e1007b3c6 server_cookie=cb02a7450a631d8 in_seq=0 out_seq=0 2024-05-09T04:53:31.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.646+0000 7f07a3e00700 10 client.24992 ms_handle_connect on v2:172.21.15.186:6836/1647607118 2024-05-09T04:53:31.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.647+0000 7f07a3e00700 1 -- 172.21.15.6:0/3037407084 <== mds.4 v2:172.21.15.186:6836/1647607118 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55811d994820 con 0x55811d9e24c0 2024-05-09T04:53:31.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.647+0000 7f07a3e00700 10 client.24992 handle_command_reply: tid=0 2024-05-09T04:53:31.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.647+0000 7f07d759cb80 1 client.24992 shutdown 2024-05-09T04:53:31.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.647+0000 7f07d759cb80 1 -- 172.21.15.6:0/3037407084 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55811d84b450 msgr2=0x55811d9d1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:31.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.647+0000 7f07d759cb80 1 --2- 172.21.15.6:0/3037407084 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55811d84b450 0x55811d9d1d50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f07b4006010 tx=0x7f07b4005610 comp rx=0 tx=0).stop 2024-05-09T04:53:31.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.647+0000 7f07d759cb80 1 -- 172.21.15.6:0/3037407084 shutdown_connections 2024-05-09T04:53:31.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.647+0000 7f07d759cb80 1 --2- 172.21.15.6:0/3037407084 >> [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] conn(0x55811d9e24c0 0x55811d9e3930 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:31.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.647+0000 7f07d759cb80 1 --2- 172.21.15.6:0/3037407084 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55811d9ce640 0x55811d9d27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:31.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.647+0000 7f07d759cb80 1 --2- 172.21.15.6:0/3037407084 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55811d84b450 0x55811d9d1d50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:31.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.647+0000 7f07d759cb80 1 --2- 172.21.15.6:0/3037407084 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55811d9c8b60 0x55811d9d2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:31.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.647+0000 7f07d759cb80 1 -- 172.21.15.6:0/3037407084 >> 172.21.15.6:0/3037407084 conn(0x55811d88fe20 msgr2=0x55811d9c3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:31.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.647+0000 7f07a3e00700 0 client.24992 ms_handle_reset on v2:172.21.15.186:6836/1647607118 2024-05-09T04:53:31.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.648+0000 7f07d759cb80 1 -- 172.21.15.6:0/3037407084 shutdown_connections 2024-05-09T04:53:31.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.648+0000 7f07d759cb80 1 -- 172.21.15.6:0/3037407084 wait complete. 2024-05-09T04:53:31.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.648+0000 7f07d759cb80 20 client.24992 trim_cache size 0 max 16384 2024-05-09T04:53:31.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.672+0000 7f07d759cb80 1 Processor -- start 2024-05-09T04:53:31.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.673+0000 7f07d759cb80 1 -- start start 2024-05-09T04:53:31.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.673+0000 7f07d759cb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55811d84b450 0x55811d9c3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:31.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.673+0000 7f07d759cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55811d9c8b60 0x55811d9d2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:31.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.673+0000 7f07d759cb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55811d9ce640 0x55811d9cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:31.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.673+0000 7f07d759cb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55811d9e1c70 con 0x55811d84b450 2024-05-09T04:53:31.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.673+0000 7f07d759cb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55811d9e2340 con 0x55811d9ce640 2024-05-09T04:53:31.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.673+0000 7f07d759cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55811d9e2a10 con 0x55811d9c8b60 2024-05-09T04:53:31.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.673+0000 7f07c1400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55811d9c8b60 0x55811d9d2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:31.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.673+0000 7f07c1e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55811d84b450 0x55811d9c3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:31.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.673+0000 7f07c1400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55811d9c8b60 0x55811d9d2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:55116/0 (socket says 172.21.15.6:55116) 2024-05-09T04:53:31.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.673+0000 7f07c1400700 1 -- 172.21.15.6:0/1642442414 learned_addr learned my addr 172.21.15.6:0/1642442414 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:31.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.673+0000 7f07c2800700 1 --2- 172.21.15.6:0/1642442414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55811d9ce640 0x55811d9cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:31.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.674+0000 7f07c1400700 1 -- 172.21.15.6:0/1642442414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55811d9ce640 msgr2=0x55811d9cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:31.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.674+0000 7f07c1400700 1 --2- 172.21.15.6:0/1642442414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55811d9ce640 0x55811d9cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:31.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.674+0000 7f07c1400700 1 -- 172.21.15.6:0/1642442414 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55811d84b450 msgr2=0x55811d9c3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:31.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.674+0000 7f07c1400700 1 --2- 172.21.15.6:0/1642442414 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55811d84b450 0x55811d9c3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:31.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.674+0000 7f07c1400700 1 -- 172.21.15.6:0/1642442414 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55811d9be2f0 con 0x55811d9c8b60 2024-05-09T04:53:31.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.674+0000 7f07c1e00700 1 --2- 172.21.15.6:0/1642442414 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55811d84b450 0x55811d9c3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:53:31.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.674+0000 7f07c1400700 1 --2- 172.21.15.6:0/1642442414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55811d9c8b60 0x55811d9d2610 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f07b00a61c0 tx=0x7f07b00e1f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e6d781e501ec25e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:31.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.675+0000 7f07aea00700 1 -- 172.21.15.6:0/1642442414 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b00c3070 con 0x55811d9c8b60 2024-05-09T04:53:31.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.675+0000 7f07aea00700 1 -- 172.21.15.6:0/1642442414 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07b00a8a60 con 0x55811d9c8b60 2024-05-09T04:53:31.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.675+0000 7f07aea00700 1 -- 172.21.15.6:0/1642442414 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b001f040 con 0x55811d9c8b60 2024-05-09T04:53:31.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.676+0000 7f07d759cb80 1 -- 172.21.15.6:0/1642442414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55811d9c8b60 msgr2=0x55811d9d2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:31.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.676+0000 7f07d759cb80 1 --2- 172.21.15.6:0/1642442414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55811d9c8b60 0x55811d9d2610 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f07b00a61c0 tx=0x7f07b00e1f90 comp rx=0 tx=0).stop 2024-05-09T04:53:31.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.676+0000 7f07d759cb80 1 -- 172.21.15.6:0/1642442414 shutdown_connections 2024-05-09T04:53:31.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.676+0000 7f07d759cb80 1 --2- 172.21.15.6:0/1642442414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55811d9ce640 0x55811d9cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:31.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.676+0000 7f07d759cb80 1 --2- 172.21.15.6:0/1642442414 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55811d84b450 0x55811d9c3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:31.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.676+0000 7f07d759cb80 1 --2- 172.21.15.6:0/1642442414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55811d9c8b60 0x55811d9d2610 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:31.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.676+0000 7f07d759cb80 1 -- 172.21.15.6:0/1642442414 >> 172.21.15.6:0/1642442414 conn(0x55811d88fe20 msgr2=0x55811d9d0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:31.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.676+0000 7f07d759cb80 1 -- 172.21.15.6:0/1642442414 shutdown_connections 2024-05-09T04:53:31.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.676+0000 7f07d759cb80 1 -- 172.21.15.6:0/1642442414 wait complete. 2024-05-09T04:53:31.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.677+0000 7f07d759cb80 1 -- start start 2024-05-09T04:53:31.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.677+0000 7f07d759cb80 1 Processor -- start 2024-05-09T04:53:31.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.677+0000 7f07d759cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55811d84b450 0x55811d9d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:31.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.678+0000 7f07d759cb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55811d9c8b60 0x55811d9d3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:31.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.678+0000 7f07d759cb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55811d9ce640 0x55811d9d3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:31.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.678+0000 7f07d759cb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55811d9e37b0 con 0x55811d9c8b60 2024-05-09T04:53:31.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.678+0000 7f07d759cb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55811d9e30e0 con 0x55811d9ce640 2024-05-09T04:53:31.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.678+0000 7f07d759cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55811d9e3e80 con 0x55811d84b450 2024-05-09T04:53:31.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.678+0000 7f07d759cb80 10 client.0 _ll_register_callbacks cb 0x55811d882540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:31.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.678+0000 7f07c1e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55811d84b450 0x55811d9d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:31.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.678+0000 7f07c1e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55811d84b450 0x55811d9d31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:55120/0 (socket says 172.21.15.6:55120) 2024-05-09T04:53:31.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.678+0000 7f07c1e00700 1 -- 172.21.15.6:0/3423924233 learned_addr learned my addr 172.21.15.6:0/3423924233 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:31.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.678+0000 7f07c2800700 1 --2- 172.21.15.6:0/3423924233 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55811d9ce640 0x55811d9d3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:31.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.678+0000 7f07c1400700 1 --2- 172.21.15.6:0/3423924233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55811d9c8b60 0x55811d9d3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:31.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.679+0000 7f07c1e00700 1 -- 172.21.15.6:0/3423924233 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55811d9ce640 msgr2=0x55811d9d3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:31.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.679+0000 7f07c1e00700 1 --2- 172.21.15.6:0/3423924233 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55811d9ce640 0x55811d9d3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:31.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.679+0000 7f07c1e00700 1 -- 172.21.15.6:0/3423924233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55811d9c8b60 msgr2=0x55811d9d3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:31.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.679+0000 7f07c1e00700 1 --2- 172.21.15.6:0/3423924233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55811d9c8b60 0x55811d9d3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:31.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.679+0000 7f07c1e00700 1 -- 172.21.15.6:0/3423924233 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55811d9ebf10 con 0x55811d84b450 2024-05-09T04:53:31.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.679+0000 7f07c2800700 1 --2- 172.21.15.6:0/3423924233 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55811d9ce640 0x55811d9d3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:53:31.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.679+0000 7f07c1400700 1 --2- 172.21.15.6:0/3423924233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55811d9c8b60 0x55811d9d3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:53:31.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.679+0000 7f07c1e00700 1 --2- 172.21.15.6:0/3423924233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55811d84b450 0x55811d9d31e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f07b4000c00 tx=0x7f07b4005800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fe4e8f81b644cc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:31.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.679+0000 7f07a3e00700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-09T04:53:31.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.679+0000 7f07a3e00700 1 -- 172.21.15.6:0/3423924233 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b40058e0 con 0x55811d84b450 2024-05-09T04:53:31.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.679+0000 7f07d759cb80 1 -- 172.21.15.6:0/3423924233 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07b00a8a60 con 0x55811d84b450 2024-05-09T04:53:31.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.679+0000 7f07a3e00700 1 -- 172.21.15.6:0/3423924233 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f07b400f820 con 0x55811d84b450 2024-05-09T04:53:31.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.680+0000 7f07a3e00700 1 -- 172.21.15.6:0/3423924233 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f07b4041050 con 0x55811d84b450 2024-05-09T04:53:31.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.680+0000 7f07d759cb80 10 client.24985 fetch_fsmap learned FSMap version 29 2024-05-09T04:53:31.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.680+0000 7f07d759cb80 1 -- 172.21.15.6:0/3423924233 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55811d9ed150 con 0x55811d84b450 2024-05-09T04:53:31.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.680+0000 7f07a3e00700 1 -- 172.21.15.6:0/3423924233 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07b00a8a60 con 0x55811d84b450 2024-05-09T04:53:31.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.681+0000 7f07a3e00700 1 -- 172.21.15.6:0/3423924233 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f07b4003680 con 0x55811d84b450 2024-05-09T04:53:31.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.681+0000 7f07d759cb80 10 client.24985 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T04:53:31.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.681+0000 7f07d759cb80 10 client.24985 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T04:53:31.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.681+0000 7f07d759cb80 20 client.24985 populate_metadata read hostname 'smithi006' 2024-05-09T04:53:31.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.681+0000 7f07d759cb80 1 --2- 172.21.15.6:0/3423924233 >> [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] conn(0x55811d9ed440 0x55811da59100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:31.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.681+0000 7f07d759cb80 4 client.24985 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T04:53:31.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.681+0000 7f07d759cb80 1 -- 172.21.15.6:0/3423924233 --> [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] -- command(tid 0: {"prefix": "status"}) -- 0x55811d8f8c40 con 0x55811d9ed440 2024-05-09T04:53:31.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.681+0000 7f07c1400700 1 --2- 172.21.15.6:0/3423924233 >> [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] conn(0x55811d9ed440 0x55811da59100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:31.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.682+0000 7f07c1400700 1 --2- 172.21.15.6:0/3423924233 >> [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] conn(0x55811d9ed440 0x55811da59100 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=694f3cbffcff83c4 server_cookie=e113edfcc28eb97f in_seq=0 out_seq=0 2024-05-09T04:53:31.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.682+0000 7f07a3e00700 10 client.24985 ms_handle_connect on v2:172.21.15.186:6836/1647607118 2024-05-09T04:53:31.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.682+0000 7f07a3e00700 1 -- 172.21.15.6:0/3423924233 <== mds.4 v2:172.21.15.186:6836/1647607118 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55811d8f8c40 con 0x55811d9ed440 2024-05-09T04:53:31.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.682+0000 7f07a3e00700 10 client.24985 handle_command_reply: tid=0 2024-05-09T04:53:31.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.682+0000 7f07d759cb80 1 client.24985 shutdown 2024-05-09T04:53:31.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.683+0000 7f07d759cb80 1 -- 172.21.15.6:0/3423924233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55811d84b450 msgr2=0x55811d9d31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:31.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.683+0000 7f07d759cb80 1 --2- 172.21.15.6:0/3423924233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55811d84b450 0x55811d9d31e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f07b4000c00 tx=0x7f07b4005800 comp rx=0 tx=0).stop 2024-05-09T04:53:31.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.683+0000 7f07d759cb80 1 -- 172.21.15.6:0/3423924233 shutdown_connections 2024-05-09T04:53:31.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.683+0000 7f07d759cb80 1 --2- 172.21.15.6:0/3423924233 >> [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] conn(0x55811d9ed440 0x55811da59100 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:31.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.683+0000 7f07d759cb80 1 --2- 172.21.15.6:0/3423924233 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55811d9ce640 0x55811d9d3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:31.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.683+0000 7f07d759cb80 1 --2- 172.21.15.6:0/3423924233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55811d84b450 0x55811d9d31e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:31.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.683+0000 7f07d759cb80 1 --2- 172.21.15.6:0/3423924233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55811d9c8b60 0x55811d9d3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:31.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.683+0000 7f07d759cb80 1 -- 172.21.15.6:0/3423924233 >> 172.21.15.6:0/3423924233 conn(0x55811d88fe20 msgr2=0x55811d9cce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:31.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.683+0000 7f07a3e00700 0 client.24985 ms_handle_reset on v2:172.21.15.186:6836/1647607118 2024-05-09T04:53:31.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.683+0000 7f07d759cb80 1 -- 172.21.15.6:0/3423924233 shutdown_connections 2024-05-09T04:53:31.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.683+0000 7f07d759cb80 1 -- 172.21.15.6:0/3423924233 wait complete. 2024-05-09T04:53:31.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.683+0000 7f07d759cb80 20 client.24985 trim_cache size 0 max 16384 2024-05-09T04:53:31.686 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-09T04:53:31.686 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "f74d813c-0dbd-11ef-bc97-c7b262605968", 2024-05-09T04:53:31.686 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 4, 2024-05-09T04:53:31.686 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24493, 2024-05-09T04:53:31.686 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-05-09T04:53:31.686 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-05-09T04:53:31.686 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-05-09T04:53:31.686 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 148.925900098, 2024-05-09T04:53:31.687 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 29, 2024-05-09T04:53:31.687 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 89, 2024-05-09T04:53:31.687 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 89, 2024-05-09T04:53:31.687 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 276.209843326 2024-05-09T04:53:31.687 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-09T04:53:31.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.684+0000 7f07c3200700 1 -- 172.21.15.6:0/157352077 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f07a80b2880 msgr2=0x7f07a80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:31.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.684+0000 7f07c3200700 1 --2- 172.21.15.6:0/157352077 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f07a80b2880 0x7f07a80b4d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x55811d9cdc10 tx=0x7f07bc00b040 comp rx=0 tx=0).stop 2024-05-09T04:53:31.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.684+0000 7f07c3200700 1 -- 172.21.15.6:0/157352077 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07c4103880 msgr2=0x7f07c419d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:31.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.684+0000 7f07c3200700 1 --2- 172.21.15.6:0/157352077 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07c4103880 0x7f07c419d440 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f07b000d950 tx=0x7f07b000dc60 comp rx=0 tx=0).stop 2024-05-09T04:53:31.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.684+0000 7f07c3200700 1 -- 172.21.15.6:0/157352077 shutdown_connections 2024-05-09T04:53:31.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.684+0000 7f07c3200700 1 --2- 172.21.15.6:0/157352077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07c4110ae0 0x7f07c41a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:31.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.684+0000 7f07c3200700 1 --2- 172.21.15.6:0/157352077 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f07a80b2880 0x7f07a80b4d90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:31.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.684+0000 7f07c3200700 1 --2- 172.21.15.6:0/157352077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07c4102f30 0x7f07c419cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:31.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.684+0000 7f07c3200700 1 --2- 172.21.15.6:0/157352077 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07c4103880 0x7f07c419d440 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:31.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.684+0000 7f07c3200700 1 -- 172.21.15.6:0/157352077 >> 172.21.15.6:0/157352077 conn(0x7f07c4078ac0 msgr2=0x7f07c4111e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:31.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.685+0000 7f07c3200700 1 -- 172.21.15.6:0/157352077 shutdown_connections 2024-05-09T04:53:31.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:31.685+0000 7f07c3200700 1 -- 172.21.15.6:0/157352077 wait complete. 2024-05-09T04:53:32.643 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-09T04:53:32.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:32 smithi006 ceph-mon[29835]: pgmap v457: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:53:33.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:32 smithi186 ceph-mon[34668]: pgmap v457: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:53:33.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:32 smithi169 ceph-mon[34075]: pgmap v457: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:53:33.121 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:53:34.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:34 smithi006 ceph-mon[29835]: pgmap v458: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:53:35.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:34 smithi186 ceph-mon[34668]: pgmap v458: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:53:35.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:34 smithi169 ceph-mon[34075]: pgmap v458: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:53:35.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.536+0000 7f7638000700 1 -- 172.21.15.6:0/2212519584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7630110ba0 msgr2=0x7f7630113000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:35.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.536+0000 7f7638000700 1 --2- 172.21.15.6:0/2212519584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7630110ba0 0x7f7630113000 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f762800b0a0 tx=0x7f762800b3b0 comp rx=0 tx=0).stop 2024-05-09T04:53:35.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.536+0000 7f7638000700 1 -- 172.21.15.6:0/2212519584 shutdown_connections 2024-05-09T04:53:35.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.536+0000 7f7638000700 1 --2- 172.21.15.6:0/2212519584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7630110ba0 0x7f7630113000 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:35.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.536+0000 7f7638000700 1 --2- 172.21.15.6:0/2212519584 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7630076e40 0x7f7630077250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:35.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.536+0000 7f7638000700 1 --2- 172.21.15.6:0/2212519584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7630077790 0x7f7630110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:35.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.536+0000 7f7638000700 1 -- 172.21.15.6:0/2212519584 >> 172.21.15.6:0/2212519584 conn(0x7f76300ffec0 msgr2=0x7f7630102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:35.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.536+0000 7f7638000700 1 -- 172.21.15.6:0/2212519584 shutdown_connections 2024-05-09T04:53:35.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.537+0000 7f7638000700 1 -- 172.21.15.6:0/2212519584 wait complete. 2024-05-09T04:53:35.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.538+0000 7f7638000700 1 Processor -- start 2024-05-09T04:53:35.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.538+0000 7f7638000700 1 -- start start 2024-05-09T04:53:35.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.538+0000 7f7638000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7630076e40 0x7f763019cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:35.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.538+0000 7f7638000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7630077790 0x7f763019d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:35.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.538+0000 7f7638000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7630110ba0 0x7f76301a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:35.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.538+0000 7f7638000700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f76301166d0 con 0x7f7630077790 2024-05-09T04:53:35.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.538+0000 7f7638000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7630116550 con 0x7f7630110ba0 2024-05-09T04:53:35.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.538+0000 7f7638000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7630116850 con 0x7f7630076e40 2024-05-09T04:53:35.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.539+0000 7f7634c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7630077790 0x7f763019d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:35.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.539+0000 7f7634c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7630077790 0x7f763019d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41428/0 (socket says 172.21.15.6:41428) 2024-05-09T04:53:35.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.539+0000 7f7634c00700 1 -- 172.21.15.6:0/1856123634 learned_addr learned my addr 172.21.15.6:0/1856123634 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:35.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.539+0000 7f7635600700 1 --2- 172.21.15.6:0/1856123634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7630076e40 0x7f763019cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:35.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.539+0000 7f7636000700 1 --2- 172.21.15.6:0/1856123634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7630110ba0 0x7f76301a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:35.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.539+0000 7f7634c00700 1 -- 172.21.15.6:0/1856123634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7630076e40 msgr2=0x7f763019cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:35.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.539+0000 7f7634c00700 1 --2- 172.21.15.6:0/1856123634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7630076e40 0x7f763019cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:35.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.539+0000 7f7634c00700 1 -- 172.21.15.6:0/1856123634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7630110ba0 msgr2=0x7f76301a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:35.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.539+0000 7f7634c00700 1 --2- 172.21.15.6:0/1856123634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7630110ba0 0x7f76301a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:35.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.539+0000 7f7634c00700 1 -- 172.21.15.6:0/1856123634 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76301a1eb0 con 0x7f7630077790 2024-05-09T04:53:35.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.539+0000 7f7636000700 1 --2- 172.21.15.6:0/1856123634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7630110ba0 0x7f76301a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:53:35.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.539+0000 7f7635600700 1 --2- 172.21.15.6:0/1856123634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7630076e40 0x7f763019cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:53:35.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.540+0000 7f7634c00700 1 --2- 172.21.15.6:0/1856123634 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7630077790 0x7f763019d580 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f761c00e9a0 tx=0x7f761c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:35.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.541+0000 7f762e000700 1 -- 172.21.15.6:0/1856123634 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761c00cbd0 con 0x7f7630077790 2024-05-09T04:53:35.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.541+0000 7f762e000700 1 -- 172.21.15.6:0/1856123634 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f761c00cd70 con 0x7f7630077790 2024-05-09T04:53:35.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.541+0000 7f762e000700 1 -- 172.21.15.6:0/1856123634 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761c018680 con 0x7f7630077790 2024-05-09T04:53:35.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.541+0000 7f7638000700 1 -- 172.21.15.6:0/1856123634 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76301a21a0 con 0x7f7630077790 2024-05-09T04:53:35.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.541+0000 7f7638000700 1 -- 172.21.15.6:0/1856123634 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76301a9ac0 con 0x7f7630077790 2024-05-09T04:53:35.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.547+0000 7f762e000700 1 -- 172.21.15.6:0/1856123634 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f761c010430 con 0x7f7630077790 2024-05-09T04:53:35.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.548+0000 7f7646bc5b80 1 Processor -- start 2024-05-09T04:53:35.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.548+0000 7f7646bc5b80 1 -- start start 2024-05-09T04:53:35.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.548+0000 7f7646bc5b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556458020450 0x5564581a2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:35.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.548+0000 7f7646bc5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564581a3410 0x55645819d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:35.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.548+0000 7f7646bc5b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55645819da00 0x55645819de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:35.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.548+0000 7f7646bc5b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5564580c0260 con 0x55645819da00 2024-05-09T04:53:35.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.548+0000 7f7646bc5b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55645805e090 con 0x556458020450 2024-05-09T04:53:35.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.548+0000 7f7646bc5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5564580bc220 con 0x5564581a3410 2024-05-09T04:53:35.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.549+0000 7f7635600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564581a3410 0x55645819d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:35.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.549+0000 7f7634c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55645819da00 0x55645819de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:35.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.549+0000 7f7635600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564581a3410 0x55645819d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:55142/0 (socket says 172.21.15.6:55142) 2024-05-09T04:53:35.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.549+0000 7f7635600700 1 -- 172.21.15.6:0/785469507 learned_addr learned my addr 172.21.15.6:0/785469507 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:35.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.549+0000 7f7636000700 1 --2- 172.21.15.6:0/785469507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556458020450 0x5564581a2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:35.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.549+0000 7f7635600700 1 -- 172.21.15.6:0/785469507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556458020450 msgr2=0x5564581a2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:35.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.549+0000 7f7635600700 1 --2- 172.21.15.6:0/785469507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556458020450 0x5564581a2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:35.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.549+0000 7f7635600700 1 -- 172.21.15.6:0/785469507 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55645819da00 msgr2=0x55645819de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:35.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.549+0000 7f7635600700 1 --2- 172.21.15.6:0/785469507 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55645819da00 0x55645819de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:35.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.549+0000 7f7635600700 1 -- 172.21.15.6:0/785469507 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556457f3b440 con 0x5564581a3410 2024-05-09T04:53:35.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.549+0000 7f7636000700 1 --2- 172.21.15.6:0/785469507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556458020450 0x5564581a2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:53:35.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.549+0000 7f762e000700 1 --2- 172.21.15.6:0/1856123634 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f76180b29d0 0x7f76180b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:35.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.550+0000 7f7635600700 1 --2- 172.21.15.6:0/785469507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564581a3410 0x55645819d4c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7624010530 tx=0x7f7624010840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9421125b3df68470 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:35.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.550+0000 7f762e000700 1 -- 172.21.15.6:0/1856123634 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f761c0e2710 con 0x7f7630077790 2024-05-09T04:53:35.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.550+0000 7f7636000700 1 --2- 172.21.15.6:0/1856123634 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f76180b29d0 0x7f76180b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:35.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.550+0000 7f7622a00700 1 -- 172.21.15.6:0/785469507 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624004220 con 0x5564581a3410 2024-05-09T04:53:35.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.550+0000 7f7622a00700 1 -- 172.21.15.6:0/785469507 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76240043c0 con 0x5564581a3410 2024-05-09T04:53:35.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.550+0000 7f7622a00700 1 -- 172.21.15.6:0/785469507 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624011740 con 0x5564581a3410 2024-05-09T04:53:35.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.551+0000 7f7646bc5b80 1 -- 172.21.15.6:0/785469507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564581a3410 msgr2=0x55645819d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:35.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.551+0000 7f7646bc5b80 1 --2- 172.21.15.6:0/785469507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564581a3410 0x55645819d4c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7624010530 tx=0x7f7624010840 comp rx=0 tx=0).stop 2024-05-09T04:53:35.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.551+0000 7f7636000700 1 --2- 172.21.15.6:0/1856123634 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f76180b29d0 0x7f76180b4ee0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f762800b780 tx=0x7f762801d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:35.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.551+0000 7f7646bc5b80 1 -- 172.21.15.6:0/785469507 shutdown_connections 2024-05-09T04:53:35.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.551+0000 7f7646bc5b80 1 --2- 172.21.15.6:0/785469507 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55645819da00 0x55645819de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:35.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.551+0000 7f7646bc5b80 1 --2- 172.21.15.6:0/785469507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556458020450 0x5564581a2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:35.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.551+0000 7f7646bc5b80 1 --2- 172.21.15.6:0/785469507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564581a3410 0x55645819d4c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:35.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.551+0000 7f7646bc5b80 1 -- 172.21.15.6:0/785469507 >> 172.21.15.6:0/785469507 conn(0x556458064e20 msgr2=0x556458197e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:35.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.551+0000 7f7646bc5b80 1 -- 172.21.15.6:0/785469507 shutdown_connections 2024-05-09T04:53:35.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.551+0000 7f7646bc5b80 1 -- 172.21.15.6:0/785469507 wait complete. 2024-05-09T04:53:35.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.552+0000 7f7646bc5b80 1 -- start start 2024-05-09T04:53:35.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.552+0000 7f7646bc5b80 1 Processor -- start 2024-05-09T04:53:35.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.553+0000 7f7646bc5b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556458020450 0x5564581a6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:35.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.553+0000 7f7646bc5b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55645819da00 0x5564581a7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:35.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.553+0000 7f7646bc5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564581a3410 0x5564581a75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:35.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.553+0000 7f7646bc5b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5564581a9f00 con 0x556458020450 2024-05-09T04:53:35.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.553+0000 7f7646bc5b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556458025670 con 0x55645819da00 2024-05-09T04:53:35.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.553+0000 7f7646bc5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5564581aa080 con 0x5564581a3410 2024-05-09T04:53:35.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.553+0000 7f7646bc5b80 10 client.0 _ll_register_callbacks cb 0x556458057540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:35.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.553+0000 7f7635600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556458020450 0x5564581a6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:35.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.553+0000 7f7635600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556458020450 0x5564581a6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41436/0 (socket says 172.21.15.6:41436) 2024-05-09T04:53:35.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.553+0000 7f7635600700 1 -- 172.21.15.6:0/2481425059 learned_addr learned my addr 172.21.15.6:0/2481425059 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:35.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.553+0000 7f7636000700 1 --2- 172.21.15.6:0/2481425059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564581a3410 0x5564581a75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:35.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.553+0000 7f7634c00700 1 --2- 172.21.15.6:0/2481425059 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55645819da00 0x5564581a7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:35.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.554+0000 7f7635600700 1 -- 172.21.15.6:0/2481425059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564581a3410 msgr2=0x5564581a75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:35.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.554+0000 7f7635600700 1 --2- 172.21.15.6:0/2481425059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564581a3410 0x5564581a75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:35.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.554+0000 7f7635600700 1 -- 172.21.15.6:0/2481425059 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55645819da00 msgr2=0x5564581a7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:35.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.554+0000 7f7635600700 1 --2- 172.21.15.6:0/2481425059 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55645819da00 0x5564581a7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:35.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.554+0000 7f7635600700 1 -- 172.21.15.6:0/2481425059 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564581b33e0 con 0x556458020450 2024-05-09T04:53:35.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.554+0000 7f7635600700 1 --2- 172.21.15.6:0/2481425059 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556458020450 0x5564581a6b20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7624010500 tx=0x7f7624003f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6246af30944af63 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:35.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.554+0000 7f7617400700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-05-09T04:53:35.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.555+0000 7f7617400700 1 -- 172.21.15.6:0/2481425059 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624029060 con 0x556458020450 2024-05-09T04:53:35.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.555+0000 7f7617400700 1 -- 172.21.15.6:0/2481425059 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f7624003680 con 0x556458020450 2024-05-09T04:53:35.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.555+0000 7f7646bc5b80 1 -- 172.21.15.6:0/2481425059 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76240043c0 con 0x556458020450 2024-05-09T04:53:35.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.555+0000 7f7617400700 1 -- 172.21.15.6:0/2481425059 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7624030050 con 0x556458020450 2024-05-09T04:53:35.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.555+0000 7f7646bc5b80 10 client.14868 fetch_fsmap learned FSMap version 29 2024-05-09T04:53:35.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.555+0000 7f7646bc5b80 1 -- 172.21.15.6:0/2481425059 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5564581b4c80 con 0x556458020450 2024-05-09T04:53:35.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.555+0000 7f7617400700 1 -- 172.21.15.6:0/2481425059 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76240043c0 con 0x556458020450 2024-05-09T04:53:35.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.555+0000 7f7617400700 1 -- 172.21.15.6:0/2481425059 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7624004870 con 0x556458020450 2024-05-09T04:53:35.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.556+0000 7f7646bc5b80 10 client.14868 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T04:53:35.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.556+0000 7f7646bc5b80 10 client.14868 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T04:53:35.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.556+0000 7f7646bc5b80 20 client.14868 populate_metadata read hostname 'smithi006' 2024-05-09T04:53:35.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.556+0000 7f7646bc5b80 1 --2- 172.21.15.6:0/2481425059 >> [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] conn(0x5564581b6ef0 0x5564581b8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:35.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.556+0000 7f7646bc5b80 4 client.14868 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:53:35.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.556+0000 7f7646bc5b80 1 -- 172.21.15.6:0/2481425059 --> [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556458169820 con 0x5564581b6ef0 2024-05-09T04:53:35.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.556+0000 7f7634c00700 1 --2- 172.21.15.6:0/2481425059 >> [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] conn(0x5564581b6ef0 0x5564581b8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:35.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.557+0000 7f7634c00700 1 --2- 172.21.15.6:0/2481425059 >> [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] conn(0x5564581b6ef0 0x5564581b8360 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a6675357798695ac server_cookie=3534f8c3fc9cee88 in_seq=0 out_seq=0 2024-05-09T04:53:35.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.557+0000 7f7617400700 10 client.14868 ms_handle_connect on v2:172.21.15.186:6838/3954883105 2024-05-09T04:53:35.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.558+0000 7f7617400700 1 -- 172.21.15.6:0/2481425059 <== mds.3 v2:172.21.15.186:6838/3954883105 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556458169820 con 0x5564581b6ef0 2024-05-09T04:53:35.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.558+0000 7f7617400700 10 client.14868 handle_command_reply: tid=0 2024-05-09T04:53:35.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.558+0000 7f7646bc5b80 1 client.14868 shutdown 2024-05-09T04:53:35.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.558+0000 7f7646bc5b80 1 -- 172.21.15.6:0/2481425059 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556458020450 msgr2=0x5564581a6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:35.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.558+0000 7f7646bc5b80 1 --2- 172.21.15.6:0/2481425059 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556458020450 0x5564581a6b20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7624010500 tx=0x7f7624003f00 comp rx=0 tx=0).stop 2024-05-09T04:53:35.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.558+0000 7f7646bc5b80 1 -- 172.21.15.6:0/2481425059 shutdown_connections 2024-05-09T04:53:35.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.558+0000 7f7646bc5b80 1 --2- 172.21.15.6:0/2481425059 >> [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] conn(0x5564581b6ef0 0x5564581b8360 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:35.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.558+0000 7f7646bc5b80 1 --2- 172.21.15.6:0/2481425059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564581a3410 0x5564581a75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:35.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.558+0000 7f7646bc5b80 1 --2- 172.21.15.6:0/2481425059 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556458020450 0x5564581a6b20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:35.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.558+0000 7f7646bc5b80 1 --2- 172.21.15.6:0/2481425059 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55645819da00 0x5564581a7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:35.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.558+0000 7f7646bc5b80 1 -- 172.21.15.6:0/2481425059 >> 172.21.15.6:0/2481425059 conn(0x556458064e20 msgr2=0x556458197e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:35.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.558+0000 7f7617400700 0 client.14868 ms_handle_reset on v2:172.21.15.186:6838/3954883105 2024-05-09T04:53:35.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.558+0000 7f7646bc5b80 1 -- 172.21.15.6:0/2481425059 shutdown_connections 2024-05-09T04:53:35.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.559+0000 7f7646bc5b80 1 -- 172.21.15.6:0/2481425059 wait complete. 2024-05-09T04:53:35.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.559+0000 7f7646bc5b80 20 client.14868 trim_cache size 0 max 16384 2024-05-09T04:53:35.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.582+0000 7f7646bc5b80 1 Processor -- start 2024-05-09T04:53:35.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.582+0000 7f7646bc5b80 1 -- start start 2024-05-09T04:53:35.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.583+0000 7f7646bc5b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556458020450 0x55645819d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:35.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.583+0000 7f7646bc5b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55645819da00 0x5564581a2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:35.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.583+0000 7f7646bc5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564581a3410 0x5564581a1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:35.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.583+0000 7f7646bc5b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5564581b66a0 con 0x556458020450 2024-05-09T04:53:35.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.583+0000 7f7646bc5b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5564581b6d70 con 0x55645819da00 2024-05-09T04:53:35.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.583+0000 7f7646bc5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5564581b7440 con 0x5564581a3410 2024-05-09T04:53:35.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.583+0000 7f7635600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556458020450 0x55645819d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:35.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.583+0000 7f7634c00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55645819da00 0x5564581a2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:35.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.583+0000 7f7635600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556458020450 0x55645819d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41450/0 (socket says 172.21.15.6:41450) 2024-05-09T04:53:35.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.583+0000 7f7635600700 1 -- 172.21.15.6:0/1974153982 learned_addr learned my addr 172.21.15.6:0/1974153982 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:35.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.583+0000 7f7636000700 1 --2- 172.21.15.6:0/1974153982 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564581a3410 0x5564581a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:35.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.584+0000 7f7634c00700 1 -- 172.21.15.6:0/1974153982 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564581a3410 msgr2=0x5564581a1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:35.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.584+0000 7f7634c00700 1 --2- 172.21.15.6:0/1974153982 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564581a3410 0x5564581a1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:35.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.584+0000 7f7634c00700 1 -- 172.21.15.6:0/1974153982 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556458020450 msgr2=0x55645819d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:35.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.584+0000 7f7634c00700 1 --2- 172.21.15.6:0/1974153982 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556458020450 0x55645819d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:35.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.584+0000 7f7634c00700 1 -- 172.21.15.6:0/1974153982 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556458198760 con 0x55645819da00 2024-05-09T04:53:35.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.584+0000 7f7635600700 1 --2- 172.21.15.6:0/1974153982 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556458020450 0x55645819d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:53:35.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.584+0000 7f7634c00700 1 --2- 172.21.15.6:0/1974153982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55645819da00 0x5564581a2e70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f761c0a6bc0 tx=0x7f761c0e4040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b439a22396af590 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:35.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.584+0000 7f7622000700 1 -- 172.21.15.6:0/1974153982 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761c0a92a0 con 0x55645819da00 2024-05-09T04:53:35.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.585+0000 7f7622000700 1 -- 172.21.15.6:0/1974153982 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f761c0b20f0 con 0x55645819da00 2024-05-09T04:53:35.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.585+0000 7f7622000700 1 -- 172.21.15.6:0/1974153982 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761c0a50e0 con 0x55645819da00 2024-05-09T04:53:35.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.585+0000 7f7646bc5b80 1 -- 172.21.15.6:0/1974153982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55645819da00 msgr2=0x5564581a2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:35.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.585+0000 7f7646bc5b80 1 --2- 172.21.15.6:0/1974153982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55645819da00 0x5564581a2e70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f761c0a6bc0 tx=0x7f761c0e4040 comp rx=0 tx=0).stop 2024-05-09T04:53:35.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.586+0000 7f7646bc5b80 1 -- 172.21.15.6:0/1974153982 shutdown_connections 2024-05-09T04:53:35.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.586+0000 7f7646bc5b80 1 --2- 172.21.15.6:0/1974153982 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564581a3410 0x5564581a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:35.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.586+0000 7f7646bc5b80 1 --2- 172.21.15.6:0/1974153982 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556458020450 0x55645819d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:35.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.586+0000 7f7646bc5b80 1 --2- 172.21.15.6:0/1974153982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55645819da00 0x5564581a2e70 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:35.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.586+0000 7f7646bc5b80 1 -- 172.21.15.6:0/1974153982 >> 172.21.15.6:0/1974153982 conn(0x556458064e20 msgr2=0x5564581a1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:35.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.586+0000 7f7646bc5b80 1 -- 172.21.15.6:0/1974153982 shutdown_connections 2024-05-09T04:53:35.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.586+0000 7f7646bc5b80 1 -- 172.21.15.6:0/1974153982 wait complete. 2024-05-09T04:53:35.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.586+0000 7f7646bc5b80 1 -- start start 2024-05-09T04:53:35.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.586+0000 7f7646bc5b80 1 Processor -- start 2024-05-09T04:53:35.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.587+0000 7f7646bc5b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556458020450 0x5564581971a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:35.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.587+0000 7f7646bc5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55645819da00 0x5564581976e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:35.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.587+0000 7f7646bc5b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5564581a3410 0x556458197c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:35.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.587+0000 7f7646bc5b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5564581b81e0 con 0x556458020450 2024-05-09T04:53:35.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.587+0000 7f7646bc5b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5564581b7b10 con 0x5564581a3410 2024-05-09T04:53:35.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.587+0000 7f7646bc5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5564581b88b0 con 0x55645819da00 2024-05-09T04:53:35.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.587+0000 7f7646bc5b80 10 client.0 _ll_register_callbacks cb 0x556458057540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:35.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.587+0000 7f7635600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556458020450 0x5564581971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:35.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.587+0000 7f7634c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55645819da00 0x5564581976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:35.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.587+0000 7f7635600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556458020450 0x5564581971a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41466/0 (socket says 172.21.15.6:41466) 2024-05-09T04:53:35.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.587+0000 7f7634c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55645819da00 0x5564581976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:55186/0 (socket says 172.21.15.6:55186) 2024-05-09T04:53:35.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.587+0000 7f7635600700 1 -- 172.21.15.6:0/687322087 learned_addr learned my addr 172.21.15.6:0/687322087 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:35.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.588+0000 7f7636000700 1 --2- 172.21.15.6:0/687322087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5564581a3410 0x556458197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:35.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.588+0000 7f7634c00700 1 -- 172.21.15.6:0/687322087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5564581a3410 msgr2=0x556458197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:35.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.588+0000 7f7634c00700 1 --2- 172.21.15.6:0/687322087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5564581a3410 0x556458197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:35.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.588+0000 7f7634c00700 1 -- 172.21.15.6:0/687322087 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556458020450 msgr2=0x5564581971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:35.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.588+0000 7f7634c00700 1 --2- 172.21.15.6:0/687322087 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556458020450 0x5564581971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:35.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.588+0000 7f7634c00700 1 -- 172.21.15.6:0/687322087 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564581bfb00 con 0x55645819da00 2024-05-09T04:53:35.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.588+0000 7f7634c00700 1 --2- 172.21.15.6:0/687322087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55645819da00 0x5564581976e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f761c0a9650 tx=0x7f761c0add90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a47b25d60ff827c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:35.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.588+0000 7f7617400700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-09T04:53:35.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.588+0000 7f7617400700 1 -- 172.21.15.6:0/687322087 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761c0a90c0 con 0x55645819da00 2024-05-09T04:53:35.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.588+0000 7f7646bc5b80 1 -- 172.21.15.6:0/687322087 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5564580cdc40 con 0x55645819da00 2024-05-09T04:53:35.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.588+0000 7f7617400700 1 -- 172.21.15.6:0/687322087 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f761c0a9bb0 con 0x55645819da00 2024-05-09T04:53:35.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.589+0000 7f7617400700 1 -- 172.21.15.6:0/687322087 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f761c0a9460 con 0x55645819da00 2024-05-09T04:53:35.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.589+0000 7f7646bc5b80 10 client.25003 fetch_fsmap learned FSMap version 29 2024-05-09T04:53:35.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.589+0000 7f7646bc5b80 1 -- 172.21.15.6:0/687322087 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556458222590 con 0x55645819da00 2024-05-09T04:53:35.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.589+0000 7f7617400700 1 -- 172.21.15.6:0/687322087 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5564580cdc40 con 0x55645819da00 2024-05-09T04:53:35.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.589+0000 7f7617400700 1 -- 172.21.15.6:0/687322087 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f761c0e1040 con 0x55645819da00 2024-05-09T04:53:35.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.589+0000 7f7646bc5b80 10 client.25003 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T04:53:35.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.590+0000 7f7646bc5b80 10 client.25003 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T04:53:35.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.590+0000 7f7646bc5b80 20 client.25003 populate_metadata read hostname 'smithi006' 2024-05-09T04:53:35.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.590+0000 7f7646bc5b80 1 --2- 172.21.15.6:0/687322087 >> [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] conn(0x5564582245d0 0x556458225a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:35.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.590+0000 7f7646bc5b80 4 client.25003 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T04:53:35.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.590+0000 7f7646bc5b80 1 -- 172.21.15.6:0/687322087 --> [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] -- command(tid 0: {"prefix": "status"}) -- 0x5564580c7ce0 con 0x5564582245d0 2024-05-09T04:53:35.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.590+0000 7f7635600700 1 --2- 172.21.15.6:0/687322087 >> [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] conn(0x5564582245d0 0x556458225a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:35.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.590+0000 7f7635600700 1 --2- 172.21.15.6:0/687322087 >> [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] conn(0x5564582245d0 0x556458225a40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=660ad6f90b253fe4 server_cookie=8e124f216fcb6175 in_seq=0 out_seq=0 2024-05-09T04:53:35.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.590+0000 7f7617400700 10 client.25003 ms_handle_connect on v2:172.21.15.186:6838/3954883105 2024-05-09T04:53:35.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.591+0000 7f7617400700 1 -- 172.21.15.6:0/687322087 <== mds.3 v2:172.21.15.186:6838/3954883105 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5564580c7ce0 con 0x5564582245d0 2024-05-09T04:53:35.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.591+0000 7f7617400700 10 client.25003 handle_command_reply: tid=0 2024-05-09T04:53:35.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.591+0000 7f7646bc5b80 1 client.25003 shutdown 2024-05-09T04:53:35.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.591+0000 7f7646bc5b80 1 -- 172.21.15.6:0/687322087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55645819da00 msgr2=0x5564581976e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:35.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.591+0000 7f7646bc5b80 1 --2- 172.21.15.6:0/687322087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55645819da00 0x5564581976e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f761c0a9650 tx=0x7f761c0add90 comp rx=0 tx=0).stop 2024-05-09T04:53:35.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.591+0000 7f7646bc5b80 1 -- 172.21.15.6:0/687322087 shutdown_connections 2024-05-09T04:53:35.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.591+0000 7f7646bc5b80 1 --2- 172.21.15.6:0/687322087 >> [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] conn(0x5564582245d0 0x556458225a40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:35.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.591+0000 7f7646bc5b80 1 --2- 172.21.15.6:0/687322087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5564581a3410 0x556458197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:35.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.591+0000 7f7646bc5b80 1 --2- 172.21.15.6:0/687322087 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556458020450 0x5564581971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:35.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.591+0000 7f7646bc5b80 1 --2- 172.21.15.6:0/687322087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55645819da00 0x5564581976e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:35.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.591+0000 7f7646bc5b80 1 -- 172.21.15.6:0/687322087 >> 172.21.15.6:0/687322087 conn(0x556458064e20 msgr2=0x5564581b2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:35.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.591+0000 7f7617400700 0 client.25003 ms_handle_reset on v2:172.21.15.186:6838/3954883105 2024-05-09T04:53:35.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.591+0000 7f7646bc5b80 1 -- 172.21.15.6:0/687322087 shutdown_connections 2024-05-09T04:53:35.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.591+0000 7f7646bc5b80 1 -- 172.21.15.6:0/687322087 wait complete. 2024-05-09T04:53:35.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.591+0000 7f7646bc5b80 20 client.25003 trim_cache size 0 max 16384 2024-05-09T04:53:35.598 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-09T04:53:35.598 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "f74d813c-0dbd-11ef-bc97-c7b262605968", 2024-05-09T04:53:35.598 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 3, 2024-05-09T04:53:35.598 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24511, 2024-05-09T04:53:35.598 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-05-09T04:53:35.598 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-05-09T04:53:35.598 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-05-09T04:53:35.598 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 153.84376164700001, 2024-05-09T04:53:35.599 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 29, 2024-05-09T04:53:35.599 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 89, 2024-05-09T04:53:35.599 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 89, 2024-05-09T04:53:35.599 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 274.31317436699999 2024-05-09T04:53:35.599 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-09T04:53:35.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.592+0000 7f7638000700 1 -- 172.21.15.6:0/1856123634 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f76180b29d0 msgr2=0x7f76180b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:35.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.592+0000 7f7638000700 1 --2- 172.21.15.6:0/1856123634 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f76180b29d0 0x7f76180b4ee0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f762800b780 tx=0x7f762801d040 comp rx=0 tx=0).stop 2024-05-09T04:53:35.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.592+0000 7f7638000700 1 -- 172.21.15.6:0/1856123634 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7630077790 msgr2=0x7f763019d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:35.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.592+0000 7f7638000700 1 --2- 172.21.15.6:0/1856123634 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7630077790 0x7f763019d580 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f761c00e9a0 tx=0x7f761c00ecb0 comp rx=0 tx=0).stop 2024-05-09T04:53:35.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.592+0000 7f7638000700 1 -- 172.21.15.6:0/1856123634 shutdown_connections 2024-05-09T04:53:35.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.592+0000 7f7638000700 1 --2- 172.21.15.6:0/1856123634 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f76180b29d0 0x7f76180b4ee0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:35.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.592+0000 7f7638000700 1 --2- 172.21.15.6:0/1856123634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7630110ba0 0x7f76301a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:35.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.592+0000 7f7638000700 1 --2- 172.21.15.6:0/1856123634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7630076e40 0x7f763019cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:35.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.592+0000 7f7638000700 1 --2- 172.21.15.6:0/1856123634 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7630077790 0x7f763019d580 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:35.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.592+0000 7f7638000700 1 -- 172.21.15.6:0/1856123634 >> 172.21.15.6:0/1856123634 conn(0x7f76300ffec0 msgr2=0x7f7630100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:35.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.593+0000 7f7638000700 1 -- 172.21.15.6:0/1856123634 shutdown_connections 2024-05-09T04:53:35.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:35.593+0000 7f7638000700 1 -- 172.21.15.6:0/1856123634 wait complete. 2024-05-09T04:53:36.475 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.h status 2024-05-09T04:53:36.953 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:53:37.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:36 smithi186 ceph-mon[34668]: pgmap v459: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:53:37.103 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:36 smithi006 ceph-mon[29835]: pgmap v459: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:53:37.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:36 smithi169 ceph-mon[34075]: pgmap v459: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:53:39.006 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:38 smithi006 ceph-mon[29835]: pgmap v460: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:53:39.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:38 smithi186 ceph-mon[34668]: pgmap v460: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:53:39.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:38 smithi169 ceph-mon[34075]: pgmap v460: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:53:39.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.362+0000 7fdf38000700 1 -- 172.21.15.6:0/3392531699 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf30075d30 msgr2=0x7fdf30076140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:39.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.362+0000 7fdf38000700 1 --2- 172.21.15.6:0/3392531699 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf30075d30 0x7fdf30076140 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400b0a0 tx=0x7fdf2400b3b0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.362+0000 7fdf38000700 1 -- 172.21.15.6:0/3392531699 shutdown_connections 2024-05-09T04:53:39.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.362+0000 7fdf38000700 1 --2- 172.21.15.6:0/3392531699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf30110ba0 0x7fdf30113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.362+0000 7fdf38000700 1 --2- 172.21.15.6:0/3392531699 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf30075d30 0x7fdf30076140 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.362+0000 7fdf38000700 1 --2- 172.21.15.6:0/3392531699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf30076680 0x7fdf30110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.362+0000 7fdf38000700 1 -- 172.21.15.6:0/3392531699 >> 172.21.15.6:0/3392531699 conn(0x7fdf30100040 msgr2=0x7fdf30102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:39.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.362+0000 7fdf38000700 1 -- 172.21.15.6:0/3392531699 shutdown_connections 2024-05-09T04:53:39.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.363+0000 7fdf38000700 1 -- 172.21.15.6:0/3392531699 wait complete. 2024-05-09T04:53:39.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.364+0000 7fdf38000700 1 Processor -- start 2024-05-09T04:53:39.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.365+0000 7fdf38000700 1 -- start start 2024-05-09T04:53:39.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.365+0000 7fdf38000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf30075d30 0x7fdf3019d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:39.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.365+0000 7fdf38000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf30076680 0x7fdf3019d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:39.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.365+0000 7fdf38000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf30110ba0 0x7fdf301a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:39.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.365+0000 7fdf38000700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fdf301166e0 con 0x7fdf30076680 2024-05-09T04:53:39.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.365+0000 7fdf38000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdf30116560 con 0x7fdf30075d30 2024-05-09T04:53:39.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.365+0000 7fdf38000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdf30116860 con 0x7fdf30110ba0 2024-05-09T04:53:39.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.365+0000 7fdf34c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf30076680 0x7fdf3019d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:39.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.365+0000 7fdf35600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf30075d30 0x7fdf3019d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:39.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.365+0000 7fdf34c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf30076680 0x7fdf3019d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56666/0 (socket says 172.21.15.6:56666) 2024-05-09T04:53:39.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.365+0000 7fdf34c00700 1 -- 172.21.15.6:0/312842529 learned_addr learned my addr 172.21.15.6:0/312842529 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:39.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.365+0000 7fdf36000700 1 --2- 172.21.15.6:0/312842529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf30110ba0 0x7fdf301a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:39.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.366+0000 7fdf34c00700 1 -- 172.21.15.6:0/312842529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf30110ba0 msgr2=0x7fdf301a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:39.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.366+0000 7fdf34c00700 1 --2- 172.21.15.6:0/312842529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf30110ba0 0x7fdf301a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.366+0000 7fdf34c00700 1 -- 172.21.15.6:0/312842529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf30075d30 msgr2=0x7fdf3019d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:39.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.366+0000 7fdf34c00700 1 --2- 172.21.15.6:0/312842529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf30075d30 0x7fdf3019d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.366+0000 7fdf34c00700 1 -- 172.21.15.6:0/312842529 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf301a1f00 con 0x7fdf30076680 2024-05-09T04:53:39.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.366+0000 7fdf36000700 1 --2- 172.21.15.6:0/312842529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf30110ba0 0x7fdf301a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:53:39.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.366+0000 7fdf34c00700 1 --2- 172.21.15.6:0/312842529 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf30076680 0x7fdf3019d5d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c00e9a0 tx=0x7fdf1c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:39.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.367+0000 7fdf2e000700 1 -- 172.21.15.6:0/312842529 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c00ca10 con 0x7fdf30076680 2024-05-09T04:53:39.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.367+0000 7fdf2e000700 1 -- 172.21.15.6:0/312842529 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf1c00cbb0 con 0x7fdf30076680 2024-05-09T04:53:39.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.367+0000 7fdf38000700 1 -- 172.21.15.6:0/312842529 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf301a21f0 con 0x7fdf30076680 2024-05-09T04:53:39.369 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.367+0000 7fdf2e000700 1 -- 172.21.15.6:0/312842529 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c018680 con 0x7fdf30076680 2024-05-09T04:53:39.369 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.367+0000 7fdf38000700 1 -- 172.21.15.6:0/312842529 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf301a9b10 con 0x7fdf30076680 2024-05-09T04:53:39.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.371+0000 7fdf2e000700 1 -- 172.21.15.6:0/312842529 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fdf1c010430 con 0x7fdf30076680 2024-05-09T04:53:39.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.372+0000 7fdf46bc1b80 1 Processor -- start 2024-05-09T04:53:39.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.372+0000 7fdf46bc1b80 1 -- start start 2024-05-09T04:53:39.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.373+0000 7fdf46bc1b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb572f5450 0x55cb57477ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:39.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.373+0000 7fdf46bc1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb57478410 0x55cb574724c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:39.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.373+0000 7fdf46bc1b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cb57472a00 0x55cb57472e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:39.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.373+0000 7fdf46bc1b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55cb57395260 con 0x55cb57472a00 2024-05-09T04:53:39.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.373+0000 7fdf46bc1b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55cb57333090 con 0x55cb572f5450 2024-05-09T04:53:39.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.373+0000 7fdf46bc1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55cb57391220 con 0x55cb57478410 2024-05-09T04:53:39.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.373+0000 7fdf36000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb572f5450 0x55cb57477ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:39.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.373+0000 7fdf34c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cb57472a00 0x55cb57472e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:39.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.373+0000 7fdf36000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb572f5450 0x55cb57477ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:47890/0 (socket says 172.21.15.6:47890) 2024-05-09T04:53:39.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.373+0000 7fdf36000700 1 -- 172.21.15.6:0/3575878406 learned_addr learned my addr 172.21.15.6:0/3575878406 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:39.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.373+0000 7fdf35600700 1 --2- 172.21.15.6:0/3575878406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb57478410 0x55cb574724c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:39.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.373+0000 7fdf2e000700 1 --2- 172.21.15.6:0/312842529 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fdf180b2900 0x7fdf180b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:39.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.374+0000 7fdf36000700 1 -- 172.21.15.6:0/3575878406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb57478410 msgr2=0x55cb574724c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:39.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.374+0000 7fdf36000700 1 --2- 172.21.15.6:0/3575878406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb57478410 0x55cb574724c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.374+0000 7fdf36000700 1 -- 172.21.15.6:0/3575878406 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cb57472a00 msgr2=0x55cb57472e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:39.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.374+0000 7fdf2e000700 1 -- 172.21.15.6:0/312842529 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fdf1c0e26e0 con 0x7fdf30076680 2024-05-09T04:53:39.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.374+0000 7fdf36000700 1 --2- 172.21.15.6:0/3575878406 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cb57472a00 0x55cb57472e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.374+0000 7fdf36000700 1 -- 172.21.15.6:0/3575878406 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb57210440 con 0x55cb572f5450 2024-05-09T04:53:39.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.374+0000 7fdf35600700 1 --2- 172.21.15.6:0/3575878406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb57478410 0x55cb574724c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:53:39.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.374+0000 7fdf36000700 1 --2- 172.21.15.6:0/312842529 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fdf180b2900 0x7fdf180b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:39.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.374+0000 7fdf36000700 1 --2- 172.21.15.6:0/3575878406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb572f5450 0x55cb57477ed0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdf28011570 tx=0x7fdf28011880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b93ed790b49eab1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:39.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.375+0000 7fdf22a00700 1 -- 172.21.15.6:0/3575878406 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf28017050 con 0x55cb572f5450 2024-05-09T04:53:39.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.375+0000 7fdf22a00700 1 -- 172.21.15.6:0/3575878406 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf2801bb30 con 0x55cb572f5450 2024-05-09T04:53:39.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.375+0000 7fdf22a00700 1 -- 172.21.15.6:0/3575878406 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf28025a70 con 0x55cb572f5450 2024-05-09T04:53:39.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.376+0000 7fdf46bc1b80 1 -- 172.21.15.6:0/3575878406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb572f5450 msgr2=0x55cb57477ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:39.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.376+0000 7fdf46bc1b80 1 --2- 172.21.15.6:0/3575878406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb572f5450 0x55cb57477ed0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdf28011570 tx=0x7fdf28011880 comp rx=0 tx=0).stop 2024-05-09T04:53:39.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.376+0000 7fdf36000700 1 --2- 172.21.15.6:0/312842529 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fdf180b2900 0x7fdf180b4e10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdf28022000 tx=0x7fdf28023000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:39.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.376+0000 7fdf46bc1b80 1 -- 172.21.15.6:0/3575878406 shutdown_connections 2024-05-09T04:53:39.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.376+0000 7fdf46bc1b80 1 --2- 172.21.15.6:0/3575878406 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cb57472a00 0x55cb57472e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.376+0000 7fdf46bc1b80 1 --2- 172.21.15.6:0/3575878406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb572f5450 0x55cb57477ed0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.376+0000 7fdf46bc1b80 1 --2- 172.21.15.6:0/3575878406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb57478410 0x55cb574724c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.376+0000 7fdf46bc1b80 1 -- 172.21.15.6:0/3575878406 >> 172.21.15.6:0/3575878406 conn(0x55cb57339e20 msgr2=0x55cb5746ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:39.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.376+0000 7fdf46bc1b80 1 -- 172.21.15.6:0/3575878406 shutdown_connections 2024-05-09T04:53:39.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.376+0000 7fdf46bc1b80 1 -- 172.21.15.6:0/3575878406 wait complete. 2024-05-09T04:53:39.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.377+0000 7fdf46bc1b80 1 -- start start 2024-05-09T04:53:39.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.377+0000 7fdf46bc1b80 1 Processor -- start 2024-05-09T04:53:39.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.378+0000 7fdf46bc1b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cb572f5450 0x55cb5747bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:39.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.378+0000 7fdf46bc1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb57472a00 0x55cb5747c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:39.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.378+0000 7fdf46bc1b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb57478410 0x55cb5747c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:39.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.378+0000 7fdf46bc1b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55cb5747ef00 con 0x55cb572f5450 2024-05-09T04:53:39.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.378+0000 7fdf46bc1b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55cb572fa670 con 0x55cb57478410 2024-05-09T04:53:39.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.378+0000 7fdf46bc1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55cb5747f080 con 0x55cb57472a00 2024-05-09T04:53:39.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.378+0000 7fdf46bc1b80 10 client.0 _ll_register_callbacks cb 0x55cb5732c540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:39.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.378+0000 7fdf35600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cb572f5450 0x55cb5747bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:39.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.378+0000 7fdf34c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb57472a00 0x55cb5747c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:39.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.378+0000 7fdf35600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cb572f5450 0x55cb5747bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56682/0 (socket says 172.21.15.6:56682) 2024-05-09T04:53:39.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.378+0000 7fdf35600700 1 -- 172.21.15.6:0/1635978829 learned_addr learned my addr 172.21.15.6:0/1635978829 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:39.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.378+0000 7fdf36000700 1 --2- 172.21.15.6:0/1635978829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb57478410 0x55cb5747c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:39.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.379+0000 7fdf34c00700 1 -- 172.21.15.6:0/1635978829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb57478410 msgr2=0x55cb5747c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:39.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.379+0000 7fdf34c00700 1 --2- 172.21.15.6:0/1635978829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb57478410 0x55cb5747c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.379+0000 7fdf34c00700 1 -- 172.21.15.6:0/1635978829 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cb572f5450 msgr2=0x55cb5747bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:39.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.379+0000 7fdf34c00700 1 --2- 172.21.15.6:0/1635978829 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cb572f5450 0x55cb5747bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.379+0000 7fdf34c00700 1 -- 172.21.15.6:0/1635978829 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb574883b0 con 0x55cb57472a00 2024-05-09T04:53:39.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.379+0000 7fdf34c00700 1 --2- 172.21.15.6:0/1635978829 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb57472a00 0x55cb5747c0e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c00e970 tx=0x7fdf1c010c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4628f4ec3d2cdec8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:39.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.379+0000 7fdf17400700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-09T04:53:39.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.379+0000 7fdf17400700 1 -- 172.21.15.6:0/1635978829 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c0a50b0 con 0x55cb57472a00 2024-05-09T04:53:39.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.380+0000 7fdf17400700 1 -- 172.21.15.6:0/1635978829 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fdf1c0cc0e0 con 0x55cb57472a00 2024-05-09T04:53:39.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.380+0000 7fdf46bc1b80 1 -- 172.21.15.6:0/1635978829 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf2801bb30 con 0x55cb57472a00 2024-05-09T04:53:39.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.380+0000 7fdf17400700 1 -- 172.21.15.6:0/1635978829 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdf1c0a9430 con 0x55cb57472a00 2024-05-09T04:53:39.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.380+0000 7fdf46bc1b80 10 client.25015 fetch_fsmap learned FSMap version 29 2024-05-09T04:53:39.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.380+0000 7fdf46bc1b80 1 -- 172.21.15.6:0/1635978829 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cb57489d30 con 0x55cb57472a00 2024-05-09T04:53:39.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.380+0000 7fdf17400700 1 -- 172.21.15.6:0/1635978829 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf2801bb30 con 0x55cb57472a00 2024-05-09T04:53:39.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.380+0000 7fdf17400700 1 -- 172.21.15.6:0/1635978829 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdf1c0cabd0 con 0x55cb57472a00 2024-05-09T04:53:39.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.381+0000 7fdf46bc1b80 10 client.25015 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T04:53:39.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.381+0000 7fdf46bc1b80 10 client.25015 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-09T04:53:39.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.381+0000 7fdf46bc1b80 20 client.25015 populate_metadata read hostname 'smithi006' 2024-05-09T04:53:39.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.381+0000 7fdf46bc1b80 1 --2- 172.21.15.6:0/1635978829 >> [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] conn(0x55cb5748c2b0 0x55cb5748d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:39.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.381+0000 7fdf46bc1b80 4 client.25015 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:53:39.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.381+0000 7fdf46bc1b80 1 -- 172.21.15.6:0/1635978829 --> [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb5743e820 con 0x55cb5748c2b0 2024-05-09T04:53:39.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.381+0000 7fdf35600700 1 --2- 172.21.15.6:0/1635978829 >> [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] conn(0x55cb5748c2b0 0x55cb5748d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:39.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.382+0000 7fdf35600700 1 --2- 172.21.15.6:0/1635978829 >> [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] conn(0x55cb5748c2b0 0x55cb5748d720 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2fa07c84e41cc199 server_cookie=d104957cc3d59406 in_seq=0 out_seq=0 2024-05-09T04:53:39.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.382+0000 7fdf17400700 10 client.25015 ms_handle_connect on v2:172.21.15.169:6836/1845715294 2024-05-09T04:53:39.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.383+0000 7fdf17400700 1 -- 172.21.15.6:0/1635978829 <== mds.2 v2:172.21.15.169:6836/1845715294 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cb5743e820 con 0x55cb5748c2b0 2024-05-09T04:53:39.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.383+0000 7fdf17400700 10 client.25015 handle_command_reply: tid=0 2024-05-09T04:53:39.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.383+0000 7fdf46bc1b80 1 client.25015 shutdown 2024-05-09T04:53:39.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.383+0000 7fdf46bc1b80 1 -- 172.21.15.6:0/1635978829 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb57472a00 msgr2=0x55cb5747c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:39.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.383+0000 7fdf46bc1b80 1 --2- 172.21.15.6:0/1635978829 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb57472a00 0x55cb5747c0e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c00e970 tx=0x7fdf1c010c10 comp rx=0 tx=0).stop 2024-05-09T04:53:39.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.383+0000 7fdf46bc1b80 1 -- 172.21.15.6:0/1635978829 shutdown_connections 2024-05-09T04:53:39.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.383+0000 7fdf46bc1b80 1 --2- 172.21.15.6:0/1635978829 >> [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] conn(0x55cb5748c2b0 0x55cb5748d720 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.383+0000 7fdf46bc1b80 1 --2- 172.21.15.6:0/1635978829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb57478410 0x55cb5747c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.383+0000 7fdf46bc1b80 1 --2- 172.21.15.6:0/1635978829 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cb572f5450 0x55cb5747bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.383+0000 7fdf46bc1b80 1 --2- 172.21.15.6:0/1635978829 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb57472a00 0x55cb5747c0e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.383+0000 7fdf17400700 0 client.25015 ms_handle_reset on v2:172.21.15.169:6836/1845715294 2024-05-09T04:53:39.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.383+0000 7fdf46bc1b80 1 -- 172.21.15.6:0/1635978829 >> 172.21.15.6:0/1635978829 conn(0x55cb57339e20 msgr2=0x55cb5746ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:39.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.384+0000 7fdf46bc1b80 1 -- 172.21.15.6:0/1635978829 shutdown_connections 2024-05-09T04:53:39.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.384+0000 7fdf46bc1b80 1 -- 172.21.15.6:0/1635978829 wait complete. 2024-05-09T04:53:39.386 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.384+0000 7fdf46bc1b80 20 client.25015 trim_cache size 0 max 16384 2024-05-09T04:53:39.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.407+0000 7fdf46bc1b80 1 Processor -- start 2024-05-09T04:53:39.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.407+0000 7fdf46bc1b80 1 -- start start 2024-05-09T04:53:39.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.407+0000 7fdf46bc1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb572f5450 0x55cb574724c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:39.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.407+0000 7fdf46bc1b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cb57472a00 0x55cb5747b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:39.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.407+0000 7fdf46bc1b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb57478410 0x55cb5746cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:39.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.407+0000 7fdf46bc1b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55cb5748ba60 con 0x55cb57472a00 2024-05-09T04:53:39.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.407+0000 7fdf46bc1b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55cb5748c130 con 0x55cb57478410 2024-05-09T04:53:39.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.407+0000 7fdf46bc1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55cb5748c800 con 0x55cb572f5450 2024-05-09T04:53:39.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.408+0000 7fdf34c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cb57472a00 0x55cb5747b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:39.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.408+0000 7fdf34c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cb57472a00 0x55cb5747b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56686/0 (socket says 172.21.15.6:56686) 2024-05-09T04:53:39.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.408+0000 7fdf34c00700 1 -- 172.21.15.6:0/1936087321 learned_addr learned my addr 172.21.15.6:0/1936087321 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:39.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.408+0000 7fdf36000700 1 --2- 172.21.15.6:0/1936087321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb57478410 0x55cb5746cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:39.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.408+0000 7fdf35600700 1 --2- 172.21.15.6:0/1936087321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb572f5450 0x55cb574724c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:39.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.408+0000 7fdf34c00700 1 -- 172.21.15.6:0/1936087321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb572f5450 msgr2=0x55cb574724c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:39.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.408+0000 7fdf34c00700 1 --2- 172.21.15.6:0/1936087321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb572f5450 0x55cb574724c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.408+0000 7fdf34c00700 1 -- 172.21.15.6:0/1936087321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb57478410 msgr2=0x55cb5746cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:39.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.408+0000 7fdf34c00700 1 --2- 172.21.15.6:0/1936087321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb57478410 0x55cb5746cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.408+0000 7fdf34c00700 1 -- 172.21.15.6:0/1936087321 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb5747f030 con 0x55cb57472a00 2024-05-09T04:53:39.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.409+0000 7fdf34c00700 1 --2- 172.21.15.6:0/1936087321 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cb57472a00 0x55cb5747b450 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c0a95d0 tx=0x7fdf1c0e8150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c67173df6b7336 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:39.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.409+0000 7fdf22000700 1 -- 172.21.15.6:0/1936087321 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c0aa020 con 0x55cb57472a00 2024-05-09T04:53:39.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.409+0000 7fdf22000700 1 -- 172.21.15.6:0/1936087321 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf1c0adcd0 con 0x55cb57472a00 2024-05-09T04:53:39.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.409+0000 7fdf22000700 1 -- 172.21.15.6:0/1936087321 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c010720 con 0x55cb57472a00 2024-05-09T04:53:39.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.410+0000 7fdf46bc1b80 1 -- 172.21.15.6:0/1936087321 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cb57472a00 msgr2=0x55cb5747b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:39.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.410+0000 7fdf46bc1b80 1 --2- 172.21.15.6:0/1936087321 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cb57472a00 0x55cb5747b450 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c0a95d0 tx=0x7fdf1c0e8150 comp rx=0 tx=0).stop 2024-05-09T04:53:39.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.410+0000 7fdf46bc1b80 1 -- 172.21.15.6:0/1936087321 shutdown_connections 2024-05-09T04:53:39.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.410+0000 7fdf46bc1b80 1 --2- 172.21.15.6:0/1936087321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb57478410 0x55cb5746cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.410+0000 7fdf46bc1b80 1 --2- 172.21.15.6:0/1936087321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb572f5450 0x55cb574724c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.410+0000 7fdf46bc1b80 1 --2- 172.21.15.6:0/1936087321 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cb57472a00 0x55cb5747b450 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.410+0000 7fdf46bc1b80 1 -- 172.21.15.6:0/1936087321 >> 172.21.15.6:0/1936087321 conn(0x55cb57339e20 msgr2=0x55cb5747a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:39.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.410+0000 7fdf46bc1b80 1 -- 172.21.15.6:0/1936087321 shutdown_connections 2024-05-09T04:53:39.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.410+0000 7fdf46bc1b80 1 -- 172.21.15.6:0/1936087321 wait complete. 2024-05-09T04:53:39.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.411+0000 7fdf46bc1b80 1 -- start start 2024-05-09T04:53:39.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.411+0000 7fdf46bc1b80 1 Processor -- start 2024-05-09T04:53:39.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.411+0000 7fdf46bc1b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb572f5450 0x55cb5746c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:39.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.411+0000 7fdf46bc1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb57472a00 0x55cb5746c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:39.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.411+0000 7fdf46bc1b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cb57478410 0x55cb5746cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:39.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.412+0000 7fdf46bc1b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55cb5748d5a0 con 0x55cb57478410 2024-05-09T04:53:39.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.412+0000 7fdf46bc1b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55cb5748ced0 con 0x55cb572f5450 2024-05-09T04:53:39.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.412+0000 7fdf46bc1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55cb5748dc70 con 0x55cb57472a00 2024-05-09T04:53:39.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.412+0000 7fdf46bc1b80 10 client.0 _ll_register_callbacks cb 0x55cb5732c540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:39.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.412+0000 7fdf35600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb572f5450 0x55cb5746c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:39.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.412+0000 7fdf34c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb57472a00 0x55cb5746c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:39.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.412+0000 7fdf36000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cb57478410 0x55cb5746cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:39.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.412+0000 7fdf35600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb572f5450 0x55cb5746c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:47916/0 (socket says 172.21.15.6:47916) 2024-05-09T04:53:39.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.412+0000 7fdf34c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb57472a00 0x55cb5746c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:47494/0 (socket says 172.21.15.6:47494) 2024-05-09T04:53:39.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.412+0000 7fdf35600700 1 -- 172.21.15.6:0/1535682210 learned_addr learned my addr 172.21.15.6:0/1535682210 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:39.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.412+0000 7fdf34c00700 1 -- 172.21.15.6:0/1535682210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb572f5450 msgr2=0x55cb5746c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:39.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.412+0000 7fdf34c00700 1 --2- 172.21.15.6:0/1535682210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb572f5450 0x55cb5746c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.412+0000 7fdf34c00700 1 -- 172.21.15.6:0/1535682210 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cb57478410 msgr2=0x55cb5746cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:39.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.412+0000 7fdf34c00700 1 --2- 172.21.15.6:0/1535682210 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cb57478410 0x55cb5746cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.412+0000 7fdf34c00700 1 -- 172.21.15.6:0/1535682210 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb574952e0 con 0x55cb57472a00 2024-05-09T04:53:39.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.412+0000 7fdf35600700 1 --2- 172.21.15.6:0/1535682210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb572f5450 0x55cb5746c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:53:39.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.413+0000 7fdf34c00700 1 --2- 172.21.15.6:0/1535682210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb57472a00 0x55cb5746c8a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x55cb57207c40 tx=0x7fdf1c0a58b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=884c6d768220b2f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:39.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.413+0000 7fdf17400700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-09T04:53:39.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.413+0000 7fdf17400700 1 -- 172.21.15.6:0/1535682210 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c0a9920 con 0x55cb57472a00 2024-05-09T04:53:39.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.413+0000 7fdf17400700 1 -- 172.21.15.6:0/1535682210 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fdf1c0a8b80 con 0x55cb57472a00 2024-05-09T04:53:39.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.413+0000 7fdf46bc1b80 1 -- 172.21.15.6:0/1535682210 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf1c0adcd0 con 0x55cb57472a00 2024-05-09T04:53:39.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.413+0000 7fdf17400700 1 -- 172.21.15.6:0/1535682210 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdf1c0a8d80 con 0x55cb57472a00 2024-05-09T04:53:39.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.413+0000 7fdf46bc1b80 10 client.25021 fetch_fsmap learned FSMap version 29 2024-05-09T04:53:39.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.413+0000 7fdf46bc1b80 1 -- 172.21.15.6:0/1535682210 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cb574f7b10 con 0x55cb57472a00 2024-05-09T04:53:39.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.414+0000 7fdf17400700 1 -- 172.21.15.6:0/1535682210 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf1c0adcd0 con 0x55cb57472a00 2024-05-09T04:53:39.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.414+0000 7fdf17400700 1 -- 172.21.15.6:0/1535682210 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdf1c0a87b0 con 0x55cb57472a00 2024-05-09T04:53:39.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.414+0000 7fdf46bc1b80 10 client.25021 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T04:53:39.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.414+0000 7fdf46bc1b80 10 client.25021 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-09T04:53:39.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.414+0000 7fdf46bc1b80 20 client.25021 populate_metadata read hostname 'smithi006' 2024-05-09T04:53:39.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.414+0000 7fdf46bc1b80 1 --2- 172.21.15.6:0/1535682210 >> [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] conn(0x55cb574f9930 0x55cb574fada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:39.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.414+0000 7fdf46bc1b80 4 client.25021 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T04:53:39.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.414+0000 7fdf46bc1b80 1 -- 172.21.15.6:0/1535682210 --> [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] -- command(tid 0: {"prefix": "status"}) -- 0x55cb573a2c40 con 0x55cb574f9930 2024-05-09T04:53:39.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.414+0000 7fdf35600700 1 --2- 172.21.15.6:0/1535682210 >> [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] conn(0x55cb574f9930 0x55cb574fada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:39.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.415+0000 7fdf35600700 1 --2- 172.21.15.6:0/1535682210 >> [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] conn(0x55cb574f9930 0x55cb574fada0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c56980f80d818aac server_cookie=3f3372211846e650 in_seq=0 out_seq=0 2024-05-09T04:53:39.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.415+0000 7fdf17400700 10 client.25021 ms_handle_connect on v2:172.21.15.169:6836/1845715294 2024-05-09T04:53:39.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.415+0000 7fdf17400700 1 -- 172.21.15.6:0/1535682210 <== mds.2 v2:172.21.15.169:6836/1845715294 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55cb573a2c40 con 0x55cb574f9930 2024-05-09T04:53:39.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.415+0000 7fdf17400700 10 client.25021 handle_command_reply: tid=0 2024-05-09T04:53:39.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.415+0000 7fdf46bc1b80 1 client.25021 shutdown 2024-05-09T04:53:39.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.416+0000 7fdf46bc1b80 1 -- 172.21.15.6:0/1535682210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb57472a00 msgr2=0x55cb5746c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:39.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.416+0000 7fdf46bc1b80 1 --2- 172.21.15.6:0/1535682210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb57472a00 0x55cb5746c8a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x55cb57207c40 tx=0x7fdf1c0a58b0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.416+0000 7fdf46bc1b80 1 -- 172.21.15.6:0/1535682210 shutdown_connections 2024-05-09T04:53:39.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.416+0000 7fdf46bc1b80 1 --2- 172.21.15.6:0/1535682210 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cb57478410 0x55cb5746cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.416+0000 7fdf46bc1b80 1 --2- 172.21.15.6:0/1535682210 >> [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] conn(0x55cb574f9930 0x55cb574fada0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.416+0000 7fdf46bc1b80 1 --2- 172.21.15.6:0/1535682210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb572f5450 0x55cb5746c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.416+0000 7fdf46bc1b80 1 --2- 172.21.15.6:0/1535682210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb57472a00 0x55cb5746c8a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.416+0000 7fdf46bc1b80 1 -- 172.21.15.6:0/1535682210 >> 172.21.15.6:0/1535682210 conn(0x55cb57339e20 msgr2=0x55cb574872d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:39.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.416+0000 7fdf17400700 0 client.25021 ms_handle_reset on v2:172.21.15.169:6836/1845715294 2024-05-09T04:53:39.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.416+0000 7fdf46bc1b80 1 -- 172.21.15.6:0/1535682210 shutdown_connections 2024-05-09T04:53:39.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.416+0000 7fdf46bc1b80 1 -- 172.21.15.6:0/1535682210 wait complete. 2024-05-09T04:53:39.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.416+0000 7fdf46bc1b80 20 client.25021 trim_cache size 0 max 16384 2024-05-09T04:53:39.420 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-09T04:53:39.420 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "f74d813c-0dbd-11ef-bc97-c7b262605968", 2024-05-09T04:53:39.420 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 2, 2024-05-09T04:53:39.420 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24512, 2024-05-09T04:53:39.420 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-05-09T04:53:39.420 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-05-09T04:53:39.420 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-05-09T04:53:39.420 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 158.68312952900001, 2024-05-09T04:53:39.420 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 29, 2024-05-09T04:53:39.420 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 89, 2024-05-09T04:53:39.420 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 89, 2024-05-09T04:53:39.420 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 281.07253347699998 2024-05-09T04:53:39.420 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-09T04:53:39.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.417+0000 7fdf38000700 1 -- 172.21.15.6:0/312842529 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fdf180b2900 msgr2=0x7fdf180b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:39.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.417+0000 7fdf38000700 1 --2- 172.21.15.6:0/312842529 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fdf180b2900 0x7fdf180b4e10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdf28022000 tx=0x7fdf28023000 comp rx=0 tx=0).stop 2024-05-09T04:53:39.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.417+0000 7fdf38000700 1 -- 172.21.15.6:0/312842529 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf30076680 msgr2=0x7fdf3019d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:39.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.417+0000 7fdf38000700 1 --2- 172.21.15.6:0/312842529 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf30076680 0x7fdf3019d5d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c00e9a0 tx=0x7fdf1c00ecb0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.417+0000 7fdf38000700 1 -- 172.21.15.6:0/312842529 shutdown_connections 2024-05-09T04:53:39.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.417+0000 7fdf38000700 1 --2- 172.21.15.6:0/312842529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf30110ba0 0x7fdf301a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.417+0000 7fdf38000700 1 --2- 172.21.15.6:0/312842529 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fdf180b2900 0x7fdf180b4e10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.417+0000 7fdf38000700 1 --2- 172.21.15.6:0/312842529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf30075d30 0x7fdf3019d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.417+0000 7fdf38000700 1 --2- 172.21.15.6:0/312842529 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf30076680 0x7fdf3019d5d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.417+0000 7fdf38000700 1 -- 172.21.15.6:0/312842529 >> 172.21.15.6:0/312842529 conn(0x7fdf30100040 msgr2=0x7fdf30101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:39.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.417+0000 7fdf38000700 1 -- 172.21.15.6:0/312842529 shutdown_connections 2024-05-09T04:53:39.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:39.417+0000 7fdf38000700 1 -- 172.21.15.6:0/312842529 wait complete. 2024-05-09T04:53:40.310 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.g status 2024-05-09T04:53:40.790 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:53:41.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:40 smithi186 ceph-mon[34668]: pgmap v461: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:53:41.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:40 smithi169 ceph-mon[34075]: pgmap v461: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:53:41.120 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:40 smithi006 ceph-mon[29835]: pgmap v461: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:53:42.909 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:42 smithi006 ceph-mon[29835]: pgmap v462: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:53:42.909 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:42 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:53:43.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:42 smithi186 ceph-mon[34668]: pgmap v462: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:53:43.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:42 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:53:43.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:42 smithi169 ceph-mon[34075]: pgmap v462: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:53:43.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:42 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:53:43.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.151+0000 7fbcb6c00700 1 -- 172.21.15.6:0/3538695232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcb0105290 msgr2=0x7fbcb01056a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:43.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.151+0000 7fbcb6c00700 1 --2- 172.21.15.6:0/3538695232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcb0105290 0x7fbcb01056a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbca4009990 tx=0x7fbca4009ca0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.151+0000 7fbcb6c00700 1 -- 172.21.15.6:0/3538695232 shutdown_connections 2024-05-09T04:53:43.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.151+0000 7fbcb6c00700 1 --2- 172.21.15.6:0/3538695232 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbcb010e700 0x7fbcb0110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.151+0000 7fbcb6c00700 1 --2- 172.21.15.6:0/3538695232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcb0105290 0x7fbcb01056a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.151+0000 7fbcb6c00700 1 --2- 172.21.15.6:0/3538695232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbcb0105be0 0x7fbcb010e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.151+0000 7fbcb6c00700 1 -- 172.21.15.6:0/3538695232 >> 172.21.15.6:0/3538695232 conn(0x7fbcb0078ac0 msgr2=0x7fbcb0078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:43.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.151+0000 7fbcb6c00700 1 -- 172.21.15.6:0/3538695232 shutdown_connections 2024-05-09T04:53:43.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.151+0000 7fbcb6c00700 1 -- 172.21.15.6:0/3538695232 wait complete. 2024-05-09T04:53:43.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.153+0000 7fbcb6c00700 1 Processor -- start 2024-05-09T04:53:43.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.153+0000 7fbcb6c00700 1 -- start start 2024-05-09T04:53:43.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.153+0000 7fbcb6c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbcb0105290 0x7fbcb0102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.154+0000 7fbcb6c00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbcb0105be0 0x7fbcb01034c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.154+0000 7fbcb6c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcb010e700 0x7fbcb0103a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.154+0000 7fbcb6c00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb0114250 con 0x7fbcb0105290 2024-05-09T04:53:43.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.154+0000 7fbcb6c00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb01140d0 con 0x7fbcb0105be0 2024-05-09T04:53:43.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.154+0000 7fbcb6c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb01143d0 con 0x7fbcb010e700 2024-05-09T04:53:43.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.154+0000 7fbcb5800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbcb0105290 0x7fbcb0102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.154+0000 7fbcb4e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbcb0105be0 0x7fbcb01034c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.154+0000 7fbcb4e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbcb0105be0 0x7fbcb01034c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:47922/0 (socket says 172.21.15.6:47922) 2024-05-09T04:53:43.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.154+0000 7fbcb4e00700 1 -- 172.21.15.6:0/744990747 learned_addr learned my addr 172.21.15.6:0/744990747 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:43.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.154+0000 7fbcb6200700 1 --2- 172.21.15.6:0/744990747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcb010e700 0x7fbcb0103a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.154+0000 7fbcb4e00700 1 -- 172.21.15.6:0/744990747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcb010e700 msgr2=0x7fbcb0103a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:43.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.154+0000 7fbcb4e00700 1 --2- 172.21.15.6:0/744990747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcb010e700 0x7fbcb0103a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.154+0000 7fbcb4e00700 1 -- 172.21.15.6:0/744990747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbcb0105290 msgr2=0x7fbcb0102f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:43.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.154+0000 7fbcb4e00700 1 --2- 172.21.15.6:0/744990747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbcb0105290 0x7fbcb0102f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.154+0000 7fbcb4e00700 1 -- 172.21.15.6:0/744990747 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb01a8fc0 con 0x7fbcb0105be0 2024-05-09T04:53:43.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.155+0000 7fbcb4e00700 1 --2- 172.21.15.6:0/744990747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbcb0105be0 0x7fbcb01034c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00e9a0 tx=0x7fbc9c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:43.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.155+0000 7fbcae000700 1 -- 172.21.15.6:0/744990747 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c00cbd0 con 0x7fbcb0105be0 2024-05-09T04:53:43.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.155+0000 7fbcb6c00700 1 -- 172.21.15.6:0/744990747 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcb01a92b0 con 0x7fbcb0105be0 2024-05-09T04:53:43.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.155+0000 7fbcae000700 1 -- 172.21.15.6:0/744990747 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc9c00cd70 con 0x7fbcb0105be0 2024-05-09T04:53:43.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.155+0000 7fbcae000700 1 -- 172.21.15.6:0/744990747 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c018680 con 0x7fbcb0105be0 2024-05-09T04:53:43.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.155+0000 7fbcb6c00700 1 -- 172.21.15.6:0/744990747 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcb01a9840 con 0x7fbcb0105be0 2024-05-09T04:53:43.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.159+0000 7fbcae000700 1 -- 172.21.15.6:0/744990747 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fbc9c010430 con 0x7fbcb0105be0 2024-05-09T04:53:43.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.160+0000 7fbcc6cf2b80 1 Processor -- start 2024-05-09T04:53:43.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.160+0000 7fbcc6cf2b80 1 -- start start 2024-05-09T04:53:43.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.161+0000 7fbcc6cf2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d2164450 0x55c4d22e7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.161+0000 7fbcc6cf2b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c4d22e7640 0x55c4d22e1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.161+0000 7fbcc6cf2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d22e1b60 0x55c4d22e1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.161+0000 7fbcc6cf2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55c4d2204260 con 0x55c4d2164450 2024-05-09T04:53:43.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.161+0000 7fbcc6cf2b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c4d21a2090 con 0x55c4d22e7640 2024-05-09T04:53:43.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.161+0000 7fbcc6cf2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55c4d2200220 con 0x55c4d22e1b60 2024-05-09T04:53:43.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.161+0000 7fbcae000700 1 --2- 172.21.15.6:0/744990747 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fbc980b2880 0x7fbc980b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.161+0000 7fbcae000700 1 -- 172.21.15.6:0/744990747 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fbc9c0e2870 con 0x7fbcb0105be0 2024-05-09T04:53:43.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.161+0000 7fbcb5800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c4d22e7640 0x55c4d22e1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.161+0000 7fbcb4e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d22e1b60 0x55c4d22e1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.161+0000 7fbcb5800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c4d22e7640 0x55c4d22e1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:47936/0 (socket says 172.21.15.6:47936) 2024-05-09T04:53:43.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.161+0000 7fbcb5800700 1 -- 172.21.15.6:0/2238340696 learned_addr learned my addr 172.21.15.6:0/2238340696 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:43.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.161+0000 7fbcb6200700 1 --2- 172.21.15.6:0/744990747 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fbc980b2880 0x7fbc980b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.162+0000 7fbcb6200700 1 --2- 172.21.15.6:0/2238340696 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d2164450 0x55c4d22e7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.162+0000 7fbcb5800700 1 -- 172.21.15.6:0/2238340696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d22e1b60 msgr2=0x55c4d22e1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:43.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.162+0000 7fbcb5800700 1 --2- 172.21.15.6:0/2238340696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d22e1b60 0x55c4d22e1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.162+0000 7fbcb5800700 1 -- 172.21.15.6:0/2238340696 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d2164450 msgr2=0x55c4d22e7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:43.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.162+0000 7fbcb5800700 1 --2- 172.21.15.6:0/2238340696 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d2164450 0x55c4d22e7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.162+0000 7fbcb5800700 1 -- 172.21.15.6:0/2238340696 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4d207f440 con 0x55c4d22e7640 2024-05-09T04:53:43.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.162+0000 7fbcb4e00700 1 --2- 172.21.15.6:0/2238340696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d22e1b60 0x55c4d22e1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:53:43.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.163+0000 7fbcb6200700 1 --2- 172.21.15.6:0/744990747 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fbc980b2880 0x7fbc980b4d90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbca8015da0 tx=0x7fbca8015400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:43.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.163+0000 7fbcb5800700 1 --2- 172.21.15.6:0/2238340696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c4d22e7640 0x55c4d22e1620 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbca4009990 tx=0x7fbca40096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a85330fb3a40a9c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:43.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.163+0000 7fbca2a00700 1 -- 172.21.15.6:0/2238340696 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca4027070 con 0x55c4d22e7640 2024-05-09T04:53:43.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.163+0000 7fbca2a00700 1 -- 172.21.15.6:0/2238340696 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbca4005630 con 0x55c4d22e7640 2024-05-09T04:53:43.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.163+0000 7fbca2a00700 1 -- 172.21.15.6:0/2238340696 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca4003be0 con 0x55c4d22e7640 2024-05-09T04:53:43.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.164+0000 7fbcc6cf2b80 1 -- 172.21.15.6:0/2238340696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c4d22e7640 msgr2=0x55c4d22e1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:43.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.164+0000 7fbcc6cf2b80 1 --2- 172.21.15.6:0/2238340696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c4d22e7640 0x55c4d22e1620 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbca4009990 tx=0x7fbca40096a0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.164+0000 7fbcc6cf2b80 1 -- 172.21.15.6:0/2238340696 shutdown_connections 2024-05-09T04:53:43.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.164+0000 7fbcc6cf2b80 1 --2- 172.21.15.6:0/2238340696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d22e1b60 0x55c4d22e1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.164+0000 7fbcc6cf2b80 1 --2- 172.21.15.6:0/2238340696 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d2164450 0x55c4d22e7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.164+0000 7fbcc6cf2b80 1 --2- 172.21.15.6:0/2238340696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c4d22e7640 0x55c4d22e1620 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.164+0000 7fbcc6cf2b80 1 -- 172.21.15.6:0/2238340696 >> 172.21.15.6:0/2238340696 conn(0x55c4d21a8e20 msgr2=0x55c4d22dc050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:43.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.164+0000 7fbcc6cf2b80 1 -- 172.21.15.6:0/2238340696 shutdown_connections 2024-05-09T04:53:43.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.164+0000 7fbcc6cf2b80 1 -- 172.21.15.6:0/2238340696 wait complete. 2024-05-09T04:53:43.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.165+0000 7fbcc6cf2b80 1 -- start start 2024-05-09T04:53:43.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.166+0000 7fbcc6cf2b80 1 Processor -- start 2024-05-09T04:53:43.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.166+0000 7fbcc6cf2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d2164450 0x55c4d22eade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.166+0000 7fbcc6cf2b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c4d22e1b60 0x55c4d22eb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.166+0000 7fbcc6cf2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d22e7640 0x55c4d22eb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.166+0000 7fbcc6cf2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55c4d22ee070 con 0x55c4d22e7640 2024-05-09T04:53:43.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.166+0000 7fbcc6cf2b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c4d2169670 con 0x55c4d22e1b60 2024-05-09T04:53:43.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.166+0000 7fbcc6cf2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55c4d22ee1f0 con 0x55c4d2164450 2024-05-09T04:53:43.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.166+0000 7fbcc6cf2b80 10 client.0 _ll_register_callbacks cb 0x55c4d219b540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:43.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.167+0000 7fbcb5800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d2164450 0x55c4d22eade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.167+0000 7fbcb5800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d2164450 0x55c4d22eade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:47536/0 (socket says 172.21.15.6:47536) 2024-05-09T04:53:43.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.167+0000 7fbcb5800700 1 -- 172.21.15.6:0/3208550316 learned_addr learned my addr 172.21.15.6:0/3208550316 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:43.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.167+0000 7fbcb4e00700 1 --2- 172.21.15.6:0/3208550316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c4d22e1b60 0x55c4d22eb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.167+0000 7fbcb5800700 1 -- 172.21.15.6:0/3208550316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c4d22e1b60 msgr2=0x55c4d22eb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:43.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.167+0000 7fbcb5800700 1 --2- 172.21.15.6:0/3208550316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c4d22e1b60 0x55c4d22eb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.167+0000 7fbcb5800700 1 -- 172.21.15.6:0/3208550316 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d22e7640 msgr2=0x55c4d22eb860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T04:53:43.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.167+0000 7fbcb6200700 1 --2- 172.21.15.6:0/3208550316 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d22e7640 0x55c4d22eb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.168+0000 7fbcb5800700 1 --2- 172.21.15.6:0/3208550316 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d22e7640 0x55c4d22eb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.168+0000 7fbcb5800700 1 -- 172.21.15.6:0/3208550316 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4d22ffd20 con 0x55c4d2164450 2024-05-09T04:53:43.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.168+0000 7fbcb6200700 1 --2- 172.21.15.6:0/3208550316 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d22e7640 0x55c4d22eb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:53:43.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.168+0000 7fbcb5800700 1 --2- 172.21.15.6:0/3208550316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d2164450 0x55c4d22eade0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbca4000c00 tx=0x7fbca4003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca54a2ffcbe33c62 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:43.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.168+0000 7fbc97400700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-09T04:53:43.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.168+0000 7fbc97400700 1 -- 172.21.15.6:0/3208550316 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca4027040 con 0x55c4d2164450 2024-05-09T04:53:43.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.169+0000 7fbc97400700 1 -- 172.21.15.6:0/3208550316 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fbca4003c20 con 0x55c4d2164450 2024-05-09T04:53:43.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.169+0000 7fbcc6cf2b80 1 -- 172.21.15.6:0/3208550316 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbca4005630 con 0x55c4d2164450 2024-05-09T04:53:43.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.169+0000 7fbc97400700 1 -- 172.21.15.6:0/3208550316 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbca400b720 con 0x55c4d2164450 2024-05-09T04:53:43.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.169+0000 7fbcc6cf2b80 10 client.25033 fetch_fsmap learned FSMap version 29 2024-05-09T04:53:43.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.169+0000 7fbcc6cf2b80 1 -- 172.21.15.6:0/3208550316 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c4d22ee070 con 0x55c4d2164450 2024-05-09T04:53:43.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.169+0000 7fbc97400700 1 -- 172.21.15.6:0/3208550316 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbca4005630 con 0x55c4d2164450 2024-05-09T04:53:43.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.169+0000 7fbc97400700 1 -- 172.21.15.6:0/3208550316 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbca4022c10 con 0x55c4d2164450 2024-05-09T04:53:43.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.170+0000 7fbcc6cf2b80 10 client.25033 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T04:53:43.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.170+0000 7fbcc6cf2b80 10 client.25033 resolve_mds: resolved name 'g' to daemon mds.g 2024-05-09T04:53:43.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.170+0000 7fbcc6cf2b80 20 client.25033 populate_metadata read hostname 'smithi006' 2024-05-09T04:53:43.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.170+0000 7fbcc6cf2b80 1 --2- 172.21.15.6:0/3208550316 >> [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] conn(0x55c4d2303700 0x55c4d2304b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.170+0000 7fbcc6cf2b80 4 client.25033 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:53:43.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.170+0000 7fbcc6cf2b80 1 -- 172.21.15.6:0/3208550316 --> [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4d22ad820 con 0x55c4d2303700 2024-05-09T04:53:43.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.170+0000 7fbcb4e00700 1 --2- 172.21.15.6:0/3208550316 >> [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] conn(0x55c4d2303700 0x55c4d2304b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.171+0000 7fbcb4e00700 1 --2- 172.21.15.6:0/3208550316 >> [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] conn(0x55c4d2303700 0x55c4d2304b70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=147696f4998716d1 server_cookie=6e570df87151ecc2 in_seq=0 out_seq=0 2024-05-09T04:53:43.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.171+0000 7fbc97400700 10 client.25033 ms_handle_connect on v2:172.21.15.6:6838/2780306492 2024-05-09T04:53:43.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.172+0000 7fbc97400700 1 -- 172.21.15.6:0/3208550316 <== mds.1 v2:172.21.15.6:6838/2780306492 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c4d22ad820 con 0x55c4d2303700 2024-05-09T04:53:43.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.172+0000 7fbc97400700 10 client.25033 handle_command_reply: tid=0 2024-05-09T04:53:43.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.172+0000 7fbcc6cf2b80 1 client.25033 shutdown 2024-05-09T04:53:43.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.173+0000 7fbcc6cf2b80 1 -- 172.21.15.6:0/3208550316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d2164450 msgr2=0x55c4d22eade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:43.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.173+0000 7fbcc6cf2b80 1 --2- 172.21.15.6:0/3208550316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d2164450 0x55c4d22eade0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbca4000c00 tx=0x7fbca4003680 comp rx=0 tx=0).stop 2024-05-09T04:53:43.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.173+0000 7fbcc6cf2b80 1 -- 172.21.15.6:0/3208550316 shutdown_connections 2024-05-09T04:53:43.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.173+0000 7fbcc6cf2b80 1 --2- 172.21.15.6:0/3208550316 >> [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] conn(0x55c4d2303700 0x55c4d2304b70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.173+0000 7fbcc6cf2b80 1 --2- 172.21.15.6:0/3208550316 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d22e7640 0x55c4d22eb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.173+0000 7fbcc6cf2b80 1 --2- 172.21.15.6:0/3208550316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d2164450 0x55c4d22eade0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.173+0000 7fbcc6cf2b80 1 --2- 172.21.15.6:0/3208550316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c4d22e1b60 0x55c4d22eb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.173+0000 7fbcc6cf2b80 1 -- 172.21.15.6:0/3208550316 >> 172.21.15.6:0/3208550316 conn(0x55c4d21a8e20 msgr2=0x55c4d22dc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:43.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.173+0000 7fbc97400700 0 client.25033 ms_handle_reset on v2:172.21.15.6:6838/2780306492 2024-05-09T04:53:43.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.173+0000 7fbcc6cf2b80 1 -- 172.21.15.6:0/3208550316 shutdown_connections 2024-05-09T04:53:43.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.173+0000 7fbcc6cf2b80 1 -- 172.21.15.6:0/3208550316 wait complete. 2024-05-09T04:53:43.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.174+0000 7fbcc6cf2b80 20 client.25033 trim_cache size 0 max 16384 2024-05-09T04:53:43.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.196+0000 7fbcc6cf2b80 1 Processor -- start 2024-05-09T04:53:43.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.196+0000 7fbcc6cf2b80 1 -- start start 2024-05-09T04:53:43.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.197+0000 7fbcc6cf2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d2164450 0x55c4d22ea120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.197+0000 7fbcc6cf2b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c4d22e1b60 0x55c4d23027e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.197+0000 7fbcc6cf2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d22e7640 0x55c4d22eb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.197+0000 7fbcc6cf2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55c4d2302eb0 con 0x55c4d2164450 2024-05-09T04:53:43.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.197+0000 7fbcc6cf2b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c4d2303580 con 0x55c4d22e1b60 2024-05-09T04:53:43.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.197+0000 7fbcc6cf2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55c4d2303c50 con 0x55c4d22e7640 2024-05-09T04:53:43.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.197+0000 7fbcb4e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c4d22e1b60 0x55c4d23027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.197+0000 7fbcb5800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d2164450 0x55c4d22ea120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.197+0000 7fbcb4e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c4d22e1b60 0x55c4d23027e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:47954/0 (socket says 172.21.15.6:47954) 2024-05-09T04:53:43.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.197+0000 7fbcb4e00700 1 -- 172.21.15.6:0/2122781426 learned_addr learned my addr 172.21.15.6:0/2122781426 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:43.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.197+0000 7fbcb6200700 1 --2- 172.21.15.6:0/2122781426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d22e7640 0x55c4d22eb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.198+0000 7fbcb5800700 1 -- 172.21.15.6:0/2122781426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d22e7640 msgr2=0x55c4d22eb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:43.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.198+0000 7fbcb5800700 1 --2- 172.21.15.6:0/2122781426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d22e7640 0x55c4d22eb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.198+0000 7fbcb5800700 1 -- 172.21.15.6:0/2122781426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c4d22e1b60 msgr2=0x55c4d23027e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:43.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.198+0000 7fbcb5800700 1 --2- 172.21.15.6:0/2122781426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c4d22e1b60 0x55c4d23027e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.198+0000 7fbcb5800700 1 -- 172.21.15.6:0/2122781426 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4d22edd40 con 0x55c4d2164450 2024-05-09T04:53:43.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.198+0000 7fbcb4e00700 1 --2- 172.21.15.6:0/2122781426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c4d22e1b60 0x55c4d23027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:53:43.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.198+0000 7fbcb5800700 1 --2- 172.21.15.6:0/2122781426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d2164450 0x55c4d22ea120 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbca40098e0 tx=0x7fbca400f8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615989f6757c6f35 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:43.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.198+0000 7fbca2000700 1 -- 172.21.15.6:0/2122781426 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca4003c70 con 0x55c4d2164450 2024-05-09T04:53:43.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.198+0000 7fbca2000700 1 -- 172.21.15.6:0/2122781426 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbca4031070 con 0x55c4d2164450 2024-05-09T04:53:43.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.198+0000 7fbca2000700 1 -- 172.21.15.6:0/2122781426 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca400be00 con 0x55c4d2164450 2024-05-09T04:53:43.201 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.199+0000 7fbcc6cf2b80 1 -- 172.21.15.6:0/2122781426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d2164450 msgr2=0x55c4d22ea120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:43.201 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.199+0000 7fbcc6cf2b80 1 --2- 172.21.15.6:0/2122781426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d2164450 0x55c4d22ea120 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbca40098e0 tx=0x7fbca400f8f0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.201 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.200+0000 7fbcc6cf2b80 1 -- 172.21.15.6:0/2122781426 shutdown_connections 2024-05-09T04:53:43.201 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.200+0000 7fbcc6cf2b80 1 --2- 172.21.15.6:0/2122781426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d22e7640 0x55c4d22eb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.200+0000 7fbcc6cf2b80 1 --2- 172.21.15.6:0/2122781426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d2164450 0x55c4d22ea120 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.200+0000 7fbcc6cf2b80 1 --2- 172.21.15.6:0/2122781426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c4d22e1b60 0x55c4d23027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.200+0000 7fbcc6cf2b80 1 -- 172.21.15.6:0/2122781426 >> 172.21.15.6:0/2122781426 conn(0x55c4d21a8e20 msgr2=0x55c4d22ddbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:43.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.200+0000 7fbcc6cf2b80 1 -- 172.21.15.6:0/2122781426 shutdown_connections 2024-05-09T04:53:43.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.200+0000 7fbcc6cf2b80 1 -- 172.21.15.6:0/2122781426 wait complete. 2024-05-09T04:53:43.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.200+0000 7fbcc6cf2b80 1 -- start start 2024-05-09T04:53:43.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.200+0000 7fbcc6cf2b80 1 Processor -- start 2024-05-09T04:53:43.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.201+0000 7fbcc6cf2b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c4d2164450 0x55c4d22dacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.201+0000 7fbcc6cf2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d22e1b60 0x55c4d22db200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.201+0000 7fbcc6cf2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d22e7640 0x55c4d22db740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.201+0000 7fbcc6cf2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55c4d23049f0 con 0x55c4d22e7640 2024-05-09T04:53:43.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.201+0000 7fbcc6cf2b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c4d2304320 con 0x55c4d2164450 2024-05-09T04:53:43.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.201+0000 7fbcc6cf2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55c4d23050c0 con 0x55c4d22e1b60 2024-05-09T04:53:43.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.201+0000 7fbcc6cf2b80 10 client.0 _ll_register_callbacks cb 0x55c4d219b540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:43.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.201+0000 7fbcb4e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d22e1b60 0x55c4d22db200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.201+0000 7fbcb4e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d22e1b60 0x55c4d22db200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:47552/0 (socket says 172.21.15.6:47552) 2024-05-09T04:53:43.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.201+0000 7fbcb4e00700 1 -- 172.21.15.6:0/162234058 learned_addr learned my addr 172.21.15.6:0/162234058 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:43.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.201+0000 7fbcb5800700 1 --2- 172.21.15.6:0/162234058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c4d2164450 0x55c4d22dacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.202+0000 7fbcb6200700 1 --2- 172.21.15.6:0/162234058 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d22e7640 0x55c4d22db740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.202+0000 7fbcb4e00700 1 -- 172.21.15.6:0/162234058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c4d2164450 msgr2=0x55c4d22dacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:43.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.202+0000 7fbcb4e00700 1 --2- 172.21.15.6:0/162234058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c4d2164450 0x55c4d22dacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.202+0000 7fbcb4e00700 1 -- 172.21.15.6:0/162234058 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d22e7640 msgr2=0x55c4d22db740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:43.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.202+0000 7fbcb4e00700 1 --2- 172.21.15.6:0/162234058 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d22e7640 0x55c4d22db740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.202+0000 7fbcb4e00700 1 -- 172.21.15.6:0/162234058 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4d230d150 con 0x55c4d22e1b60 2024-05-09T04:53:43.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.202+0000 7fbcb6200700 1 --2- 172.21.15.6:0/162234058 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d22e7640 0x55c4d22db740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:53:43.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.202+0000 7fbcb4e00700 1 --2- 172.21.15.6:0/162234058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d22e1b60 0x55c4d22db200 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c0a6d20 tx=0x7fbc9c009790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87360b383d5f3c6c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:43.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.202+0000 7fbc97400700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-09T04:53:43.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.202+0000 7fbc97400700 1 -- 172.21.15.6:0/162234058 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c0a9ab0 con 0x55c4d22e1b60 2024-05-09T04:53:43.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.202+0000 7fbc97400700 1 -- 172.21.15.6:0/162234058 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fbc9c0a60a0 con 0x55c4d22e1b60 2024-05-09T04:53:43.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.202+0000 7fbcc6cf2b80 1 -- 172.21.15.6:0/162234058 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbca4031070 con 0x55c4d22e1b60 2024-05-09T04:53:43.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.202+0000 7fbc97400700 1 -- 172.21.15.6:0/162234058 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbc9c0ade60 con 0x55c4d22e1b60 2024-05-09T04:53:43.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.202+0000 7fbcc6cf2b80 10 client.25039 fetch_fsmap learned FSMap version 29 2024-05-09T04:53:43.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.202+0000 7fbcc6cf2b80 1 -- 172.21.15.6:0/162234058 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c4d230e3f0 con 0x55c4d22e1b60 2024-05-09T04:53:43.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.203+0000 7fbc97400700 1 -- 172.21.15.6:0/162234058 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbca4031070 con 0x55c4d22e1b60 2024-05-09T04:53:43.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.203+0000 7fbc97400700 1 -- 172.21.15.6:0/162234058 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbc9c0b2b60 con 0x55c4d22e1b60 2024-05-09T04:53:43.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.203+0000 7fbcc6cf2b80 10 client.25039 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T04:53:43.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.203+0000 7fbcc6cf2b80 10 client.25039 resolve_mds: resolved name 'g' to daemon mds.g 2024-05-09T04:53:43.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.203+0000 7fbcc6cf2b80 20 client.25039 populate_metadata read hostname 'smithi006' 2024-05-09T04:53:43.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.203+0000 7fbcc6cf2b80 1 --2- 172.21.15.6:0/162234058 >> [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] conn(0x55c4d230e6e0 0x55c4d2371c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.203+0000 7fbcc6cf2b80 4 client.25039 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T04:53:43.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.203+0000 7fbcc6cf2b80 1 -- 172.21.15.6:0/162234058 --> [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] -- command(tid 0: {"prefix": "status"}) -- 0x55c4d2211c40 con 0x55c4d230e6e0 2024-05-09T04:53:43.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.203+0000 7fbcb5800700 1 --2- 172.21.15.6:0/162234058 >> [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] conn(0x55c4d230e6e0 0x55c4d2371c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.204+0000 7fbcb5800700 1 --2- 172.21.15.6:0/162234058 >> [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] conn(0x55c4d230e6e0 0x55c4d2371c50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4c9533e6fcb18aed server_cookie=36fc1db46814af60 in_seq=0 out_seq=0 2024-05-09T04:53:43.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.204+0000 7fbc97400700 10 client.25039 ms_handle_connect on v2:172.21.15.6:6838/2780306492 2024-05-09T04:53:43.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.204+0000 7fbc97400700 1 -- 172.21.15.6:0/162234058 <== mds.1 v2:172.21.15.6:6838/2780306492 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c4d2211c40 con 0x55c4d230e6e0 2024-05-09T04:53:43.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.204+0000 7fbc97400700 10 client.25039 handle_command_reply: tid=0 2024-05-09T04:53:43.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.204+0000 7fbcc6cf2b80 1 client.25039 shutdown 2024-05-09T04:53:43.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.205+0000 7fbcc6cf2b80 1 -- 172.21.15.6:0/162234058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d22e1b60 msgr2=0x55c4d22db200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:43.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.205+0000 7fbcc6cf2b80 1 --2- 172.21.15.6:0/162234058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d22e1b60 0x55c4d22db200 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c0a6d20 tx=0x7fbc9c009790 comp rx=0 tx=0).stop 2024-05-09T04:53:43.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.205+0000 7fbcc6cf2b80 1 -- 172.21.15.6:0/162234058 shutdown_connections 2024-05-09T04:53:43.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.205+0000 7fbcc6cf2b80 1 --2- 172.21.15.6:0/162234058 >> [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] conn(0x55c4d230e6e0 0x55c4d2371c50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.205+0000 7fbcc6cf2b80 1 --2- 172.21.15.6:0/162234058 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d22e7640 0x55c4d22db740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.205+0000 7fbcc6cf2b80 1 --2- 172.21.15.6:0/162234058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c4d2164450 0x55c4d22dacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.205+0000 7fbcc6cf2b80 1 --2- 172.21.15.6:0/162234058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d22e1b60 0x55c4d22db200 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.205+0000 7fbc97400700 0 client.25039 ms_handle_reset on v2:172.21.15.6:6838/2780306492 2024-05-09T04:53:43.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.205+0000 7fbcc6cf2b80 1 -- 172.21.15.6:0/162234058 >> 172.21.15.6:0/162234058 conn(0x55c4d21a8e20 msgr2=0x55c4d22e4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:43.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.205+0000 7fbcc6cf2b80 1 -- 172.21.15.6:0/162234058 shutdown_connections 2024-05-09T04:53:43.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.205+0000 7fbcc6cf2b80 1 -- 172.21.15.6:0/162234058 wait complete. 2024-05-09T04:53:43.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.205+0000 7fbcc6cf2b80 20 client.25039 trim_cache size 0 max 16384 2024-05-09T04:53:43.209 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-09T04:53:43.209 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "f74d813c-0dbd-11ef-bc97-c7b262605968", 2024-05-09T04:53:43.209 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 1, 2024-05-09T04:53:43.209 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24530, 2024-05-09T04:53:43.209 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-05-09T04:53:43.210 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-05-09T04:53:43.210 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-05-09T04:53:43.210 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 163.478400314, 2024-05-09T04:53:43.210 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 29, 2024-05-09T04:53:43.210 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 89, 2024-05-09T04:53:43.210 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 89, 2024-05-09T04:53:43.210 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 276.51977178800001 2024-05-09T04:53:43.210 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-09T04:53:43.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.206+0000 7fbcb6c00700 1 -- 172.21.15.6:0/744990747 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fbc980b2880 msgr2=0x7fbc980b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:43.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.206+0000 7fbcb6c00700 1 --2- 172.21.15.6:0/744990747 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fbc980b2880 0x7fbc980b4d90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbca8015da0 tx=0x7fbca8015400 comp rx=0 tx=0).stop 2024-05-09T04:53:43.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.206+0000 7fbcb6c00700 1 -- 172.21.15.6:0/744990747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbcb0105be0 msgr2=0x7fbcb01034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:43.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.206+0000 7fbcb6c00700 1 --2- 172.21.15.6:0/744990747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbcb0105be0 0x7fbcb01034c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00e9a0 tx=0x7fbc9c00ecb0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.206+0000 7fbcb6c00700 1 -- 172.21.15.6:0/744990747 shutdown_connections 2024-05-09T04:53:43.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.206+0000 7fbcb6c00700 1 --2- 172.21.15.6:0/744990747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcb010e700 0x7fbcb0103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.206+0000 7fbcb6c00700 1 --2- 172.21.15.6:0/744990747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbcb0105290 0x7fbcb0102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.206+0000 7fbcb6c00700 1 --2- 172.21.15.6:0/744990747 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fbc980b2880 0x7fbc980b4d90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.206+0000 7fbcb6c00700 1 --2- 172.21.15.6:0/744990747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbcb0105be0 0x7fbcb01034c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.206+0000 7fbcb6c00700 1 -- 172.21.15.6:0/744990747 >> 172.21.15.6:0/744990747 conn(0x7fbcb0078ac0 msgr2=0x7fbcb00fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:43.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.206+0000 7fbcb6c00700 1 -- 172.21.15.6:0/744990747 shutdown_connections 2024-05-09T04:53:43.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:43.206+0000 7fbcb6c00700 1 -- 172.21.15.6:0/744990747 wait complete. 2024-05-09T04:53:43.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:43 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:53:43.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:43 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:53:43.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:43 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:53:44.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:43 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:53:44.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:43 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:53:44.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:43 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:53:44.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:43 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:53:44.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:43 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:53:44.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:43 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:53:44.165 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-09T04:53:44.650 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:53:45.024 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:44 smithi006 ceph-mon[29835]: pgmap v463: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:53:45.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:44 smithi186 ceph-mon[34668]: pgmap v463: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:53:45.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:44 smithi169 ceph-mon[34075]: pgmap v463: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:53:46.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.964+0000 7f4d8a200700 1 -- 172.21.15.6:0/4147084637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d84105db0 msgr2=0x7f4d8410e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:46.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.964+0000 7f4d8a200700 1 --2- 172.21.15.6:0/4147084637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d84105db0 0x7f4d8410e300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4d700098f0 tx=0x7f4d70009c00 comp rx=0 tx=0).stop 2024-05-09T04:53:46.966 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:46 smithi006 ceph-mon[29835]: pgmap v464: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:53:46.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.964+0000 7f4d8a200700 1 -- 172.21.15.6:0/4147084637 shutdown_connections 2024-05-09T04:53:46.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.964+0000 7f4d8a200700 1 --2- 172.21.15.6:0/4147084637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d8410e8d0 0x7f4d84110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:46.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.964+0000 7f4d8a200700 1 --2- 172.21.15.6:0/4147084637 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d84105460 0x7f4d84105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:46.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.964+0000 7f4d8a200700 1 --2- 172.21.15.6:0/4147084637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d84105db0 0x7f4d8410e300 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:46.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.964+0000 7f4d8a200700 1 -- 172.21.15.6:0/4147084637 >> 172.21.15.6:0/4147084637 conn(0x7f4d840fedf0 msgr2=0x7f4d84101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:46.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.964+0000 7f4d8a200700 1 -- 172.21.15.6:0/4147084637 shutdown_connections 2024-05-09T04:53:46.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.964+0000 7f4d8a200700 1 -- 172.21.15.6:0/4147084637 wait complete. 2024-05-09T04:53:46.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.965+0000 7f4d8a200700 1 Processor -- start 2024-05-09T04:53:46.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.966+0000 7f4d8a200700 1 -- start start 2024-05-09T04:53:46.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.966+0000 7f4d8a200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d84105460 0x7f4d8419d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:46.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.966+0000 7f4d8a200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d84105db0 0x7f4d8419d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:46.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.966+0000 7f4d8a200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d8410e8d0 0x7f4d841a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:46.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.966+0000 7f4d8a200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4d841144d0 con 0x7f4d84105460 2024-05-09T04:53:46.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.966+0000 7f4d8a200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4d84114350 con 0x7f4d84105db0 2024-05-09T04:53:46.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.966+0000 7f4d8a200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4d84114650 con 0x7f4d8410e8d0 2024-05-09T04:53:46.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.967+0000 7f4d83e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d8410e8d0 0x7f4d841a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:46.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.967+0000 7f4d83400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d84105460 0x7f4d8419d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:46.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.967+0000 7f4d83e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d8410e8d0 0x7f4d841a18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:50580/0 (socket says 172.21.15.6:50580) 2024-05-09T04:53:46.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.967+0000 7f4d82a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d84105db0 0x7f4d8419d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:46.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.967+0000 7f4d83e00700 1 -- 172.21.15.6:0/1668220028 learned_addr learned my addr 172.21.15.6:0/1668220028 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:46.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.967+0000 7f4d83e00700 1 -- 172.21.15.6:0/1668220028 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d84105db0 msgr2=0x7f4d8419d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:46.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.967+0000 7f4d83e00700 1 --2- 172.21.15.6:0/1668220028 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d84105db0 0x7f4d8419d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:46.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.967+0000 7f4d83e00700 1 -- 172.21.15.6:0/1668220028 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d84105460 msgr2=0x7f4d8419d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:46.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.967+0000 7f4d83e00700 1 --2- 172.21.15.6:0/1668220028 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d84105460 0x7f4d8419d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:46.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.967+0000 7f4d83e00700 1 -- 172.21.15.6:0/1668220028 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d841a1f70 con 0x7f4d8410e8d0 2024-05-09T04:53:46.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.967+0000 7f4d83e00700 1 --2- 172.21.15.6:0/1668220028 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d8410e8d0 0x7f4d841a18d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c00ef30 tx=0x7f4d7c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:46.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.968+0000 7f4d7be00700 1 -- 172.21.15.6:0/1668220028 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c0092e0 con 0x7f4d8410e8d0 2024-05-09T04:53:46.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.968+0000 7f4d7be00700 1 -- 172.21.15.6:0/1668220028 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d7c004bb0 con 0x7f4d8410e8d0 2024-05-09T04:53:46.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.968+0000 7f4d8a200700 1 -- 172.21.15.6:0/1668220028 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d841a2200 con 0x7f4d8410e8d0 2024-05-09T04:53:46.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.968+0000 7f4d7be00700 1 -- 172.21.15.6:0/1668220028 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c004020 con 0x7f4d8410e8d0 2024-05-09T04:53:46.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.968+0000 7f4d8a200700 1 -- 172.21.15.6:0/1668220028 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d841a2710 con 0x7f4d8410e8d0 2024-05-09T04:53:46.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.972+0000 7f4d7be00700 1 -- 172.21.15.6:0/1668220028 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f4d7c01b030 con 0x7f4d8410e8d0 2024-05-09T04:53:46.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.973+0000 7f4d98deab80 1 Processor -- start 2024-05-09T04:53:46.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.973+0000 7f4d98deab80 1 -- start start 2024-05-09T04:53:46.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.973+0000 7f4d7be00700 1 --2- 172.21.15.6:0/1668220028 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4d680b2880 0x7f4d680b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:46.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.974+0000 7f4d98deab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f21315450 0x555f21497ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:46.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.974+0000 7f4d98deab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f21498410 0x555f214924c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:46.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.974+0000 7f4d98deab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555f21492a00 0x555f21492e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:46.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.974+0000 7f4d98deab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x555f213b5260 con 0x555f21498410 2024-05-09T04:53:46.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.974+0000 7f4d98deab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x555f21353090 con 0x555f21492a00 2024-05-09T04:53:46.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.974+0000 7f4d98deab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x555f213b1220 con 0x555f21315450 2024-05-09T04:53:46.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.974+0000 7f4d7be00700 1 -- 172.21.15.6:0/1668220028 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f4d7c0e1fe0 con 0x7f4d8410e8d0 2024-05-09T04:53:46.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.974+0000 7f4d82a00700 1 --2- 172.21.15.6:0/1668220028 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4d680b2880 0x7f4d680b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:46.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.974+0000 7f4d83400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f21315450 0x555f21497ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:46.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.974+0000 7f4d82a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f21498410 0x555f214924c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:46.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.974+0000 7f4d83e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555f21492a00 0x555f21492e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:46.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.974+0000 7f4d83e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555f21492a00 0x555f21492e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:48536/0 (socket says 172.21.15.6:48536) 2024-05-09T04:53:46.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.974+0000 7f4d83e00700 1 -- 172.21.15.6:0/457747294 learned_addr learned my addr 172.21.15.6:0/457747294 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:46.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.975+0000 7f4d83e00700 1 -- 172.21.15.6:0/457747294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f21315450 msgr2=0x555f21497ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:46.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.975+0000 7f4d83e00700 1 --2- 172.21.15.6:0/457747294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f21315450 0x555f21497ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:46.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.975+0000 7f4d83e00700 1 -- 172.21.15.6:0/457747294 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f21498410 msgr2=0x555f214924c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:46.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.975+0000 7f4d83e00700 1 --2- 172.21.15.6:0/457747294 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f21498410 0x555f214924c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:46.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.975+0000 7f4d83e00700 1 -- 172.21.15.6:0/457747294 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f21230440 con 0x555f21492a00 2024-05-09T04:53:46.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.975+0000 7f4d82a00700 1 --2- 172.21.15.6:0/1668220028 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4d680b2880 0x7f4d680b4d90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4d70009fd0 tx=0x7f4d700095a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:46.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.975+0000 7f4d83e00700 1 --2- 172.21.15.6:0/457747294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555f21492a00 0x555f21492e10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0a5330 tx=0x7f4d7c0a8d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d99c10f043cb66b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:46.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.975+0000 7f4d78c00700 1 -- 172.21.15.6:0/457747294 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c0ad660 con 0x555f21492a00 2024-05-09T04:53:46.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.976+0000 7f4d78c00700 1 -- 172.21.15.6:0/457747294 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d7c0b2080 con 0x555f21492a00 2024-05-09T04:53:46.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.976+0000 7f4d78c00700 1 -- 172.21.15.6:0/457747294 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c0dfbc0 con 0x555f21492a00 2024-05-09T04:53:46.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.977+0000 7f4d98deab80 1 -- 172.21.15.6:0/457747294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555f21492a00 msgr2=0x555f21492e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:46.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.977+0000 7f4d98deab80 1 --2- 172.21.15.6:0/457747294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555f21492a00 0x555f21492e10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0a5330 tx=0x7f4d7c0a8d30 comp rx=0 tx=0).stop 2024-05-09T04:53:46.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.977+0000 7f4d98deab80 1 -- 172.21.15.6:0/457747294 shutdown_connections 2024-05-09T04:53:46.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.977+0000 7f4d98deab80 1 --2- 172.21.15.6:0/457747294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555f21492a00 0x555f21492e10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:46.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.977+0000 7f4d98deab80 1 --2- 172.21.15.6:0/457747294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f21315450 0x555f21497ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:46.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.977+0000 7f4d98deab80 1 --2- 172.21.15.6:0/457747294 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f21498410 0x555f214924c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:46.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.977+0000 7f4d98deab80 1 -- 172.21.15.6:0/457747294 >> 172.21.15.6:0/457747294 conn(0x555f21359e20 msgr2=0x555f2148ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:46.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.977+0000 7f4d98deab80 1 -- 172.21.15.6:0/457747294 shutdown_connections 2024-05-09T04:53:46.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.977+0000 7f4d98deab80 1 -- 172.21.15.6:0/457747294 wait complete. 2024-05-09T04:53:46.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.977+0000 7f4d98deab80 1 -- start start 2024-05-09T04:53:46.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.978+0000 7f4d98deab80 1 Processor -- start 2024-05-09T04:53:46.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.978+0000 7f4d98deab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f21315450 0x555f2149bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:46.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.978+0000 7f4d98deab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555f21492a00 0x555f2149c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:46.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.978+0000 7f4d98deab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f21498410 0x555f2149c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:46.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.978+0000 7f4d98deab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x555f2149ef00 con 0x555f21498410 2024-05-09T04:53:46.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.978+0000 7f4d98deab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x555f2131a670 con 0x555f21492a00 2024-05-09T04:53:46.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.978+0000 7f4d98deab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x555f2149f080 con 0x555f21315450 2024-05-09T04:53:46.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.979+0000 7f4d98deab80 10 client.0 _ll_register_callbacks cb 0x555f2134c540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:46.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.979+0000 7f4d83e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f21498410 0x555f2149c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:46.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.979+0000 7f4d83400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f21315450 0x555f2149bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:46.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.979+0000 7f4d82a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555f21492a00 0x555f2149c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:46.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.979+0000 7f4d83400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f21315450 0x555f2149bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:50600/0 (socket says 172.21.15.6:50600) 2024-05-09T04:53:46.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.979+0000 7f4d83400700 1 -- 172.21.15.6:0/3745008368 learned_addr learned my addr 172.21.15.6:0/3745008368 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:46.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.979+0000 7f4d83400700 1 -- 172.21.15.6:0/3745008368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555f21492a00 msgr2=0x555f2149c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:46.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.979+0000 7f4d83400700 1 --2- 172.21.15.6:0/3745008368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555f21492a00 0x555f2149c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:46.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.979+0000 7f4d83400700 1 -- 172.21.15.6:0/3745008368 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f21498410 msgr2=0x555f2149c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:46.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.979+0000 7f4d83400700 1 --2- 172.21.15.6:0/3745008368 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f21498410 0x555f2149c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:46.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.979+0000 7f4d83400700 1 -- 172.21.15.6:0/3745008368 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f214a83b0 con 0x555f21315450 2024-05-09T04:53:46.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.980+0000 7f4d83400700 1 --2- 172.21.15.6:0/3745008368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f21315450 0x555f2149bb40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4d740127f0 tx=0x7f4d74012b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2fd9b7c63e320d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:46.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.980+0000 7f4d6d600700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-09T04:53:46.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.980+0000 7f4d6d600700 1 -- 172.21.15.6:0/3745008368 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d74004050 con 0x555f21315450 2024-05-09T04:53:46.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.980+0000 7f4d98deab80 1 -- 172.21.15.6:0/3745008368 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d7c0b2080 con 0x555f21315450 2024-05-09T04:53:46.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.980+0000 7f4d6d600700 1 -- 172.21.15.6:0/3745008368 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f4d740041f0 con 0x555f21315450 2024-05-09T04:53:46.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.981+0000 7f4d6d600700 1 -- 172.21.15.6:0/3745008368 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4d74025050 con 0x555f21315450 2024-05-09T04:53:46.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.981+0000 7f4d98deab80 10 client.25051 fetch_fsmap learned FSMap version 29 2024-05-09T04:53:46.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.981+0000 7f4d98deab80 1 -- 172.21.15.6:0/3745008368 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555f214a9d20 con 0x555f21315450 2024-05-09T04:53:46.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.981+0000 7f4d6d600700 1 -- 172.21.15.6:0/3745008368 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d7c0b2080 con 0x555f21315450 2024-05-09T04:53:46.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.981+0000 7f4d6d600700 1 -- 172.21.15.6:0/3745008368 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4d7400f570 con 0x555f21315450 2024-05-09T04:53:46.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.981+0000 7f4d98deab80 10 client.25051 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T04:53:46.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.982+0000 7f4d98deab80 10 client.25051 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T04:53:46.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.982+0000 7f4d98deab80 20 client.25051 populate_metadata read hostname 'smithi006' 2024-05-09T04:53:46.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.982+0000 7f4d98deab80 1 --2- 172.21.15.6:0/3745008368 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x555f214ac3b0 0x555f214ad820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:46.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.982+0000 7f4d98deab80 4 client.25051 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:53:46.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.982+0000 7f4d98deab80 1 -- 172.21.15.6:0/3745008368 --> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f2145e820 con 0x555f214ac3b0 2024-05-09T04:53:46.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.982+0000 7f4d82a00700 1 --2- 172.21.15.6:0/3745008368 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x555f214ac3b0 0x555f214ad820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:46.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.983+0000 7f4d6d600700 10 client.25051 ms_handle_connect on v2:172.21.15.169:6838/614388771 2024-05-09T04:53:46.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.983+0000 7f4d82a00700 1 --2- 172.21.15.6:0/3745008368 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x555f214ac3b0 0x555f214ad820 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60f5f786e4c50c14 server_cookie=9b1d3e6ccd683d29 in_seq=0 out_seq=0 2024-05-09T04:53:46.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.984+0000 7f4d6d600700 1 -- 172.21.15.6:0/3745008368 <== mds.0 v2:172.21.15.169:6838/614388771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555f2145e820 con 0x555f214ac3b0 2024-05-09T04:53:46.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.984+0000 7f4d6d600700 10 client.25051 handle_command_reply: tid=0 2024-05-09T04:53:46.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.984+0000 7f4d98deab80 1 client.25051 shutdown 2024-05-09T04:53:46.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.984+0000 7f4d98deab80 1 -- 172.21.15.6:0/3745008368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f21315450 msgr2=0x555f2149bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:46.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.984+0000 7f4d98deab80 1 --2- 172.21.15.6:0/3745008368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f21315450 0x555f2149bb40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4d740127f0 tx=0x7f4d74012b00 comp rx=0 tx=0).stop 2024-05-09T04:53:46.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.984+0000 7f4d98deab80 1 -- 172.21.15.6:0/3745008368 shutdown_connections 2024-05-09T04:53:46.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.984+0000 7f4d98deab80 1 --2- 172.21.15.6:0/3745008368 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f21498410 0x555f2149c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:46.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.984+0000 7f4d98deab80 1 --2- 172.21.15.6:0/3745008368 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x555f214ac3b0 0x555f214ad820 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:46.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.984+0000 7f4d98deab80 1 --2- 172.21.15.6:0/3745008368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f21315450 0x555f2149bb40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:46.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.984+0000 7f4d98deab80 1 --2- 172.21.15.6:0/3745008368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555f21492a00 0x555f2149c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:46.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.984+0000 7f4d6d600700 0 client.25051 ms_handle_reset on v2:172.21.15.169:6838/614388771 2024-05-09T04:53:46.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.984+0000 7f4d98deab80 1 -- 172.21.15.6:0/3745008368 >> 172.21.15.6:0/3745008368 conn(0x555f21359e20 msgr2=0x555f2148ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:46.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.984+0000 7f4d98deab80 1 -- 172.21.15.6:0/3745008368 shutdown_connections 2024-05-09T04:53:46.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.984+0000 7f4d98deab80 1 -- 172.21.15.6:0/3745008368 wait complete. 2024-05-09T04:53:46.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:46.984+0000 7f4d98deab80 20 client.25051 trim_cache size 0 max 16384 2024-05-09T04:53:47.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.008+0000 7f4d98deab80 1 Processor -- start 2024-05-09T04:53:47.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.008+0000 7f4d98deab80 1 -- start start 2024-05-09T04:53:47.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.008+0000 7f4d98deab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f21315450 0x555f214924c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:47.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.008+0000 7f4d98deab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555f21492a00 0x555f2149aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:47.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.008+0000 7f4d98deab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f21498410 0x555f2149c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:47.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.008+0000 7f4d98deab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x555f214abb60 con 0x555f21315450 2024-05-09T04:53:47.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.008+0000 7f4d98deab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x555f214ac230 con 0x555f21492a00 2024-05-09T04:53:47.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.008+0000 7f4d98deab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x555f214ac900 con 0x555f21498410 2024-05-09T04:53:47.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.009+0000 7f4d83e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f21498410 0x555f2149c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:47.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.009+0000 7f4d82a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555f21492a00 0x555f2149aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:47.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.009+0000 7f4d83e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f21498410 0x555f2149c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:50608/0 (socket says 172.21.15.6:50608) 2024-05-09T04:53:47.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.009+0000 7f4d83e00700 1 -- 172.21.15.6:0/411577557 learned_addr learned my addr 172.21.15.6:0/411577557 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:47.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.009+0000 7f4d83400700 1 --2- 172.21.15.6:0/411577557 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f21315450 0x555f214924c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:47.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.009+0000 7f4d82a00700 1 -- 172.21.15.6:0/411577557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f21498410 msgr2=0x555f2149c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:47.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.009+0000 7f4d82a00700 1 --2- 172.21.15.6:0/411577557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f21498410 0x555f2149c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:47.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.009+0000 7f4d82a00700 1 -- 172.21.15.6:0/411577557 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f21315450 msgr2=0x555f214924c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:47.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.009+0000 7f4d82a00700 1 --2- 172.21.15.6:0/411577557 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f21315450 0x555f214924c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:47.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.009+0000 7f4d82a00700 1 -- 172.21.15.6:0/411577557 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f2149f030 con 0x555f21492a00 2024-05-09T04:53:47.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.009+0000 7f4d83e00700 1 --2- 172.21.15.6:0/411577557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f21498410 0x555f2149c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:53:47.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.010+0000 7f4d82a00700 1 --2- 172.21.15.6:0/411577557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555f21492a00 0x555f2149aaa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4d70000c00 tx=0x7f4d7001c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13650c6f5092fc47 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:47.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.010+0000 7f4d6fe00700 1 -- 172.21.15.6:0/411577557 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d700053a0 con 0x555f21492a00 2024-05-09T04:53:47.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.010+0000 7f4d6fe00700 1 -- 172.21.15.6:0/411577557 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d70038070 con 0x555f21492a00 2024-05-09T04:53:47.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.010+0000 7f4d6fe00700 1 -- 172.21.15.6:0/411577557 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d700038e0 con 0x555f21492a00 2024-05-09T04:53:47.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.011+0000 7f4d98deab80 1 -- 172.21.15.6:0/411577557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555f21492a00 msgr2=0x555f2149aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:47.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.011+0000 7f4d98deab80 1 --2- 172.21.15.6:0/411577557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555f21492a00 0x555f2149aaa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4d70000c00 tx=0x7f4d7001c000 comp rx=0 tx=0).stop 2024-05-09T04:53:47.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.011+0000 7f4d98deab80 1 -- 172.21.15.6:0/411577557 shutdown_connections 2024-05-09T04:53:47.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.011+0000 7f4d98deab80 1 --2- 172.21.15.6:0/411577557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f21498410 0x555f2149c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:47.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.011+0000 7f4d98deab80 1 --2- 172.21.15.6:0/411577557 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f21315450 0x555f214924c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:47.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.011+0000 7f4d98deab80 1 --2- 172.21.15.6:0/411577557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555f21492a00 0x555f2149aaa0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:47.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.011+0000 7f4d98deab80 1 -- 172.21.15.6:0/411577557 >> 172.21.15.6:0/411577557 conn(0x555f21359e20 msgr2=0x555f21495900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:47.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.011+0000 7f4d98deab80 1 -- 172.21.15.6:0/411577557 shutdown_connections 2024-05-09T04:53:47.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.011+0000 7f4d98deab80 1 -- 172.21.15.6:0/411577557 wait complete. 2024-05-09T04:53:47.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.012+0000 7f4d98deab80 1 -- start start 2024-05-09T04:53:47.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.012+0000 7f4d98deab80 1 Processor -- start 2024-05-09T04:53:47.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.012+0000 7f4d98deab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f21315450 0x555f214b6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:47.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.012+0000 7f4d98deab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555f21492a00 0x555f214b71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:47.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.013+0000 7f4d98deab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f21498410 0x555f214b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:47.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.013+0000 7f4d98deab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x555f214ad6a0 con 0x555f21498410 2024-05-09T04:53:47.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.013+0000 7f4d98deab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x555f214acfd0 con 0x555f21492a00 2024-05-09T04:53:47.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.013+0000 7f4d98deab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x555f214add70 con 0x555f21315450 2024-05-09T04:53:47.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.013+0000 7f4d98deab80 10 client.0 _ll_register_callbacks cb 0x555f2134c540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:47.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.013+0000 7f4d82a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555f21492a00 0x555f214b71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:47.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.013+0000 7f4d83400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f21315450 0x555f214b6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:47.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.013+0000 7f4d83e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f21498410 0x555f214b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:47.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.013+0000 7f4d82a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555f21492a00 0x555f214b71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:48558/0 (socket says 172.21.15.6:48558) 2024-05-09T04:53:47.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.013+0000 7f4d82a00700 1 -- 172.21.15.6:0/1642356311 learned_addr learned my addr 172.21.15.6:0/1642356311 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:47.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.013+0000 7f4d82a00700 1 -- 172.21.15.6:0/1642356311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f21315450 msgr2=0x555f214b6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:47.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.013+0000 7f4d82a00700 1 --2- 172.21.15.6:0/1642356311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f21315450 0x555f214b6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:47.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.013+0000 7f4d82a00700 1 -- 172.21.15.6:0/1642356311 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f21498410 msgr2=0x555f214b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:47.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.013+0000 7f4d82a00700 1 --2- 172.21.15.6:0/1642356311 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f21498410 0x555f214b1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:47.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.013+0000 7f4d82a00700 1 -- 172.21.15.6:0/1642356311 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f214b21c0 con 0x555f21492a00 2024-05-09T04:53:47.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.013+0000 7f4d83400700 1 --2- 172.21.15.6:0/1642356311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f21315450 0x555f214b6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:53:47.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.014+0000 7f4d82a00700 1 --2- 172.21.15.6:0/1642356311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555f21492a00 0x555f214b71e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4d70005690 tx=0x7f4d70005500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2683c1b08e374ed server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:47.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.014+0000 7f4d6d600700 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-05-09T04:53:47.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.014+0000 7f4d6d600700 1 -- 172.21.15.6:0/1642356311 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70041060 con 0x555f21492a00 2024-05-09T04:53:47.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.014+0000 7f4d98deab80 1 -- 172.21.15.6:0/1642356311 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d70038070 con 0x555f21492a00 2024-05-09T04:53:47.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.014+0000 7f4d6d600700 1 -- 172.21.15.6:0/1642356311 <== mon.2 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f4d70033640 con 0x555f21492a00 2024-05-09T04:53:47.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.015+0000 7f4d6d600700 1 -- 172.21.15.6:0/1642356311 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4d70048050 con 0x555f21492a00 2024-05-09T04:53:47.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.015+0000 7f4d98deab80 10 client.25070 fetch_fsmap learned FSMap version 29 2024-05-09T04:53:47.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.015+0000 7f4d98deab80 1 -- 172.21.15.6:0/1642356311 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555f21517fc0 con 0x555f21492a00 2024-05-09T04:53:47.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.015+0000 7f4d6d600700 1 -- 172.21.15.6:0/1642356311 <== mon.2 v2:172.21.15.169:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d70038070 con 0x555f21492a00 2024-05-09T04:53:47.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.015+0000 7f4d6d600700 1 -- 172.21.15.6:0/1642356311 <== mon.2 v2:172.21.15.169:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4d70026040 con 0x555f21492a00 2024-05-09T04:53:47.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.015+0000 7f4d98deab80 10 client.25070 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T04:53:47.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.016+0000 7f4d98deab80 10 client.25070 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T04:53:47.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.016+0000 7f4d98deab80 20 client.25070 populate_metadata read hostname 'smithi006' 2024-05-09T04:53:47.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.016+0000 7f4d98deab80 1 --2- 172.21.15.6:0/1642356311 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x555f21519ea0 0x555f2151b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:47.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.016+0000 7f4d98deab80 4 client.25070 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T04:53:47.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.016+0000 7f4d98deab80 1 -- 172.21.15.6:0/1642356311 --> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] -- command(tid 0: {"prefix": "status"}) -- 0x555f213c2c40 con 0x555f21519ea0 2024-05-09T04:53:47.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.016+0000 7f4d83400700 1 --2- 172.21.15.6:0/1642356311 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x555f21519ea0 0x555f2151b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:47.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.017+0000 7f4d83400700 1 --2- 172.21.15.6:0/1642356311 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x555f21519ea0 0x555f2151b310 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd4a6607e9baf961 server_cookie=e7258fee09862408 in_seq=0 out_seq=0 2024-05-09T04:53:47.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.017+0000 7f4d6d600700 10 client.25070 ms_handle_connect on v2:172.21.15.169:6838/614388771 2024-05-09T04:53:47.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.017+0000 7f4d6d600700 1 -- 172.21.15.6:0/1642356311 <== mds.0 v2:172.21.15.169:6838/614388771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x555f213c2c40 con 0x555f21519ea0 2024-05-09T04:53:47.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.017+0000 7f4d6d600700 10 client.25070 handle_command_reply: tid=0 2024-05-09T04:53:47.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.017+0000 7f4d98deab80 1 client.25070 shutdown 2024-05-09T04:53:47.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.017+0000 7f4d98deab80 1 -- 172.21.15.6:0/1642356311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555f21492a00 msgr2=0x555f214b71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:47.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.017+0000 7f4d98deab80 1 --2- 172.21.15.6:0/1642356311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555f21492a00 0x555f214b71e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4d70005690 tx=0x7f4d70005500 comp rx=0 tx=0).stop 2024-05-09T04:53:47.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.017+0000 7f4d98deab80 1 -- 172.21.15.6:0/1642356311 shutdown_connections 2024-05-09T04:53:47.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.017+0000 7f4d98deab80 1 --2- 172.21.15.6:0/1642356311 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f21498410 0x555f214b1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:47.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.017+0000 7f4d98deab80 1 --2- 172.21.15.6:0/1642356311 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x555f21519ea0 0x555f2151b310 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:47.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.017+0000 7f4d98deab80 1 --2- 172.21.15.6:0/1642356311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f21315450 0x555f214b6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:47.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.017+0000 7f4d98deab80 1 --2- 172.21.15.6:0/1642356311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555f21492a00 0x555f214b71e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:47.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.017+0000 7f4d6d600700 0 client.25070 ms_handle_reset on v2:172.21.15.169:6838/614388771 2024-05-09T04:53:47.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.017+0000 7f4d98deab80 1 -- 172.21.15.6:0/1642356311 >> 172.21.15.6:0/1642356311 conn(0x555f21359e20 msgr2=0x555f214a72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:47.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.018+0000 7f4d98deab80 1 -- 172.21.15.6:0/1642356311 shutdown_connections 2024-05-09T04:53:47.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.018+0000 7f4d98deab80 1 -- 172.21.15.6:0/1642356311 wait complete. 2024-05-09T04:53:47.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.018+0000 7f4d98deab80 20 client.25070 trim_cache size 0 max 16384 2024-05-09T04:53:47.021 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-09T04:53:47.022 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "f74d813c-0dbd-11ef-bc97-c7b262605968", 2024-05-09T04:53:47.022 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 0, 2024-05-09T04:53:47.022 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24554, 2024-05-09T04:53:47.022 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-05-09T04:53:47.022 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-05-09T04:53:47.022 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-05-09T04:53:47.022 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 249.86117624600001, 2024-05-09T04:53:47.022 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 29, 2024-05-09T04:53:47.022 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 89, 2024-05-09T04:53:47.022 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 89, 2024-05-09T04:53:47.022 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 271.16457921300002 2024-05-09T04:53:47.023 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-09T04:53:47.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.018+0000 7f4d8a200700 1 -- 172.21.15.6:0/1668220028 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4d680b2880 msgr2=0x7f4d680b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:47.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.018+0000 7f4d8a200700 1 --2- 172.21.15.6:0/1668220028 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4d680b2880 0x7f4d680b4d90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4d70009fd0 tx=0x7f4d700095a0 comp rx=0 tx=0).stop 2024-05-09T04:53:47.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.018+0000 7f4d8a200700 1 -- 172.21.15.6:0/1668220028 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d8410e8d0 msgr2=0x7f4d841a18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:47.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.018+0000 7f4d8a200700 1 --2- 172.21.15.6:0/1668220028 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d8410e8d0 0x7f4d841a18d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c00ef30 tx=0x7f4d7c00c3a0 comp rx=0 tx=0).stop 2024-05-09T04:53:47.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.019+0000 7f4d8a200700 1 -- 172.21.15.6:0/1668220028 shutdown_connections 2024-05-09T04:53:47.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.019+0000 7f4d8a200700 1 --2- 172.21.15.6:0/1668220028 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d8410e8d0 0x7f4d841a18d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:47.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.019+0000 7f4d8a200700 1 --2- 172.21.15.6:0/1668220028 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d84105460 0x7f4d8419d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:47.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.019+0000 7f4d8a200700 1 --2- 172.21.15.6:0/1668220028 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4d680b2880 0x7f4d680b4d90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:47.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.019+0000 7f4d8a200700 1 --2- 172.21.15.6:0/1668220028 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d84105db0 0x7f4d8419d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:47.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.019+0000 7f4d8a200700 1 -- 172.21.15.6:0/1668220028 >> 172.21.15.6:0/1668220028 conn(0x7f4d840fedf0 msgr2=0x7f4d841030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:47.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.019+0000 7f4d8a200700 1 -- 172.21.15.6:0/1668220028 shutdown_connections 2024-05-09T04:53:47.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:47.019+0000 7f4d8a200700 1 -- 172.21.15.6:0/1668220028 wait complete. 2024-05-09T04:53:47.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:46 smithi186 ceph-mon[34668]: pgmap v464: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:53:47.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:46 smithi169 ceph-mon[34075]: pgmap v464: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:53:48.065 INFO:tasks.cephadm:Setup complete, yielding 2024-05-09T04:53:48.065 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-09T04:53:48.075 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi006.front.sepia.ceph.com 2024-05-09T04:53:48.075 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- bash -c 'ceph orch status' 2024-05-09T04:53:48.546 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:53:49.034 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:48 smithi006 ceph-mon[29835]: pgmap v465: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:53:49.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:48 smithi186 ceph-mon[34668]: pgmap v465: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:53:49.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:48 smithi169 ceph-mon[34075]: pgmap v465: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:53:50.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.872+0000 7f9a8a200700 1 -- 172.21.15.6:0/106357716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a84078100 msgr2=0x7f9a84078510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:50.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.872+0000 7f9a8a200700 1 --2- 172.21.15.6:0/106357716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a84078100 0x7f9a84078510 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9a78009990 tx=0x7f9a78009ca0 comp rx=0 tx=0).stop 2024-05-09T04:53:50.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.873+0000 7f9a8a200700 1 -- 172.21.15.6:0/106357716 shutdown_connections 2024-05-09T04:53:50.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.873+0000 7f9a8a200700 1 --2- 172.21.15.6:0/106357716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a84079bd0 0x7f9a8407a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:50.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.873+0000 7f9a8a200700 1 --2- 172.21.15.6:0/106357716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a84078100 0x7f9a84078510 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:50.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.873+0000 7f9a8a200700 1 --2- 172.21.15.6:0/106357716 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a84079280 0x7f9a84079690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:50.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.873+0000 7f9a8a200700 1 -- 172.21.15.6:0/106357716 >> 172.21.15.6:0/106357716 conn(0x7f9a841019c0 msgr2=0x7f9a84103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:50.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.873+0000 7f9a8a200700 1 -- 172.21.15.6:0/106357716 shutdown_connections 2024-05-09T04:53:50.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.873+0000 7f9a8a200700 1 -- 172.21.15.6:0/106357716 wait complete. 2024-05-09T04:53:50.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.874+0000 7f9a8a200700 1 Processor -- start 2024-05-09T04:53:50.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.874+0000 7f9a8a200700 1 -- start start 2024-05-09T04:53:50.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.875+0000 7f9a8a200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a84078100 0x7f9a841a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:50.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.875+0000 7f9a8a200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a84079280 0x7f9a841a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:50.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.875+0000 7f9a8a200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a84079bd0 0x7f9a841a8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:50.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.875+0000 7f9a8a200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9a84116260 con 0x7f9a84078100 2024-05-09T04:53:50.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.875+0000 7f9a8a200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9a841160e0 con 0x7f9a84079bd0 2024-05-09T04:53:50.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.875+0000 7f9a8a200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9a841163e0 con 0x7f9a84079280 2024-05-09T04:53:50.875 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:50 smithi006 ceph-mon[29835]: pgmap v466: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:53:50.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.875+0000 7f9a88e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a84078100 0x7f9a841a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:50.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.875+0000 7f9a88e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a84078100 0x7f9a841a0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41958/0 (socket says 172.21.15.6:41958) 2024-05-09T04:53:50.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.875+0000 7f9a88e00700 1 -- 172.21.15.6:0/4290717074 learned_addr learned my addr 172.21.15.6:0/4290717074 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:50.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.875+0000 7f9a89800700 1 --2- 172.21.15.6:0/4290717074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a84079bd0 0x7f9a841a8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:50.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.875+0000 7f9a83e00700 1 --2- 172.21.15.6:0/4290717074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a84079280 0x7f9a841a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:50.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.876+0000 7f9a89800700 1 -- 172.21.15.6:0/4290717074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a84079280 msgr2=0x7f9a841a12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:50.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.876+0000 7f9a89800700 1 --2- 172.21.15.6:0/4290717074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a84079280 0x7f9a841a12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:50.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.876+0000 7f9a89800700 1 -- 172.21.15.6:0/4290717074 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a84078100 msgr2=0x7f9a841a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:50.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.876+0000 7f9a89800700 1 --2- 172.21.15.6:0/4290717074 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a84078100 0x7f9a841a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:50.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.876+0000 7f9a89800700 1 -- 172.21.15.6:0/4290717074 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a841a88e0 con 0x7f9a84079bd0 2024-05-09T04:53:50.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.876+0000 7f9a88e00700 1 --2- 172.21.15.6:0/4290717074 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a84078100 0x7f9a841a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:53:50.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.876+0000 7f9a89800700 1 --2- 172.21.15.6:0/4290717074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a84079bd0 0x7f9a841a8210 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c00efb0 tx=0x7f9a7c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:50.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.876+0000 7f9a81600700 1 -- 172.21.15.6:0/4290717074 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c00ca50 con 0x7f9a84079bd0 2024-05-09T04:53:50.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.876+0000 7f9a81600700 1 -- 172.21.15.6:0/4290717074 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a7c00cbf0 con 0x7f9a84079bd0 2024-05-09T04:53:50.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.876+0000 7f9a8a200700 1 -- 172.21.15.6:0/4290717074 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a841a8bd0 con 0x7f9a84079bd0 2024-05-09T04:53:50.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.877+0000 7f9a81600700 1 -- 172.21.15.6:0/4290717074 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c004890 con 0x7f9a84079bd0 2024-05-09T04:53:50.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.877+0000 7f9a8a200700 1 -- 172.21.15.6:0/4290717074 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a841a9110 con 0x7f9a84079bd0 2024-05-09T04:53:50.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.878+0000 7f9a8a200700 1 -- 172.21.15.6:0/4290717074 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a840512c0 con 0x7f9a84079bd0 2024-05-09T04:53:50.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.881+0000 7f9a81600700 1 -- 172.21.15.6:0/4290717074 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f9a7c012050 con 0x7f9a84079bd0 2024-05-09T04:53:50.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.882+0000 7f9a81600700 1 --2- 172.21.15.6:0/4290717074 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9a6c0b2900 0x7f9a6c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:50.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.882+0000 7f9a81600700 1 -- 172.21.15.6:0/4290717074 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f9a7c0e1eb0 con 0x7f9a84079bd0 2024-05-09T04:53:50.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.883+0000 7f9a88e00700 1 --2- 172.21.15.6:0/4290717074 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9a6c0b2900 0x7f9a6c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:50.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.883+0000 7f9a88e00700 1 --2- 172.21.15.6:0/4290717074 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9a6c0b2900 0x7f9a6c0b4e10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800b2e0 tx=0x7f9a78005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:50.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:50.887+0000 7f9a81600700 1 -- 172.21.15.6:0/4290717074 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9a7c022060 con 0x7f9a84079bd0 2024-05-09T04:53:51.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:50 smithi186 ceph-mon[34668]: pgmap v466: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:53:51.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:50 smithi169 ceph-mon[34075]: pgmap v466: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:53:51.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:51.141+0000 7f9a8a200700 1 -- 172.21.15.6:0/4290717074 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f9a84063c00 con 0x7f9a6c0b2900 2024-05-09T04:53:51.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:51.144+0000 7f9a81600700 1 -- 172.21.15.6:0/4290717074 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f9a84063c00 con 0x7f9a6c0b2900 2024-05-09T04:53:51.144 INFO:teuthology.orchestra.run.smithi006.stdout:Backend: cephadm 2024-05-09T04:53:51.144 INFO:teuthology.orchestra.run.smithi006.stdout:Available: Yes 2024-05-09T04:53:51.144 INFO:teuthology.orchestra.run.smithi006.stdout:Paused: No 2024-05-09T04:53:51.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:51.147+0000 7f9a8a200700 1 -- 172.21.15.6:0/4290717074 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9a6c0b2900 msgr2=0x7f9a6c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:51.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:51.147+0000 7f9a8a200700 1 --2- 172.21.15.6:0/4290717074 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9a6c0b2900 0x7f9a6c0b4e10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800b2e0 tx=0x7f9a78005fb0 comp rx=0 tx=0).stop 2024-05-09T04:53:51.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:51.147+0000 7f9a8a200700 1 -- 172.21.15.6:0/4290717074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a84079bd0 msgr2=0x7f9a841a8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:51.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:51.147+0000 7f9a8a200700 1 --2- 172.21.15.6:0/4290717074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a84079bd0 0x7f9a841a8210 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c00efb0 tx=0x7f9a7c00c3a0 comp rx=0 tx=0).stop 2024-05-09T04:53:51.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:51.147+0000 7f9a8a200700 1 -- 172.21.15.6:0/4290717074 shutdown_connections 2024-05-09T04:53:51.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:51.147+0000 7f9a8a200700 1 --2- 172.21.15.6:0/4290717074 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9a6c0b2900 0x7f9a6c0b4e10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:51.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:51.147+0000 7f9a8a200700 1 --2- 172.21.15.6:0/4290717074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a84079bd0 0x7f9a841a8210 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:51.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:51.147+0000 7f9a8a200700 1 --2- 172.21.15.6:0/4290717074 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a84078100 0x7f9a841a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:51.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:51.147+0000 7f9a8a200700 1 --2- 172.21.15.6:0/4290717074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a84079280 0x7f9a841a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:51.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:51.147+0000 7f9a8a200700 1 -- 172.21.15.6:0/4290717074 >> 172.21.15.6:0/4290717074 conn(0x7f9a841019c0 msgr2=0x7f9a84110650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:51.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:51.148+0000 7f9a8a200700 1 -- 172.21.15.6:0/4290717074 shutdown_connections 2024-05-09T04:53:51.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:51.148+0000 7f9a8a200700 1 -- 172.21.15.6:0/4290717074 wait complete. 2024-05-09T04:53:52.122 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-09T04:53:52.592 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:53:53.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:52 smithi186 ceph-mon[34668]: from='client.25076 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:53:53.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:52 smithi186 ceph-mon[34668]: pgmap v467: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:53:53.110 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:52 smithi006 ceph-mon[29835]: from='client.25076 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:53:53.110 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:52 smithi006 ceph-mon[29835]: pgmap v467: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:53:53.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:52 smithi169 ceph-mon[34075]: from='client.25076 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:53:53.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:52 smithi169 ceph-mon[34075]: pgmap v467: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:53:54.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:54 smithi006 ceph-mon[29835]: pgmap v468: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:53:54.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.946+0000 7f701d800700 1 -- 172.21.15.6:0/1706853644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7018108020 msgr2=0x7f701810e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:54.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.946+0000 7f701d800700 1 --2- 172.21.15.6:0/1706853644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7018108020 0x7f701810e8d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7010009990 tx=0x7f7010009ca0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.947+0000 7f701d800700 1 -- 172.21.15.6:0/1706853644 shutdown_connections 2024-05-09T04:53:54.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.947+0000 7f701d800700 1 --2- 172.21.15.6:0/1706853644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7018108020 0x7f701810e8d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.947+0000 7f701d800700 1 --2- 172.21.15.6:0/1706853644 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7018106550 0x7f7018106960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.947+0000 7f701d800700 1 --2- 172.21.15.6:0/1706853644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7018107680 0x7f7018107a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.947+0000 7f701d800700 1 -- 172.21.15.6:0/1706853644 >> 172.21.15.6:0/1706853644 conn(0x7f7018101b90 msgr2=0x7f7018104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:54.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.947+0000 7f701d800700 1 -- 172.21.15.6:0/1706853644 shutdown_connections 2024-05-09T04:53:54.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.947+0000 7f701d800700 1 -- 172.21.15.6:0/1706853644 wait complete. 2024-05-09T04:53:54.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.948+0000 7f701d800700 1 Processor -- start 2024-05-09T04:53:54.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.949+0000 7f701d800700 1 -- start start 2024-05-09T04:53:54.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.949+0000 7f701d800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7018106550 0x7f701819cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:54.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.949+0000 7f701d800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7018107680 0x7f701819d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:54.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.949+0000 7f701d800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7018108020 0x7f70181a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:54.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.949+0000 7f701d800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7018112000 con 0x7f7018107680 2024-05-09T04:53:54.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.949+0000 7f701d800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7018111e80 con 0x7f7018108020 2024-05-09T04:53:54.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.949+0000 7f701d800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7018112180 con 0x7f7018106550 2024-05-09T04:53:54.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.949+0000 7f7016a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7018106550 0x7f701819cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:54.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.950+0000 7f700e000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7018107680 0x7f701819d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:54.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.950+0000 7f7016a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7018106550 0x7f701819cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:50656/0 (socket says 172.21.15.6:50656) 2024-05-09T04:53:54.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.950+0000 7f7016a00700 1 -- 172.21.15.6:0/2369052215 learned_addr learned my addr 172.21.15.6:0/2369052215 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:54.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.950+0000 7f7017400700 1 --2- 172.21.15.6:0/2369052215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7018108020 0x7f70181a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:54.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.950+0000 7f7016a00700 1 -- 172.21.15.6:0/2369052215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7018108020 msgr2=0x7f70181a3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:54.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.950+0000 7f7016a00700 1 --2- 172.21.15.6:0/2369052215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7018108020 0x7f70181a3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.950+0000 7f7016a00700 1 -- 172.21.15.6:0/2369052215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7018107680 msgr2=0x7f701819d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:54.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.950+0000 7f7016a00700 1 --2- 172.21.15.6:0/2369052215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7018107680 0x7f701819d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.950+0000 7f7016a00700 1 -- 172.21.15.6:0/2369052215 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70181a4640 con 0x7f7018106550 2024-05-09T04:53:54.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.950+0000 7f700e000700 1 --2- 172.21.15.6:0/2369052215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7018107680 0x7f701819d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:53:54.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.951+0000 7f7016a00700 1 --2- 172.21.15.6:0/2369052215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7018106550 0x7f701819cae0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f700800ef70 tx=0x7f700800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:54.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.951+0000 7f700fe00700 1 -- 172.21.15.6:0/2369052215 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70080092e0 con 0x7f7018106550 2024-05-09T04:53:54.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.951+0000 7f700fe00700 1 -- 172.21.15.6:0/2369052215 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f700800c9c0 con 0x7f7018106550 2024-05-09T04:53:54.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.951+0000 7f701d800700 1 -- 172.21.15.6:0/2369052215 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70181a48d0 con 0x7f7018106550 2024-05-09T04:53:54.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.951+0000 7f700fe00700 1 -- 172.21.15.6:0/2369052215 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7008004550 con 0x7f7018106550 2024-05-09T04:53:54.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.951+0000 7f701d800700 1 -- 172.21.15.6:0/2369052215 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70181a4e60 con 0x7f7018106550 2024-05-09T04:53:54.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.952+0000 7f701d800700 1 -- 172.21.15.6:0/2369052215 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70180512c0 con 0x7f7018106550 2024-05-09T04:53:54.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.955+0000 7f700fe00700 1 -- 172.21.15.6:0/2369052215 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f700801b030 con 0x7f7018106550 2024-05-09T04:53:54.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.956+0000 7f700fe00700 1 --2- 172.21.15.6:0/2369052215 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f6ffc0b2900 0x7f6ffc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:54.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.957+0000 7f700fe00700 1 -- 172.21.15.6:0/2369052215 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f70080e2d60 con 0x7f7018106550 2024-05-09T04:53:54.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.957+0000 7f700e000700 1 --2- 172.21.15.6:0/2369052215 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f6ffc0b2900 0x7f6ffc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:54.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.957+0000 7f700e000700 1 --2- 172.21.15.6:0/2369052215 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f6ffc0b2900 0x7f6ffc0b4e10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f701819db90 tx=0x7f7000009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:54.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:54.958+0000 7f700fe00700 1 -- 172.21.15.6:0/2369052215 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7008022020 con 0x7f7018106550 2024-05-09T04:53:55.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:54 smithi186 ceph-mon[34668]: pgmap v468: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:53:55.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:54 smithi169 ceph-mon[34075]: pgmap v468: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:53:55.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:55.205+0000 7f701d800700 1 -- 172.21.15.6:0/2369052215 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f7018063cc0 con 0x7f6ffc0b2900 2024-05-09T04:53:55.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:55.229+0000 7f700fe00700 1 -- 172.21.15.6:0/2369052215 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f7018063cc0 con 0x7f6ffc0b2900 2024-05-09T04:53:55.230 INFO:teuthology.orchestra.run.smithi006.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T04:53:55.230 INFO:teuthology.orchestra.run.smithi006.stdout:mds.a smithi006 running (4m) 2m ago 4m 80.2M - 19.0.0-3594-gc9da4049 db794d706f49 1a921a25b562 2024-05-09T04:53:55.230 INFO:teuthology.orchestra.run.smithi006.stdout:mds.b smithi169 running (4m) 2m ago 4m 93.2M - 19.0.0-3594-gc9da4049 db794d706f49 a0c62ed0cc42 2024-05-09T04:53:55.230 INFO:teuthology.orchestra.run.smithi006.stdout:mds.c smithi186 running (5m) 2m ago 4m 86.9M - 19.0.0-3594-gc9da4049 db794d706f49 c5741dc9744b 2024-05-09T04:53:55.230 INFO:teuthology.orchestra.run.smithi006.stdout:mds.d smithi006 running (5m) 2m ago 5m 82.3M - 19.0.0-3594-gc9da4049 db794d706f49 507be399f6da 2024-05-09T04:53:55.230 INFO:teuthology.orchestra.run.smithi006.stdout:mds.e smithi169 running (5m) 2m ago 5m 87.2M - 19.0.0-3594-gc9da4049 db794d706f49 67aa99d0163a 2024-05-09T04:53:55.230 INFO:teuthology.orchestra.run.smithi006.stdout:mds.f smithi186 running (5m) 2m ago 5m 82.9M - 19.0.0-3594-gc9da4049 db794d706f49 6d28b7e38c06 2024-05-09T04:53:55.230 INFO:teuthology.orchestra.run.smithi006.stdout:mds.g smithi006 running (4m) 2m ago 4m 84.2M - 19.0.0-3594-gc9da4049 db794d706f49 1e1230aa6c00 2024-05-09T04:53:55.230 INFO:teuthology.orchestra.run.smithi006.stdout:mds.h smithi169 running (4m) 2m ago 4m 85.5M - 19.0.0-3594-gc9da4049 db794d706f49 03ff7e9c27ce 2024-05-09T04:53:55.231 INFO:teuthology.orchestra.run.smithi006.stdout:mds.i smithi186 running (4m) 2m ago 4m 84.3M - 19.0.0-3594-gc9da4049 db794d706f49 4e2839b809cc 2024-05-09T04:53:55.231 INFO:teuthology.orchestra.run.smithi006.stdout:mds.j smithi006 running (5m) 2m ago 5m 80.5M - 19.0.0-3594-gc9da4049 db794d706f49 443ae74a980b 2024-05-09T04:53:55.231 INFO:teuthology.orchestra.run.smithi006.stdout:mds.k smithi169 running (5m) 2m ago 5m 83.8M - 19.0.0-3594-gc9da4049 db794d706f49 92ceeac7bec6 2024-05-09T04:53:55.231 INFO:teuthology.orchestra.run.smithi006.stdout:mds.l smithi186 running (5m) 2m ago 5m 86.3M - 19.0.0-3594-gc9da4049 db794d706f49 1123264bf8e8 2024-05-09T04:53:55.231 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.x smithi006 *:9283,8765 running (14m) 2m ago 14m 598M - 19.0.0-3594-gc9da4049 db794d706f49 ede2170d178c 2024-05-09T04:53:55.231 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.y smithi169 *:8443,8765 running (11m) 2m ago 11m 537M - 19.0.0-3594-gc9da4049 db794d706f49 2029de5ec1e1 2024-05-09T04:53:55.231 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.z smithi186 *:8443,8765 running (11m) 2m ago 11m 538M - 19.0.0-3594-gc9da4049 db794d706f49 2681ae679c05 2024-05-09T04:53:55.231 INFO:teuthology.orchestra.run.smithi006.stdout:mon.a smithi006 running (14m) 2m ago 14m 117M 2048M 19.0.0-3594-gc9da4049 db794d706f49 34c0cb98e2cc 2024-05-09T04:53:55.231 INFO:teuthology.orchestra.run.smithi006.stdout:mon.b smithi169 running (11m) 2m ago 11m 114M 2048M 19.0.0-3594-gc9da4049 db794d706f49 1ca93128349d 2024-05-09T04:53:55.231 INFO:teuthology.orchestra.run.smithi006.stdout:mon.c smithi186 running (12m) 2m ago 12m 110M 2048M 19.0.0-3594-gc9da4049 db794d706f49 1bc9bdc0322d 2024-05-09T04:53:55.231 INFO:teuthology.orchestra.run.smithi006.stdout:osd.0 smithi006 running (11m) 2m ago 11m 114M 4273M 19.0.0-3594-gc9da4049 db794d706f49 4bc898547b80 2024-05-09T04:53:55.231 INFO:teuthology.orchestra.run.smithi006.stdout:osd.1 smithi169 running (10m) 2m ago 10m 113M 4272M 19.0.0-3594-gc9da4049 db794d706f49 980ad70d4ffe 2024-05-09T04:53:55.231 INFO:teuthology.orchestra.run.smithi006.stdout:osd.2 smithi186 running (10m) 2m ago 10m 125M 4272M 19.0.0-3594-gc9da4049 db794d706f49 be5c698b34c6 2024-05-09T04:53:55.231 INFO:teuthology.orchestra.run.smithi006.stdout:osd.3 smithi006 running (9m) 2m ago 9m 121M 4273M 19.0.0-3594-gc9da4049 db794d706f49 3c1301617b18 2024-05-09T04:53:55.231 INFO:teuthology.orchestra.run.smithi006.stdout:osd.4 smithi169 running (9m) 2m ago 9m 120M 4272M 19.0.0-3594-gc9da4049 db794d706f49 ce788644115f 2024-05-09T04:53:55.232 INFO:teuthology.orchestra.run.smithi006.stdout:osd.5 smithi186 running (8m) 2m ago 8m 114M 4272M 19.0.0-3594-gc9da4049 db794d706f49 2e63acb9f074 2024-05-09T04:53:55.232 INFO:teuthology.orchestra.run.smithi006.stdout:osd.6 smithi006 running (8m) 2m ago 8m 110M 4273M 19.0.0-3594-gc9da4049 db794d706f49 007f7da53666 2024-05-09T04:53:55.232 INFO:teuthology.orchestra.run.smithi006.stdout:osd.7 smithi169 running (7m) 2m ago 7m 111M 4272M 19.0.0-3594-gc9da4049 db794d706f49 e67e07953657 2024-05-09T04:53:55.232 INFO:teuthology.orchestra.run.smithi006.stdout:osd.8 smithi186 running (7m) 2m ago 7m 119M 4272M 19.0.0-3594-gc9da4049 db794d706f49 0af8d4b19537 2024-05-09T04:53:55.232 INFO:teuthology.orchestra.run.smithi006.stdout:osd.9 smithi006 running (6m) 2m ago 6m 109M 4273M 19.0.0-3594-gc9da4049 db794d706f49 2a522e6a0ace 2024-05-09T04:53:55.232 INFO:teuthology.orchestra.run.smithi006.stdout:osd.10 smithi169 running (6m) 2m ago 6m 119M 4272M 19.0.0-3594-gc9da4049 db794d706f49 311ce3fec2fa 2024-05-09T04:53:55.232 INFO:teuthology.orchestra.run.smithi006.stdout:osd.11 smithi186 running (5m) 2m ago 5m 109M 4272M 19.0.0-3594-gc9da4049 db794d706f49 9f23ba9a373b 2024-05-09T04:53:55.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:55.233+0000 7f701d800700 1 -- 172.21.15.6:0/2369052215 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f6ffc0b2900 msgr2=0x7f6ffc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:55.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:55.233+0000 7f701d800700 1 --2- 172.21.15.6:0/2369052215 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f6ffc0b2900 0x7f6ffc0b4e10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f701819db90 tx=0x7f7000009260 comp rx=0 tx=0).stop 2024-05-09T04:53:55.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:55.233+0000 7f701d800700 1 -- 172.21.15.6:0/2369052215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7018106550 msgr2=0x7f701819cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:55.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:55.233+0000 7f701d800700 1 --2- 172.21.15.6:0/2369052215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7018106550 0x7f701819cae0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f700800ef70 tx=0x7f700800c3a0 comp rx=0 tx=0).stop 2024-05-09T04:53:55.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:55.233+0000 7f701d800700 1 -- 172.21.15.6:0/2369052215 shutdown_connections 2024-05-09T04:53:55.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:55.233+0000 7f701d800700 1 --2- 172.21.15.6:0/2369052215 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f6ffc0b2900 0x7f6ffc0b4e10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:55.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:55.233+0000 7f701d800700 1 --2- 172.21.15.6:0/2369052215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7018108020 0x7f70181a3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:55.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:55.233+0000 7f701d800700 1 --2- 172.21.15.6:0/2369052215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7018106550 0x7f701819cae0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:55.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:55.233+0000 7f701d800700 1 --2- 172.21.15.6:0/2369052215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7018107680 0x7f701819d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:55.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:55.233+0000 7f701d800700 1 -- 172.21.15.6:0/2369052215 >> 172.21.15.6:0/2369052215 conn(0x7f7018101b90 msgr2=0x7f701810a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:55.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:55.233+0000 7f701d800700 1 -- 172.21.15.6:0/2369052215 shutdown_connections 2024-05-09T04:53:55.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:55.233+0000 7f701d800700 1 -- 172.21.15.6:0/2369052215 wait complete. 2024-05-09T04:53:56.082 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-09T04:53:56.559 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:53:57.037 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:56 smithi006 ceph-mon[29835]: from='client.25069 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:53:57.038 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:56 smithi006 ceph-mon[29835]: pgmap v469: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:53:57.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:56 smithi186 ceph-mon[34668]: from='client.25069 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:53:57.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:56 smithi186 ceph-mon[34668]: pgmap v469: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:53:57.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:56 smithi169 ceph-mon[34075]: from='client.25069 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:53:57.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:56 smithi169 ceph-mon[34075]: pgmap v469: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:53:59.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.005+0000 7f9b75a00700 1 -- 172.21.15.6:0/4165459306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b701077b0 msgr2=0x7f9b70109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:59.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.005+0000 7f9b75a00700 1 --2- 172.21.15.6:0/4165459306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b701077b0 0x7f9b70109bd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9b580098f0 tx=0x7f9b58009c00 comp rx=0 tx=0).stop 2024-05-09T04:53:59.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.005+0000 7f9b75a00700 1 -- 172.21.15.6:0/4165459306 shutdown_connections 2024-05-09T04:53:59.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.005+0000 7f9b75a00700 1 --2- 172.21.15.6:0/4165459306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b7010a110 0x7f9b7010c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:59.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.005+0000 7f9b75a00700 1 --2- 172.21.15.6:0/4165459306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9b70104e50 0x7f9b70107270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:59.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.005+0000 7f9b75a00700 1 --2- 172.21.15.6:0/4165459306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b701077b0 0x7f9b70109bd0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:59.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.005+0000 7f9b75a00700 1 -- 172.21.15.6:0/4165459306 >> 172.21.15.6:0/4165459306 conn(0x7f9b700fe890 msgr2=0x7f9b70100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:59.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.005+0000 7f9b75a00700 1 -- 172.21.15.6:0/4165459306 shutdown_connections 2024-05-09T04:53:59.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.005+0000 7f9b75a00700 1 -- 172.21.15.6:0/4165459306 wait complete. 2024-05-09T04:53:59.007 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:53:58 smithi006 ceph-mon[29835]: pgmap v470: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:53:59.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.007+0000 7f9b75a00700 1 Processor -- start 2024-05-09T04:53:59.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.007+0000 7f9b75a00700 1 -- start start 2024-05-09T04:53:59.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.008+0000 7f9b75a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b70104e50 0x7f9b701046b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:59.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.008+0000 7f9b75a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b701077b0 0x7f9b70102d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:59.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.008+0000 7f9b75a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9b7010a110 0x7f9b701032a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:59.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.008+0000 7f9b75a00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7010fdf0 con 0x7f9b7010a110 2024-05-09T04:53:59.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.008+0000 7f9b75a00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7010fc70 con 0x7f9b70104e50 2024-05-09T04:53:59.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.008+0000 7f9b75a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7010ff70 con 0x7f9b701077b0 2024-05-09T04:53:59.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.008+0000 7f9b6f400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9b7010a110 0x7f9b701032a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:59.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.008+0000 7f9b67e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b701077b0 0x7f9b70102d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:59.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.008+0000 7f9b6f400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9b7010a110 0x7f9b701032a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:47772/0 (socket says 172.21.15.6:47772) 2024-05-09T04:53:59.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.008+0000 7f9b6f400700 1 -- 172.21.15.6:0/4247740297 learned_addr learned my addr 172.21.15.6:0/4247740297 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:53:59.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.008+0000 7f9b6ea00700 1 --2- 172.21.15.6:0/4247740297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b70104e50 0x7f9b701046b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:59.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.009+0000 7f9b67e00700 1 -- 172.21.15.6:0/4247740297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b70104e50 msgr2=0x7f9b701046b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:59.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.009+0000 7f9b67e00700 1 --2- 172.21.15.6:0/4247740297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b70104e50 0x7f9b701046b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:59.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.009+0000 7f9b67e00700 1 -- 172.21.15.6:0/4247740297 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9b7010a110 msgr2=0x7f9b701032a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:59.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.009+0000 7f9b67e00700 1 --2- 172.21.15.6:0/4247740297 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9b7010a110 0x7f9b701032a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:59.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.009+0000 7f9b67e00700 1 -- 172.21.15.6:0/4247740297 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b70103a00 con 0x7f9b701077b0 2024-05-09T04:53:59.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.009+0000 7f9b6ea00700 1 --2- 172.21.15.6:0/4247740297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b70104e50 0x7f9b701046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:53:59.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.009+0000 7f9b67e00700 1 --2- 172.21.15.6:0/4247740297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b701077b0 0x7f9b70102d60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9b58000c00 tx=0x7f9b58005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:59.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.009+0000 7f9b67400700 1 -- 172.21.15.6:0/4247740297 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b58025070 con 0x7f9b701077b0 2024-05-09T04:53:59.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.010+0000 7f9b67400700 1 -- 172.21.15.6:0/4247740297 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b580055f0 con 0x7f9b701077b0 2024-05-09T04:53:59.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.010+0000 7f9b75a00700 1 -- 172.21.15.6:0/4247740297 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b701a4a70 con 0x7f9b701077b0 2024-05-09T04:53:59.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.010+0000 7f9b67400700 1 -- 172.21.15.6:0/4247740297 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b58003a80 con 0x7f9b701077b0 2024-05-09T04:53:59.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.010+0000 7f9b75a00700 1 -- 172.21.15.6:0/4247740297 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b701a4e10 con 0x7f9b701077b0 2024-05-09T04:53:59.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.011+0000 7f9b75a00700 1 -- 172.21.15.6:0/4247740297 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b700512c0 con 0x7f9b701077b0 2024-05-09T04:53:59.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.014+0000 7f9b67400700 1 -- 172.21.15.6:0/4247740297 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f9b58028070 con 0x7f9b701077b0 2024-05-09T04:53:59.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.015+0000 7f9b67400700 1 --2- 172.21.15.6:0/4247740297 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9b540b2880 0x7f9b540b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:59.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.015+0000 7f9b67400700 1 -- 172.21.15.6:0/4247740297 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f9b580e3070 con 0x7f9b701077b0 2024-05-09T04:53:59.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.016+0000 7f9b6ea00700 1 --2- 172.21.15.6:0/4247740297 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9b540b2880 0x7f9b540b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:59.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.016+0000 7f9b6ea00700 1 --2- 172.21.15.6:0/4247740297 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9b540b2880 0x7f9b540b4d90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9b70076410 tx=0x7f9b60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:59.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.017+0000 7f9b67400700 1 -- 172.21.15.6:0/4247740297 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9b5802f080 con 0x7f9b701077b0 2024-05-09T04:53:59.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:53:58 smithi186 ceph-mon[34668]: pgmap v470: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:53:59.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:53:58 smithi169 ceph-mon[34075]: pgmap v470: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:53:59.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.266+0000 7f9b75a00700 1 -- 172.21.15.6:0/4247740297 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f9b70063c00 con 0x7f9b540b2880 2024-05-09T04:53:59.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.283+0000 7f9b67400700 1 -- 172.21.15.6:0/4247740297 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f9b70063c00 con 0x7f9b540b2880 2024-05-09T04:53:59.283 INFO:teuthology.orchestra.run.smithi006.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-09T04:53:59.283 INFO:teuthology.orchestra.run.smithi006.stdout:mds.all 12/12 2m ago 5m smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi169=b;smithi169=e;smithi169=h;smithi169=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2024-05-09T04:53:59.283 INFO:teuthology.orchestra.run.smithi006.stdout:mgr 3/3 2m ago 11m smithi006=x;smithi169=y;smithi186=z;count:3 2024-05-09T04:53:59.283 INFO:teuthology.orchestra.run.smithi006.stdout:mon 3/3 2m ago 12m smithi006:172.21.15.6=a;smithi169:172.21.15.169=b;smithi186:172.21.15.186=c;count:3 2024-05-09T04:53:59.283 INFO:teuthology.orchestra.run.smithi006.stdout:osd 12 2m ago - 2024-05-09T04:53:59.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.286+0000 7f9b75a00700 1 -- 172.21.15.6:0/4247740297 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9b540b2880 msgr2=0x7f9b540b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:59.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.286+0000 7f9b75a00700 1 --2- 172.21.15.6:0/4247740297 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9b540b2880 0x7f9b540b4d90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9b70076410 tx=0x7f9b60009260 comp rx=0 tx=0).stop 2024-05-09T04:53:59.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.286+0000 7f9b75a00700 1 -- 172.21.15.6:0/4247740297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b701077b0 msgr2=0x7f9b70102d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:59.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.286+0000 7f9b75a00700 1 --2- 172.21.15.6:0/4247740297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b701077b0 0x7f9b70102d60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9b58000c00 tx=0x7f9b58005270 comp rx=0 tx=0).stop 2024-05-09T04:53:59.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.287+0000 7f9b75a00700 1 -- 172.21.15.6:0/4247740297 shutdown_connections 2024-05-09T04:53:59.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.287+0000 7f9b75a00700 1 --2- 172.21.15.6:0/4247740297 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9b7010a110 0x7f9b701032a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:59.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.287+0000 7f9b75a00700 1 --2- 172.21.15.6:0/4247740297 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9b540b2880 0x7f9b540b4d90 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:59.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.287+0000 7f9b75a00700 1 --2- 172.21.15.6:0/4247740297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b70104e50 0x7f9b701046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:59.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.287+0000 7f9b75a00700 1 --2- 172.21.15.6:0/4247740297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b701077b0 0x7f9b70102d60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:59.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.287+0000 7f9b75a00700 1 -- 172.21.15.6:0/4247740297 >> 172.21.15.6:0/4247740297 conn(0x7f9b700fe890 msgr2=0x7f9b70106e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:59.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.287+0000 7f9b75a00700 1 -- 172.21.15.6:0/4247740297 shutdown_connections 2024-05-09T04:53:59.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:53:59.287+0000 7f9b75a00700 1 -- 172.21.15.6:0/4247740297 wait complete. 2024-05-09T04:54:00.205 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-09T04:54:00.682 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:54:01.053 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:00 smithi006 ceph-mon[29835]: from='client.25075 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:54:01.053 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:00 smithi006 ceph-mon[29835]: pgmap v471: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:01.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:00 smithi186 ceph-mon[34668]: from='client.25075 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:54:01.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:00 smithi186 ceph-mon[34668]: pgmap v471: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:01.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:00 smithi169 ceph-mon[34075]: from='client.25075 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:54:01.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:00 smithi169 ceph-mon[34075]: pgmap v471: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:03.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.000+0000 7f1602c00700 1 -- 172.21.15.6:0/1450735929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15fc105700 msgr2=0x7f15fc105b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:03.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.000+0000 7f1602c00700 1 --2- 172.21.15.6:0/1450735929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15fc105700 0x7f15fc105b10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f15f0009990 tx=0x7f15f0009ca0 comp rx=0 tx=0).stop 2024-05-09T04:54:03.002 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:02 smithi006 ceph-mon[29835]: pgmap v472: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:54:03.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.000+0000 7f1602c00700 1 -- 172.21.15.6:0/1450735929 shutdown_connections 2024-05-09T04:54:03.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.000+0000 7f1602c00700 1 --2- 172.21.15.6:0/1450735929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15fc102aa0 0x7f15fc10c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:03.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.000+0000 7f1602c00700 1 --2- 172.21.15.6:0/1450735929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15fc105700 0x7f15fc105b10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:03.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.000+0000 7f1602c00700 1 --2- 172.21.15.6:0/1450735929 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15fc102150 0x7f15fc102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:03.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.000+0000 7f1602c00700 1 -- 172.21.15.6:0/1450735929 >> 172.21.15.6:0/1450735929 conn(0x7f15fc0fde60 msgr2=0x7f15fc1002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:03.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.000+0000 7f1602c00700 1 -- 172.21.15.6:0/1450735929 shutdown_connections 2024-05-09T04:54:03.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.000+0000 7f1602c00700 1 -- 172.21.15.6:0/1450735929 wait complete. 2024-05-09T04:54:03.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.002+0000 7f1602c00700 1 Processor -- start 2024-05-09T04:54:03.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.002+0000 7f1602c00700 1 -- start start 2024-05-09T04:54:03.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.003+0000 7f1602c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15fc102150 0x7f15fc19ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:03.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.003+0000 7f1602c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15fc102aa0 0x7f15fc19cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:03.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.003+0000 7f1602c00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15fc105700 0x7f15fc1a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:03.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.003+0000 7f1602c00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f15fc10fc00 con 0x7f15fc102aa0 2024-05-09T04:54:03.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.003+0000 7f1602c00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f15fc10fa80 con 0x7f15fc105700 2024-05-09T04:54:03.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.003+0000 7f1602c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f15fc10fd80 con 0x7f15fc102150 2024-05-09T04:54:03.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.003+0000 7f1600e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15fc102aa0 0x7f15fc19cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:03.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.003+0000 7f1600e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15fc102aa0 0x7f15fc19cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:47788/0 (socket says 172.21.15.6:47788) 2024-05-09T04:54:03.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.003+0000 7f1600e00700 1 -- 172.21.15.6:0/3737260013 learned_addr learned my addr 172.21.15.6:0/3737260013 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:03.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.003+0000 7f1601800700 1 --2- 172.21.15.6:0/3737260013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15fc102150 0x7f15fc19ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:03.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.003+0000 7f1602200700 1 --2- 172.21.15.6:0/3737260013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15fc105700 0x7f15fc1a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:03.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.004+0000 7f1601800700 1 -- 172.21.15.6:0/3737260013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15fc105700 msgr2=0x7f15fc1a3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:03.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.004+0000 7f1601800700 1 --2- 172.21.15.6:0/3737260013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15fc105700 0x7f15fc1a3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:03.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.004+0000 7f1601800700 1 -- 172.21.15.6:0/3737260013 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15fc102aa0 msgr2=0x7f15fc19cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:03.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.004+0000 7f1601800700 1 --2- 172.21.15.6:0/3737260013 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15fc102aa0 0x7f15fc19cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:03.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.004+0000 7f1601800700 1 -- 172.21.15.6:0/3737260013 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15fc1a45c0 con 0x7f15fc102150 2024-05-09T04:54:03.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.004+0000 7f1601800700 1 --2- 172.21.15.6:0/3737260013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15fc102150 0x7f15fc19ca60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f15f00098e0 tx=0x7f15f0009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:03.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.004+0000 7f15fa000700 1 -- 172.21.15.6:0/3737260013 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f0026070 con 0x7f15fc102150 2024-05-09T04:54:03.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.004+0000 7f15fa000700 1 -- 172.21.15.6:0/3737260013 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15f0021410 con 0x7f15fc102150 2024-05-09T04:54:03.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.004+0000 7f1602c00700 1 -- 172.21.15.6:0/3737260013 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15fc1a4850 con 0x7f15fc102150 2024-05-09T04:54:03.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.005+0000 7f15fa000700 1 -- 172.21.15.6:0/3737260013 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f0017650 con 0x7f15fc102150 2024-05-09T04:54:03.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.005+0000 7f1602c00700 1 -- 172.21.15.6:0/3737260013 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15fc1a4ca0 con 0x7f15fc102150 2024-05-09T04:54:03.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.006+0000 7f1602c00700 1 -- 172.21.15.6:0/3737260013 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15fc0512c0 con 0x7f15fc102150 2024-05-09T04:54:03.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.008+0000 7f15fa000700 1 -- 172.21.15.6:0/3737260013 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f15f0029070 con 0x7f15fc102150 2024-05-09T04:54:03.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.010+0000 7f15fa000700 1 --2- 172.21.15.6:0/3737260013 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f15e40b2880 0x7f15e40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:03.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.010+0000 7f15fa000700 1 -- 172.21.15.6:0/3737260013 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f15f00e3830 con 0x7f15fc102150 2024-05-09T04:54:03.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.010+0000 7f1600e00700 1 --2- 172.21.15.6:0/3737260013 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f15e40b2880 0x7f15e40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:03.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.011+0000 7f1600e00700 1 --2- 172.21.15.6:0/3737260013 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f15e40b2880 0x7f15e40b4d90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f15fc19db10 tx=0x7f15e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:03.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.012+0000 7f15fa000700 1 -- 172.21.15.6:0/3737260013 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f15f0030080 con 0x7f15fc102150 2024-05-09T04:54:03.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:02 smithi186 ceph-mon[34668]: pgmap v472: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:54:03.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:02 smithi169 ceph-mon[34075]: pgmap v472: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:54:03.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.261+0000 7f1602c00700 1 -- 172.21.15.6:0/3737260013 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f15fc104980 con 0x7f15e40b2880 2024-05-09T04:54:03.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.264+0000 7f15fa000700 1 -- 172.21.15.6:0/3737260013 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f15fc104980 con 0x7f15e40b2880 2024-05-09T04:54:03.264 INFO:teuthology.orchestra.run.smithi006.stdout:HOST ADDR LABELS STATUS 2024-05-09T04:54:03.265 INFO:teuthology.orchestra.run.smithi006.stdout:smithi006 172.21.15.6 2024-05-09T04:54:03.265 INFO:teuthology.orchestra.run.smithi006.stdout:smithi169 172.21.15.169 2024-05-09T04:54:03.265 INFO:teuthology.orchestra.run.smithi006.stdout:smithi186 172.21.15.186 2024-05-09T04:54:03.265 INFO:teuthology.orchestra.run.smithi006.stdout:3 hosts in cluster 2024-05-09T04:54:03.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.267+0000 7f1602c00700 1 -- 172.21.15.6:0/3737260013 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f15e40b2880 msgr2=0x7f15e40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:03.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.267+0000 7f1602c00700 1 --2- 172.21.15.6:0/3737260013 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f15e40b2880 0x7f15e40b4d90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f15fc19db10 tx=0x7f15e8009260 comp rx=0 tx=0).stop 2024-05-09T04:54:03.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.267+0000 7f1602c00700 1 -- 172.21.15.6:0/3737260013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15fc102150 msgr2=0x7f15fc19ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:03.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.267+0000 7f1602c00700 1 --2- 172.21.15.6:0/3737260013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15fc102150 0x7f15fc19ca60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f15f00098e0 tx=0x7f15f0009580 comp rx=0 tx=0).stop 2024-05-09T04:54:03.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.268+0000 7f1602c00700 1 -- 172.21.15.6:0/3737260013 shutdown_connections 2024-05-09T04:54:03.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.268+0000 7f1602c00700 1 --2- 172.21.15.6:0/3737260013 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f15e40b2880 0x7f15e40b4d90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:03.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.268+0000 7f1602c00700 1 --2- 172.21.15.6:0/3737260013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15fc105700 0x7f15fc1a3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:03.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.268+0000 7f1602c00700 1 --2- 172.21.15.6:0/3737260013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15fc102150 0x7f15fc19ca60 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:03.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.268+0000 7f1602c00700 1 --2- 172.21.15.6:0/3737260013 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f15fc102aa0 0x7f15fc19cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:03.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.268+0000 7f1602c00700 1 -- 172.21.15.6:0/3737260013 >> 172.21.15.6:0/3737260013 conn(0x7f15fc0fde60 msgr2=0x7f15fc103260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:03.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.268+0000 7f1602c00700 1 -- 172.21.15.6:0/3737260013 shutdown_connections 2024-05-09T04:54:03.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:03.268+0000 7f1602c00700 1 -- 172.21.15.6:0/3737260013 wait complete. 2024-05-09T04:54:04.180 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-09T04:54:04.659 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:54:05.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:04 smithi186 ceph-mon[34668]: from='client.25081 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:54:05.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:04 smithi186 ceph-mon[34668]: pgmap v473: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:05.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:04 smithi169 ceph-mon[34075]: from='client.25081 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:54:05.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:04 smithi169 ceph-mon[34075]: pgmap v473: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:05.139 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:04 smithi006 ceph-mon[29835]: from='client.25081 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:54:05.139 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:04 smithi006 ceph-mon[29835]: pgmap v473: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:07.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:06 smithi186 ceph-mon[34668]: pgmap v474: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:54:07.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:06 smithi169 ceph-mon[34075]: pgmap v474: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:54:07.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.140+0000 7ff5a8400700 1 -- 172.21.15.6:0/4100857040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5a0103580 msgr2=0x7ff5a0103990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:07.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.140+0000 7ff5a8400700 1 --2- 172.21.15.6:0/4100857040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5a0103580 0x7ff5a0103990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff590009990 tx=0x7ff590009ca0 comp rx=0 tx=0).stop 2024-05-09T04:54:07.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.140+0000 7ff5a8400700 1 -- 172.21.15.6:0/4100857040 shutdown_connections 2024-05-09T04:54:07.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.140+0000 7ff5a8400700 1 --2- 172.21.15.6:0/4100857040 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5a0103ed0 0x7ff5a01042e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:07.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.140+0000 7ff5a8400700 1 --2- 172.21.15.6:0/4100857040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5a0102c30 0x7ff5a0103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:07.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.140+0000 7ff5a8400700 1 --2- 172.21.15.6:0/4100857040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5a0103580 0x7ff5a0103990 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:07.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.140+0000 7ff5a8400700 1 -- 172.21.15.6:0/4100857040 >> 172.21.15.6:0/4100857040 conn(0x7ff5a00fe880 msgr2=0x7ff5a0100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:07.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.140+0000 7ff5a8400700 1 -- 172.21.15.6:0/4100857040 shutdown_connections 2024-05-09T04:54:07.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.140+0000 7ff5a8400700 1 -- 172.21.15.6:0/4100857040 wait complete. 2024-05-09T04:54:07.143 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:06 smithi006 ceph-mon[29835]: pgmap v474: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:54:07.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.141+0000 7ff5a8400700 1 Processor -- start 2024-05-09T04:54:07.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.142+0000 7ff5a8400700 1 -- start start 2024-05-09T04:54:07.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.142+0000 7ff5a8400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5a0102c30 0x7ff5a019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:07.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.142+0000 7ff5a8400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5a0103580 0x7ff5a019d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:07.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.142+0000 7ff5a8400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5a0103ed0 0x7ff5a01a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:07.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.142+0000 7ff5a8400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a0111ff0 con 0x7ff5a0103ed0 2024-05-09T04:54:07.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.142+0000 7ff5a8400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a0111e70 con 0x7ff5a0103580 2024-05-09T04:54:07.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.142+0000 7ff5a8400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a0112170 con 0x7ff5a0102c30 2024-05-09T04:54:07.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.142+0000 7ff5a5a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5a0102c30 0x7ff5a019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:07.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.142+0000 7ff5a5a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5a0102c30 0x7ff5a019cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:38254/0 (socket says 172.21.15.6:38254) 2024-05-09T04:54:07.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.142+0000 7ff5a5a00700 1 -- 172.21.15.6:0/4118243136 learned_addr learned my addr 172.21.15.6:0/4118243136 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:07.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.143+0000 7ff5a5000700 1 --2- 172.21.15.6:0/4118243136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5a0103580 0x7ff5a019d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:07.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.143+0000 7ff5a6400700 1 --2- 172.21.15.6:0/4118243136 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5a0103ed0 0x7ff5a01a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:07.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.143+0000 7ff5a5a00700 1 -- 172.21.15.6:0/4118243136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5a0103580 msgr2=0x7ff5a019d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:07.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.143+0000 7ff5a5a00700 1 --2- 172.21.15.6:0/4118243136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5a0103580 0x7ff5a019d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:07.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.143+0000 7ff5a5a00700 1 -- 172.21.15.6:0/4118243136 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5a0103ed0 msgr2=0x7ff5a01a4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:07.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.143+0000 7ff5a5a00700 1 --2- 172.21.15.6:0/4118243136 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5a0103ed0 0x7ff5a01a4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:07.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.143+0000 7ff5a5a00700 1 -- 172.21.15.6:0/4118243136 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a01a4680 con 0x7ff5a0102c30 2024-05-09T04:54:07.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.143+0000 7ff5a5a00700 1 --2- 172.21.15.6:0/4118243136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5a0102c30 0x7ff5a019cb90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff59400e970 tx=0x7ff59400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:07.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.144+0000 7ff59e000700 1 -- 172.21.15.6:0/4118243136 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59400cbd0 con 0x7ff5a0102c30 2024-05-09T04:54:07.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.144+0000 7ff59e000700 1 -- 172.21.15.6:0/4118243136 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff59400cd70 con 0x7ff5a0102c30 2024-05-09T04:54:07.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.144+0000 7ff5a8400700 1 -- 172.21.15.6:0/4118243136 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5a01a4970 con 0x7ff5a0102c30 2024-05-09T04:54:07.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.144+0000 7ff59e000700 1 -- 172.21.15.6:0/4118243136 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff594005680 con 0x7ff5a0102c30 2024-05-09T04:54:07.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.144+0000 7ff5a8400700 1 -- 172.21.15.6:0/4118243136 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5a01a4f00 con 0x7ff5a0102c30 2024-05-09T04:54:07.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.145+0000 7ff5a8400700 1 -- 172.21.15.6:0/4118243136 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5a00512c0 con 0x7ff5a0102c30 2024-05-09T04:54:07.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.148+0000 7ff59e000700 1 -- 172.21.15.6:0/4118243136 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7ff594004830 con 0x7ff5a0102c30 2024-05-09T04:54:07.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.149+0000 7ff59e000700 1 --2- 172.21.15.6:0/4118243136 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff5880b2900 0x7ff5880b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:07.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.149+0000 7ff59e000700 1 -- 172.21.15.6:0/4118243136 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7ff5940e2790 con 0x7ff5a0102c30 2024-05-09T04:54:07.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.150+0000 7ff5a5000700 1 --2- 172.21.15.6:0/4118243136 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff5880b2900 0x7ff5880b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:07.151 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.150+0000 7ff5a5000700 1 --2- 172.21.15.6:0/4118243136 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff5880b2900 0x7ff5880b4e10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff5a019dc40 tx=0x7ff59000b2a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:07.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.154+0000 7ff59e000700 1 -- 172.21.15.6:0/4118243136 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff5940aa0d0 con 0x7ff5a0102c30 2024-05-09T04:54:07.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.400+0000 7ff5a8400700 1 -- 172.21.15.6:0/4118243136 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7ff5a0063c00 con 0x7ff5880b2900 2024-05-09T04:54:07.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.403+0000 7ff59e000700 1 -- 172.21.15.6:0/4118243136 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7ff5a0063c00 con 0x7ff5880b2900 2024-05-09T04:54:07.404 INFO:teuthology.orchestra.run.smithi006.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-09T04:54:07.404 INFO:teuthology.orchestra.run.smithi006.stdout:smithi006 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330000H400BGN 372G No 6m ago Has a FileSystem, LVM detected 2024-05-09T04:54:07.404 INFO:teuthology.orchestra.run.smithi006.stdout:smithi169 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300LN400BGN 372G No 6m ago Has a FileSystem, LVM detected 2024-05-09T04:54:07.404 INFO:teuthology.orchestra.run.smithi006.stdout:smithi186 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CB400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-05-09T04:54:07.407 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.406+0000 7ff5a8400700 1 -- 172.21.15.6:0/4118243136 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff5880b2900 msgr2=0x7ff5880b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:07.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.406+0000 7ff5a8400700 1 --2- 172.21.15.6:0/4118243136 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff5880b2900 0x7ff5880b4e10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff5a019dc40 tx=0x7ff59000b2a0 comp rx=0 tx=0).stop 2024-05-09T04:54:07.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.406+0000 7ff5a8400700 1 -- 172.21.15.6:0/4118243136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5a0102c30 msgr2=0x7ff5a019cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:07.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.406+0000 7ff5a8400700 1 --2- 172.21.15.6:0/4118243136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5a0102c30 0x7ff5a019cb90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff59400e970 tx=0x7ff59400ec80 comp rx=0 tx=0).stop 2024-05-09T04:54:07.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.406+0000 7ff5a8400700 1 -- 172.21.15.6:0/4118243136 shutdown_connections 2024-05-09T04:54:07.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.406+0000 7ff5a8400700 1 --2- 172.21.15.6:0/4118243136 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5a0103ed0 0x7ff5a01a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:07.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.406+0000 7ff5a8400700 1 --2- 172.21.15.6:0/4118243136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5a0102c30 0x7ff5a019cb90 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:07.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.406+0000 7ff5a8400700 1 --2- 172.21.15.6:0/4118243136 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff5880b2900 0x7ff5880b4e10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:07.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.406+0000 7ff5a8400700 1 --2- 172.21.15.6:0/4118243136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5a0103580 0x7ff5a019d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:07.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.406+0000 7ff5a8400700 1 -- 172.21.15.6:0/4118243136 >> 172.21.15.6:0/4118243136 conn(0x7ff5a00fe880 msgr2=0x7ff5a010b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:07.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.407+0000 7ff5a8400700 1 -- 172.21.15.6:0/4118243136 shutdown_connections 2024-05-09T04:54:07.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:07.407+0000 7ff5a8400700 1 -- 172.21.15.6:0/4118243136 wait complete. 2024-05-09T04:54:08.325 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-09T04:54:08.334 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi006.front.sepia.ceph.com 2024-05-09T04:54:08.334 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-09T04:54:08.804 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:54:09.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:08 smithi186 ceph-mon[34668]: from='client.25087 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:54:09.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:08 smithi186 ceph-mon[34668]: pgmap v475: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:54:09.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:08 smithi169 ceph-mon[34075]: from='client.25087 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:54:09.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:08 smithi169 ceph-mon[34075]: pgmap v475: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:54:09.163 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:08 smithi006 ceph-mon[29835]: from='client.25087 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:54:09.163 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:08 smithi006 ceph-mon[29835]: pgmap v475: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:54:11.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:10 smithi186 ceph-mon[34668]: pgmap v476: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:11.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:10 smithi169 ceph-mon[34075]: pgmap v476: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:11.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.131+0000 7f03b2200700 1 -- 172.21.15.6:0/755295022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03ac104e30 msgr2=0x7f03ac107250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:11.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.131+0000 7f03b2200700 1 --2- 172.21.15.6:0/755295022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03ac104e30 0x7f03ac107250 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f039c009990 tx=0x7f039c009ca0 comp rx=0 tx=0).stop 2024-05-09T04:54:11.133 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:10 smithi006 ceph-mon[29835]: pgmap v476: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:11.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.131+0000 7f03b2200700 1 -- 172.21.15.6:0/755295022 shutdown_connections 2024-05-09T04:54:11.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.131+0000 7f03b2200700 1 --2- 172.21.15.6:0/755295022 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03ac10a0f0 0x7f03ac10c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:11.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.131+0000 7f03b2200700 1 --2- 172.21.15.6:0/755295022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03ac104e30 0x7f03ac107250 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:11.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.131+0000 7f03b2200700 1 --2- 172.21.15.6:0/755295022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03ac107790 0x7f03ac109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:11.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.131+0000 7f03b2200700 1 -- 172.21.15.6:0/755295022 >> 172.21.15.6:0/755295022 conn(0x7f03ac0fe890 msgr2=0x7f03ac100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:11.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.131+0000 7f03b2200700 1 -- 172.21.15.6:0/755295022 shutdown_connections 2024-05-09T04:54:11.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.131+0000 7f03b2200700 1 -- 172.21.15.6:0/755295022 wait complete. 2024-05-09T04:54:11.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.132+0000 7f03b2200700 1 Processor -- start 2024-05-09T04:54:11.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.133+0000 7f03b2200700 1 -- start start 2024-05-09T04:54:11.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.133+0000 7f03b2200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03ac104e30 0x7f03ac104690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:11.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.133+0000 7f03b2200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03ac107790 0x7f03ac102d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:11.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.133+0000 7f03b2200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03ac10a0f0 0x7f03ac103280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:11.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.133+0000 7f03b2200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac10fd70 con 0x7f03ac104e30 2024-05-09T04:54:11.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.133+0000 7f03b2200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac10fbf0 con 0x7f03ac107790 2024-05-09T04:54:11.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.133+0000 7f03b2200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac10fef0 con 0x7f03ac10a0f0 2024-05-09T04:54:11.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.134+0000 7f03ab400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03ac104e30 0x7f03ac104690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:11.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.134+0000 7f03aaa00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03ac107790 0x7f03ac102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:11.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.134+0000 7f03ab400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03ac104e30 0x7f03ac104690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49414/0 (socket says 172.21.15.6:49414) 2024-05-09T04:54:11.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.134+0000 7f03ab400700 1 -- 172.21.15.6:0/4116689179 learned_addr learned my addr 172.21.15.6:0/4116689179 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:11.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.134+0000 7f03abe00700 1 --2- 172.21.15.6:0/4116689179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03ac10a0f0 0x7f03ac103280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:11.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.134+0000 7f03aaa00700 1 -- 172.21.15.6:0/4116689179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03ac10a0f0 msgr2=0x7f03ac103280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:11.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.134+0000 7f03aaa00700 1 --2- 172.21.15.6:0/4116689179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03ac10a0f0 0x7f03ac103280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:11.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.134+0000 7f03aaa00700 1 -- 172.21.15.6:0/4116689179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03ac104e30 msgr2=0x7f03ac104690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:11.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.134+0000 7f03aaa00700 1 --2- 172.21.15.6:0/4116689179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03ac104e30 0x7f03ac104690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:11.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.134+0000 7f03aaa00700 1 -- 172.21.15.6:0/4116689179 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03ac1039e0 con 0x7f03ac107790 2024-05-09T04:54:11.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.134+0000 7f03abe00700 1 --2- 172.21.15.6:0/4116689179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03ac10a0f0 0x7f03ac103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:54:11.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.134+0000 7f03ab400700 1 --2- 172.21.15.6:0/4116689179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03ac104e30 0x7f03ac104690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:54:11.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.135+0000 7f03aaa00700 1 --2- 172.21.15.6:0/4116689179 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03ac107790 0x7f03ac102d40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f039800e9f0 tx=0x7f039800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:11.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.135+0000 7f03a3e00700 1 -- 172.21.15.6:0/4116689179 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039800c9c0 con 0x7f03ac107790 2024-05-09T04:54:11.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.135+0000 7f03a3e00700 1 -- 172.21.15.6:0/4116689179 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f039800cb60 con 0x7f03ac107790 2024-05-09T04:54:11.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.135+0000 7f03a3e00700 1 -- 172.21.15.6:0/4116689179 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0398018680 con 0x7f03ac107790 2024-05-09T04:54:11.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.135+0000 7f03b2200700 1 -- 172.21.15.6:0/4116689179 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03ac19a570 con 0x7f03ac107790 2024-05-09T04:54:11.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.136+0000 7f03b2200700 1 -- 172.21.15.6:0/4116689179 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03ac19aa60 con 0x7f03ac107790 2024-05-09T04:54:11.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.137+0000 7f03b2200700 1 -- 172.21.15.6:0/4116689179 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03ac0512c0 con 0x7f03ac107790 2024-05-09T04:54:11.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.141+0000 7f03a3e00700 1 -- 172.21.15.6:0/4116689179 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f0398010430 con 0x7f03ac107790 2024-05-09T04:54:11.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.142+0000 7f03a3e00700 1 --2- 172.21.15.6:0/4116689179 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f03900b2900 0x7f03900b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:11.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.142+0000 7f03a3e00700 1 -- 172.21.15.6:0/4116689179 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f03980e2950 con 0x7f03ac107790 2024-05-09T04:54:11.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.143+0000 7f03ab400700 1 --2- 172.21.15.6:0/4116689179 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f03900b2900 0x7f03900b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:11.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.144+0000 7f03a3e00700 1 -- 172.21.15.6:0/4116689179 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f03980aa290 con 0x7f03ac107790 2024-05-09T04:54:11.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.144+0000 7f03ab400700 1 --2- 172.21.15.6:0/4116689179 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f03900b2900 0x7f03900b4e10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f039c00b2e0 tx=0x7f039c00b7c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:11.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.430+0000 7f03b2200700 1 -- 172.21.15.6:0/4116689179 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f03ac02e8f0 con 0x7f03ac107790 2024-05-09T04:54:11.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.431+0000 7f03a3e00700 1 -- 172.21.15.6:0/4116689179 <== mon.2 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) ==== 76+0+3088 (secure 0 0 0) 0x7f03980a9b90 con 0x7f03ac107790 2024-05-09T04:54:11.432 INFO:teuthology.orchestra.run.smithi006.stdout:e29 2024-05-09T04:54:11.432 INFO:teuthology.orchestra.run.smithi006.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T04:54:11.432 INFO:teuthology.orchestra.run.smithi006.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T04:54:11.433 INFO:teuthology.orchestra.run.smithi006.stdout:legacy client fscid: 1 2024-05-09T04:54:11.433 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:54:11.433 INFO:teuthology.orchestra.run.smithi006.stdout:Filesystem 'cephfs' (1) 2024-05-09T04:54:11.433 INFO:teuthology.orchestra.run.smithi006.stdout:fs_name cephfs 2024-05-09T04:54:11.433 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 29 2024-05-09T04:54:11.433 INFO:teuthology.orchestra.run.smithi006.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-09T04:54:11.433 INFO:teuthology.orchestra.run.smithi006.stdout:created 2024-05-09T04:49:36.920376+0000 2024-05-09T04:54:11.433 INFO:teuthology.orchestra.run.smithi006.stdout:modified 2024-05-09T04:51:07.589004+0000 2024-05-09T04:54:11.433 INFO:teuthology.orchestra.run.smithi006.stdout:tableserver 0 2024-05-09T04:54:11.433 INFO:teuthology.orchestra.run.smithi006.stdout:root 0 2024-05-09T04:54:11.433 INFO:teuthology.orchestra.run.smithi006.stdout:session_timeout 300 2024-05-09T04:54:11.433 INFO:teuthology.orchestra.run.smithi006.stdout:session_autoclose 300 2024-05-09T04:54:11.433 INFO:teuthology.orchestra.run.smithi006.stdout:max_file_size 1099511627776 2024-05-09T04:54:11.433 INFO:teuthology.orchestra.run.smithi006.stdout:max_xattr_size 65536 2024-05-09T04:54:11.433 INFO:teuthology.orchestra.run.smithi006.stdout:required_client_features {} 2024-05-09T04:54:11.434 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure 0 2024-05-09T04:54:11.434 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure_osd_epoch 0 2024-05-09T04:54:11.434 INFO:teuthology.orchestra.run.smithi006.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T04:54:11.434 INFO:teuthology.orchestra.run.smithi006.stdout:max_mds 5 2024-05-09T04:54:11.434 INFO:teuthology.orchestra.run.smithi006.stdout:in 0,1,2,3,4 2024-05-09T04:54:11.434 INFO:teuthology.orchestra.run.smithi006.stdout:up {0=24554,1=24530,2=24512,3=24511,4=24493} 2024-05-09T04:54:11.434 INFO:teuthology.orchestra.run.smithi006.stdout:failed 2024-05-09T04:54:11.434 INFO:teuthology.orchestra.run.smithi006.stdout:damaged 2024-05-09T04:54:11.434 INFO:teuthology.orchestra.run.smithi006.stdout:stopped 2024-05-09T04:54:11.434 INFO:teuthology.orchestra.run.smithi006.stdout:data_pools [3,4] 2024-05-09T04:54:11.434 INFO:teuthology.orchestra.run.smithi006.stdout:metadata_pool 2 2024-05-09T04:54:11.434 INFO:teuthology.orchestra.run.smithi006.stdout:inline_data disabled 2024-05-09T04:54:11.434 INFO:teuthology.orchestra.run.smithi006.stdout:balancer 2024-05-09T04:54:11.434 INFO:teuthology.orchestra.run.smithi006.stdout:bal_rank_mask -1 2024-05-09T04:54:11.435 INFO:teuthology.orchestra.run.smithi006.stdout:standby_count_wanted 0 2024-05-09T04:54:11.435 INFO:teuthology.orchestra.run.smithi006.stdout:qdb_cluster leader: 24554 members: 24493,24511,24530,24554,24512 2024-05-09T04:54:11.435 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.b{0:24554} state up:active seq 7 addr [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T04:54:11.435 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.e{0:24491} state up:standby-replay seq 1 addr [v2:172.21.15.169:6834/3811005052,v1:172.21.15.169:6835/3811005052] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T04:54:11.435 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.g{1:24530} state up:active seq 30 addr [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T04:54:11.435 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.f{1:24481} state up:standby-replay seq 1 addr [v2:172.21.15.186:6834/2268322007,v1:172.21.15.186:6835/2268322007] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T04:54:11.435 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.h{2:24512} state up:active seq 32 addr [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T04:54:11.435 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.d{2:24479} state up:standby-replay seq 1 addr [v2:172.21.15.6:6836/1281521088,v1:172.21.15.6:6837/1281521088] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T04:54:11.435 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.i{3:24511} state up:active seq 32 addr [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T04:54:11.435 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.k{3:24461} state up:standby-replay seq 1 addr [v2:172.21.15.169:6832/133345948,v1:172.21.15.169:6833/133345948] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T04:54:11.435 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.c{4:24493} state up:active seq 33 addr [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T04:54:11.435 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.l{4:24442} state up:standby-replay seq 1 addr [v2:172.21.15.186:6832/2977145206,v1:172.21.15.186:6833/2977145206] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T04:54:11.435 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:54:11.436 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:54:11.436 INFO:teuthology.orchestra.run.smithi006.stdout:Standby daemons: 2024-05-09T04:54:11.436 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:54:11.436 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.j{-1:14361} state up:standby seq 1 addr [v2:172.21.15.6:6834/162094599,v1:172.21.15.6:6835/162094599] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T04:54:11.436 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.a{-1:14400} state up:standby seq 1 addr [v2:172.21.15.6:6840/589294363,v1:172.21.15.6:6841/589294363] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T04:54:11.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.434+0000 7f03b2200700 1 -- 172.21.15.6:0/4116689179 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f03900b2900 msgr2=0x7f03900b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:11.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.434+0000 7f03b2200700 1 --2- 172.21.15.6:0/4116689179 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f03900b2900 0x7f03900b4e10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f039c00b2e0 tx=0x7f039c00b7c0 comp rx=0 tx=0).stop 2024-05-09T04:54:11.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.434+0000 7f03b2200700 1 -- 172.21.15.6:0/4116689179 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03ac107790 msgr2=0x7f03ac102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:11.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.434+0000 7f03b2200700 1 --2- 172.21.15.6:0/4116689179 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03ac107790 0x7f03ac102d40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f039800e9f0 tx=0x7f039800ed00 comp rx=0 tx=0).stop 2024-05-09T04:54:11.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.435+0000 7f03b2200700 1 -- 172.21.15.6:0/4116689179 shutdown_connections 2024-05-09T04:54:11.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.435+0000 7f03b2200700 1 --2- 172.21.15.6:0/4116689179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03ac10a0f0 0x7f03ac103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:11.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.435+0000 7f03b2200700 1 --2- 172.21.15.6:0/4116689179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03ac104e30 0x7f03ac104690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:11.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.435+0000 7f03b2200700 1 --2- 172.21.15.6:0/4116689179 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f03900b2900 0x7f03900b4e10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:11.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.435+0000 7f03b2200700 1 --2- 172.21.15.6:0/4116689179 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03ac107790 0x7f03ac102d40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:11.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.435+0000 7f03b2200700 1 -- 172.21.15.6:0/4116689179 >> 172.21.15.6:0/4116689179 conn(0x7f03ac0fe890 msgr2=0x7f03ac105900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:11.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.435+0000 7f03b2200700 1 -- 172.21.15.6:0/4116689179 shutdown_connections 2024-05-09T04:54:11.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:11.435+0000 7f03b2200700 1 -- 172.21.15.6:0/4116689179 wait complete. 2024-05-09T04:54:11.438 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 29 2024-05-09T04:54:12.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:11 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/4116689179' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T04:54:12.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:11 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/4116689179' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T04:54:12.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:11 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/4116689179' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T04:54:12.411 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-09T04:54:12.877 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:54:13.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:12 smithi186 ceph-mon[34668]: pgmap v477: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:54:13.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:12 smithi169 ceph-mon[34075]: pgmap v477: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:54:13.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:12 smithi006 ceph-mon[29835]: pgmap v477: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:54:14.981 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:14 smithi006 ceph-mon[29835]: pgmap v478: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:15.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:14 smithi169 ceph-mon[34075]: pgmap v478: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:15.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.330+0000 7f2607200700 1 -- 172.21.15.6:0/3744082547 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2608105c20 msgr2=0x7f260810c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:15.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.330+0000 7f2607200700 1 --2- 172.21.15.6:0/3744082547 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2608105c20 0x7f260810c4d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f260000b0a0 tx=0x7f260000b3b0 comp rx=0 tx=0).stop 2024-05-09T04:54:15.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.330+0000 7f2607200700 1 -- 172.21.15.6:0/3744082547 shutdown_connections 2024-05-09T04:54:15.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.330+0000 7f2607200700 1 --2- 172.21.15.6:0/3744082547 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2608105c20 0x7f260810c4d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:15.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.330+0000 7f2607200700 1 --2- 172.21.15.6:0/3744082547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2608104150 0x7f2608104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:15.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.330+0000 7f2607200700 1 --2- 172.21.15.6:0/3744082547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2608105280 0x7f2608105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:15.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.330+0000 7f2607200700 1 -- 172.21.15.6:0/3744082547 >> 172.21.15.6:0/3744082547 conn(0x7f26080ff7b0 msgr2=0x7f2608101c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:15.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.330+0000 7f2607200700 1 -- 172.21.15.6:0/3744082547 shutdown_connections 2024-05-09T04:54:15.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.330+0000 7f2607200700 1 -- 172.21.15.6:0/3744082547 wait complete. 2024-05-09T04:54:15.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.331+0000 7f2607200700 1 Processor -- start 2024-05-09T04:54:15.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.331+0000 7f2607200700 1 -- start start 2024-05-09T04:54:15.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.332+0000 7f2607200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2608104150 0x7f260819c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:15.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.332+0000 7f2607200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2608105280 0x7f260819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:15.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.332+0000 7f2607200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2608105c20 0x7f26081a3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:15.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.332+0000 7f2607200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f260810fbd0 con 0x7f2608105280 2024-05-09T04:54:15.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.332+0000 7f2607200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f260810fa50 con 0x7f2608105c20 2024-05-09T04:54:15.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.332+0000 7f2607200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f260810fd50 con 0x7f2608104150 2024-05-09T04:54:15.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.332+0000 7f2605400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2608105280 0x7f260819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:15.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.332+0000 7f2605400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2608105280 0x7f260819ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49436/0 (socket says 172.21.15.6:49436) 2024-05-09T04:54:15.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.332+0000 7f2605400700 1 -- 172.21.15.6:0/3897900528 learned_addr learned my addr 172.21.15.6:0/3897900528 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:15.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.332+0000 7f2605e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2608104150 0x7f260819c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:15.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.332+0000 7f2606800700 1 --2- 172.21.15.6:0/3897900528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2608105c20 0x7f26081a3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:15.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.333+0000 7f2605400700 1 -- 172.21.15.6:0/3897900528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2608104150 msgr2=0x7f260819c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:15.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.333+0000 7f2605400700 1 --2- 172.21.15.6:0/3897900528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2608104150 0x7f260819c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:15.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.333+0000 7f2605400700 1 -- 172.21.15.6:0/3897900528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2608105c20 msgr2=0x7f26081a3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:15.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.333+0000 7f2605400700 1 --2- 172.21.15.6:0/3897900528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2608105c20 0x7f26081a3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:15.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.333+0000 7f2605400700 1 -- 172.21.15.6:0/3897900528 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26081a4430 con 0x7f2608105280 2024-05-09T04:54:15.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.333+0000 7f2605400700 1 --2- 172.21.15.6:0/3897900528 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2608105280 0x7f260819ce10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f25f400e9a0 tx=0x7f25f400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:15.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.334+0000 7f25fea00700 1 -- 172.21.15.6:0/3897900528 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f400ca10 con 0x7f2608105280 2024-05-09T04:54:15.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.334+0000 7f25fea00700 1 -- 172.21.15.6:0/3897900528 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25f400cbb0 con 0x7f2608105280 2024-05-09T04:54:15.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.334+0000 7f2607200700 1 -- 172.21.15.6:0/3897900528 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26081a4720 con 0x7f2608105280 2024-05-09T04:54:15.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.334+0000 7f25fea00700 1 -- 172.21.15.6:0/3897900528 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f4018680 con 0x7f2608105280 2024-05-09T04:54:15.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.334+0000 7f2607200700 1 -- 172.21.15.6:0/3897900528 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26081a4c60 con 0x7f2608105280 2024-05-09T04:54:15.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.335+0000 7f2607200700 1 -- 172.21.15.6:0/3897900528 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26080512c0 con 0x7f2608105280 2024-05-09T04:54:15.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.338+0000 7f25fea00700 1 -- 172.21.15.6:0/3897900528 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f25f4010430 con 0x7f2608105280 2024-05-09T04:54:15.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.340+0000 7f25fea00700 1 --2- 172.21.15.6:0/3897900528 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f25ec0b2880 0x7f25ec0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:15.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.340+0000 7f25fea00700 1 -- 172.21.15.6:0/3897900528 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f25f40e2a80 con 0x7f2608105280 2024-05-09T04:54:15.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.340+0000 7f2605e00700 1 --2- 172.21.15.6:0/3897900528 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f25ec0b2880 0x7f25ec0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:15.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.341+0000 7f2605e00700 1 --2- 172.21.15.6:0/3897900528 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f25ec0b2880 0x7f25ec0b4d90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2608105180 tx=0x7f25f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:15.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.342+0000 7f25fea00700 1 -- 172.21.15.6:0/3897900528 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f25f40aa3c0 con 0x7f2608105280 2024-05-09T04:54:15.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:14 smithi186 ceph-mon[34668]: pgmap v478: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:15.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.582+0000 7f2607200700 1 -- 172.21.15.6:0/3897900528 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f26081a5080 con 0x7f2608105280 2024-05-09T04:54:15.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.585+0000 7f25fea00700 1 -- 172.21.15.6:0/3897900528 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v89) ==== 56+0+5603 (secure 0 0 0) 0x7f25f40a9cc0 con 0x7f2608105280 2024-05-09T04:54:15.587 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 89 2024-05-09T04:54:15.587 INFO:teuthology.orchestra.run.smithi006.stdout:fsid f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:54:15.587 INFO:teuthology.orchestra.run.smithi006.stdout:created 2024-05-09T04:39:35.287470+0000 2024-05-09T04:54:15.588 INFO:teuthology.orchestra.run.smithi006.stdout:modified 2024-05-09T04:50:57.686719+0000 2024-05-09T04:54:15.588 INFO:teuthology.orchestra.run.smithi006.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-09T04:54:15.588 INFO:teuthology.orchestra.run.smithi006.stdout:crush_version 27 2024-05-09T04:54:15.588 INFO:teuthology.orchestra.run.smithi006.stdout:full_ratio 0.95 2024-05-09T04:54:15.588 INFO:teuthology.orchestra.run.smithi006.stdout:backfillfull_ratio 0.9 2024-05-09T04:54:15.588 INFO:teuthology.orchestra.run.smithi006.stdout:nearfull_ratio 0.85 2024-05-09T04:54:15.589 INFO:teuthology.orchestra.run.smithi006.stdout:require_min_compat_client luminous 2024-05-09T04:54:15.589 INFO:teuthology.orchestra.run.smithi006.stdout:min_compat_client luminous 2024-05-09T04:54:15.589 INFO:teuthology.orchestra.run.smithi006.stdout:require_osd_release squid 2024-05-09T04:54:15.589 INFO:teuthology.orchestra.run.smithi006.stdout:stretch_mode_enabled false 2024-05-09T04:54:15.590 INFO:teuthology.orchestra.run.smithi006.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-09T04:54:15.590 INFO:teuthology.orchestra.run.smithi006.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-09T04:54:15.590 INFO:teuthology.orchestra.run.smithi006.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-05-09T04:54:15.590 INFO:teuthology.orchestra.run.smithi006.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-05-09T04:54:15.590 INFO:teuthology.orchestra.run.smithi006.stdout:max_osd 12 2024-05-09T04:54:15.590 INFO:teuthology.orchestra.run.smithi006.stdout:osd.0 up in weight 1 up_from 8 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.6:6802/1738750914,v1:172.21.15.6:6803/1738750914] [v2:172.21.15.6:6804/1738750914,v1:172.21.15.6:6805/1738750914] exists,up 1b732fd3-e648-4b0d-aece-083bdaaa5d68 2024-05-09T04:54:15.591 INFO:teuthology.orchestra.run.smithi006.stdout:osd.1 up in weight 1 up_from 13 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.169:6800/2552207737,v1:172.21.15.169:6801/2552207737] [v2:172.21.15.169:6802/2552207737,v1:172.21.15.169:6803/2552207737] exists,up 9770c07d-4243-43bf-a0cd-9d2ac558556c 2024-05-09T04:54:15.591 INFO:teuthology.orchestra.run.smithi006.stdout:osd.2 up in weight 1 up_from 18 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6800/231435105,v1:172.21.15.186:6801/231435105] [v2:172.21.15.186:6802/231435105,v1:172.21.15.186:6803/231435105] exists,up 3f50ed89-b2a8-415e-b78d-8510d98d9368 2024-05-09T04:54:15.591 INFO:teuthology.orchestra.run.smithi006.stdout:osd.3 up in weight 1 up_from 26 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.6:6810/2846613703,v1:172.21.15.6:6811/2846613703] [v2:172.21.15.6:6812/2846613703,v1:172.21.15.6:6813/2846613703] exists,up afcbc36b-5669-4044-8dc5-2ebfe7970774 2024-05-09T04:54:15.591 INFO:teuthology.orchestra.run.smithi006.stdout:osd.4 up in weight 1 up_from 32 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.169:6808/1928722453,v1:172.21.15.169:6809/1928722453] [v2:172.21.15.169:6810/1928722453,v1:172.21.15.169:6811/1928722453] exists,up bb84cc5e-0297-4e23-be50-7ee2f7e59665 2024-05-09T04:54:15.591 INFO:teuthology.orchestra.run.smithi006.stdout:osd.5 up in weight 1 up_from 37 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6808/3644445439,v1:172.21.15.186:6809/3644445439] [v2:172.21.15.186:6810/3644445439,v1:172.21.15.186:6811/3644445439] exists,up 830dc7f3-352e-4b01-8d4c-a9b68dcb40d9 2024-05-09T04:54:15.591 INFO:teuthology.orchestra.run.smithi006.stdout:osd.6 up in weight 1 up_from 43 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.6:6818/3068252891,v1:172.21.15.6:6819/3068252891] [v2:172.21.15.6:6820/3068252891,v1:172.21.15.6:6821/3068252891] exists,up 2e8478cb-c8c9-47ce-a3a5-48ffc959ab24 2024-05-09T04:54:15.592 INFO:teuthology.orchestra.run.smithi006.stdout:osd.7 up in weight 1 up_from 48 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.169:6816/1303584711,v1:172.21.15.169:6817/1303584711] [v2:172.21.15.169:6818/1303584711,v1:172.21.15.169:6819/1303584711] exists,up 0f432ee2-42a4-4efa-9233-8e5a2fd65d10 2024-05-09T04:54:15.592 INFO:teuthology.orchestra.run.smithi006.stdout:osd.8 up in weight 1 up_from 54 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6816/3475163771,v1:172.21.15.186:6817/3475163771] [v2:172.21.15.186:6818/3475163771,v1:172.21.15.186:6819/3475163771] exists,up cf5dca81-131b-48e7-9417-d01d4be47316 2024-05-09T04:54:15.592 INFO:teuthology.orchestra.run.smithi006.stdout:osd.9 up in weight 1 up_from 60 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.6:6826/3590990973,v1:172.21.15.6:6827/3590990973] [v2:172.21.15.6:6828/3590990973,v1:172.21.15.6:6829/3590990973] exists,up bca7bedd-1a57-4465-9033-064ca468add2 2024-05-09T04:54:15.592 INFO:teuthology.orchestra.run.smithi006.stdout:osd.10 up in weight 1 up_from 65 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.169:6824/61007341,v1:172.21.15.169:6825/61007341] [v2:172.21.15.169:6826/61007341,v1:172.21.15.169:6827/61007341] exists,up 97b0e058-64f8-42ee-9b99-3ef8e39c8a3d 2024-05-09T04:54:15.592 INFO:teuthology.orchestra.run.smithi006.stdout:osd.11 up in weight 1 up_from 71 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6824/1138940441,v1:172.21.15.186:6825/1138940441] [v2:172.21.15.186:6826/1138940441,v1:172.21.15.186:6827/1138940441] exists,up 7f2220b0-24cf-48b2-95ae-1d66e00e57fc 2024-05-09T04:54:15.592 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 2.a [2,11,8,1] 2024-05-09T04:54:15.593 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 2.e [2,4] 2024-05-09T04:54:15.593 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 2.10 [8,11] 2024-05-09T04:54:15.593 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 3.0 [4,8] 2024-05-09T04:54:15.593 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 3.2 [3,9] 2024-05-09T04:54:15.593 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 3.3 [4,9] 2024-05-09T04:54:15.593 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 3.21 [5,9] 2024-05-09T04:54:15.593 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 4.8 [5,8] 2024-05-09T04:54:15.594 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 4.16 [9,8] 2024-05-09T04:54:15.594 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 4.19 [9,10] 2024-05-09T04:54:15.594 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 4.1f [9,8] 2024-05-09T04:54:15.594 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:0/3742411767 expires 2024-05-10T04:40:55.357057+0000 2024-05-09T04:54:15.594 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:0/2410771702 expires 2024-05-10T04:40:55.357057+0000 2024-05-09T04:54:15.594 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:6801/937713898 expires 2024-05-10T04:40:55.357057+0000 2024-05-09T04:54:15.595 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:6800/937713898 expires 2024-05-10T04:40:55.357057+0000 2024-05-09T04:54:15.595 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:6801/3021503778 expires 2024-05-10T04:40:19.639151+0000 2024-05-09T04:54:15.595 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:6800/3021503778 expires 2024-05-10T04:40:19.639151+0000 2024-05-09T04:54:15.595 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:0/2559510850 expires 2024-05-10T04:40:19.639151+0000 2024-05-09T04:54:15.595 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:0/3810474743 expires 2024-05-10T04:40:19.639151+0000 2024-05-09T04:54:15.595 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:0/4133970555 expires 2024-05-10T04:40:55.357057+0000 2024-05-09T04:54:15.595 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:0/664817743 expires 2024-05-10T04:40:19.639151+0000 2024-05-09T04:54:15.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.588+0000 7f2607200700 1 -- 172.21.15.6:0/3897900528 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f25ec0b2880 msgr2=0x7f25ec0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:15.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.588+0000 7f2607200700 1 --2- 172.21.15.6:0/3897900528 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f25ec0b2880 0x7f25ec0b4d90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2608105180 tx=0x7f25f8009260 comp rx=0 tx=0).stop 2024-05-09T04:54:15.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.588+0000 7f2607200700 1 -- 172.21.15.6:0/3897900528 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2608105280 msgr2=0x7f260819ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:15.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.588+0000 7f2607200700 1 --2- 172.21.15.6:0/3897900528 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2608105280 0x7f260819ce10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f25f400e9a0 tx=0x7f25f400ecb0 comp rx=0 tx=0).stop 2024-05-09T04:54:15.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.588+0000 7f2607200700 1 -- 172.21.15.6:0/3897900528 shutdown_connections 2024-05-09T04:54:15.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.588+0000 7f2607200700 1 --2- 172.21.15.6:0/3897900528 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f25ec0b2880 0x7f25ec0b4d90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:15.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.588+0000 7f2607200700 1 --2- 172.21.15.6:0/3897900528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2608105c20 0x7f26081a3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:15.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.588+0000 7f2607200700 1 --2- 172.21.15.6:0/3897900528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2608104150 0x7f260819c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:15.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.588+0000 7f2607200700 1 --2- 172.21.15.6:0/3897900528 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2608105280 0x7f260819ce10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:15.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.589+0000 7f2607200700 1 -- 172.21.15.6:0/3897900528 >> 172.21.15.6:0/3897900528 conn(0x7f26080ff7b0 msgr2=0x7f26080759f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:15.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.589+0000 7f2607200700 1 -- 172.21.15.6:0/3897900528 shutdown_connections 2024-05-09T04:54:15.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:15.589+0000 7f2607200700 1 -- 172.21.15.6:0/3897900528 wait complete. 2024-05-09T04:54:16.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:15 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/3897900528' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-09T04:54:16.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:15 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/3897900528' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-09T04:54:16.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:15 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/3897900528' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-09T04:54:16.426 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-09T04:54:16.441 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:54:16.909 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:54:17.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:16 smithi006 ceph-mon[29835]: pgmap v479: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:54:17.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:16 smithi186 ceph-mon[34668]: pgmap v479: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:54:17.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:16 smithi169 ceph-mon[34075]: pgmap v479: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:54:19.084 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:18 smithi006 ceph-mon[29835]: pgmap v480: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:54:19.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.334+0000 7fdccba00700 1 -- 172.21.15.6:0/2929019172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcc4075d30 msgr2=0x7fdcc4076140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:19.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.334+0000 7fdccba00700 1 --2- 172.21.15.6:0/2929019172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcc4075d30 0x7fdcc4076140 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdcac0098f0 tx=0x7fdcac009c00 comp rx=0 tx=0).stop 2024-05-09T04:54:19.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.334+0000 7fdccba00700 1 -- 172.21.15.6:0/2929019172 shutdown_connections 2024-05-09T04:54:19.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.334+0000 7fdccba00700 1 --2- 172.21.15.6:0/2929019172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdcc4110ba0 0x7fdcc4113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:19.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.334+0000 7fdccba00700 1 --2- 172.21.15.6:0/2929019172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcc4075d30 0x7fdcc4076140 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:19.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.334+0000 7fdccba00700 1 --2- 172.21.15.6:0/2929019172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcc4076680 0x7fdcc4110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:19.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.334+0000 7fdccba00700 1 -- 172.21.15.6:0/2929019172 >> 172.21.15.6:0/2929019172 conn(0x7fdcc4100040 msgr2=0x7fdcc4102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:19.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.335+0000 7fdccba00700 1 -- 172.21.15.6:0/2929019172 shutdown_connections 2024-05-09T04:54:19.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.335+0000 7fdccba00700 1 -- 172.21.15.6:0/2929019172 wait complete. 2024-05-09T04:54:19.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.336+0000 7fdccba00700 1 Processor -- start 2024-05-09T04:54:19.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.336+0000 7fdccba00700 1 -- start start 2024-05-09T04:54:19.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.337+0000 7fdccba00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcc4075d30 0x7fdcc419d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:19.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.337+0000 7fdccba00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcc4076680 0x7fdcc419d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:19.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.337+0000 7fdccba00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdcc4110ba0 0x7fdcc41a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:19.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.337+0000 7fdccba00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc41166e0 con 0x7fdcc4110ba0 2024-05-09T04:54:19.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.337+0000 7fdccba00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc4116560 con 0x7fdcc4075d30 2024-05-09T04:54:19.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.337+0000 7fdccba00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc4116860 con 0x7fdcc4076680 2024-05-09T04:54:19.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.337+0000 7fdcbbe00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcc4076680 0x7fdcc419d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:19.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.337+0000 7fdcc9000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcc4075d30 0x7fdcc419d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:19.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.337+0000 7fdcbbe00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcc4076680 0x7fdcc419d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:47626/0 (socket says 172.21.15.6:47626) 2024-05-09T04:54:19.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.337+0000 7fdcbbe00700 1 -- 172.21.15.6:0/3571864029 learned_addr learned my addr 172.21.15.6:0/3571864029 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:19.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.337+0000 7fdcc9000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcc4075d30 0x7fdcc419d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:40024/0 (socket says 172.21.15.6:40024) 2024-05-09T04:54:19.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.337+0000 7fdcc9a00700 1 --2- 172.21.15.6:0/3571864029 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdcc4110ba0 0x7fdcc41a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:19.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.337+0000 7fdcbbe00700 1 -- 172.21.15.6:0/3571864029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcc4075d30 msgr2=0x7fdcc419d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:19.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.338+0000 7fdcbbe00700 1 --2- 172.21.15.6:0/3571864029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcc4075d30 0x7fdcc419d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:19.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.338+0000 7fdcbbe00700 1 -- 172.21.15.6:0/3571864029 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdcc4110ba0 msgr2=0x7fdcc41a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:19.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.338+0000 7fdcbbe00700 1 --2- 172.21.15.6:0/3571864029 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdcc4110ba0 0x7fdcc41a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:19.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.338+0000 7fdcbbe00700 1 -- 172.21.15.6:0/3571864029 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc41a1f80 con 0x7fdcc4076680 2024-05-09T04:54:19.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.338+0000 7fdcc9000700 1 --2- 172.21.15.6:0/3571864029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcc4075d30 0x7fdcc419d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:54:19.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.338+0000 7fdcbbe00700 1 --2- 172.21.15.6:0/3571864029 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcc4076680 0x7fdcc419d650 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400e9c0 tx=0x7fdcb400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:19.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.338+0000 7fdcc2000700 1 -- 172.21.15.6:0/3571864029 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4004a30 con 0x7fdcc4076680 2024-05-09T04:54:19.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.338+0000 7fdccba00700 1 -- 172.21.15.6:0/3571864029 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc41a2270 con 0x7fdcc4076680 2024-05-09T04:54:19.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.338+0000 7fdcc2000700 1 -- 172.21.15.6:0/3571864029 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcb400cc70 con 0x7fdcc4076680 2024-05-09T04:54:19.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.338+0000 7fdcc2000700 1 -- 172.21.15.6:0/3571864029 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb40054a0 con 0x7fdcc4076680 2024-05-09T04:54:19.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.338+0000 7fdccba00700 1 -- 172.21.15.6:0/3571864029 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc41a9b90 con 0x7fdcc4076680 2024-05-09T04:54:19.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.340+0000 7fdccba00700 1 -- 172.21.15.6:0/3571864029 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcc40512c0 con 0x7fdcc4076680 2024-05-09T04:54:19.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.342+0000 7fdcc2000700 1 -- 172.21.15.6:0/3571864029 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fdcb4004bd0 con 0x7fdcc4076680 2024-05-09T04:54:19.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.344+0000 7fdcc2000700 1 --2- 172.21.15.6:0/3571864029 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fdca40b2880 0x7fdca40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:19.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.344+0000 7fdcc2000700 1 -- 172.21.15.6:0/3571864029 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fdcb40e26a0 con 0x7fdcc4076680 2024-05-09T04:54:19.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.344+0000 7fdcc9000700 1 --2- 172.21.15.6:0/3571864029 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fdca40b2880 0x7fdca40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:19.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.345+0000 7fdcc9000700 1 --2- 172.21.15.6:0/3571864029 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fdca40b2880 0x7fdca40b4d90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdcac0098c0 tx=0x7fdcac019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:19.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.348+0000 7fdcc2000700 1 -- 172.21.15.6:0/3571864029 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdcb40a9fe0 con 0x7fdcc4076680 2024-05-09T04:54:19.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:18 smithi186 ceph-mon[34668]: pgmap v480: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:54:19.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:18 smithi169 ceph-mon[34075]: pgmap v480: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:54:19.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.637+0000 7fdccba00700 1 -- 172.21.15.6:0/3571864029 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdcc419e410 con 0x7fdcc4076680 2024-05-09T04:54:19.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.638+0000 7fdcc2000700 1 -- 172.21.15.6:0/3571864029 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11927 (secure 0 0 0) 0x7fdcb40a98e0 con 0x7fdcc4076680 2024-05-09T04:54:19.639 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:54:19.640 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/162094599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":162094599},{"type":"v1","addr":"172.21.15.6:6835","nonce":162094599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/589294363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":589294363},{"type":"v1","addr":"172.21.15.6:6841","nonce":589294363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:49:36.920376+0000","modified":"2024-05-09T04:51:07.589004+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24554,"mds_1":24530,"mds_2":24512,"mds_3":24511,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24442":{"gid":24442,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6833/2977145206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2977145206},{"type":"v1","addr":"172.21.15.186:6833","nonce":2977145206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6833/133345948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":133345948},{"type":"v1","addr":"172.21.15.169:6833","nonce":133345948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.6:6837/1281521088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":1281521088},{"type":"v1","addr":"172.21.15.6:6837","nonce":1281521088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2268322007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2268322007},{"type":"v1","addr":"172.21.15.186:6835","nonce":2268322007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/3811005052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3811005052},{"type":"v1","addr":"172.21.15.169:6835","nonce":3811005052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.186:6837/1647607118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1647607118},{"type":"v1","addr":"172.21.15.186:6837","nonce":1647607118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":32,"addr":"172.21.15.186:6839/3954883105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":3954883105},{"type":"v1","addr":"172.21.15.186:6839","nonce":3954883105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.169:6837/1845715294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1845715294},{"type":"v1","addr":"172.21.15.169:6837","nonce":1845715294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.6:6839/2780306492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":2780306492},{"type":"v1","addr":"172.21.15.6:6839","nonce":2780306492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.169:6839/614388771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":614388771},{"type":"v1","addr":"172.21.15.169:6839","nonce":614388771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24493,24511,24530,24554,24512]},"id":1}]} 2024-05-09T04:54:19.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.643+0000 7fdccba00700 1 -- 172.21.15.6:0/3571864029 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fdca40b2880 msgr2=0x7fdca40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:19.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.643+0000 7fdccba00700 1 --2- 172.21.15.6:0/3571864029 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fdca40b2880 0x7fdca40b4d90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdcac0098c0 tx=0x7fdcac019040 comp rx=0 tx=0).stop 2024-05-09T04:54:19.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.643+0000 7fdccba00700 1 -- 172.21.15.6:0/3571864029 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcc4076680 msgr2=0x7fdcc419d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:19.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.643+0000 7fdccba00700 1 --2- 172.21.15.6:0/3571864029 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcc4076680 0x7fdcc419d650 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400e9c0 tx=0x7fdcb400ecd0 comp rx=0 tx=0).stop 2024-05-09T04:54:19.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.643+0000 7fdccba00700 1 -- 172.21.15.6:0/3571864029 shutdown_connections 2024-05-09T04:54:19.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.643+0000 7fdccba00700 1 --2- 172.21.15.6:0/3571864029 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdcc4110ba0 0x7fdcc41a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:19.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.643+0000 7fdccba00700 1 --2- 172.21.15.6:0/3571864029 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fdca40b2880 0x7fdca40b4d90 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:19.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.643+0000 7fdccba00700 1 --2- 172.21.15.6:0/3571864029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcc4075d30 0x7fdcc419d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:19.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.643+0000 7fdccba00700 1 --2- 172.21.15.6:0/3571864029 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcc4076680 0x7fdcc419d650 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:19.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.643+0000 7fdccba00700 1 -- 172.21.15.6:0/3571864029 >> 172.21.15.6:0/3571864029 conn(0x7fdcc4100040 msgr2=0x7fdcc4100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:19.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.643+0000 7fdccba00700 1 -- 172.21.15.6:0/3571864029 shutdown_connections 2024-05-09T04:54:19.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:19.644+0000 7fdccba00700 1 -- 172.21.15.6:0/3571864029 wait complete. 2024-05-09T04:54:19.646 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 29 2024-05-09T04:54:20.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:19 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/3571864029' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:54:20.268 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:19 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/3571864029' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:54:20.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:19 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/3571864029' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:54:20.529 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:54:20.990 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:54:21.147 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:20 smithi006 ceph-mon[29835]: pgmap v481: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:21.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:20 smithi186 ceph-mon[34668]: pgmap v481: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:21.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:20 smithi169 ceph-mon[34075]: pgmap v481: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:23.026 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:22 smithi006 ceph-mon[29835]: pgmap v482: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:54:23.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.279+0000 7f16f1400700 1 -- 172.21.15.6:0/2436495241 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f16ec10e8d0 msgr2=0x7f16ec110db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:23.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.279+0000 7f16f1400700 1 --2- 172.21.15.6:0/2436495241 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f16ec10e8d0 0x7f16ec110db0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f16e400b0a0 tx=0x7f16e400b3b0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.279+0000 7f16f1400700 1 -- 172.21.15.6:0/2436495241 shutdown_connections 2024-05-09T04:54:23.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.279+0000 7f16f1400700 1 --2- 172.21.15.6:0/2436495241 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f16ec10e8d0 0x7f16ec110db0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.279+0000 7f16f1400700 1 --2- 172.21.15.6:0/2436495241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16ec105460 0x7f16ec105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.279+0000 7f16f1400700 1 --2- 172.21.15.6:0/2436495241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16ec105db0 0x7f16ec10e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.279+0000 7f16f1400700 1 -- 172.21.15.6:0/2436495241 >> 172.21.15.6:0/2436495241 conn(0x7f16ec0fedc0 msgr2=0x7f16ec1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:23.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.279+0000 7f16f1400700 1 -- 172.21.15.6:0/2436495241 shutdown_connections 2024-05-09T04:54:23.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.279+0000 7f16f1400700 1 -- 172.21.15.6:0/2436495241 wait complete. 2024-05-09T04:54:23.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.281+0000 7f16f1400700 1 Processor -- start 2024-05-09T04:54:23.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.281+0000 7f16f1400700 1 -- start start 2024-05-09T04:54:23.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.282+0000 7f16f1400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f16ec105460 0x7f16ec19cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:23.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.282+0000 7f16f1400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16ec105db0 0x7f16ec19d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:23.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.282+0000 7f16f1400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16ec10e8d0 0x7f16ec1a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:23.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.282+0000 7f16f1400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f16ec114480 con 0x7f16ec105460 2024-05-09T04:54:23.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.282+0000 7f16f1400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f16ec114300 con 0x7f16ec10e8d0 2024-05-09T04:54:23.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.282+0000 7f16f1400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f16ec114600 con 0x7f16ec105db0 2024-05-09T04:54:23.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.282+0000 7f16e9e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16ec105db0 0x7f16ec19d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:23.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.282+0000 7f16ea800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f16ec105460 0x7f16ec19cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:23.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.282+0000 7f16e9e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16ec105db0 0x7f16ec19d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:47656/0 (socket says 172.21.15.6:47656) 2024-05-09T04:54:23.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.282+0000 7f16e9e00700 1 -- 172.21.15.6:0/2295312567 learned_addr learned my addr 172.21.15.6:0/2295312567 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:23.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.282+0000 7f16eb200700 1 --2- 172.21.15.6:0/2295312567 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16ec10e8d0 0x7f16ec1a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:23.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.283+0000 7f16e9e00700 1 -- 172.21.15.6:0/2295312567 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16ec10e8d0 msgr2=0x7f16ec1a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:23.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.283+0000 7f16e9e00700 1 --2- 172.21.15.6:0/2295312567 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16ec10e8d0 0x7f16ec1a17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.283+0000 7f16e9e00700 1 -- 172.21.15.6:0/2295312567 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f16ec105460 msgr2=0x7f16ec19cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:23.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.283+0000 7f16e9e00700 1 --2- 172.21.15.6:0/2295312567 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f16ec105460 0x7f16ec19cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.283+0000 7f16e9e00700 1 -- 172.21.15.6:0/2295312567 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16ec1a1e60 con 0x7f16ec105db0 2024-05-09T04:54:23.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.283+0000 7f16e9e00700 1 --2- 172.21.15.6:0/2295312567 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16ec105db0 0x7f16ec19d530 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f16d800e9a0 tx=0x7f16d800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:23.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.283+0000 7f16e3400700 1 -- 172.21.15.6:0/2295312567 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d800ca10 con 0x7f16ec105db0 2024-05-09T04:54:23.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.283+0000 7f16e3400700 1 -- 172.21.15.6:0/2295312567 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16d800cbb0 con 0x7f16ec105db0 2024-05-09T04:54:23.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.283+0000 7f16f1400700 1 -- 172.21.15.6:0/2295312567 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16ec1a2150 con 0x7f16ec105db0 2024-05-09T04:54:23.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.284+0000 7f16e3400700 1 -- 172.21.15.6:0/2295312567 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d8018680 con 0x7f16ec105db0 2024-05-09T04:54:23.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.284+0000 7f16f1400700 1 -- 172.21.15.6:0/2295312567 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16ec1a9a70 con 0x7f16ec105db0 2024-05-09T04:54:23.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.285+0000 7f16f1400700 1 -- 172.21.15.6:0/2295312567 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16ec0512c0 con 0x7f16ec105db0 2024-05-09T04:54:23.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.288+0000 7f16e3400700 1 -- 172.21.15.6:0/2295312567 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f16d8010430 con 0x7f16ec105db0 2024-05-09T04:54:23.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.290+0000 7f16e3400700 1 --2- 172.21.15.6:0/2295312567 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f16d00b2900 0x7f16d00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:23.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.290+0000 7f16e3400700 1 -- 172.21.15.6:0/2295312567 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f16d80e2980 con 0x7f16ec105db0 2024-05-09T04:54:23.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.290+0000 7f16ea800700 1 --2- 172.21.15.6:0/2295312567 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f16d00b2900 0x7f16d00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:23.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.291+0000 7f16ea800700 1 --2- 172.21.15.6:0/2295312567 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f16d00b2900 0x7f16d00b4e10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f16dc009c60 tx=0x7f16dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:23.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.291+0000 7f16e3400700 1 -- 172.21.15.6:0/2295312567 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f16d80aa320 con 0x7f16ec105db0 2024-05-09T04:54:23.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:22 smithi186 ceph-mon[34668]: pgmap v482: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:54:23.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:22 smithi169 ceph-mon[34075]: pgmap v482: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:54:23.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.581+0000 7f16f1400700 1 -- 172.21.15.6:0/2295312567 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16ec1a9c40 con 0x7f16ec105db0 2024-05-09T04:54:23.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.582+0000 7f16e3400700 1 -- 172.21.15.6:0/2295312567 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11927 (secure 0 0 0) 0x7f16d80b23a0 con 0x7f16ec105db0 2024-05-09T04:54:23.584 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:54:23.584 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/162094599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":162094599},{"type":"v1","addr":"172.21.15.6:6835","nonce":162094599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/589294363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":589294363},{"type":"v1","addr":"172.21.15.6:6841","nonce":589294363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:49:36.920376+0000","modified":"2024-05-09T04:51:07.589004+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24554,"mds_1":24530,"mds_2":24512,"mds_3":24511,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24442":{"gid":24442,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6833/2977145206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2977145206},{"type":"v1","addr":"172.21.15.186:6833","nonce":2977145206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6833/133345948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":133345948},{"type":"v1","addr":"172.21.15.169:6833","nonce":133345948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.6:6837/1281521088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":1281521088},{"type":"v1","addr":"172.21.15.6:6837","nonce":1281521088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2268322007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2268322007},{"type":"v1","addr":"172.21.15.186:6835","nonce":2268322007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/3811005052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3811005052},{"type":"v1","addr":"172.21.15.169:6835","nonce":3811005052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.186:6837/1647607118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1647607118},{"type":"v1","addr":"172.21.15.186:6837","nonce":1647607118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":32,"addr":"172.21.15.186:6839/3954883105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":3954883105},{"type":"v1","addr":"172.21.15.186:6839","nonce":3954883105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.169:6837/1845715294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1845715294},{"type":"v1","addr":"172.21.15.169:6837","nonce":1845715294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.6:6839/2780306492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":2780306492},{"type":"v1","addr":"172.21.15.6:6839","nonce":2780306492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.169:6839/614388771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":614388771},{"type":"v1","addr":"172.21.15.169:6839","nonce":614388771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24493,24511,24530,24554,24512]},"id":1}]} 2024-05-09T04:54:23.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.587+0000 7f16f1400700 1 -- 172.21.15.6:0/2295312567 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f16d00b2900 msgr2=0x7f16d00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:23.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.587+0000 7f16f1400700 1 --2- 172.21.15.6:0/2295312567 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f16d00b2900 0x7f16d00b4e10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f16dc009c60 tx=0x7f16dc009260 comp rx=0 tx=0).stop 2024-05-09T04:54:23.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.587+0000 7f16f1400700 1 -- 172.21.15.6:0/2295312567 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16ec105db0 msgr2=0x7f16ec19d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:23.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.587+0000 7f16f1400700 1 --2- 172.21.15.6:0/2295312567 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16ec105db0 0x7f16ec19d530 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f16d800e9a0 tx=0x7f16d800ecb0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.587+0000 7f16f1400700 1 -- 172.21.15.6:0/2295312567 shutdown_connections 2024-05-09T04:54:23.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.587+0000 7f16f1400700 1 --2- 172.21.15.6:0/2295312567 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f16d00b2900 0x7f16d00b4e10 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.587+0000 7f16f1400700 1 --2- 172.21.15.6:0/2295312567 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16ec10e8d0 0x7f16ec1a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.587+0000 7f16f1400700 1 --2- 172.21.15.6:0/2295312567 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f16ec105460 0x7f16ec19cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.587+0000 7f16f1400700 1 --2- 172.21.15.6:0/2295312567 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16ec105db0 0x7f16ec19d530 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.587+0000 7f16f1400700 1 -- 172.21.15.6:0/2295312567 >> 172.21.15.6:0/2295312567 conn(0x7f16ec0fedc0 msgr2=0x7f16ec103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:23.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.588+0000 7f16f1400700 1 -- 172.21.15.6:0/2295312567 shutdown_connections 2024-05-09T04:54:23.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:23.588+0000 7f16f1400700 1 -- 172.21.15.6:0/2295312567 wait complete. 2024-05-09T04:54:23.590 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 29 2024-05-09T04:54:24.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:23 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/2295312567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:54:24.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:23 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/2295312567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:54:24.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:23 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/2295312567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:54:24.544 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T04:54:25.033 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:54:25.180 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:24 smithi006 ceph-mon[29835]: pgmap v483: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:25.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:24 smithi186 ceph-mon[34668]: pgmap v483: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:25.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:24 smithi169 ceph-mon[34075]: pgmap v483: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:27.116 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:26 smithi006 ceph-mon[29835]: pgmap v484: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:54:27.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:26 smithi186 ceph-mon[34668]: pgmap v484: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:54:27.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:26 smithi169 ceph-mon[34075]: pgmap v484: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:54:27.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.483+0000 7fb43d000700 1 -- 172.21.15.6:0/43924216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4381045f0 msgr2=0x7fb438104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:27.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.483+0000 7fb43d000700 1 --2- 172.21.15.6:0/43924216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4381045f0 0x7fb438104a00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb4280099c0 tx=0x7fb428009cd0 comp rx=0 tx=0).stop 2024-05-09T04:54:27.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.483+0000 7fb43d000700 1 -- 172.21.15.6:0/43924216 shutdown_connections 2024-05-09T04:54:27.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.483+0000 7fb43d000700 1 --2- 172.21.15.6:0/43924216 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb438109870 0x7fb4381111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:27.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.483+0000 7fb43d000700 1 --2- 172.21.15.6:0/43924216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4381045f0 0x7fb438104a00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:27.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.483+0000 7fb43d000700 1 --2- 172.21.15.6:0/43924216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb438108f20 0x7fb438109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:27.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.483+0000 7fb43d000700 1 -- 172.21.15.6:0/43924216 >> 172.21.15.6:0/43924216 conn(0x7fb438100040 msgr2=0x7fb438102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:27.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.483+0000 7fb43d000700 1 -- 172.21.15.6:0/43924216 shutdown_connections 2024-05-09T04:54:27.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.483+0000 7fb43d000700 1 -- 172.21.15.6:0/43924216 wait complete. 2024-05-09T04:54:27.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.485+0000 7fb43d000700 1 Processor -- start 2024-05-09T04:54:27.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.485+0000 7fb43d000700 1 -- start start 2024-05-09T04:54:27.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.485+0000 7fb43d000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4381045f0 0x7fb43819ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:27.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.485+0000 7fb43d000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb438108f20 0x7fb43819b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:27.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.485+0000 7fb43d000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb438109870 0x7fb43819f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:27.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.485+0000 7fb43d000700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb4381144f0 con 0x7fb438108f20 2024-05-09T04:54:27.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.485+0000 7fb43d000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb438114370 con 0x7fb4381045f0 2024-05-09T04:54:27.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.485+0000 7fb43d000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb438114670 con 0x7fb438109870 2024-05-09T04:54:27.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.486+0000 7fb436800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb438109870 0x7fb43819f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:27.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.486+0000 7fb42fe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb438108f20 0x7fb43819b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:27.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.486+0000 7fb436800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb438109870 0x7fb43819f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:44274/0 (socket says 172.21.15.6:44274) 2024-05-09T04:54:27.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.486+0000 7fb435e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4381045f0 0x7fb43819ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:27.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.486+0000 7fb436800700 1 -- 172.21.15.6:0/1181071384 learned_addr learned my addr 172.21.15.6:0/1181071384 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:27.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.486+0000 7fb42fe00700 1 -- 172.21.15.6:0/1181071384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb438109870 msgr2=0x7fb43819f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:27.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.486+0000 7fb42fe00700 1 --2- 172.21.15.6:0/1181071384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb438109870 0x7fb43819f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:27.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.486+0000 7fb42fe00700 1 -- 172.21.15.6:0/1181071384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4381045f0 msgr2=0x7fb43819ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:27.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.486+0000 7fb42fe00700 1 --2- 172.21.15.6:0/1181071384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4381045f0 0x7fb43819ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:27.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.486+0000 7fb42fe00700 1 -- 172.21.15.6:0/1181071384 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb43819fd30 con 0x7fb438108f20 2024-05-09T04:54:27.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.486+0000 7fb436800700 1 --2- 172.21.15.6:0/1181071384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb438109870 0x7fb43819f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:54:27.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.486+0000 7fb435e00700 1 --2- 172.21.15.6:0/1181071384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4381045f0 0x7fb43819ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:54:27.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.486+0000 7fb42fe00700 1 --2- 172.21.15.6:0/1181071384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb438108f20 0x7fb43819b400 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb420009c20 tx=0x7fb420009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:27.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.487+0000 7fb42ea00700 1 -- 172.21.15.6:0/1181071384 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42001e070 con 0x7fb438108f20 2024-05-09T04:54:27.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.487+0000 7fb43d000700 1 -- 172.21.15.6:0/1181071384 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4381a0020 con 0x7fb438108f20 2024-05-09T04:54:27.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.487+0000 7fb42ea00700 1 -- 172.21.15.6:0/1181071384 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb420019410 con 0x7fb438108f20 2024-05-09T04:54:27.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.487+0000 7fb42ea00700 1 -- 172.21.15.6:0/1181071384 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb420018610 con 0x7fb438108f20 2024-05-09T04:54:27.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.487+0000 7fb43d000700 1 -- 172.21.15.6:0/1181071384 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4380759f0 con 0x7fb438108f20 2024-05-09T04:54:27.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.488+0000 7fb43d000700 1 -- 172.21.15.6:0/1181071384 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4380512c0 con 0x7fb438108f20 2024-05-09T04:54:27.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.494+0000 7fb42ea00700 1 -- 172.21.15.6:0/1181071384 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fb420021070 con 0x7fb438108f20 2024-05-09T04:54:27.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.495+0000 7fb42ea00700 1 --2- 172.21.15.6:0/1181071384 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fb41c0b2880 0x7fb41c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:27.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.495+0000 7fb42ea00700 1 -- 172.21.15.6:0/1181071384 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fb4200e5fb0 con 0x7fb438108f20 2024-05-09T04:54:27.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.496+0000 7fb435e00700 1 --2- 172.21.15.6:0/1181071384 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fb41c0b2880 0x7fb41c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:27.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.497+0000 7fb435e00700 1 --2- 172.21.15.6:0/1181071384 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fb41c0b2880 0x7fb41c0b4d90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb428000c00 tx=0x7fb42801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:27.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.498+0000 7fb42ea00700 1 -- 172.21.15.6:0/1181071384 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb4200ad8f0 con 0x7fb438108f20 2024-05-09T04:54:27.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.745+0000 7fb43d000700 1 -- 172.21.15.6:0/1181071384 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb43819c1c0 con 0x7fb438108f20 2024-05-09T04:54:27.754 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.753+0000 7fb42ea00700 1 -- 172.21.15.6:0/1181071384 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24659 (secure 0 0 0) 0x7fb4200ad1f0 con 0x7fb438108f20 2024-05-09T04:54:27.755 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:54:27.755 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":89,"fsid":"f74d813c-0dbd-11ef-bc97-c7b262605968","created":"2024-05-09T04:39:35.287470+0000","modified":"2024-05-09T04:50:57.686719+0000","last_up_change":"2024-05-09T04:48:16.919842+0000","last_in_change":"2024-05-09T04:47:56.452645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:43:47.538306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:49:28.805447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T04:49:32.970287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T04:49:45.637000+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1b732fd3-e648-4b0d-aece-083bdaaa5d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6803","nonce":1738750914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6805","nonce":1738750914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6809","nonce":1738750914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6807","nonce":1738750914}]},"public_addr":"172.21.15.6:6803/1738750914","cluster_addr":"172.21.15.6:6805/1738750914","heartbeat_back_addr":"172.21.15.6:6809/1738750914","heartbeat_front_addr":"172.21.15.6:6807/1738750914","state":["exists","up"]},{"osd":1,"uuid":"9770c07d-4243-43bf-a0cd-9d2ac558556c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6801","nonce":2552207737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6803","nonce":2552207737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6807","nonce":2552207737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6805","nonce":2552207737}]},"public_addr":"172.21.15.169:6801/2552207737","cluster_addr":"172.21.15.169:6803/2552207737","heartbeat_back_addr":"172.21.15.169:6807/2552207737","heartbeat_front_addr":"172.21.15.169:6805/2552207737","state":["exists","up"]},{"osd":2,"uuid":"3f50ed89-b2a8-415e-b78d-8510d98d9368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6801","nonce":231435105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6803","nonce":231435105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6807","nonce":231435105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6805","nonce":231435105}]},"public_addr":"172.21.15.186:6801/231435105","cluster_addr":"172.21.15.186:6803/231435105","heartbeat_back_addr":"172.21.15.186:6807/231435105","heartbeat_front_addr":"172.21.15.186:6805/231435105","state":["exists","up"]},{"osd":3,"uuid":"afcbc36b-5669-4044-8dc5-2ebfe7970774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6811","nonce":2846613703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6813","nonce":2846613703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6817","nonce":2846613703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6815","nonce":2846613703}]},"public_addr":"172.21.15.6:6811/2846613703","cluster_addr":"172.21.15.6:6813/2846613703","heartbeat_back_addr":"172.21.15.6:6817/2846613703","heartbeat_front_addr":"172.21.15.6:6815/2846613703","state":["exists","up"]},{"osd":4,"uuid":"bb84cc5e-0297-4e23-be50-7ee2f7e59665","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6809","nonce":1928722453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6811","nonce":1928722453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6815","nonce":1928722453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6813","nonce":1928722453}]},"public_addr":"172.21.15.169:6809/1928722453","cluster_addr":"172.21.15.169:6811/1928722453","heartbeat_back_addr":"172.21.15.169:6815/1928722453","heartbeat_front_addr":"172.21.15.169:6813/1928722453","state":["exists","up"]},{"osd":5,"uuid":"830dc7f3-352e-4b01-8d4c-a9b68dcb40d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6809","nonce":3644445439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6811","nonce":3644445439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6815","nonce":3644445439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6813","nonce":3644445439}]},"public_addr":"172.21.15.186:6809/3644445439","cluster_addr":"172.21.15.186:6811/3644445439","heartbeat_back_addr":"172.21.15.186:6815/3644445439","heartbeat_front_addr":"172.21.15.186:6813/3644445439","state":["exists","up"]},{"osd":6,"uuid":"2e8478cb-c8c9-47ce-a3a5-48ffc959ab24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6819","nonce":3068252891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6821","nonce":3068252891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6825","nonce":3068252891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6823","nonce":3068252891}]},"public_addr":"172.21.15.6:6819/3068252891","cluster_addr":"172.21.15.6:6821/3068252891","heartbeat_back_addr":"172.21.15.6:6825/3068252891","heartbeat_front_addr":"172.21.15.6:6823/3068252891","state":["exists","up"]},{"osd":7,"uuid":"0f432ee2-42a4-4efa-9233-8e5a2fd65d10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6817","nonce":1303584711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6819","nonce":1303584711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6823","nonce":1303584711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6821","nonce":1303584711}]},"public_addr":"172.21.15.169:6817/1303584711","cluster_addr":"172.21.15.169:6819/1303584711","heartbeat_back_addr":"172.21.15.169:6823/1303584711","heartbeat_front_addr":"172.21.15.169:6821/1303584711","state":["exists","up"]},{"osd":8,"uuid":"cf5dca81-131b-48e7-9417-d01d4be47316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6817","nonce":3475163771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6819","nonce":3475163771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6823","nonce":3475163771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6821","nonce":3475163771}]},"public_addr":"172.21.15.186:6817/3475163771","cluster_addr":"172.21.15.186:6819/3475163771","heartbeat_back_addr":"172.21.15.186:6823/3475163771","heartbeat_front_addr":"172.21.15.186:6821/3475163771","state":["exists","up"]},{"osd":9,"uuid":"bca7bedd-1a57-4465-9033-064ca468add2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6827","nonce":3590990973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6829","nonce":3590990973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6833","nonce":3590990973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6831","nonce":3590990973}]},"public_addr":"172.21.15.6:6827/3590990973","cluster_addr":"172.21.15.6:6829/3590990973","heartbeat_back_addr":"172.21.15.6:6833/3590990973","heartbeat_front_addr":"172.21.15.6:6831/3590990973","state":["exists","up"]},{"osd":10,"uuid":"97b0e058-64f8-42ee-9b99-3ef8e39c8a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6825","nonce":61007341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6827","nonce":61007341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6831","nonce":61007341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6829","nonce":61007341}]},"public_addr":"172.21.15.169:6825/61007341","cluster_addr":"172.21.15.169:6827/61007341","heartbeat_back_addr":"172.21.15.169:6831/61007341","heartbeat_front_addr":"172.21.15.169:6829/61007341","state":["exists","up"]},{"osd":11,"uuid":"7f2220b0-24cf-48b2-95ae-1d66e00e57fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6825","nonce":1138940441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6827","nonce":1138940441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6831","nonce":1138940441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6829","nonce":1138940441}]},"public_addr":"172.21.15.186:6825/1138940441","cluster_addr":"172.21.15.186:6827/1138940441","heartbeat_back_addr":"172.21.15.186:6831/1138940441","heartbeat_front_addr":"172.21.15.186:6829/1138940441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:53.766295+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:20.195243+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:44.153812+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:20.472086+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:48.415371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:13.795415+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:50.236398+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:18.066623+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:43.781009+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:47:20.084719+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:47:48.219901+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:48:14.965536+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/3742411767":"2024-05-10T04:40:55.357057+0000","172.21.15.6:0/2410771702":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6801/937713898":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6800/937713898":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6801/3021503778":"2024-05-10T04:40:19.639151+0000","172.21.15.6:6800/3021503778":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/2559510850":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/3810474743":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/4133970555":"2024-05-10T04:40:55.357057+0000","172.21.15.6:0/664817743":"2024-05-10T04:40:19.639151+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:54:27.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.756+0000 7fb43d000700 1 -- 172.21.15.6:0/1181071384 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fb41c0b2880 msgr2=0x7fb41c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:27.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.756+0000 7fb43d000700 1 --2- 172.21.15.6:0/1181071384 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fb41c0b2880 0x7fb41c0b4d90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb428000c00 tx=0x7fb42801a040 comp rx=0 tx=0).stop 2024-05-09T04:54:27.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.756+0000 7fb43d000700 1 -- 172.21.15.6:0/1181071384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb438108f20 msgr2=0x7fb43819b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:27.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.756+0000 7fb43d000700 1 --2- 172.21.15.6:0/1181071384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb438108f20 0x7fb43819b400 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb420009c20 tx=0x7fb420009f30 comp rx=0 tx=0).stop 2024-05-09T04:54:27.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.757+0000 7fb43d000700 1 -- 172.21.15.6:0/1181071384 shutdown_connections 2024-05-09T04:54:27.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.757+0000 7fb43d000700 1 --2- 172.21.15.6:0/1181071384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb438109870 0x7fb43819f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:27.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.757+0000 7fb43d000700 1 --2- 172.21.15.6:0/1181071384 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fb41c0b2880 0x7fb41c0b4d90 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:27.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.757+0000 7fb43d000700 1 --2- 172.21.15.6:0/1181071384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4381045f0 0x7fb43819ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:27.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.757+0000 7fb43d000700 1 --2- 172.21.15.6:0/1181071384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb438108f20 0x7fb43819b400 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:27.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.757+0000 7fb43d000700 1 -- 172.21.15.6:0/1181071384 >> 172.21.15.6:0/1181071384 conn(0x7fb438100040 msgr2=0x7fb438102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:27.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.757+0000 7fb43d000700 1 -- 172.21.15.6:0/1181071384 shutdown_connections 2024-05-09T04:54:27.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:27.757+0000 7fb43d000700 1 -- 172.21.15.6:0/1181071384 wait complete. 2024-05-09T04:54:28.139 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:27 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1181071384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:54:28.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:27 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/1181071384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:54:28.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:27 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/1181071384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:54:28.573 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-09T04:49:36.920376+0000', 'modified': '2024-05-09T04:51:07.589004+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24554, 'mds_1': 24530, 'mds_2': 24512, 'mds_3': 24511, 'mds_4': 24493}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24442': {'gid': 24442, 'name': 'l', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6833/2977145206', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6832', 'nonce': 2977145206}, {'type': 'v1', 'addr': '172.21.15.186:6833', 'nonce': 2977145206}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24461': {'gid': 24461, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.169:6833/133345948', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6832', 'nonce': 133345948}, {'type': 'v1', 'addr': '172.21.15.169:6833', 'nonce': 133345948}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'd', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.6:6837/1281521088', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.6:6836', 'nonce': 1281521088}, {'type': 'v1', 'addr': '172.21.15.6:6837', 'nonce': 1281521088}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6835/2268322007', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6834', 'nonce': 2268322007}, {'type': 'v1', 'addr': '172.21.15.186:6835', 'nonce': 2268322007}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24491': {'gid': 24491, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.169:6835/3811005052', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6834', 'nonce': 3811005052}, {'type': 'v1', 'addr': '172.21.15.169:6835', 'nonce': 3811005052}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'c', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 33, 'addr': '172.21.15.186:6837/1647607118', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6836', 'nonce': 1647607118}, {'type': 'v1', 'addr': '172.21.15.186:6837', 'nonce': 1647607118}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'i', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 32, 'addr': '172.21.15.186:6839/3954883105', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6838', 'nonce': 3954883105}, {'type': 'v1', 'addr': '172.21.15.186:6839', 'nonce': 3954883105}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 32, 'addr': '172.21.15.169:6837/1845715294', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6836', 'nonce': 1845715294}, {'type': 'v1', 'addr': '172.21.15.169:6837', 'nonce': 1845715294}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24530': {'gid': 24530, 'name': 'g', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 30, 'addr': '172.21.15.6:6839/2780306492', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.6:6838', 'nonce': 2780306492}, {'type': 'v1', 'addr': '172.21.15.6:6839', 'nonce': 2780306492}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24554': {'gid': 24554, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.169:6839/614388771', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6838', 'nonce': 614388771}, {'type': 'v1', 'addr': '172.21.15.169:6839', 'nonce': 614388771}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24554, 'qdb_cluster': [24493, 24511, 24530, 24554, 24512]} 2024-05-09T04:54:28.573 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-09T04:54:28.573 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.c status 2024-05-09T04:54:29.059 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:54:29.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:28 smithi006 ceph-mon[29835]: pgmap v485: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:54:29.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:28 smithi186 ceph-mon[34668]: pgmap v485: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:54:29.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:28 smithi169 ceph-mon[34075]: pgmap v485: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:54:31.064 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:30 smithi006 ceph-mon[29835]: pgmap v486: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:31.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:30 smithi186 ceph-mon[34668]: pgmap v486: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:31.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:30 smithi169 ceph-mon[34075]: pgmap v486: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:31.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.433+0000 7f602e400700 1 -- 172.21.15.6:0/3730398802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f602810f6c0 msgr2=0x7f602810fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:31.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.433+0000 7f602e400700 1 --2- 172.21.15.6:0/3730398802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f602810f6c0 0x7f602810fad0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f601c009990 tx=0x7f601c009ca0 comp rx=0 tx=0).stop 2024-05-09T04:54:31.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.434+0000 7f602e400700 1 -- 172.21.15.6:0/3730398802 shutdown_connections 2024-05-09T04:54:31.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.434+0000 7f602e400700 1 --2- 172.21.15.6:0/3730398802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f602810b110 0x7f6028117620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:31.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.434+0000 7f602e400700 1 --2- 172.21.15.6:0/3730398802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f602810f6c0 0x7f602810fad0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:31.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.434+0000 7f602e400700 1 --2- 172.21.15.6:0/3730398802 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f602810a7c0 0x7f602810abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:31.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.434+0000 7f602e400700 1 -- 172.21.15.6:0/3730398802 >> 172.21.15.6:0/3730398802 conn(0x7f6028076b00 msgr2=0x7f6028076f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:31.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.434+0000 7f602e400700 1 -- 172.21.15.6:0/3730398802 shutdown_connections 2024-05-09T04:54:31.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.434+0000 7f602e400700 1 -- 172.21.15.6:0/3730398802 wait complete. 2024-05-09T04:54:31.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.435+0000 7f602e400700 1 Processor -- start 2024-05-09T04:54:31.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.435+0000 7f602e400700 1 -- start start 2024-05-09T04:54:31.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.436+0000 7f602e400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f602810a7c0 0x7f60281a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:31.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.436+0000 7f602e400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f602810b110 0x7f60281a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:31.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.436+0000 7f602e400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f602810f6c0 0x7f60281a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:31.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.436+0000 7f602e400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f602811a900 con 0x7f602810f6c0 2024-05-09T04:54:31.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.436+0000 7f602e400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f602811a780 con 0x7f602810a7c0 2024-05-09T04:54:31.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.436+0000 7f602e400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f602811aa80 con 0x7f602810b110 2024-05-09T04:54:31.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.436+0000 7f602da00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f602810f6c0 0x7f60281a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:31.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.436+0000 7f602d000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f602810a7c0 0x7f60281a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:31.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.436+0000 7f6027e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f602810b110 0x7f60281a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:31.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.436+0000 7f6027e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f602810b110 0x7f60281a17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:44292/0 (socket says 172.21.15.6:44292) 2024-05-09T04:54:31.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.436+0000 7f6027e00700 1 -- 172.21.15.6:0/2408770488 learned_addr learned my addr 172.21.15.6:0/2408770488 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:31.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.437+0000 7f6027e00700 1 -- 172.21.15.6:0/2408770488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f602810a7c0 msgr2=0x7f60281a1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:31.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.437+0000 7f6027e00700 1 --2- 172.21.15.6:0/2408770488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f602810a7c0 0x7f60281a1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:31.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.437+0000 7f6027e00700 1 -- 172.21.15.6:0/2408770488 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f602810f6c0 msgr2=0x7f60281a5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:31.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.437+0000 7f6027e00700 1 --2- 172.21.15.6:0/2408770488 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f602810f6c0 0x7f60281a5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:31.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.437+0000 7f6027e00700 1 -- 172.21.15.6:0/2408770488 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60281a6120 con 0x7f602810b110 2024-05-09T04:54:31.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.437+0000 7f6027e00700 1 --2- 172.21.15.6:0/2408770488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f602810b110 0x7f60281a17f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f601400eb30 tx=0x7f601400ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:31.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.437+0000 7f6025600700 1 -- 172.21.15.6:0/2408770488 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601400cbd0 con 0x7f602810b110 2024-05-09T04:54:31.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.437+0000 7f6025600700 1 -- 172.21.15.6:0/2408770488 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60140044d0 con 0x7f602810b110 2024-05-09T04:54:31.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.437+0000 7f6025600700 1 -- 172.21.15.6:0/2408770488 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6014003e30 con 0x7f602810b110 2024-05-09T04:54:31.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.438+0000 7f602e400700 1 -- 172.21.15.6:0/2408770488 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60281a6410 con 0x7f602810b110 2024-05-09T04:54:31.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.438+0000 7f602e400700 1 -- 172.21.15.6:0/2408770488 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60281add30 con 0x7f602810b110 2024-05-09T04:54:31.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.442+0000 7f6025600700 1 -- 172.21.15.6:0/2408770488 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f6014004000 con 0x7f602810b110 2024-05-09T04:54:31.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.442+0000 7f603e4d5b80 1 Processor -- start 2024-05-09T04:54:31.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.443+0000 7f603e4d5b80 1 -- start start 2024-05-09T04:54:31.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.443+0000 7f603e4d5b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56555e5ff450 0x56555e782100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:31.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.443+0000 7f603e4d5b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56555e782640 0x56555e77c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:31.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.443+0000 7f603e4d5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56555e77cb60 0x56555e77cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:31.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.443+0000 7f603e4d5b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56555e69f260 con 0x56555e782640 2024-05-09T04:54:31.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.443+0000 7f603e4d5b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56555e63d090 con 0x56555e5ff450 2024-05-09T04:54:31.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.443+0000 7f603e4d5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56555e69b220 con 0x56555e77cb60 2024-05-09T04:54:31.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.443+0000 7f602d000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56555e782640 0x56555e77c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:31.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.443+0000 7f602da00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56555e5ff450 0x56555e782100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:31.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.444+0000 7f602d000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56555e782640 0x56555e77c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43604/0 (socket says 172.21.15.6:43604) 2024-05-09T04:54:31.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.444+0000 7f602d000700 1 -- 172.21.15.6:0/99696366 learned_addr learned my addr 172.21.15.6:0/99696366 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:31.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.444+0000 7f6027e00700 1 --2- 172.21.15.6:0/99696366 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56555e77cb60 0x56555e77cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:31.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.444+0000 7f6025600700 1 --2- 172.21.15.6:0/2408770488 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f60100b2880 0x7f60100b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:31.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.444+0000 7f602da00700 1 -- 172.21.15.6:0/99696366 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56555e77cb60 msgr2=0x56555e77cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:31.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.444+0000 7f602da00700 1 --2- 172.21.15.6:0/99696366 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56555e77cb60 0x56555e77cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:31.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.444+0000 7f602da00700 1 -- 172.21.15.6:0/99696366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56555e782640 msgr2=0x56555e77c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:31.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.444+0000 7f602da00700 1 --2- 172.21.15.6:0/99696366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56555e782640 0x56555e77c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:31.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.444+0000 7f602da00700 1 -- 172.21.15.6:0/99696366 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56555e51a440 con 0x56555e5ff450 2024-05-09T04:54:31.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.444+0000 7f6025600700 1 -- 172.21.15.6:0/2408770488 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f60140e2980 con 0x7f602810b110 2024-05-09T04:54:31.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.445+0000 7f602da00700 1 --2- 172.21.15.6:0/2408770488 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f60100b2880 0x7f60100b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:31.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.445+0000 7f602da00700 1 --2- 172.21.15.6:0/99696366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56555e5ff450 0x56555e782100 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6020011580 tx=0x7f6020009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1602c1c6c5f530c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:31.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.445+0000 7f601a000700 1 -- 172.21.15.6:0/99696366 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6020003a00 con 0x56555e5ff450 2024-05-09T04:54:31.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.445+0000 7f601a000700 1 -- 172.21.15.6:0/99696366 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f602001a070 con 0x56555e5ff450 2024-05-09T04:54:31.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.445+0000 7f601a000700 1 -- 172.21.15.6:0/99696366 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60200079f0 con 0x56555e5ff450 2024-05-09T04:54:31.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.446+0000 7f603e4d5b80 1 -- 172.21.15.6:0/99696366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56555e5ff450 msgr2=0x56555e782100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:31.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.446+0000 7f603e4d5b80 1 --2- 172.21.15.6:0/99696366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56555e5ff450 0x56555e782100 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6020011580 tx=0x7f6020009920 comp rx=0 tx=0).stop 2024-05-09T04:54:31.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.446+0000 7f602da00700 1 --2- 172.21.15.6:0/2408770488 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f60100b2880 0x7f60100b4d90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6020012010 tx=0x7f6020020470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:31.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.446+0000 7f603e4d5b80 1 -- 172.21.15.6:0/99696366 shutdown_connections 2024-05-09T04:54:31.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.446+0000 7f603e4d5b80 1 --2- 172.21.15.6:0/99696366 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56555e77cb60 0x56555e77cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:31.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.446+0000 7f603e4d5b80 1 --2- 172.21.15.6:0/99696366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56555e5ff450 0x56555e782100 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:31.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.446+0000 7f603e4d5b80 1 --2- 172.21.15.6:0/99696366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56555e782640 0x56555e77c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:31.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.446+0000 7f603e4d5b80 1 -- 172.21.15.6:0/99696366 >> 172.21.15.6:0/99696366 conn(0x56555e643e20 msgr2=0x56555e777050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:31.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.446+0000 7f603e4d5b80 1 -- 172.21.15.6:0/99696366 shutdown_connections 2024-05-09T04:54:31.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.446+0000 7f603e4d5b80 1 -- 172.21.15.6:0/99696366 wait complete. 2024-05-09T04:54:31.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.447+0000 7f603e4d5b80 1 -- start start 2024-05-09T04:54:31.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.447+0000 7f603e4d5b80 1 Processor -- start 2024-05-09T04:54:31.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.448+0000 7f603e4d5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56555e5ff450 0x56555e785ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:31.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.448+0000 7f603e4d5b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56555e77cb60 0x56555e7863e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:31.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.448+0000 7f603e4d5b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56555e782640 0x56555e786920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:31.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.448+0000 7f603e4d5b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56555e7890d0 con 0x56555e77cb60 2024-05-09T04:54:31.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.448+0000 7f603e4d5b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56555e604670 con 0x56555e782640 2024-05-09T04:54:31.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.448+0000 7f603e4d5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56555e789250 con 0x56555e5ff450 2024-05-09T04:54:31.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.448+0000 7f603e4d5b80 10 client.0 _ll_register_callbacks cb 0x56555e636540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:31.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.448+0000 7f602d000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56555e5ff450 0x56555e785ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:31.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.448+0000 7f602d000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56555e5ff450 0x56555e785ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:44302/0 (socket says 172.21.15.6:44302) 2024-05-09T04:54:31.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.448+0000 7f6027e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56555e77cb60 0x56555e7863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:31.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.448+0000 7f602d000700 1 -- 172.21.15.6:0/2618796238 learned_addr learned my addr 172.21.15.6:0/2618796238 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:31.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.448+0000 7f602da00700 1 --2- 172.21.15.6:0/2618796238 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56555e782640 0x56555e786920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:31.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.449+0000 7f602da00700 1 -- 172.21.15.6:0/2618796238 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56555e5ff450 msgr2=0x56555e785ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:31.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.449+0000 7f602da00700 1 --2- 172.21.15.6:0/2618796238 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56555e5ff450 0x56555e785ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:31.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.449+0000 7f602da00700 1 -- 172.21.15.6:0/2618796238 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56555e77cb60 msgr2=0x56555e7863e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:31.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.449+0000 7f602da00700 1 --2- 172.21.15.6:0/2618796238 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56555e77cb60 0x56555e7863e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:31.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.449+0000 7f602da00700 1 -- 172.21.15.6:0/2618796238 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56555e7925a0 con 0x56555e782640 2024-05-09T04:54:31.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.449+0000 7f602da00700 1 --2- 172.21.15.6:0/2618796238 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56555e782640 0x56555e786920 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6020011580 tx=0x7f6020017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2caa0efe6ed5422e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:31.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.449+0000 7f600ea00700 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-05-09T04:54:31.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.450+0000 7f600ea00700 1 -- 172.21.15.6:0/2618796238 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6020030070 con 0x56555e782640 2024-05-09T04:54:31.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.450+0000 7f603e4d5b80 1 -- 172.21.15.6:0/2618796238 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f602001a070 con 0x56555e782640 2024-05-09T04:54:31.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.450+0000 7f600ea00700 1 -- 172.21.15.6:0/2618796238 <== mon.2 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f6020003d80 con 0x56555e782640 2024-05-09T04:54:31.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.450+0000 7f600ea00700 1 -- 172.21.15.6:0/2618796238 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f602003d050 con 0x56555e782640 2024-05-09T04:54:31.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.450+0000 7f603e4d5b80 10 client.25142 fetch_fsmap learned FSMap version 29 2024-05-09T04:54:31.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.450+0000 7f603e4d5b80 1 -- 172.21.15.6:0/2618796238 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56555e793f40 con 0x56555e782640 2024-05-09T04:54:31.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.450+0000 7f600ea00700 1 -- 172.21.15.6:0/2618796238 <== mon.2 v2:172.21.15.169:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f602001a070 con 0x56555e782640 2024-05-09T04:54:31.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.451+0000 7f600ea00700 1 -- 172.21.15.6:0/2618796238 <== mon.2 v2:172.21.15.169:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6020004040 con 0x56555e782640 2024-05-09T04:54:31.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.451+0000 7f603e4d5b80 10 client.25142 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T04:54:31.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.451+0000 7f603e4d5b80 10 client.25142 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T04:54:31.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.451+0000 7f603e4d5b80 20 client.25142 populate_metadata read hostname 'smithi006' 2024-05-09T04:54:31.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.451+0000 7f603e4d5b80 1 --2- 172.21.15.6:0/2618796238 >> [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] conn(0x56555e796220 0x56555e797690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:31.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.451+0000 7f603e4d5b80 4 client.25142 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:54:31.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.451+0000 7f603e4d5b80 1 -- 172.21.15.6:0/2618796238 --> [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56555e748820 con 0x56555e796220 2024-05-09T04:54:31.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.451+0000 7f602d000700 1 --2- 172.21.15.6:0/2618796238 >> [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] conn(0x56555e796220 0x56555e797690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:31.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.452+0000 7f602d000700 1 --2- 172.21.15.6:0/2618796238 >> [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] conn(0x56555e796220 0x56555e797690 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9d792b62863c5f0b server_cookie=bb4e049e10ecfcfd in_seq=0 out_seq=0 2024-05-09T04:54:31.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.452+0000 7f600ea00700 10 client.25142 ms_handle_connect on v2:172.21.15.186:6836/1647607118 2024-05-09T04:54:31.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.453+0000 7f600ea00700 1 -- 172.21.15.6:0/2618796238 <== mds.4 v2:172.21.15.186:6836/1647607118 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56555e748820 con 0x56555e796220 2024-05-09T04:54:31.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.453+0000 7f600ea00700 10 client.25142 handle_command_reply: tid=0 2024-05-09T04:54:31.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.453+0000 7f603e4d5b80 1 client.25142 shutdown 2024-05-09T04:54:31.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.453+0000 7f603e4d5b80 1 -- 172.21.15.6:0/2618796238 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56555e782640 msgr2=0x56555e786920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:31.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.453+0000 7f603e4d5b80 1 --2- 172.21.15.6:0/2618796238 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56555e782640 0x56555e786920 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6020011580 tx=0x7f6020017000 comp rx=0 tx=0).stop 2024-05-09T04:54:31.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.454+0000 7f603e4d5b80 1 -- 172.21.15.6:0/2618796238 shutdown_connections 2024-05-09T04:54:31.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.454+0000 7f603e4d5b80 1 --2- 172.21.15.6:0/2618796238 >> [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] conn(0x56555e796220 0x56555e797690 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:31.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.454+0000 7f603e4d5b80 1 --2- 172.21.15.6:0/2618796238 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56555e782640 0x56555e786920 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:31.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.454+0000 7f603e4d5b80 1 --2- 172.21.15.6:0/2618796238 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56555e5ff450 0x56555e785ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:31.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.454+0000 7f603e4d5b80 1 --2- 172.21.15.6:0/2618796238 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56555e77cb60 0x56555e7863e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:31.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.454+0000 7f603e4d5b80 1 -- 172.21.15.6:0/2618796238 >> 172.21.15.6:0/2618796238 conn(0x56555e643e20 msgr2=0x56555e777010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:31.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.454+0000 7f600ea00700 0 client.25142 ms_handle_reset on v2:172.21.15.186:6836/1647607118 2024-05-09T04:54:31.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.454+0000 7f603e4d5b80 1 -- 172.21.15.6:0/2618796238 shutdown_connections 2024-05-09T04:54:31.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.454+0000 7f603e4d5b80 1 -- 172.21.15.6:0/2618796238 wait complete. 2024-05-09T04:54:31.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.454+0000 7f603e4d5b80 20 client.25142 trim_cache size 0 max 16384 2024-05-09T04:54:31.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.478+0000 7f603e4d5b80 1 Processor -- start 2024-05-09T04:54:31.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.478+0000 7f603e4d5b80 1 -- start start 2024-05-09T04:54:31.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.478+0000 7f603e4d5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56555e5ff450 0x56555e791be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:31.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.478+0000 7f603e4d5b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56555e77cb60 0x56555e786410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:31.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.479+0000 7f603e4d5b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56555e782640 0x56555e776cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:31.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.479+0000 7f603e4d5b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56555e7959d0 con 0x56555e77cb60 2024-05-09T04:54:31.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.479+0000 7f603e4d5b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56555e7960a0 con 0x56555e782640 2024-05-09T04:54:31.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.479+0000 7f603e4d5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56555e796770 con 0x56555e5ff450 2024-05-09T04:54:31.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.479+0000 7f6027e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56555e77cb60 0x56555e786410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:31.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.479+0000 7f6027e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56555e77cb60 0x56555e786410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43618/0 (socket says 172.21.15.6:43618) 2024-05-09T04:54:31.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.479+0000 7f6027e00700 1 -- 172.21.15.6:0/1858587774 learned_addr learned my addr 172.21.15.6:0/1858587774 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:31.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.479+0000 7f602d000700 1 --2- 172.21.15.6:0/1858587774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56555e5ff450 0x56555e791be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:31.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.479+0000 7f602da00700 1 --2- 172.21.15.6:0/1858587774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56555e782640 0x56555e776cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:31.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.479+0000 7f6027e00700 1 -- 172.21.15.6:0/1858587774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56555e5ff450 msgr2=0x56555e791be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:31.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.479+0000 7f6027e00700 1 --2- 172.21.15.6:0/1858587774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56555e5ff450 0x56555e791be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:31.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.479+0000 7f6027e00700 1 -- 172.21.15.6:0/1858587774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56555e782640 msgr2=0x56555e776cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:31.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.479+0000 7f6027e00700 1 --2- 172.21.15.6:0/1858587774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56555e782640 0x56555e776cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:31.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.480+0000 7f6027e00700 1 -- 172.21.15.6:0/1858587774 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56555e777c20 con 0x56555e77cb60 2024-05-09T04:54:31.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.480+0000 7f602da00700 1 --2- 172.21.15.6:0/1858587774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56555e782640 0x56555e776cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:54:31.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.480+0000 7f6027e00700 1 --2- 172.21.15.6:0/1858587774 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56555e77cb60 0x56555e786410 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f60140a5c40 tx=0x7f60140a5ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce269b15d8887a4f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:31.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.480+0000 7f6019600700 1 -- 172.21.15.6:0/1858587774 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60140a9510 con 0x56555e77cb60 2024-05-09T04:54:31.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.480+0000 7f6019600700 1 -- 172.21.15.6:0/1858587774 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60140b2370 con 0x56555e77cb60 2024-05-09T04:54:31.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.480+0000 7f6019600700 1 -- 172.21.15.6:0/1858587774 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60140a5350 con 0x56555e77cb60 2024-05-09T04:54:31.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.481+0000 7f603e4d5b80 1 -- 172.21.15.6:0/1858587774 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56555e77cb60 msgr2=0x56555e786410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:31.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.481+0000 7f603e4d5b80 1 --2- 172.21.15.6:0/1858587774 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56555e77cb60 0x56555e786410 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f60140a5c40 tx=0x7f60140a5ba0 comp rx=0 tx=0).stop 2024-05-09T04:54:31.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.481+0000 7f603e4d5b80 1 -- 172.21.15.6:0/1858587774 shutdown_connections 2024-05-09T04:54:31.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.481+0000 7f603e4d5b80 1 --2- 172.21.15.6:0/1858587774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56555e782640 0x56555e776cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:31.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.481+0000 7f603e4d5b80 1 --2- 172.21.15.6:0/1858587774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56555e5ff450 0x56555e791be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:31.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.481+0000 7f603e4d5b80 1 --2- 172.21.15.6:0/1858587774 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56555e77cb60 0x56555e786410 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:31.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.481+0000 7f603e4d5b80 1 -- 172.21.15.6:0/1858587774 >> 172.21.15.6:0/1858587774 conn(0x56555e643e20 msgr2=0x56555e7914d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:31.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.481+0000 7f603e4d5b80 1 -- 172.21.15.6:0/1858587774 shutdown_connections 2024-05-09T04:54:31.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.481+0000 7f603e4d5b80 1 -- 172.21.15.6:0/1858587774 wait complete. 2024-05-09T04:54:31.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.482+0000 7f603e4d5b80 1 -- start start 2024-05-09T04:54:31.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.482+0000 7f603e4d5b80 1 Processor -- start 2024-05-09T04:54:31.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.483+0000 7f603e4d5b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56555e5ff450 0x56555e776730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:31.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.483+0000 7f603e4d5b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56555e77cb60 0x56555e776c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:31.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.483+0000 7f603e4d5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56555e782640 0x56555e79f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:31.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.483+0000 7f603e4d5b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56555e797510 con 0x56555e77cb60 2024-05-09T04:54:31.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.483+0000 7f603e4d5b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56555e796e40 con 0x56555e5ff450 2024-05-09T04:54:31.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.483+0000 7f603e4d5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56555e797be0 con 0x56555e782640 2024-05-09T04:54:31.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.483+0000 7f603e4d5b80 10 client.0 _ll_register_callbacks cb 0x56555e636540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:31.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.483+0000 7f6027e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56555e77cb60 0x56555e776c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:31.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.483+0000 7f6027e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56555e77cb60 0x56555e776c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43634/0 (socket says 172.21.15.6:43634) 2024-05-09T04:54:31.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.483+0000 7f602d000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56555e5ff450 0x56555e776730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:31.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.483+0000 7f602da00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56555e782640 0x56555e79f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:31.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.483+0000 7f6027e00700 1 -- 172.21.15.6:0/630378793 learned_addr learned my addr 172.21.15.6:0/630378793 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:31.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.484+0000 7f602da00700 1 -- 172.21.15.6:0/630378793 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56555e5ff450 msgr2=0x56555e776730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:31.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.484+0000 7f602da00700 1 --2- 172.21.15.6:0/630378793 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56555e5ff450 0x56555e776730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:31.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.484+0000 7f602da00700 1 -- 172.21.15.6:0/630378793 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56555e77cb60 msgr2=0x56555e776c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:31.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.484+0000 7f602da00700 1 --2- 172.21.15.6:0/630378793 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56555e77cb60 0x56555e776c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:31.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.484+0000 7f602da00700 1 -- 172.21.15.6:0/630378793 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56555e777550 con 0x56555e782640 2024-05-09T04:54:31.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.484+0000 7f602d000700 1 --2- 172.21.15.6:0/630378793 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56555e5ff450 0x56555e776730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:54:31.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.484+0000 7f602da00700 1 --2- 172.21.15.6:0/630378793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56555e782640 0x56555e79f200 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6020004440 tx=0x7f6020003ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d92072c1de1de149 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:31.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.484+0000 7f600ea00700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-09T04:54:31.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.484+0000 7f600ea00700 1 -- 172.21.15.6:0/630378793 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6020030070 con 0x56555e782640 2024-05-09T04:54:31.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.484+0000 7f603e4d5b80 1 -- 172.21.15.6:0/630378793 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60140b2370 con 0x56555e782640 2024-05-09T04:54:31.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.484+0000 7f600ea00700 1 -- 172.21.15.6:0/630378793 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f60200047d0 con 0x56555e782640 2024-05-09T04:54:31.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.485+0000 7f600ea00700 1 -- 172.21.15.6:0/630378793 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6020004a10 con 0x56555e782640 2024-05-09T04:54:31.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.485+0000 7f603e4d5b80 10 client.25135 fetch_fsmap learned FSMap version 29 2024-05-09T04:54:31.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.485+0000 7f603e4d5b80 1 -- 172.21.15.6:0/630378793 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56555e801c30 con 0x56555e782640 2024-05-09T04:54:31.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.485+0000 7f600ea00700 1 -- 172.21.15.6:0/630378793 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60140b2370 con 0x56555e782640 2024-05-09T04:54:31.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.485+0000 7f600ea00700 1 -- 172.21.15.6:0/630378793 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6020040030 con 0x56555e782640 2024-05-09T04:54:31.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.486+0000 7f603e4d5b80 10 client.25135 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T04:54:31.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.486+0000 7f603e4d5b80 10 client.25135 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T04:54:31.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.486+0000 7f603e4d5b80 20 client.25135 populate_metadata read hostname 'smithi006' 2024-05-09T04:54:31.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.486+0000 7f603e4d5b80 1 --2- 172.21.15.6:0/630378793 >> [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] conn(0x56555e803f80 0x56555e8053f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:31.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.486+0000 7f603e4d5b80 4 client.25135 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T04:54:31.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.486+0000 7f603e4d5b80 1 -- 172.21.15.6:0/630378793 --> [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] -- command(tid 0: {"prefix": "status"}) -- 0x56555e6acc40 con 0x56555e803f80 2024-05-09T04:54:31.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.486+0000 7f602d000700 1 --2- 172.21.15.6:0/630378793 >> [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] conn(0x56555e803f80 0x56555e8053f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:31.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.487+0000 7f602d000700 1 --2- 172.21.15.6:0/630378793 >> [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] conn(0x56555e803f80 0x56555e8053f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1678ad07bf6d7504 server_cookie=4d940f43309d088a in_seq=0 out_seq=0 2024-05-09T04:54:31.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.487+0000 7f600ea00700 10 client.25135 ms_handle_connect on v2:172.21.15.186:6836/1647607118 2024-05-09T04:54:31.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.487+0000 7f600ea00700 1 -- 172.21.15.6:0/630378793 <== mds.4 v2:172.21.15.186:6836/1647607118 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56555e6acc40 con 0x56555e803f80 2024-05-09T04:54:31.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.487+0000 7f600ea00700 10 client.25135 handle_command_reply: tid=0 2024-05-09T04:54:31.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.487+0000 7f603e4d5b80 1 client.25135 shutdown 2024-05-09T04:54:31.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.487+0000 7f603e4d5b80 1 -- 172.21.15.6:0/630378793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56555e782640 msgr2=0x56555e79f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:31.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.487+0000 7f603e4d5b80 1 --2- 172.21.15.6:0/630378793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56555e782640 0x56555e79f200 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6020004440 tx=0x7f6020003ef0 comp rx=0 tx=0).stop 2024-05-09T04:54:31.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.488+0000 7f603e4d5b80 1 -- 172.21.15.6:0/630378793 shutdown_connections 2024-05-09T04:54:31.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.488+0000 7f603e4d5b80 1 --2- 172.21.15.6:0/630378793 >> [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] conn(0x56555e803f80 0x56555e8053f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:31.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.488+0000 7f603e4d5b80 1 --2- 172.21.15.6:0/630378793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56555e782640 0x56555e79f200 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:31.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.488+0000 7f603e4d5b80 1 --2- 172.21.15.6:0/630378793 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56555e5ff450 0x56555e776730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:31.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.488+0000 7f603e4d5b80 1 --2- 172.21.15.6:0/630378793 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56555e77cb60 0x56555e776c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:31.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.488+0000 7f603e4d5b80 1 -- 172.21.15.6:0/630378793 >> 172.21.15.6:0/630378793 conn(0x56555e643e20 msgr2=0x56555e7914d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:31.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.488+0000 7f600ea00700 0 client.25135 ms_handle_reset on v2:172.21.15.186:6836/1647607118 2024-05-09T04:54:31.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.488+0000 7f603e4d5b80 1 -- 172.21.15.6:0/630378793 shutdown_connections 2024-05-09T04:54:31.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.488+0000 7f603e4d5b80 1 -- 172.21.15.6:0/630378793 wait complete. 2024-05-09T04:54:31.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.488+0000 7f603e4d5b80 20 client.25135 trim_cache size 0 max 16384 2024-05-09T04:54:31.491 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-09T04:54:31.491 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "f74d813c-0dbd-11ef-bc97-c7b262605968", 2024-05-09T04:54:31.491 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 4, 2024-05-09T04:54:31.491 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24493, 2024-05-09T04:54:31.491 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-05-09T04:54:31.491 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-05-09T04:54:31.491 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-05-09T04:54:31.491 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 208.73091498599999, 2024-05-09T04:54:31.492 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 29, 2024-05-09T04:54:31.492 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 89, 2024-05-09T04:54:31.492 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 89, 2024-05-09T04:54:31.492 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 336.01485848200002 2024-05-09T04:54:31.492 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-09T04:54:31.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.489+0000 7f602e400700 1 -- 172.21.15.6:0/2408770488 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f60100b2880 msgr2=0x7f60100b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:31.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.489+0000 7f602e400700 1 --2- 172.21.15.6:0/2408770488 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f60100b2880 0x7f60100b4d90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6020012010 tx=0x7f6020020470 comp rx=0 tx=0).stop 2024-05-09T04:54:31.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.489+0000 7f602e400700 1 -- 172.21.15.6:0/2408770488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f602810b110 msgr2=0x7f60281a17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:31.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.489+0000 7f602e400700 1 --2- 172.21.15.6:0/2408770488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f602810b110 0x7f60281a17f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f601400eb30 tx=0x7f601400ee40 comp rx=0 tx=0).stop 2024-05-09T04:54:31.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.489+0000 7f602e400700 1 -- 172.21.15.6:0/2408770488 shutdown_connections 2024-05-09T04:54:31.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.489+0000 7f602e400700 1 --2- 172.21.15.6:0/2408770488 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f602810f6c0 0x7f60281a5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:31.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.489+0000 7f602e400700 1 --2- 172.21.15.6:0/2408770488 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f60100b2880 0x7f60100b4d90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:31.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.489+0000 7f602e400700 1 --2- 172.21.15.6:0/2408770488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f602810a7c0 0x7f60281a1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:31.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.489+0000 7f602e400700 1 --2- 172.21.15.6:0/2408770488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f602810b110 0x7f60281a17f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:31.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.489+0000 7f602e400700 1 -- 172.21.15.6:0/2408770488 >> 172.21.15.6:0/2408770488 conn(0x7f6028076b00 msgr2=0x7f6028108860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:31.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.489+0000 7f602e400700 1 -- 172.21.15.6:0/2408770488 shutdown_connections 2024-05-09T04:54:31.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:31.489+0000 7f602e400700 1 -- 172.21.15.6:0/2408770488 wait complete. 2024-05-09T04:54:32.306 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-09T04:54:32.771 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:54:33.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:32 smithi006 ceph-mon[29835]: pgmap v487: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:54:33.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:32 smithi186 ceph-mon[34668]: pgmap v487: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:54:33.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:32 smithi169 ceph-mon[34075]: pgmap v487: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:54:35.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:34 smithi006 ceph-mon[29835]: pgmap v488: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:35.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.202+0000 7f3c11400700 1 -- 172.21.15.6:0/3996804208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c0c108560 msgr2=0x7f3c0c10fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:35.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.202+0000 7f3c11400700 1 --2- 172.21.15.6:0/3996804208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c0c108560 0x7f3c0c10fec0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400b0d0 tx=0x7f3c0400b3e0 comp rx=0 tx=0).stop 2024-05-09T04:54:35.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.202+0000 7f3c11400700 1 -- 172.21.15.6:0/3996804208 shutdown_connections 2024-05-09T04:54:35.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.202+0000 7f3c11400700 1 --2- 172.21.15.6:0/3996804208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c0c108560 0x7f3c0c10fec0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:35.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.202+0000 7f3c11400700 1 --2- 172.21.15.6:0/3996804208 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c0c1032e0 0x7f3c0c1036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:35.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.202+0000 7f3c11400700 1 --2- 172.21.15.6:0/3996804208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c0c107c10 0x7f3c0c108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:35.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.202+0000 7f3c11400700 1 -- 172.21.15.6:0/3996804208 >> 172.21.15.6:0/3996804208 conn(0x7f3c0c078ac0 msgr2=0x7f3c0c078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:35.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.202+0000 7f3c11400700 1 -- 172.21.15.6:0/3996804208 shutdown_connections 2024-05-09T04:54:35.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.202+0000 7f3c11400700 1 -- 172.21.15.6:0/3996804208 wait complete. 2024-05-09T04:54:35.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.203+0000 7f3c11400700 1 Processor -- start 2024-05-09T04:54:35.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.203+0000 7f3c11400700 1 -- start start 2024-05-09T04:54:35.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.204+0000 7f3c11400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c0c1032e0 0x7f3c0c100430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:35.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.204+0000 7f3c11400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c0c107c10 0x7f3c0c100970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:35.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.204+0000 7f3c11400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c0c108560 0x7f3c0c100eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:35.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.204+0000 7f3c11400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c1131e0 con 0x7f3c0c107c10 2024-05-09T04:54:35.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.204+0000 7f3c11400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c113060 con 0x7f3c0c108560 2024-05-09T04:54:35.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.204+0000 7f3c11400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c113360 con 0x7f3c0c1032e0 2024-05-09T04:54:35.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.204+0000 7f3c0be00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c0c1032e0 0x7f3c0c100430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:35.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.204+0000 7f3c0b400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c0c107c10 0x7f3c0c100970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:35.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.204+0000 7f3c0be00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c0c1032e0 0x7f3c0c100430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:44320/0 (socket says 172.21.15.6:44320) 2024-05-09T04:54:35.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.204+0000 7f3c0be00700 1 -- 172.21.15.6:0/149225708 learned_addr learned my addr 172.21.15.6:0/149225708 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:35.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.204+0000 7f3c10a00700 1 --2- 172.21.15.6:0/149225708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c0c108560 0x7f3c0c100eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:35.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.205+0000 7f3c10a00700 1 -- 172.21.15.6:0/149225708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c0c1032e0 msgr2=0x7f3c0c100430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:35.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.205+0000 7f3c10a00700 1 --2- 172.21.15.6:0/149225708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c0c1032e0 0x7f3c0c100430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:35.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.205+0000 7f3c10a00700 1 -- 172.21.15.6:0/149225708 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c0c107c10 msgr2=0x7f3c0c100970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:35.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.205+0000 7f3c10a00700 1 --2- 172.21.15.6:0/149225708 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c0c107c10 0x7f3c0c100970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:35.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.205+0000 7f3c10a00700 1 -- 172.21.15.6:0/149225708 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c0c1a8fe0 con 0x7f3c0c108560 2024-05-09T04:54:35.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.205+0000 7f3c10a00700 1 --2- 172.21.15.6:0/149225708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c0c108560 0x7f3c0c100eb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400b770 tx=0x7f3c04007980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:35.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.206+0000 7f3c08c00700 1 -- 172.21.15.6:0/149225708 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0401f070 con 0x7f3c0c108560 2024-05-09T04:54:35.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.206+0000 7f3c08c00700 1 -- 172.21.15.6:0/149225708 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c04007dc0 con 0x7f3c0c108560 2024-05-09T04:54:35.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.206+0000 7f3c11400700 1 -- 172.21.15.6:0/149225708 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c0c1a9270 con 0x7f3c0c108560 2024-05-09T04:54:35.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.206+0000 7f3c08c00700 1 -- 172.21.15.6:0/149225708 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0401b430 con 0x7f3c0c108560 2024-05-09T04:54:35.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.206+0000 7f3c11400700 1 -- 172.21.15.6:0/149225708 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c0c1a9720 con 0x7f3c0c108560 2024-05-09T04:54:35.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.211+0000 7f3c08c00700 1 -- 172.21.15.6:0/149225708 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f3c04022070 con 0x7f3c0c108560 2024-05-09T04:54:35.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.212+0000 7f3c214d4b80 1 Processor -- start 2024-05-09T04:54:35.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.212+0000 7f3c214d4b80 1 -- start start 2024-05-09T04:54:35.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.213+0000 7f3c214d4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3d24af450 0x55e3d2631e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:35.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.213+0000 7f3c214d4b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e3d2632360 0x55e3d262c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:35.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.213+0000 7f3c214d4b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3d262c6b0 0x55e3d262cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:35.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.213+0000 7f3c214d4b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55e3d254f200 con 0x55e3d2632360 2024-05-09T04:54:35.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.213+0000 7f3c214d4b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e3d24ecfd0 con 0x55e3d262c6b0 2024-05-09T04:54:35.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.213+0000 7f3c214d4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55e3d254b1c0 con 0x55e3d24af450 2024-05-09T04:54:35.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.213+0000 7f3c0b400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3d24af450 0x55e3d2631e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:35.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.213+0000 7f3c0be00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e3d2632360 0x55e3d262c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:35.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.213+0000 7f3c0b400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3d24af450 0x55e3d2631e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:44322/0 (socket says 172.21.15.6:44322) 2024-05-09T04:54:35.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.213+0000 7f3c0b400700 1 -- 172.21.15.6:0/1982214852 learned_addr learned my addr 172.21.15.6:0/1982214852 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:35.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.213+0000 7f3c08c00700 1 --2- 172.21.15.6:0/149225708 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f3bf00b2900 0x7f3bf00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:35.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.213+0000 7f3c0b400700 1 --2- 172.21.15.6:0/1982214852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3d262c6b0 0x55e3d262cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:35.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.213+0000 7f3c08c00700 1 -- 172.21.15.6:0/149225708 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f3c0400f020 con 0x7f3c0c108560 2024-05-09T04:54:35.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.214+0000 7f3c10a00700 1 --2- 172.21.15.6:0/149225708 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f3bf00b2900 0x7f3bf00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:35.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.214+0000 7f3c0b400700 1 -- 172.21.15.6:0/1982214852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3d262c6b0 msgr2=0x55e3d262cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:35.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.214+0000 7f3c0b400700 1 --2- 172.21.15.6:0/1982214852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3d262c6b0 0x55e3d262cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:35.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.214+0000 7f3c0b400700 1 -- 172.21.15.6:0/1982214852 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e3d2632360 msgr2=0x55e3d262c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:35.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.214+0000 7f3c0b400700 1 --2- 172.21.15.6:0/1982214852 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e3d2632360 0x55e3d262c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:35.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.214+0000 7f3c0b400700 1 -- 172.21.15.6:0/1982214852 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3d23ca440 con 0x55e3d24af450 2024-05-09T04:54:35.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.214+0000 7f3c0be00700 1 --2- 172.21.15.6:0/1982214852 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e3d2632360 0x55e3d262c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:54:35.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.214+0000 7f3c0b400700 1 --2- 172.21.15.6:0/1982214852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3d24af450 0x55e3d2631e20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8005950 tx=0x7f3bf80058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd8144c8ed938cad server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:35.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.215+0000 7f3c01600700 1 -- 172.21.15.6:0/1982214852 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf8013e20 con 0x55e3d24af450 2024-05-09T04:54:35.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.215+0000 7f3c01600700 1 -- 172.21.15.6:0/1982214852 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bf8004ce0 con 0x55e3d24af450 2024-05-09T04:54:35.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.215+0000 7f3c01600700 1 -- 172.21.15.6:0/1982214852 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf8010420 con 0x55e3d24af450 2024-05-09T04:54:35.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.215+0000 7f3c10a00700 1 --2- 172.21.15.6:0/149225708 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f3bf00b2900 0x7f3bf00b4e10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3c04009fd0 tx=0x7f3c0400b3e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:35.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.216+0000 7f3c214d4b80 1 -- 172.21.15.6:0/1982214852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3d24af450 msgr2=0x55e3d2631e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:35.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.216+0000 7f3c214d4b80 1 --2- 172.21.15.6:0/1982214852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3d24af450 0x55e3d2631e20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8005950 tx=0x7f3bf80058e0 comp rx=0 tx=0).stop 2024-05-09T04:54:35.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.216+0000 7f3c214d4b80 1 -- 172.21.15.6:0/1982214852 shutdown_connections 2024-05-09T04:54:35.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.216+0000 7f3c214d4b80 1 --2- 172.21.15.6:0/1982214852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3d262c6b0 0x55e3d262cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:35.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.216+0000 7f3c214d4b80 1 --2- 172.21.15.6:0/1982214852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3d24af450 0x55e3d2631e20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:35.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.216+0000 7f3c214d4b80 1 --2- 172.21.15.6:0/1982214852 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e3d2632360 0x55e3d262c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:35.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.216+0000 7f3c214d4b80 1 -- 172.21.15.6:0/1982214852 >> 172.21.15.6:0/1982214852 conn(0x55e3d24b0740 msgr2=0x55e3d2626b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:35.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.216+0000 7f3c214d4b80 1 -- 172.21.15.6:0/1982214852 shutdown_connections 2024-05-09T04:54:35.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.216+0000 7f3c214d4b80 1 -- 172.21.15.6:0/1982214852 wait complete. 2024-05-09T04:54:35.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.217+0000 7f3c214d4b80 1 -- start start 2024-05-09T04:54:35.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.217+0000 7f3c214d4b80 1 Processor -- start 2024-05-09T04:54:35.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.218+0000 7f3c214d4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3d24af450 0x55e3d2635aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:35.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.218+0000 7f3c214d4b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e3d262c6b0 0x55e3d2635fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:35.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.218+0000 7f3c214d4b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3d2632360 0x55e3d2636520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:35.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.218+0000 7f3c214d4b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55e3d2638d70 con 0x55e3d262c6b0 2024-05-09T04:54:35.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.218+0000 7f3c214d4b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e3d24e9280 con 0x55e3d2632360 2024-05-09T04:54:35.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.218+0000 7f3c214d4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55e3d2638ef0 con 0x55e3d24af450 2024-05-09T04:54:35.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.218+0000 7f3c214d4b80 10 client.0 _ll_register_callbacks cb 0x55e3d24e6540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:35.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.218+0000 7f3c0be00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e3d262c6b0 0x55e3d2635fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:35.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.218+0000 7f3c0be00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e3d262c6b0 0x55e3d2635fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43666/0 (socket says 172.21.15.6:43666) 2024-05-09T04:54:35.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.218+0000 7f3c0be00700 1 -- 172.21.15.6:0/2897356777 learned_addr learned my addr 172.21.15.6:0/2897356777 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:35.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.218+0000 7f3c0b400700 1 --2- 172.21.15.6:0/2897356777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3d24af450 0x55e3d2635aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:35.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.218+0000 7f3c0b400700 1 --2- 172.21.15.6:0/2897356777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3d2632360 0x55e3d2636520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:35.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.219+0000 7f3c0be00700 1 -- 172.21.15.6:0/2897356777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3d24af450 msgr2=0x55e3d2635aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:35.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.219+0000 7f3c0be00700 1 --2- 172.21.15.6:0/2897356777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3d24af450 0x55e3d2635aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:35.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.219+0000 7f3c0be00700 1 -- 172.21.15.6:0/2897356777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3d2632360 msgr2=0x55e3d2636520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:35.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.219+0000 7f3c0be00700 1 --2- 172.21.15.6:0/2897356777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3d2632360 0x55e3d2636520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:35.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.219+0000 7f3c0be00700 1 -- 172.21.15.6:0/2897356777 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3d2514800 con 0x55e3d262c6b0 2024-05-09T04:54:35.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.219+0000 7f3c0b400700 1 --2- 172.21.15.6:0/2897356777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3d2632360 0x55e3d2636520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:54:35.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.219+0000 7f3c0be00700 1 --2- 172.21.15.6:0/2897356777 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e3d262c6b0 0x55e3d2635fe0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc012860 tx=0x7f3bfc012b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace53501446393d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:35.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.219+0000 7f3bf6000700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-05-09T04:54:35.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.219+0000 7f3bf6000700 1 -- 172.21.15.6:0/2897356777 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc004050 con 0x55e3d262c6b0 2024-05-09T04:54:35.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.219+0000 7f3bf6000700 1 -- 172.21.15.6:0/2897356777 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f3bfc0041f0 con 0x55e3d262c6b0 2024-05-09T04:54:35.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.219+0000 7f3c214d4b80 1 -- 172.21.15.6:0/2897356777 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bf8004ce0 con 0x55e3d262c6b0 2024-05-09T04:54:35.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.220+0000 7f3bf6000700 1 -- 172.21.15.6:0/2897356777 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3bfc0043f0 con 0x55e3d262c6b0 2024-05-09T04:54:35.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.220+0000 7f3c214d4b80 10 client.15012 fetch_fsmap learned FSMap version 29 2024-05-09T04:54:35.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.220+0000 7f3c214d4b80 1 -- 172.21.15.6:0/2897356777 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e3d2643a00 con 0x55e3d262c6b0 2024-05-09T04:54:35.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.220+0000 7f3bf6000700 1 -- 172.21.15.6:0/2897356777 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bf8004ce0 con 0x55e3d262c6b0 2024-05-09T04:54:35.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.220+0000 7f3bf6000700 1 -- 172.21.15.6:0/2897356777 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3bfc00f880 con 0x55e3d262c6b0 2024-05-09T04:54:35.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.220+0000 7f3c214d4b80 10 client.15012 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T04:54:35.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.221+0000 7f3c214d4b80 10 client.15012 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T04:54:35.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.221+0000 7f3c214d4b80 20 client.15012 populate_metadata read hostname 'smithi006' 2024-05-09T04:54:35.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.221+0000 7f3c214d4b80 1 --2- 172.21.15.6:0/2897356777 >> [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] conn(0x55e3d2645fe0 0x55e3d2647450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:35.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.221+0000 7f3c214d4b80 4 client.15012 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:54:35.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.221+0000 7f3c214d4b80 1 -- 172.21.15.6:0/2897356777 --> [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3d255cbe0 con 0x55e3d2645fe0 2024-05-09T04:54:35.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.221+0000 7f3c0b400700 1 --2- 172.21.15.6:0/2897356777 >> [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] conn(0x55e3d2645fe0 0x55e3d2647450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:35.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.222+0000 7f3c0b400700 1 --2- 172.21.15.6:0/2897356777 >> [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] conn(0x55e3d2645fe0 0x55e3d2647450 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d3e0bf25c0cb1ccf server_cookie=d97e8bb92017bfb5 in_seq=0 out_seq=0 2024-05-09T04:54:35.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.222+0000 7f3bf6000700 10 client.15012 ms_handle_connect on v2:172.21.15.186:6838/3954883105 2024-05-09T04:54:35.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.222+0000 7f3bf6000700 1 -- 172.21.15.6:0/2897356777 <== mds.3 v2:172.21.15.186:6838/3954883105 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e3d255cbe0 con 0x55e3d2645fe0 2024-05-09T04:54:35.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.222+0000 7f3bf6000700 10 client.15012 handle_command_reply: tid=0 2024-05-09T04:54:35.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.223+0000 7f3c214d4b80 1 client.15012 shutdown 2024-05-09T04:54:35.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.223+0000 7f3c214d4b80 1 -- 172.21.15.6:0/2897356777 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e3d262c6b0 msgr2=0x55e3d2635fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:35.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.223+0000 7f3c214d4b80 1 --2- 172.21.15.6:0/2897356777 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e3d262c6b0 0x55e3d2635fe0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc012860 tx=0x7f3bfc012b70 comp rx=0 tx=0).stop 2024-05-09T04:54:35.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.223+0000 7f3c214d4b80 1 -- 172.21.15.6:0/2897356777 shutdown_connections 2024-05-09T04:54:35.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.223+0000 7f3c214d4b80 1 --2- 172.21.15.6:0/2897356777 >> [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] conn(0x55e3d2645fe0 0x55e3d2647450 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:35.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.223+0000 7f3c214d4b80 1 --2- 172.21.15.6:0/2897356777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3d2632360 0x55e3d2636520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:35.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.223+0000 7f3c214d4b80 1 --2- 172.21.15.6:0/2897356777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3d24af450 0x55e3d2635aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:35.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.223+0000 7f3c214d4b80 1 --2- 172.21.15.6:0/2897356777 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e3d262c6b0 0x55e3d2635fe0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:35.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.223+0000 7f3c214d4b80 1 -- 172.21.15.6:0/2897356777 >> 172.21.15.6:0/2897356777 conn(0x55e3d24b0740 msgr2=0x55e3d26261a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:35.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.223+0000 7f3bf6000700 0 client.15012 ms_handle_reset on v2:172.21.15.186:6838/3954883105 2024-05-09T04:54:35.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.223+0000 7f3c214d4b80 1 -- 172.21.15.6:0/2897356777 shutdown_connections 2024-05-09T04:54:35.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.223+0000 7f3c214d4b80 1 -- 172.21.15.6:0/2897356777 wait complete. 2024-05-09T04:54:35.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.223+0000 7f3c214d4b80 20 client.15012 trim_cache size 0 max 16384 2024-05-09T04:54:35.248 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.247+0000 7f3c214d4b80 1 Processor -- start 2024-05-09T04:54:35.248 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.247+0000 7f3c214d4b80 1 -- start start 2024-05-09T04:54:35.248 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.247+0000 7f3c214d4b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3d24af450 0x55e3d262c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:35.248 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.247+0000 7f3c214d4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3d262c6b0 0x55e3d26421f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:35.248 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.247+0000 7f3c214d4b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e3d2632360 0x55e3d2636360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:35.248 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.247+0000 7f3c214d4b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55e3d2645790 con 0x55e3d2632360 2024-05-09T04:54:35.248 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.247+0000 7f3c214d4b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e3d2645e60 con 0x55e3d24af450 2024-05-09T04:54:35.248 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.247+0000 7f3c214d4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55e3d2646530 con 0x55e3d262c6b0 2024-05-09T04:54:35.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.248+0000 7f3c0b400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3d24af450 0x55e3d262c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:35.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.248+0000 7f3c0be00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3d262c6b0 0x55e3d26421f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:35.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.248+0000 7f3c0b400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3d24af450 0x55e3d262c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:56444/0 (socket says 172.21.15.6:56444) 2024-05-09T04:54:35.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.248+0000 7f3c0b400700 1 -- 172.21.15.6:0/3192255035 learned_addr learned my addr 172.21.15.6:0/3192255035 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:35.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.248+0000 7f3c0b400700 1 --2- 172.21.15.6:0/3192255035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e3d2632360 0x55e3d2636360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:35.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.248+0000 7f3c0b400700 1 -- 172.21.15.6:0/3192255035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3d262c6b0 msgr2=0x55e3d26421f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:35.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.248+0000 7f3c0b400700 1 --2- 172.21.15.6:0/3192255035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3d262c6b0 0x55e3d26421f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:35.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.248+0000 7f3c0b400700 1 -- 172.21.15.6:0/3192255035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e3d2632360 msgr2=0x55e3d2636360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:35.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.248+0000 7f3c0b400700 1 --2- 172.21.15.6:0/3192255035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e3d2632360 0x55e3d2636360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:35.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.248+0000 7f3c0b400700 1 -- 172.21.15.6:0/3192255035 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3d26368a0 con 0x55e3d24af450 2024-05-09T04:54:35.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.249+0000 7f3c0b400700 1 --2- 172.21.15.6:0/3192255035 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3d24af450 0x55e3d262c170 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3bf80095f0 tx=0x7f3bf8018040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d08bfce5eb089d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:35.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.249+0000 7f3c00c00700 1 -- 172.21.15.6:0/3192255035 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf8005010 con 0x55e3d24af450 2024-05-09T04:54:35.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.249+0000 7f3c00c00700 1 -- 172.21.15.6:0/3192255035 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bf80051b0 con 0x55e3d24af450 2024-05-09T04:54:35.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.249+0000 7f3c00c00700 1 -- 172.21.15.6:0/3192255035 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf802d2d0 con 0x55e3d24af450 2024-05-09T04:54:35.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.250+0000 7f3c214d4b80 1 -- 172.21.15.6:0/3192255035 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3d24af450 msgr2=0x55e3d262c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:35.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.250+0000 7f3c214d4b80 1 --2- 172.21.15.6:0/3192255035 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3d24af450 0x55e3d262c170 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3bf80095f0 tx=0x7f3bf8018040 comp rx=0 tx=0).stop 2024-05-09T04:54:35.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.250+0000 7f3c214d4b80 1 -- 172.21.15.6:0/3192255035 shutdown_connections 2024-05-09T04:54:35.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.250+0000 7f3c214d4b80 1 --2- 172.21.15.6:0/3192255035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e3d2632360 0x55e3d2636360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:35.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.250+0000 7f3c214d4b80 1 --2- 172.21.15.6:0/3192255035 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3d24af450 0x55e3d262c170 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:35.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.250+0000 7f3c214d4b80 1 --2- 172.21.15.6:0/3192255035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3d262c6b0 0x55e3d26421f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:35.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.250+0000 7f3c214d4b80 1 -- 172.21.15.6:0/3192255035 >> 172.21.15.6:0/3192255035 conn(0x55e3d24b0740 msgr2=0x55e3d262f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:35.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.250+0000 7f3c214d4b80 1 -- 172.21.15.6:0/3192255035 shutdown_connections 2024-05-09T04:54:35.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.250+0000 7f3c214d4b80 1 -- 172.21.15.6:0/3192255035 wait complete. 2024-05-09T04:54:35.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.251+0000 7f3c214d4b80 1 -- start start 2024-05-09T04:54:35.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.251+0000 7f3c214d4b80 1 Processor -- start 2024-05-09T04:54:35.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.251+0000 7f3c214d4b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e3d24af450 0x55e3d26508d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:35.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.251+0000 7f3c214d4b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3d262c6b0 0x55e3d2650e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:35.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.251+0000 7f3c214d4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3d2632360 0x55e3d264b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:35.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.252+0000 7f3c214d4b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55e3d26472d0 con 0x55e3d24af450 2024-05-09T04:54:35.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.252+0000 7f3c214d4b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e3d2646c00 con 0x55e3d262c6b0 2024-05-09T04:54:35.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.252+0000 7f3c214d4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55e3d26479a0 con 0x55e3d2632360 2024-05-09T04:54:35.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.252+0000 7f3c214d4b80 10 client.0 _ll_register_callbacks cb 0x55e3d24e6540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:35.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.252+0000 7f3c0b400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e3d24af450 0x55e3d26508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:35.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.252+0000 7f3c0b400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e3d24af450 0x55e3d26508d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43692/0 (socket says 172.21.15.6:43692) 2024-05-09T04:54:35.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.252+0000 7f3c0b400700 1 -- 172.21.15.6:0/2014841171 learned_addr learned my addr 172.21.15.6:0/2014841171 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:35.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.252+0000 7f3c0be00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3d262c6b0 0x55e3d2650e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:35.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.252+0000 7f3c0b400700 1 --2- 172.21.15.6:0/2014841171 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3d2632360 0x55e3d264b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:35.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.252+0000 7f3c0be00700 1 -- 172.21.15.6:0/2014841171 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3d2632360 msgr2=0x55e3d264b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:35.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.252+0000 7f3c0be00700 1 --2- 172.21.15.6:0/2014841171 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3d2632360 0x55e3d264b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:35.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.252+0000 7f3c0be00700 1 -- 172.21.15.6:0/2014841171 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e3d24af450 msgr2=0x55e3d26508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:35.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.252+0000 7f3c0be00700 1 --2- 172.21.15.6:0/2014841171 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e3d24af450 0x55e3d26508d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:35.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.252+0000 7f3c0be00700 1 -- 172.21.15.6:0/2014841171 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3d264be70 con 0x55e3d262c6b0 2024-05-09T04:54:35.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.252+0000 7f3c0b400700 1 --2- 172.21.15.6:0/2014841171 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e3d24af450 0x55e3d26508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:54:35.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.253+0000 7f3c0be00700 1 --2- 172.21.15.6:0/2014841171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3d262c6b0 0x55e3d2650e10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc00fe60 tx=0x7f3bfc01c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf8fe0349f551212 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:35.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.253+0000 7f3bf6000700 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-05-09T04:54:35.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.253+0000 7f3bf6000700 1 -- 172.21.15.6:0/2014841171 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc01ca80 con 0x55e3d262c6b0 2024-05-09T04:54:35.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.253+0000 7f3c214d4b80 1 -- 172.21.15.6:0/2014841171 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bf80051b0 con 0x55e3d262c6b0 2024-05-09T04:54:35.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.253+0000 7f3bf6000700 1 -- 172.21.15.6:0/2014841171 <== mon.2 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f3bfc01cc20 con 0x55e3d262c6b0 2024-05-09T04:54:35.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.254+0000 7f3bf6000700 1 -- 172.21.15.6:0/2014841171 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3bfc03a050 con 0x55e3d262c6b0 2024-05-09T04:54:35.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.254+0000 7f3c214d4b80 10 client.25166 fetch_fsmap learned FSMap version 29 2024-05-09T04:54:35.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.254+0000 7f3c214d4b80 1 -- 172.21.15.6:0/2014841171 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e3d26ba480 con 0x55e3d262c6b0 2024-05-09T04:54:35.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.254+0000 7f3bf6000700 1 -- 172.21.15.6:0/2014841171 <== mon.2 v2:172.21.15.169:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bfc02e020 con 0x55e3d262c6b0 2024-05-09T04:54:35.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.254+0000 7f3bf6000700 1 -- 172.21.15.6:0/2014841171 <== mon.2 v2:172.21.15.169:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3bfc004650 con 0x55e3d262c6b0 2024-05-09T04:54:35.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.254+0000 7f3c214d4b80 10 client.25166 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T04:54:35.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.254+0000 7f3c214d4b80 10 client.25166 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T04:54:35.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.254+0000 7f3c214d4b80 20 client.25166 populate_metadata read hostname 'smithi006' 2024-05-09T04:54:35.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.254+0000 7f3c214d4b80 1 --2- 172.21.15.6:0/2014841171 >> [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] conn(0x55e3d26bc920 0x55e3d26bdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:35.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.254+0000 7f3c214d4b80 4 client.25166 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T04:54:35.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.254+0000 7f3c214d4b80 1 -- 172.21.15.6:0/2014841171 --> [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] -- command(tid 0: {"prefix": "status"}) -- 0x55e3d2556c80 con 0x55e3d26bc920 2024-05-09T04:54:35.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.255+0000 7f3c0b400700 1 --2- 172.21.15.6:0/2014841171 >> [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] conn(0x55e3d26bc920 0x55e3d26bdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:35.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.255+0000 7f3c0b400700 1 --2- 172.21.15.6:0/2014841171 >> [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] conn(0x55e3d26bc920 0x55e3d26bdd90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7b7d02d3aa971432 server_cookie=1bacf3e880ced8bf in_seq=0 out_seq=0 2024-05-09T04:54:35.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.255+0000 7f3bf6000700 10 client.25166 ms_handle_connect on v2:172.21.15.186:6838/3954883105 2024-05-09T04:54:35.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.256+0000 7f3bf6000700 1 -- 172.21.15.6:0/2014841171 <== mds.3 v2:172.21.15.186:6838/3954883105 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e3d2556c80 con 0x55e3d26bc920 2024-05-09T04:54:35.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.256+0000 7f3bf6000700 10 client.25166 handle_command_reply: tid=0 2024-05-09T04:54:35.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.256+0000 7f3c214d4b80 1 client.25166 shutdown 2024-05-09T04:54:35.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.256+0000 7f3c214d4b80 1 -- 172.21.15.6:0/2014841171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3d262c6b0 msgr2=0x55e3d2650e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:35.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.256+0000 7f3c214d4b80 1 --2- 172.21.15.6:0/2014841171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3d262c6b0 0x55e3d2650e10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc00fe60 tx=0x7f3bfc01c650 comp rx=0 tx=0).stop 2024-05-09T04:54:35.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.256+0000 7f3c214d4b80 1 -- 172.21.15.6:0/2014841171 shutdown_connections 2024-05-09T04:54:35.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.256+0000 7f3c214d4b80 1 --2- 172.21.15.6:0/2014841171 >> [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] conn(0x55e3d26bc920 0x55e3d26bdd90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:35.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.256+0000 7f3c214d4b80 1 --2- 172.21.15.6:0/2014841171 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e3d2632360 0x55e3d264b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:35.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.256+0000 7f3c214d4b80 1 --2- 172.21.15.6:0/2014841171 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e3d24af450 0x55e3d26508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:35.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.256+0000 7f3c214d4b80 1 --2- 172.21.15.6:0/2014841171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3d262c6b0 0x55e3d2650e10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:35.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.256+0000 7f3bf6000700 0 client.25166 ms_handle_reset on v2:172.21.15.186:6838/3954883105 2024-05-09T04:54:35.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.256+0000 7f3c214d4b80 1 -- 172.21.15.6:0/2014841171 >> 172.21.15.6:0/2014841171 conn(0x55e3d24b0740 msgr2=0x55e3d262b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:35.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.256+0000 7f3c214d4b80 1 -- 172.21.15.6:0/2014841171 shutdown_connections 2024-05-09T04:54:35.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.256+0000 7f3c214d4b80 1 -- 172.21.15.6:0/2014841171 wait complete. 2024-05-09T04:54:35.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.256+0000 7f3c214d4b80 20 client.25166 trim_cache size 0 max 16384 2024-05-09T04:54:35.260 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-09T04:54:35.260 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "f74d813c-0dbd-11ef-bc97-c7b262605968", 2024-05-09T04:54:35.260 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 3, 2024-05-09T04:54:35.260 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24511, 2024-05-09T04:54:35.260 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-05-09T04:54:35.260 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-05-09T04:54:35.260 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-05-09T04:54:35.260 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 213.50873862500001, 2024-05-09T04:54:35.260 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 29, 2024-05-09T04:54:35.260 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 89, 2024-05-09T04:54:35.260 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 89, 2024-05-09T04:54:35.260 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 333.97815060599999 2024-05-09T04:54:35.261 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-09T04:54:35.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.257+0000 7f3c11400700 1 -- 172.21.15.6:0/149225708 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f3bf00b2900 msgr2=0x7f3bf00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:35.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.257+0000 7f3c11400700 1 --2- 172.21.15.6:0/149225708 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f3bf00b2900 0x7f3bf00b4e10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3c04009fd0 tx=0x7f3c0400b3e0 comp rx=0 tx=0).stop 2024-05-09T04:54:35.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.257+0000 7f3c11400700 1 -- 172.21.15.6:0/149225708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c0c108560 msgr2=0x7f3c0c100eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:35.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.257+0000 7f3c11400700 1 --2- 172.21.15.6:0/149225708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c0c108560 0x7f3c0c100eb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400b770 tx=0x7f3c04007980 comp rx=0 tx=0).stop 2024-05-09T04:54:35.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.257+0000 7f3c11400700 1 -- 172.21.15.6:0/149225708 shutdown_connections 2024-05-09T04:54:35.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.257+0000 7f3c11400700 1 --2- 172.21.15.6:0/149225708 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f3bf00b2900 0x7f3bf00b4e10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:35.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.257+0000 7f3c11400700 1 --2- 172.21.15.6:0/149225708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c0c108560 0x7f3c0c100eb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:35.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.257+0000 7f3c11400700 1 --2- 172.21.15.6:0/149225708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c0c1032e0 0x7f3c0c100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:35.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.257+0000 7f3c11400700 1 --2- 172.21.15.6:0/149225708 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c0c107c10 0x7f3c0c100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:35.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.257+0000 7f3c11400700 1 -- 172.21.15.6:0/149225708 >> 172.21.15.6:0/149225708 conn(0x7f3c0c078ac0 msgr2=0x7f3c0c06b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:35.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.257+0000 7f3c11400700 1 -- 172.21.15.6:0/149225708 shutdown_connections 2024-05-09T04:54:35.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:35.257+0000 7f3c11400700 1 -- 172.21.15.6:0/149225708 wait complete. 2024-05-09T04:54:35.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:34 smithi186 ceph-mon[34668]: pgmap v488: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:35.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:34 smithi169 ceph-mon[34075]: pgmap v488: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:36.222 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.h status 2024-05-09T04:54:36.728 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:54:37.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:36 smithi006 ceph-mon[29835]: pgmap v489: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:54:37.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:36 smithi186 ceph-mon[34668]: pgmap v489: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:54:37.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:36 smithi169 ceph-mon[34075]: pgmap v489: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:54:39.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.132+0000 7f07e9400700 1 -- 172.21.15.6:0/3354552558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07e410c5b0 msgr2=0x7f07e410c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:39.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.132+0000 7f07e9400700 1 --2- 172.21.15.6:0/3354552558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07e410c5b0 0x7f07e410c9c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f07d4009990 tx=0x7f07d4009ca0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.133 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:38 smithi006 ceph-mon[29835]: pgmap v490: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:54:39.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.132+0000 7f07e9400700 1 -- 172.21.15.6:0/3354552558 shutdown_connections 2024-05-09T04:54:39.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.132+0000 7f07e9400700 1 --2- 172.21.15.6:0/3354552558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07e4106ec0 0x7f07e4113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.132+0000 7f07e9400700 1 --2- 172.21.15.6:0/3354552558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07e410c5b0 0x7f07e410c9c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.132+0000 7f07e9400700 1 --2- 172.21.15.6:0/3354552558 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07e4106570 0x7f07e4106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.132+0000 7f07e9400700 1 -- 172.21.15.6:0/3354552558 >> 172.21.15.6:0/3354552558 conn(0x7f07e4102050 msgr2=0x7f07e41044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:39.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.132+0000 7f07e9400700 1 -- 172.21.15.6:0/3354552558 shutdown_connections 2024-05-09T04:54:39.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.132+0000 7f07e9400700 1 -- 172.21.15.6:0/3354552558 wait complete. 2024-05-09T04:54:39.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.134+0000 7f07e9400700 1 Processor -- start 2024-05-09T04:54:39.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.134+0000 7f07e9400700 1 -- start start 2024-05-09T04:54:39.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.134+0000 7f07e9400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07e4106570 0x7f07e419d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.134+0000 7f07e9400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07e4106ec0 0x7f07e419d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.134+0000 7f07e9400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07e410c5b0 0x7f07e41a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.134+0000 7f07e9400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f07e4116710 con 0x7f07e4106570 2024-05-09T04:54:39.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.134+0000 7f07e9400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f07e4116590 con 0x7f07e4106ec0 2024-05-09T04:54:39.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.134+0000 7f07e9400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f07e4116890 con 0x7f07e410c5b0 2024-05-09T04:54:39.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.135+0000 7f07e2800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07e4106570 0x7f07e419d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.135+0000 7f07e2800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07e4106570 0x7f07e419d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58032/0 (socket says 172.21.15.6:58032) 2024-05-09T04:54:39.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.135+0000 7f07e2800700 1 -- 172.21.15.6:0/1925331287 learned_addr learned my addr 172.21.15.6:0/1925331287 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:39.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.135+0000 7f07dbe00700 1 --2- 172.21.15.6:0/1925331287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07e4106ec0 0x7f07e419d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.135+0000 7f07e3200700 1 --2- 172.21.15.6:0/1925331287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07e410c5b0 0x7f07e41a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.135+0000 7f07dbe00700 1 -- 172.21.15.6:0/1925331287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07e410c5b0 msgr2=0x7f07e41a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:39.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.135+0000 7f07dbe00700 1 --2- 172.21.15.6:0/1925331287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07e410c5b0 0x7f07e41a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.135+0000 7f07dbe00700 1 -- 172.21.15.6:0/1925331287 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07e4106570 msgr2=0x7f07e419d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:39.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.135+0000 7f07dbe00700 1 --2- 172.21.15.6:0/1925331287 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07e4106570 0x7f07e419d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.135+0000 7f07dbe00700 1 -- 172.21.15.6:0/1925331287 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e41a1f30 con 0x7f07e4106ec0 2024-05-09T04:54:39.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.135+0000 7f07e2800700 1 --2- 172.21.15.6:0/1925331287 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07e4106570 0x7f07e419d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:54:39.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.135+0000 7f07e3200700 1 --2- 172.21.15.6:0/1925331287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07e410c5b0 0x7f07e41a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:54:39.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.136+0000 7f07dbe00700 1 --2- 172.21.15.6:0/1925331287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07e4106ec0 0x7f07e419d600 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f07e4075b80 tx=0x7f07cc00eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:39.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.136+0000 7f07db400700 1 -- 172.21.15.6:0/1925331287 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc010900 con 0x7f07e4106ec0 2024-05-09T04:54:39.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.136+0000 7f07db400700 1 -- 172.21.15.6:0/1925331287 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07cc00c430 con 0x7f07e4106ec0 2024-05-09T04:54:39.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.136+0000 7f07db400700 1 -- 172.21.15.6:0/1925331287 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc018680 con 0x7f07e4106ec0 2024-05-09T04:54:39.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.136+0000 7f07e9400700 1 -- 172.21.15.6:0/1925331287 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07e41a2220 con 0x7f07e4106ec0 2024-05-09T04:54:39.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.136+0000 7f07e9400700 1 -- 172.21.15.6:0/1925331287 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07e41a9b40 con 0x7f07e4106ec0 2024-05-09T04:54:39.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.140+0000 7f07db400700 1 -- 172.21.15.6:0/1925331287 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f07cc018820 con 0x7f07e4106ec0 2024-05-09T04:54:39.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.141+0000 7f07f7ed2b80 1 Processor -- start 2024-05-09T04:54:39.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.141+0000 7f07f7ed2b80 1 -- start start 2024-05-09T04:54:39.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.142+0000 7f07f7ed2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbc5c56450 0x55cbc5dd8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.142+0000 7f07f7ed2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cbc5dd9410 0x55cbc5dd34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.142+0000 7f07f7ed2b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cbc5dd3a00 0x55cbc5dd3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.142+0000 7f07db400700 1 --2- 172.21.15.6:0/1925331287 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f07c80b2900 0x7f07c80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.142+0000 7f07f7ed2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55cbc5cf6260 con 0x55cbc5dd9410 2024-05-09T04:54:39.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.142+0000 7f07f7ed2b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55cbc5c94090 con 0x55cbc5dd3a00 2024-05-09T04:54:39.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.142+0000 7f07f7ed2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55cbc5cf2220 con 0x55cbc5c56450 2024-05-09T04:54:39.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.142+0000 7f07e3200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbc5c56450 0x55cbc5dd8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.142+0000 7f07db400700 1 -- 172.21.15.6:0/1925331287 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f07cc0e2670 con 0x7f07e4106ec0 2024-05-09T04:54:39.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.142+0000 7f07e2800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cbc5dd9410 0x55cbc5dd34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.142+0000 7f07e3200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbc5c56450 0x55cbc5dd8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:51370/0 (socket says 172.21.15.6:51370) 2024-05-09T04:54:39.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.142+0000 7f07e3200700 1 -- 172.21.15.6:0/2670522249 learned_addr learned my addr 172.21.15.6:0/2670522249 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:39.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.142+0000 7f07e3200700 1 --2- 172.21.15.6:0/2670522249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cbc5dd3a00 0x55cbc5dd3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.143+0000 7f07dbe00700 1 --2- 172.21.15.6:0/1925331287 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f07c80b2900 0x7f07c80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.143+0000 7f07e3200700 1 -- 172.21.15.6:0/2670522249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cbc5dd3a00 msgr2=0x55cbc5dd3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:39.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.143+0000 7f07e3200700 1 --2- 172.21.15.6:0/2670522249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cbc5dd3a00 0x55cbc5dd3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.143+0000 7f07e3200700 1 -- 172.21.15.6:0/2670522249 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cbc5dd9410 msgr2=0x55cbc5dd34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:39.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.143+0000 7f07e3200700 1 --2- 172.21.15.6:0/2670522249 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cbc5dd9410 0x55cbc5dd34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.143+0000 7f07e3200700 1 -- 172.21.15.6:0/2670522249 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbc5b71440 con 0x55cbc5c56450 2024-05-09T04:54:39.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.143+0000 7f07e3200700 1 --2- 172.21.15.6:0/2670522249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbc5c56450 0x55cbc5dd8ed0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f07dc008cb0 tx=0x7f07dc009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=424a8587cbe03a43 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:39.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.144+0000 7f07d3e00700 1 -- 172.21.15.6:0/2670522249 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07dc020070 con 0x55cbc5c56450 2024-05-09T04:54:39.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.144+0000 7f07d3e00700 1 -- 172.21.15.6:0/2670522249 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07dc008320 con 0x55cbc5c56450 2024-05-09T04:54:39.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.144+0000 7f07d3e00700 1 -- 172.21.15.6:0/2670522249 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07dc0079b0 con 0x55cbc5c56450 2024-05-09T04:54:39.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.145+0000 7f07f7ed2b80 1 -- 172.21.15.6:0/2670522249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbc5c56450 msgr2=0x55cbc5dd8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:39.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.145+0000 7f07f7ed2b80 1 --2- 172.21.15.6:0/2670522249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbc5c56450 0x55cbc5dd8ed0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f07dc008cb0 tx=0x7f07dc009920 comp rx=0 tx=0).stop 2024-05-09T04:54:39.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.145+0000 7f07dbe00700 1 --2- 172.21.15.6:0/1925331287 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f07c80b2900 0x7f07c80b4e10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f07cc00cfd0 tx=0x7f07cc009790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:39.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.145+0000 7f07f7ed2b80 1 -- 172.21.15.6:0/2670522249 shutdown_connections 2024-05-09T04:54:39.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.145+0000 7f07f7ed2b80 1 --2- 172.21.15.6:0/2670522249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cbc5dd3a00 0x55cbc5dd3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.145+0000 7f07f7ed2b80 1 --2- 172.21.15.6:0/2670522249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbc5c56450 0x55cbc5dd8ed0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.145+0000 7f07f7ed2b80 1 --2- 172.21.15.6:0/2670522249 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cbc5dd9410 0x55cbc5dd34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.145+0000 7f07f7ed2b80 1 -- 172.21.15.6:0/2670522249 >> 172.21.15.6:0/2670522249 conn(0x55cbc5c9ae20 msgr2=0x55cbc5dcde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:39.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.145+0000 7f07f7ed2b80 1 -- 172.21.15.6:0/2670522249 shutdown_connections 2024-05-09T04:54:39.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.145+0000 7f07f7ed2b80 1 -- 172.21.15.6:0/2670522249 wait complete. 2024-05-09T04:54:39.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.146+0000 7f07f7ed2b80 1 -- start start 2024-05-09T04:54:39.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.146+0000 7f07f7ed2b80 1 Processor -- start 2024-05-09T04:54:39.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.146+0000 7f07f7ed2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cbc5c56450 0x55cbc5ddcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.147+0000 7f07f7ed2b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cbc5dd3a00 0x55cbc5ddd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.147+0000 7f07f7ed2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbc5dd9410 0x55cbc5ddd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.147+0000 7f07f7ed2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55cbc5ddff00 con 0x55cbc5c56450 2024-05-09T04:54:39.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.147+0000 7f07f7ed2b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55cbc5c5b670 con 0x55cbc5dd3a00 2024-05-09T04:54:39.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.147+0000 7f07f7ed2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55cbc5de0080 con 0x55cbc5dd9410 2024-05-09T04:54:39.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.147+0000 7f07e3200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cbc5c56450 0x55cbc5ddcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.147+0000 7f07f7ed2b80 10 client.0 _ll_register_callbacks cb 0x55cbc5c8d540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:39.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.147+0000 7f07e2800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cbc5dd3a00 0x55cbc5ddd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.147+0000 7f07e2800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cbc5dd3a00 0x55cbc5ddd0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:52828/0 (socket says 172.21.15.6:52828) 2024-05-09T04:54:39.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.147+0000 7f07e3200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cbc5c56450 0x55cbc5ddcbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58038/0 (socket says 172.21.15.6:58038) 2024-05-09T04:54:39.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.147+0000 7f07e2800700 1 -- 172.21.15.6:0/1779397501 learned_addr learned my addr 172.21.15.6:0/1779397501 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:39.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.147+0000 7f07e3200700 1 --2- 172.21.15.6:0/1779397501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbc5dd9410 0x55cbc5ddd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.147+0000 7f07e2800700 1 -- 172.21.15.6:0/1779397501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbc5dd9410 msgr2=0x55cbc5ddd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:39.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.148+0000 7f07e2800700 1 --2- 172.21.15.6:0/1779397501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbc5dd9410 0x55cbc5ddd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.148+0000 7f07e2800700 1 -- 172.21.15.6:0/1779397501 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cbc5c56450 msgr2=0x55cbc5ddcbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:39.151 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.148+0000 7f07e2800700 1 --2- 172.21.15.6:0/1779397501 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cbc5c56450 0x55cbc5ddcbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.151 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.148+0000 7f07e2800700 1 -- 172.21.15.6:0/1779397501 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbc5de93b0 con 0x55cbc5dd3a00 2024-05-09T04:54:39.151 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.148+0000 7f07e3200700 1 --2- 172.21.15.6:0/1779397501 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cbc5c56450 0x55cbc5ddcbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:54:39.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.148+0000 7f07e2800700 1 --2- 172.21.15.6:0/1779397501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cbc5dd3a00 0x55cbc5ddd0f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f07d4005910 tx=0x7f07d400bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1bc45ecf837e522 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:39.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.148+0000 7f07d0c00700 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-05-09T04:54:39.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.148+0000 7f07d0c00700 1 -- 172.21.15.6:0/1779397501 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d40056e0 con 0x55cbc5dd3a00 2024-05-09T04:54:39.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.148+0000 7f07d0c00700 1 -- 172.21.15.6:0/1779397501 <== mon.2 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f07d4003a90 con 0x55cbc5dd3a00 2024-05-09T04:54:39.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.148+0000 7f07f7ed2b80 1 -- 172.21.15.6:0/1779397501 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07dc008320 con 0x55cbc5dd3a00 2024-05-09T04:54:39.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.149+0000 7f07d0c00700 1 -- 172.21.15.6:0/1779397501 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f07d4031050 con 0x55cbc5dd3a00 2024-05-09T04:54:39.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.149+0000 7f07f7ed2b80 10 client.25178 fetch_fsmap learned FSMap version 29 2024-05-09T04:54:39.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.149+0000 7f07f7ed2b80 1 -- 172.21.15.6:0/1779397501 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cbc5deacf0 con 0x55cbc5dd3a00 2024-05-09T04:54:39.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.149+0000 7f07d0c00700 1 -- 172.21.15.6:0/1779397501 <== mon.2 v2:172.21.15.169:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07dc008320 con 0x55cbc5dd3a00 2024-05-09T04:54:39.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.149+0000 7f07d0c00700 1 -- 172.21.15.6:0/1779397501 <== mon.2 v2:172.21.15.169:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f07d400f460 con 0x55cbc5dd3a00 2024-05-09T04:54:39.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.150+0000 7f07f7ed2b80 10 client.25178 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T04:54:39.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.150+0000 7f07f7ed2b80 10 client.25178 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-09T04:54:39.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.150+0000 7f07f7ed2b80 20 client.25178 populate_metadata read hostname 'smithi006' 2024-05-09T04:54:39.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.150+0000 7f07f7ed2b80 1 --2- 172.21.15.6:0/1779397501 >> [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] conn(0x55cbc5ded3c0 0x55cbc5dee830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.150+0000 7f07f7ed2b80 4 client.25178 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:54:39.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.150+0000 7f07f7ed2b80 1 -- 172.21.15.6:0/1779397501 --> [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbc5d9f820 con 0x55cbc5ded3c0 2024-05-09T04:54:39.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.150+0000 7f07e3200700 1 --2- 172.21.15.6:0/1779397501 >> [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] conn(0x55cbc5ded3c0 0x55cbc5dee830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.151+0000 7f07e3200700 1 --2- 172.21.15.6:0/1779397501 >> [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] conn(0x55cbc5ded3c0 0x55cbc5dee830 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3a4517c2b3226670 server_cookie=19b5b3bd6980494d in_seq=0 out_seq=0 2024-05-09T04:54:39.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.151+0000 7f07d0c00700 10 client.25178 ms_handle_connect on v2:172.21.15.169:6836/1845715294 2024-05-09T04:54:39.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.152+0000 7f07d0c00700 1 -- 172.21.15.6:0/1779397501 <== mds.2 v2:172.21.15.169:6836/1845715294 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cbc5d9f820 con 0x55cbc5ded3c0 2024-05-09T04:54:39.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.152+0000 7f07d0c00700 10 client.25178 handle_command_reply: tid=0 2024-05-09T04:54:39.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.152+0000 7f07f7ed2b80 1 client.25178 shutdown 2024-05-09T04:54:39.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.152+0000 7f07f7ed2b80 1 -- 172.21.15.6:0/1779397501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cbc5dd3a00 msgr2=0x55cbc5ddd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:39.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.152+0000 7f07f7ed2b80 1 --2- 172.21.15.6:0/1779397501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cbc5dd3a00 0x55cbc5ddd0f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f07d4005910 tx=0x7f07d400bfd0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.152+0000 7f07f7ed2b80 1 -- 172.21.15.6:0/1779397501 shutdown_connections 2024-05-09T04:54:39.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.152+0000 7f07f7ed2b80 1 --2- 172.21.15.6:0/1779397501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbc5dd9410 0x55cbc5ddd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.152+0000 7f07f7ed2b80 1 --2- 172.21.15.6:0/1779397501 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cbc5c56450 0x55cbc5ddcbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.152+0000 7f07f7ed2b80 1 --2- 172.21.15.6:0/1779397501 >> [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] conn(0x55cbc5ded3c0 0x55cbc5dee830 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.152+0000 7f07f7ed2b80 1 --2- 172.21.15.6:0/1779397501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cbc5dd3a00 0x55cbc5ddd0f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.152+0000 7f07d0c00700 0 client.25178 ms_handle_reset on v2:172.21.15.169:6836/1845715294 2024-05-09T04:54:39.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.152+0000 7f07f7ed2b80 1 -- 172.21.15.6:0/1779397501 >> 172.21.15.6:0/1779397501 conn(0x55cbc5c9ae20 msgr2=0x55cbc5dcde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:39.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.153+0000 7f07f7ed2b80 1 -- 172.21.15.6:0/1779397501 shutdown_connections 2024-05-09T04:54:39.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.153+0000 7f07f7ed2b80 1 -- 172.21.15.6:0/1779397501 wait complete. 2024-05-09T04:54:39.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.153+0000 7f07f7ed2b80 20 client.25178 trim_cache size 0 max 16384 2024-05-09T04:54:39.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.176+0000 7f07f7ed2b80 1 Processor -- start 2024-05-09T04:54:39.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.176+0000 7f07f7ed2b80 1 -- start start 2024-05-09T04:54:39.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.177+0000 7f07f7ed2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cbc5c9ae20 0x55cbc5dd34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.177+0000 7f07f7ed2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbc5c56450 0x55cbc5ddbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.177+0000 7f07f7ed2b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cbc5dd3a00 0x55cbc5ddef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.177+0000 7f07f7ed2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55cbc5decb70 con 0x55cbc5c9ae20 2024-05-09T04:54:39.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.177+0000 7f07f7ed2b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55cbc5ded240 con 0x55cbc5dd3a00 2024-05-09T04:54:39.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.177+0000 7f07f7ed2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55cbc5ded910 con 0x55cbc5c56450 2024-05-09T04:54:39.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.177+0000 7f07e3200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cbc5c9ae20 0x55cbc5dd34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.177+0000 7f07e2800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbc5c56450 0x55cbc5ddbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.178+0000 7f07e2800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbc5c56450 0x55cbc5ddbef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:51398/0 (socket says 172.21.15.6:51398) 2024-05-09T04:54:39.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.178+0000 7f07e3200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cbc5c9ae20 0x55cbc5dd34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58040/0 (socket says 172.21.15.6:58040) 2024-05-09T04:54:39.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.178+0000 7f07e2800700 1 -- 172.21.15.6:0/3584760523 learned_addr learned my addr 172.21.15.6:0/3584760523 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:39.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.178+0000 7f07e3200700 1 --2- 172.21.15.6:0/3584760523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cbc5dd3a00 0x55cbc5ddef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.178+0000 7f07e2800700 1 -- 172.21.15.6:0/3584760523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cbc5dd3a00 msgr2=0x55cbc5ddef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:39.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.178+0000 7f07e2800700 1 --2- 172.21.15.6:0/3584760523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cbc5dd3a00 0x55cbc5ddef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.178+0000 7f07e2800700 1 -- 172.21.15.6:0/3584760523 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cbc5c9ae20 msgr2=0x55cbc5dd34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:39.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.178+0000 7f07e2800700 1 --2- 172.21.15.6:0/3584760523 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cbc5c9ae20 0x55cbc5dd34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.178+0000 7f07e2800700 1 -- 172.21.15.6:0/3584760523 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbc5dce760 con 0x55cbc5c56450 2024-05-09T04:54:39.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.178+0000 7f07e3200700 1 --2- 172.21.15.6:0/3584760523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cbc5dd3a00 0x55cbc5ddef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:54:39.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.179+0000 7f07e2800700 1 --2- 172.21.15.6:0/3584760523 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbc5c56450 0x55cbc5ddbef0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f07d400fa20 tx=0x7f07d4009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f83f23a59181cf1d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:39.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.179+0000 7f07d3400700 1 -- 172.21.15.6:0/3584760523 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d400f460 con 0x55cbc5c56450 2024-05-09T04:54:39.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.179+0000 7f07d3400700 1 -- 172.21.15.6:0/3584760523 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07d4028070 con 0x55cbc5c56450 2024-05-09T04:54:39.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.179+0000 7f07d3400700 1 -- 172.21.15.6:0/3584760523 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d40236d0 con 0x55cbc5c56450 2024-05-09T04:54:39.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.180+0000 7f07f7ed2b80 1 -- 172.21.15.6:0/3584760523 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbc5c56450 msgr2=0x55cbc5ddbef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:39.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.180+0000 7f07f7ed2b80 1 --2- 172.21.15.6:0/3584760523 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbc5c56450 0x55cbc5ddbef0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f07d400fa20 tx=0x7f07d4009310 comp rx=0 tx=0).stop 2024-05-09T04:54:39.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.180+0000 7f07f7ed2b80 1 -- 172.21.15.6:0/3584760523 shutdown_connections 2024-05-09T04:54:39.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.180+0000 7f07f7ed2b80 1 --2- 172.21.15.6:0/3584760523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cbc5dd3a00 0x55cbc5ddef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.180+0000 7f07f7ed2b80 1 --2- 172.21.15.6:0/3584760523 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cbc5c9ae20 0x55cbc5dd34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.180+0000 7f07f7ed2b80 1 --2- 172.21.15.6:0/3584760523 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbc5c56450 0x55cbc5ddbef0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.180+0000 7f07f7ed2b80 1 -- 172.21.15.6:0/3584760523 >> 172.21.15.6:0/3584760523 conn(0x55cbc5dce170 msgr2=0x55cbc5dd6900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:39.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.180+0000 7f07f7ed2b80 1 -- 172.21.15.6:0/3584760523 shutdown_connections 2024-05-09T04:54:39.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.180+0000 7f07f7ed2b80 1 -- 172.21.15.6:0/3584760523 wait complete. 2024-05-09T04:54:39.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.181+0000 7f07f7ed2b80 1 -- start start 2024-05-09T04:54:39.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.181+0000 7f07f7ed2b80 1 Processor -- start 2024-05-09T04:54:39.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.181+0000 7f07f7ed2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cbc5c56450 0x55cbc5df7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.181+0000 7f07f7ed2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbc5c9ae20 0x55cbc5df81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.181+0000 7f07f7ed2b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cbc5dd3a00 0x55cbc5df2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.181+0000 7f07f7ed2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55cbc5dee6b0 con 0x55cbc5c56450 2024-05-09T04:54:39.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.181+0000 7f07f7ed2b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55cbc5dedfe0 con 0x55cbc5dd3a00 2024-05-09T04:54:39.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.181+0000 7f07f7ed2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55cbc5deed80 con 0x55cbc5c9ae20 2024-05-09T04:54:39.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.182+0000 7f07f7ed2b80 10 client.0 _ll_register_callbacks cb 0x55cbc5c8d540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:39.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.182+0000 7f07e3200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cbc5c56450 0x55cbc5df7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.182+0000 7f07e3200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cbc5c56450 0x55cbc5df7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58054/0 (socket says 172.21.15.6:58054) 2024-05-09T04:54:39.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.182+0000 7f07e3200700 1 -- 172.21.15.6:0/3520522393 learned_addr learned my addr 172.21.15.6:0/3520522393 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:39.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.182+0000 7f07e2800700 1 --2- 172.21.15.6:0/3520522393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbc5c9ae20 0x55cbc5df81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.182+0000 7f07e3200700 1 --2- 172.21.15.6:0/3520522393 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cbc5dd3a00 0x55cbc5df2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.182+0000 7f07e3200700 1 -- 172.21.15.6:0/3520522393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbc5c9ae20 msgr2=0x55cbc5df81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:39.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.182+0000 7f07e3200700 1 --2- 172.21.15.6:0/3520522393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbc5c9ae20 0x55cbc5df81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.182+0000 7f07e3200700 1 -- 172.21.15.6:0/3520522393 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cbc5dd3a00 msgr2=0x55cbc5df2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:39.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.182+0000 7f07e3200700 1 --2- 172.21.15.6:0/3520522393 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cbc5dd3a00 0x55cbc5df2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.182+0000 7f07e3200700 1 -- 172.21.15.6:0/3520522393 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbc5df3350 con 0x55cbc5c56450 2024-05-09T04:54:39.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.183+0000 7f07e3200700 1 --2- 172.21.15.6:0/3520522393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cbc5c56450 0x55cbc5df7cb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f07dc00c010 tx=0x7f07dc0060b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c45338ca56212242 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:39.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.183+0000 7f07d0c00700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-05-09T04:54:39.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.183+0000 7f07d0c00700 1 -- 172.21.15.6:0/3520522393 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07dc020040 con 0x55cbc5c56450 2024-05-09T04:54:39.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.183+0000 7f07f7ed2b80 1 -- 172.21.15.6:0/3520522393 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07d4028070 con 0x55cbc5c56450 2024-05-09T04:54:39.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.183+0000 7f07d0c00700 1 -- 172.21.15.6:0/3520522393 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f07dc003cc0 con 0x55cbc5c56450 2024-05-09T04:54:39.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.183+0000 7f07d0c00700 1 -- 172.21.15.6:0/3520522393 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f07dc03d050 con 0x55cbc5c56450 2024-05-09T04:54:39.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.183+0000 7f07f7ed2b80 10 client.15036 fetch_fsmap learned FSMap version 29 2024-05-09T04:54:39.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.183+0000 7f07f7ed2b80 1 -- 172.21.15.6:0/3520522393 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cbc5e59150 con 0x55cbc5c56450 2024-05-09T04:54:39.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.184+0000 7f07d0c00700 1 -- 172.21.15.6:0/3520522393 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07d4028070 con 0x55cbc5c56450 2024-05-09T04:54:39.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.184+0000 7f07d0c00700 1 -- 172.21.15.6:0/3520522393 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f07dc004200 con 0x55cbc5c56450 2024-05-09T04:54:39.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.184+0000 7f07f7ed2b80 10 client.15036 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T04:54:39.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.184+0000 7f07f7ed2b80 10 client.15036 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-09T04:54:39.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.184+0000 7f07f7ed2b80 20 client.15036 populate_metadata read hostname 'smithi006' 2024-05-09T04:54:39.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.184+0000 7f07f7ed2b80 1 --2- 172.21.15.6:0/3520522393 >> [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] conn(0x55cbc5dd9410 0x55cbc5e5c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.184+0000 7f07f7ed2b80 4 client.15036 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T04:54:39.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.184+0000 7f07f7ed2b80 1 -- 172.21.15.6:0/3520522393 --> [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] -- command(tid 0: {"prefix": "status"}) -- 0x55cbc5d03c40 con 0x55cbc5dd9410 2024-05-09T04:54:39.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.185+0000 7f07e2800700 1 --2- 172.21.15.6:0/3520522393 >> [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] conn(0x55cbc5dd9410 0x55cbc5e5c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.185+0000 7f07e2800700 1 --2- 172.21.15.6:0/3520522393 >> [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] conn(0x55cbc5dd9410 0x55cbc5e5c5c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=262132d5829c9ca0 server_cookie=30082f5a12997295 in_seq=0 out_seq=0 2024-05-09T04:54:39.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.185+0000 7f07d0c00700 10 client.15036 ms_handle_connect on v2:172.21.15.169:6836/1845715294 2024-05-09T04:54:39.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.185+0000 7f07d0c00700 1 -- 172.21.15.6:0/3520522393 <== mds.2 v2:172.21.15.169:6836/1845715294 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55cbc5d03c40 con 0x55cbc5dd9410 2024-05-09T04:54:39.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.185+0000 7f07d0c00700 10 client.15036 handle_command_reply: tid=0 2024-05-09T04:54:39.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.185+0000 7f07f7ed2b80 1 client.15036 shutdown 2024-05-09T04:54:39.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.186+0000 7f07f7ed2b80 1 -- 172.21.15.6:0/3520522393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cbc5c56450 msgr2=0x55cbc5df7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:39.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.186+0000 7f07f7ed2b80 1 --2- 172.21.15.6:0/3520522393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cbc5c56450 0x55cbc5df7cb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f07dc00c010 tx=0x7f07dc0060b0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.186+0000 7f07f7ed2b80 1 -- 172.21.15.6:0/3520522393 shutdown_connections 2024-05-09T04:54:39.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.186+0000 7f07f7ed2b80 1 --2- 172.21.15.6:0/3520522393 >> [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] conn(0x55cbc5dd9410 0x55cbc5e5c5c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.186+0000 7f07f7ed2b80 1 --2- 172.21.15.6:0/3520522393 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cbc5dd3a00 0x55cbc5df2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.186+0000 7f07f7ed2b80 1 --2- 172.21.15.6:0/3520522393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cbc5c56450 0x55cbc5df7cb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.186+0000 7f07f7ed2b80 1 --2- 172.21.15.6:0/3520522393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cbc5c9ae20 0x55cbc5df81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.186+0000 7f07f7ed2b80 1 -- 172.21.15.6:0/3520522393 >> 172.21.15.6:0/3520522393 conn(0x55cbc5dce170 msgr2=0x55cbc5ddbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:39.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.186+0000 7f07d0c00700 0 client.15036 ms_handle_reset on v2:172.21.15.169:6836/1845715294 2024-05-09T04:54:39.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.186+0000 7f07f7ed2b80 1 -- 172.21.15.6:0/3520522393 shutdown_connections 2024-05-09T04:54:39.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.186+0000 7f07f7ed2b80 1 -- 172.21.15.6:0/3520522393 wait complete. 2024-05-09T04:54:39.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.186+0000 7f07f7ed2b80 20 client.15036 trim_cache size 0 max 16384 2024-05-09T04:54:39.190 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-09T04:54:39.190 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "f74d813c-0dbd-11ef-bc97-c7b262605968", 2024-05-09T04:54:39.190 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 2, 2024-05-09T04:54:39.190 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24512, 2024-05-09T04:54:39.190 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-05-09T04:54:39.190 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-05-09T04:54:39.190 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-05-09T04:54:39.190 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 218.453203086, 2024-05-09T04:54:39.190 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 29, 2024-05-09T04:54:39.191 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 89, 2024-05-09T04:54:39.191 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 89, 2024-05-09T04:54:39.191 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 340.84260685999999 2024-05-09T04:54:39.191 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-09T04:54:39.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.187+0000 7f07e9400700 1 -- 172.21.15.6:0/1925331287 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f07c80b2900 msgr2=0x7f07c80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:39.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.187+0000 7f07e9400700 1 --2- 172.21.15.6:0/1925331287 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f07c80b2900 0x7f07c80b4e10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f07cc00cfd0 tx=0x7f07cc009790 comp rx=0 tx=0).stop 2024-05-09T04:54:39.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.187+0000 7f07e9400700 1 -- 172.21.15.6:0/1925331287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07e4106ec0 msgr2=0x7f07e419d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:39.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.187+0000 7f07e9400700 1 --2- 172.21.15.6:0/1925331287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07e4106ec0 0x7f07e419d600 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f07e4075b80 tx=0x7f07cc00eca0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.187+0000 7f07e9400700 1 -- 172.21.15.6:0/1925331287 shutdown_connections 2024-05-09T04:54:39.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.187+0000 7f07e9400700 1 --2- 172.21.15.6:0/1925331287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07e410c5b0 0x7f07e41a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.187+0000 7f07e9400700 1 --2- 172.21.15.6:0/1925331287 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07e4106570 0x7f07e419d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.187+0000 7f07e9400700 1 --2- 172.21.15.6:0/1925331287 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f07c80b2900 0x7f07c80b4e10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.187+0000 7f07e9400700 1 --2- 172.21.15.6:0/1925331287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07e4106ec0 0x7f07e419d600 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.187+0000 7f07e9400700 1 -- 172.21.15.6:0/1925331287 >> 172.21.15.6:0/1925331287 conn(0x7f07e4102050 msgr2=0x7f07e41035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:39.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.187+0000 7f07e9400700 1 -- 172.21.15.6:0/1925331287 shutdown_connections 2024-05-09T04:54:39.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:39.188+0000 7f07e9400700 1 -- 172.21.15.6:0/1925331287 wait complete. 2024-05-09T04:54:39.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:38 smithi186 ceph-mon[34668]: pgmap v490: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:54:39.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:38 smithi169 ceph-mon[34075]: pgmap v490: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:54:40.134 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.g status 2024-05-09T04:54:40.611 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:54:41.118 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:40 smithi006 ceph-mon[29835]: pgmap v491: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:41.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:40 smithi186 ceph-mon[34668]: pgmap v491: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:41.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:40 smithi169 ceph-mon[34075]: pgmap v491: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:42.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.956+0000 7f72c7200700 1 -- 172.21.15.6:0/560055833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72c01054f0 msgr2=0x7f72c0105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:42.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.956+0000 7f72c7200700 1 --2- 172.21.15.6:0/560055833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72c01054f0 0x7f72c0105900 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f72b0009990 tx=0x7f72b0009ca0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.956+0000 7f72c7200700 1 -- 172.21.15.6:0/560055833 shutdown_connections 2024-05-09T04:54:42.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.956+0000 7f72c7200700 1 --2- 172.21.15.6:0/560055833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72c010e920 0x7f72c0110e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.956+0000 7f72c7200700 1 --2- 172.21.15.6:0/560055833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72c01054f0 0x7f72c0105900 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.956+0000 7f72c7200700 1 --2- 172.21.15.6:0/560055833 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72c0105e40 0x7f72c010e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.956+0000 7f72c7200700 1 -- 172.21.15.6:0/560055833 >> 172.21.15.6:0/560055833 conn(0x7f72c0078e00 msgr2=0x7f72c0079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:42.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.957+0000 7f72c7200700 1 -- 172.21.15.6:0/560055833 shutdown_connections 2024-05-09T04:54:42.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.957+0000 7f72c7200700 1 -- 172.21.15.6:0/560055833 wait complete. 2024-05-09T04:54:42.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.958+0000 7f72c7200700 1 Processor -- start 2024-05-09T04:54:42.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.958+0000 7f72c7200700 1 -- start start 2024-05-09T04:54:42.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.958+0000 7f72c7200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72c01054f0 0x7f72c01031e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:42.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.959+0000 7f72c7200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72c0105e40 0x7f72c0103720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:42.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.959+0000 7f72c7200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72c010e920 0x7f72c0103c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:42.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.959+0000 7f72c7200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f72c0114490 con 0x7f72c01054f0 2024-05-09T04:54:42.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.959+0000 7f72c7200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f72c0114310 con 0x7f72c0105e40 2024-05-09T04:54:42.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.959+0000 7f72c7200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f72c0114610 con 0x7f72c010e920 2024-05-09T04:54:42.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.959+0000 7f72bfe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72c01054f0 0x7f72c01031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:42.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.959+0000 7f72bfe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72c01054f0 0x7f72c01031e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58066/0 (socket says 172.21.15.6:58066) 2024-05-09T04:54:42.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.959+0000 7f72bfe00700 1 -- 172.21.15.6:0/1352833276 learned_addr learned my addr 172.21.15.6:0/1352833276 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:42.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.959+0000 7f72c5200700 1 --2- 172.21.15.6:0/1352833276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72c010e920 0x7f72c0103c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:42.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.959+0000 7f72bf400700 1 --2- 172.21.15.6:0/1352833276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72c0105e40 0x7f72c0103720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:42.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.959+0000 7f72bfe00700 1 -- 172.21.15.6:0/1352833276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72c010e920 msgr2=0x7f72c0103c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:42.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.959+0000 7f72bfe00700 1 --2- 172.21.15.6:0/1352833276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72c010e920 0x7f72c0103c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.959+0000 7f72bfe00700 1 -- 172.21.15.6:0/1352833276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72c0105e40 msgr2=0x7f72c0103720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:42.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.960+0000 7f72bfe00700 1 --2- 172.21.15.6:0/1352833276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72c0105e40 0x7f72c0103720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.960+0000 7f72bfe00700 1 -- 172.21.15.6:0/1352833276 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72c01a9280 con 0x7f72c01054f0 2024-05-09T04:54:42.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.960+0000 7f72bf400700 1 --2- 172.21.15.6:0/1352833276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72c0105e40 0x7f72c0103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:54:42.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.960+0000 7f72c5200700 1 --2- 172.21.15.6:0/1352833276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72c010e920 0x7f72c0103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:54:42.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.960+0000 7f72bfe00700 1 --2- 172.21.15.6:0/1352833276 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72c01054f0 0x7f72c01031e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f72b00057f0 tx=0x7f72b000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:42.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.960+0000 7f72bcc00700 1 -- 172.21.15.6:0/1352833276 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b0026070 con 0x7f72c01054f0 2024-05-09T04:54:42.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.960+0000 7f72bcc00700 1 -- 172.21.15.6:0/1352833276 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72b0021410 con 0x7f72c01054f0 2024-05-09T04:54:42.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.960+0000 7f72bcc00700 1 -- 172.21.15.6:0/1352833276 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b0017650 con 0x7f72c01054f0 2024-05-09T04:54:42.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.960+0000 7f72c7200700 1 -- 172.21.15.6:0/1352833276 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72c01a9510 con 0x7f72c01054f0 2024-05-09T04:54:42.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.961+0000 7f72c7200700 1 -- 172.21.15.6:0/1352833276 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72c01a99a0 con 0x7f72c01054f0 2024-05-09T04:54:42.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.964+0000 7f72bcc00700 1 -- 172.21.15.6:0/1352833276 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f72b0029070 con 0x7f72c01054f0 2024-05-09T04:54:42.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.965+0000 7f72d5e30b80 1 Processor -- start 2024-05-09T04:54:42.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.966+0000 7f72d5e30b80 1 -- start start 2024-05-09T04:54:42.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.966+0000 7f72d5e30b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639a1308450 0x5639a148acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:42.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.966+0000 7f72d5e30b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a148b200 0x5639a1485010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:42.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.966+0000 7f72d5e30b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5639a1485550 0x5639a1485960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:42.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.966+0000 7f72d5e30b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5639a13a8200 con 0x5639a1485550 2024-05-09T04:54:42.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.966+0000 7f72d5e30b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5639a1345fd0 con 0x5639a1308450 2024-05-09T04:54:42.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.966+0000 7f72d5e30b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5639a13a41c0 con 0x5639a148b200 2024-05-09T04:54:42.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.967+0000 7f72c5200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639a1308450 0x5639a148acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:42.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.967+0000 7f72c5200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639a1308450 0x5639a148acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:52880/0 (socket says 172.21.15.6:52880) 2024-05-09T04:54:42.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.967+0000 7f72bfe00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a148b200 0x5639a1485010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:42.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.967+0000 7f72c5200700 1 -- 172.21.15.6:0/1930534470 learned_addr learned my addr 172.21.15.6:0/1930534470 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:42.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.967+0000 7f72bf400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5639a1485550 0x5639a1485960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:42.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.967+0000 7f72bcc00700 1 --2- 172.21.15.6:0/1352833276 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f72a40b2880 0x7f72a40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:42.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.967+0000 7f72bcc00700 1 -- 172.21.15.6:0/1352833276 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f72b00e3500 con 0x7f72c01054f0 2024-05-09T04:54:42.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.967+0000 7f72c5200700 1 --2- 172.21.15.6:0/1352833276 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f72a40b2880 0x7f72a40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:42.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.967+0000 7f72bfe00700 1 -- 172.21.15.6:0/1930534470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639a1308450 msgr2=0x5639a148acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:42.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.967+0000 7f72bfe00700 1 --2- 172.21.15.6:0/1930534470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639a1308450 0x5639a148acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.967+0000 7f72bfe00700 1 -- 172.21.15.6:0/1930534470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5639a1485550 msgr2=0x5639a1485960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:42.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.967+0000 7f72bfe00700 1 --2- 172.21.15.6:0/1930534470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5639a1485550 0x5639a1485960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.967+0000 7f72bfe00700 1 -- 172.21.15.6:0/1930534470 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639a1223440 con 0x5639a148b200 2024-05-09T04:54:42.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.968+0000 7f72bfe00700 1 --2- 172.21.15.6:0/1930534470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a148b200 0x5639a1485010 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f72b00e0bf0 tx=0x7f72b0009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9faf1b5ac39fe8a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:42.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.968+0000 7f72c5200700 1 --2- 172.21.15.6:0/1352833276 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f72a40b2880 0x7f72a40b4d90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x5639a148a740 tx=0x7f72b8004b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:42.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.977+0000 7f72b5600700 1 -- 172.21.15.6:0/1930534470 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b00aa5a0 con 0x5639a148b200 2024-05-09T04:54:42.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.977+0000 7f72b5600700 1 -- 172.21.15.6:0/1930534470 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72b00aa2e0 con 0x5639a148b200 2024-05-09T04:54:42.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.977+0000 7f72b5600700 1 -- 172.21.15.6:0/1930534470 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b00b10e0 con 0x5639a148b200 2024-05-09T04:54:42.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.978+0000 7f72d5e30b80 1 -- 172.21.15.6:0/1930534470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a148b200 msgr2=0x5639a1485010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:42.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.978+0000 7f72d5e30b80 1 --2- 172.21.15.6:0/1930534470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a148b200 0x5639a1485010 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f72b00e0bf0 tx=0x7f72b0009ca0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.978+0000 7f72d5e30b80 1 -- 172.21.15.6:0/1930534470 shutdown_connections 2024-05-09T04:54:42.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.978+0000 7f72d5e30b80 1 --2- 172.21.15.6:0/1930534470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5639a1485550 0x5639a1485960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.978+0000 7f72d5e30b80 1 --2- 172.21.15.6:0/1930534470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639a1308450 0x5639a148acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.978+0000 7f72d5e30b80 1 --2- 172.21.15.6:0/1930534470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a148b200 0x5639a1485010 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.978+0000 7f72d5e30b80 1 -- 172.21.15.6:0/1930534470 >> 172.21.15.6:0/1930534470 conn(0x5639a1309740 msgr2=0x5639a147f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:42.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.978+0000 7f72d5e30b80 1 -- 172.21.15.6:0/1930534470 shutdown_connections 2024-05-09T04:54:42.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.978+0000 7f72d5e30b80 1 -- 172.21.15.6:0/1930534470 wait complete. 2024-05-09T04:54:42.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.979+0000 7f72d5e30b80 1 -- start start 2024-05-09T04:54:42.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.979+0000 7f72d5e30b80 1 Processor -- start 2024-05-09T04:54:42.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.980+0000 7f72d5e30b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a1308450 0x5639a148e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:42.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.980+0000 7f72d5e30b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5639a1485550 0x5639a148edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:42.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.980+0000 7f72d5e30b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639a148b200 0x5639a148f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:42.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.980+0000 7f72d5e30b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5639a1491aa0 con 0x5639a1485550 2024-05-09T04:54:42.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.980+0000 7f72d5e30b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5639a1342280 con 0x5639a148b200 2024-05-09T04:54:42.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.980+0000 7f72d5e30b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5639a1491c20 con 0x5639a1308450 2024-05-09T04:54:42.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.980+0000 7f72d5e30b80 10 client.0 _ll_register_callbacks cb 0x5639a133f540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:42.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.980+0000 7f72bf400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5639a1485550 0x5639a148edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:42.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.980+0000 7f72bfe00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a1308450 0x5639a148e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:42.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.980+0000 7f72bf400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5639a1485550 0x5639a148edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58084/0 (socket says 172.21.15.6:58084) 2024-05-09T04:54:42.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.980+0000 7f72bf400700 1 -- 172.21.15.6:0/4034881002 learned_addr learned my addr 172.21.15.6:0/4034881002 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:42.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.980+0000 7f72c5200700 1 --2- 172.21.15.6:0/4034881002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639a148b200 0x5639a148f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:42.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.981+0000 7f72bfe00700 1 -- 172.21.15.6:0/4034881002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639a148b200 msgr2=0x5639a148f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:42.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.981+0000 7f72bfe00700 1 --2- 172.21.15.6:0/4034881002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639a148b200 0x5639a148f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.981+0000 7f72bfe00700 1 -- 172.21.15.6:0/4034881002 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5639a1485550 msgr2=0x5639a148edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:42.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.981+0000 7f72bfe00700 1 --2- 172.21.15.6:0/4034881002 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5639a1485550 0x5639a148edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.981+0000 7f72bfe00700 1 -- 172.21.15.6:0/4034881002 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639a136d800 con 0x5639a1308450 2024-05-09T04:54:42.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.981+0000 7f72bf400700 1 --2- 172.21.15.6:0/4034881002 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5639a1485550 0x5639a148edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:54:42.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.981+0000 7f72bfe00700 1 --2- 172.21.15.6:0/4034881002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a1308450 0x5639a148e880 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f72b0091000 tx=0x7f72b00a6f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb49b5ee5b6b7656 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:42.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.981+0000 7f72aa000700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-09T04:54:42.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.982+0000 7f72aa000700 1 -- 172.21.15.6:0/4034881002 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b00a9f40 con 0x5639a1308450 2024-05-09T04:54:42.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.982+0000 7f72aa000700 1 -- 172.21.15.6:0/4034881002 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f72b00b3110 con 0x5639a1308450 2024-05-09T04:54:42.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.982+0000 7f72d5e30b80 1 -- 172.21.15.6:0/4034881002 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72b00aa2e0 con 0x5639a1308450 2024-05-09T04:54:42.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.982+0000 7f72aa000700 1 -- 172.21.15.6:0/4034881002 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f72b00aa120 con 0x5639a1308450 2024-05-09T04:54:42.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.982+0000 7f72d5e30b80 10 client.25183 fetch_fsmap learned FSMap version 29 2024-05-09T04:54:42.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.982+0000 7f72d5e30b80 1 -- 172.21.15.6:0/4034881002 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5639a1491aa0 con 0x5639a1308450 2024-05-09T04:54:42.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.982+0000 7f72aa000700 1 -- 172.21.15.6:0/4034881002 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72b00aa2e0 con 0x5639a1308450 2024-05-09T04:54:42.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.983+0000 7f72aa000700 1 -- 172.21.15.6:0/4034881002 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f72b00920d0 con 0x5639a1308450 2024-05-09T04:54:42.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.983+0000 7f72d5e30b80 10 client.25183 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T04:54:42.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.983+0000 7f72d5e30b80 10 client.25183 resolve_mds: resolved name 'g' to daemon mds.g 2024-05-09T04:54:42.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.983+0000 7f72d5e30b80 20 client.25183 populate_metadata read hostname 'smithi006' 2024-05-09T04:54:42.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.983+0000 7f72d5e30b80 1 --2- 172.21.15.6:0/4034881002 >> [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] conn(0x5639a149e8e0 0x5639a149fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:42.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.983+0000 7f72d5e30b80 4 client.25183 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:54:42.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.983+0000 7f72d5e30b80 1 -- 172.21.15.6:0/4034881002 --> [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639a13b5be0 con 0x5639a149e8e0 2024-05-09T04:54:42.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.983+0000 7f72bf400700 1 --2- 172.21.15.6:0/4034881002 >> [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] conn(0x5639a149e8e0 0x5639a149fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:42.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.984+0000 7f72bf400700 1 --2- 172.21.15.6:0/4034881002 >> [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] conn(0x5639a149e8e0 0x5639a149fd50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e6ae38bcd5689abe server_cookie=75f59dd0c8c352a8 in_seq=0 out_seq=0 2024-05-09T04:54:42.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.984+0000 7f72aa000700 10 client.25183 ms_handle_connect on v2:172.21.15.6:6838/2780306492 2024-05-09T04:54:42.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.985+0000 7f72aa000700 1 -- 172.21.15.6:0/4034881002 <== mds.1 v2:172.21.15.6:6838/2780306492 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5639a13b5be0 con 0x5639a149e8e0 2024-05-09T04:54:42.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.985+0000 7f72aa000700 10 client.25183 handle_command_reply: tid=0 2024-05-09T04:54:42.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.985+0000 7f72d5e30b80 1 client.25183 shutdown 2024-05-09T04:54:42.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.985+0000 7f72d5e30b80 1 -- 172.21.15.6:0/4034881002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a1308450 msgr2=0x5639a148e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:42.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.985+0000 7f72d5e30b80 1 --2- 172.21.15.6:0/4034881002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a1308450 0x5639a148e880 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f72b0091000 tx=0x7f72b00a6f10 comp rx=0 tx=0).stop 2024-05-09T04:54:42.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.985+0000 7f72d5e30b80 1 -- 172.21.15.6:0/4034881002 shutdown_connections 2024-05-09T04:54:42.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.985+0000 7f72d5e30b80 1 --2- 172.21.15.6:0/4034881002 >> [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] conn(0x5639a149e8e0 0x5639a149fd50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.985+0000 7f72d5e30b80 1 --2- 172.21.15.6:0/4034881002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639a148b200 0x5639a148f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.985+0000 7f72d5e30b80 1 --2- 172.21.15.6:0/4034881002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a1308450 0x5639a148e880 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.985+0000 7f72d5e30b80 1 --2- 172.21.15.6:0/4034881002 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5639a1485550 0x5639a148edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.985+0000 7f72aa000700 0 client.25183 ms_handle_reset on v2:172.21.15.6:6838/2780306492 2024-05-09T04:54:42.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.985+0000 7f72d5e30b80 1 -- 172.21.15.6:0/4034881002 >> 172.21.15.6:0/4034881002 conn(0x5639a1309740 msgr2=0x5639a147f980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:42.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.985+0000 7f72d5e30b80 1 -- 172.21.15.6:0/4034881002 shutdown_connections 2024-05-09T04:54:42.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.986+0000 7f72d5e30b80 1 -- 172.21.15.6:0/4034881002 wait complete. 2024-05-09T04:54:42.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:42.986+0000 7f72d5e30b80 20 client.25183 trim_cache size 0 max 16384 2024-05-09T04:54:43.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.009+0000 7f72d5e30b80 1 Processor -- start 2024-05-09T04:54:43.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.009+0000 7f72d5e30b80 1 -- start start 2024-05-09T04:54:43.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.009+0000 7f72d5e30b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5639a1308450 0x5639a1485010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:43.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.009+0000 7f72d5e30b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a1485550 0x5639a148f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:43.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.009+0000 7f72d5e30b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639a148b200 0x5639a148d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:43.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.009+0000 7f72d5e30b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5639a149e090 con 0x5639a1308450 2024-05-09T04:54:43.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.010+0000 7f72d5e30b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5639a149e760 con 0x5639a148b200 2024-05-09T04:54:43.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.010+0000 7f72d5e30b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5639a149ee30 con 0x5639a1485550 2024-05-09T04:54:43.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.010+0000 7f72bf400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a1485550 0x5639a148f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:43.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.010+0000 7f72bfe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5639a1308450 0x5639a1485010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:43.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.010+0000 7f72bf400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a1485550 0x5639a148f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:51448/0 (socket says 172.21.15.6:51448) 2024-05-09T04:54:43.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.010+0000 7f72bf400700 1 -- 172.21.15.6:0/955938084 learned_addr learned my addr 172.21.15.6:0/955938084 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:43.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.010+0000 7f72c5200700 1 --2- 172.21.15.6:0/955938084 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639a148b200 0x5639a148d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:43.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.010+0000 7f72bf400700 1 -- 172.21.15.6:0/955938084 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639a148b200 msgr2=0x5639a148d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:43.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.010+0000 7f72bf400700 1 --2- 172.21.15.6:0/955938084 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639a148b200 0x5639a148d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:43.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.010+0000 7f72bf400700 1 -- 172.21.15.6:0/955938084 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5639a1308450 msgr2=0x5639a1485010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:43.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.010+0000 7f72bf400700 1 --2- 172.21.15.6:0/955938084 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5639a1308450 0x5639a1485010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:43.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.010+0000 7f72bf400700 1 -- 172.21.15.6:0/955938084 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639a14918d0 con 0x5639a1485550 2024-05-09T04:54:43.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.011+0000 7f72bfe00700 1 --2- 172.21.15.6:0/955938084 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5639a1308450 0x5639a1485010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:54:43.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.011+0000 7f72bf400700 1 --2- 172.21.15.6:0/955938084 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a1485550 0x5639a148f270 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x5639a1491570 tx=0x7f72ac009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5380026c243f5377 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:43.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.011+0000 7f72b4c00700 1 -- 172.21.15.6:0/955938084 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac027070 con 0x5639a1485550 2024-05-09T04:54:43.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.011+0000 7f72b4c00700 1 -- 172.21.15.6:0/955938084 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72ac003680 con 0x5639a1485550 2024-05-09T04:54:43.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.011+0000 7f72b4c00700 1 -- 172.21.15.6:0/955938084 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac021500 con 0x5639a1485550 2024-05-09T04:54:43.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.012+0000 7f72d5e30b80 1 -- 172.21.15.6:0/955938084 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a1485550 msgr2=0x5639a148f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:43.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.012+0000 7f72d5e30b80 1 --2- 172.21.15.6:0/955938084 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a1485550 0x5639a148f270 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x5639a1491570 tx=0x7f72ac009260 comp rx=0 tx=0).stop 2024-05-09T04:54:43.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.012+0000 7f72d5e30b80 1 -- 172.21.15.6:0/955938084 shutdown_connections 2024-05-09T04:54:43.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.012+0000 7f72d5e30b80 1 --2- 172.21.15.6:0/955938084 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639a148b200 0x5639a148d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:43.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.012+0000 7f72d5e30b80 1 --2- 172.21.15.6:0/955938084 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5639a1308450 0x5639a1485010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:43.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.012+0000 7f72d5e30b80 1 --2- 172.21.15.6:0/955938084 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a1485550 0x5639a148f270 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:43.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.012+0000 7f72d5e30b80 1 -- 172.21.15.6:0/955938084 >> 172.21.15.6:0/955938084 conn(0x5639a1309740 msgr2=0x5639a1489a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:43.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.012+0000 7f72d5e30b80 1 -- 172.21.15.6:0/955938084 shutdown_connections 2024-05-09T04:54:43.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.012+0000 7f72d5e30b80 1 -- 172.21.15.6:0/955938084 wait complete. 2024-05-09T04:54:43.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.013+0000 7f72d5e30b80 1 -- start start 2024-05-09T04:54:43.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.013+0000 7f72d5e30b80 1 Processor -- start 2024-05-09T04:54:43.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.013+0000 7f72d5e30b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a1308450 0x5639a1484200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:43.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.014+0000 7f72d5e30b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5639a1485550 0x5639a1484740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:43.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.014+0000 7f72d5e30b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639a148b200 0x5639a1484c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:43.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.014+0000 7f72d5e30b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5639a149fbd0 con 0x5639a1485550 2024-05-09T04:54:43.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.014+0000 7f72d5e30b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5639a149f500 con 0x5639a148b200 2024-05-09T04:54:43.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.014+0000 7f72d5e30b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5639a14a02a0 con 0x5639a1308450 2024-05-09T04:54:43.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.014+0000 7f72d5e30b80 10 client.0 _ll_register_callbacks cb 0x5639a133f540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:43.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.014+0000 7f72bf400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5639a1485550 0x5639a1484740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:43.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.014+0000 7f72bf400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5639a1485550 0x5639a1484740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58108/0 (socket says 172.21.15.6:58108) 2024-05-09T04:54:43.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.014+0000 7f72bf400700 1 -- 172.21.15.6:0/3490182575 learned_addr learned my addr 172.21.15.6:0/3490182575 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:43.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.014+0000 7f72bfe00700 1 --2- 172.21.15.6:0/3490182575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a1308450 0x5639a1484200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:43.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.014+0000 7f72c5200700 1 --2- 172.21.15.6:0/3490182575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639a148b200 0x5639a1484c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:43.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.014+0000 7f72bf400700 1 -- 172.21.15.6:0/3490182575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a1308450 msgr2=0x5639a1484200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:43.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.014+0000 7f72bf400700 1 --2- 172.21.15.6:0/3490182575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a1308450 0x5639a1484200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:43.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.014+0000 7f72bf400700 1 -- 172.21.15.6:0/3490182575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639a148b200 msgr2=0x5639a1484c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:43.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.014+0000 7f72bf400700 1 --2- 172.21.15.6:0/3490182575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639a148b200 0x5639a1484c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:43.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.014+0000 7f72bf400700 1 -- 172.21.15.6:0/3490182575 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639a14a7640 con 0x5639a1485550 2024-05-09T04:54:43.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.015+0000 7f72bf400700 1 --2- 172.21.15.6:0/3490182575 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5639a1485550 0x5639a1484740 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f72ac025040 tx=0x7f72ac004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f00b2f971d3578 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:43.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.015+0000 7f72aa000700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-05-09T04:54:43.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.015+0000 7f72aa000700 1 -- 172.21.15.6:0/3490182575 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac02f020 con 0x5639a1485550 2024-05-09T04:54:43.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.015+0000 7f72d5e30b80 1 -- 172.21.15.6:0/3490182575 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72ac003680 con 0x5639a1485550 2024-05-09T04:54:43.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.015+0000 7f72aa000700 1 -- 172.21.15.6:0/3490182575 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f72ac003830 con 0x5639a1485550 2024-05-09T04:54:43.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.016+0000 7f72aa000700 1 -- 172.21.15.6:0/3490182575 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f72ac03a050 con 0x5639a1485550 2024-05-09T04:54:43.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.016+0000 7f72d5e30b80 10 client.15054 fetch_fsmap learned FSMap version 29 2024-05-09T04:54:43.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.016+0000 7f72d5e30b80 1 -- 172.21.15.6:0/3490182575 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5639a150a040 con 0x5639a1485550 2024-05-09T04:54:43.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.016+0000 7f72aa000700 1 -- 172.21.15.6:0/3490182575 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72ac003680 con 0x5639a1485550 2024-05-09T04:54:43.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.016+0000 7f72aa000700 1 -- 172.21.15.6:0/3490182575 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f72ac02a050 con 0x5639a1485550 2024-05-09T04:54:43.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.016+0000 7f72d5e30b80 10 client.15054 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T04:54:43.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.016+0000 7f72d5e30b80 10 client.15054 resolve_mds: resolved name 'g' to daemon mds.g 2024-05-09T04:54:43.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.016+0000 7f72d5e30b80 20 client.15054 populate_metadata read hostname 'smithi006' 2024-05-09T04:54:43.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.016+0000 7f72d5e30b80 1 --2- 172.21.15.6:0/3490182575 >> [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] conn(0x5639a150c130 0x5639a150d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:43.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.016+0000 7f72d5e30b80 4 client.15054 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T04:54:43.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.016+0000 7f72d5e30b80 1 -- 172.21.15.6:0/3490182575 --> [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] -- command(tid 0: {"prefix": "status"}) -- 0x5639a13afc80 con 0x5639a150c130 2024-05-09T04:54:43.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.017+0000 7f72bfe00700 1 --2- 172.21.15.6:0/3490182575 >> [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] conn(0x5639a150c130 0x5639a150d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:43.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.018+0000 7f72bfe00700 1 --2- 172.21.15.6:0/3490182575 >> [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] conn(0x5639a150c130 0x5639a150d5a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d8e6c164c7b1190 server_cookie=fe8edd10b06f712a in_seq=0 out_seq=0 2024-05-09T04:54:43.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.018+0000 7f72aa000700 10 client.15054 ms_handle_connect on v2:172.21.15.6:6838/2780306492 2024-05-09T04:54:43.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.018+0000 7f72aa000700 1 -- 172.21.15.6:0/3490182575 <== mds.1 v2:172.21.15.6:6838/2780306492 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5639a13afc80 con 0x5639a150c130 2024-05-09T04:54:43.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.018+0000 7f72aa000700 10 client.15054 handle_command_reply: tid=0 2024-05-09T04:54:43.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.018+0000 7f72d5e30b80 1 client.15054 shutdown 2024-05-09T04:54:43.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.018+0000 7f72d5e30b80 1 -- 172.21.15.6:0/3490182575 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5639a1485550 msgr2=0x5639a1484740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:43.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.018+0000 7f72d5e30b80 1 --2- 172.21.15.6:0/3490182575 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5639a1485550 0x5639a1484740 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f72ac025040 tx=0x7f72ac004340 comp rx=0 tx=0).stop 2024-05-09T04:54:43.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.018+0000 7f72d5e30b80 1 -- 172.21.15.6:0/3490182575 shutdown_connections 2024-05-09T04:54:43.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.018+0000 7f72d5e30b80 1 --2- 172.21.15.6:0/3490182575 >> [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] conn(0x5639a150c130 0x5639a150d5a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:43.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.018+0000 7f72d5e30b80 1 --2- 172.21.15.6:0/3490182575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639a148b200 0x5639a1484c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:43.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.018+0000 7f72d5e30b80 1 --2- 172.21.15.6:0/3490182575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a1308450 0x5639a1484200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:43.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.018+0000 7f72aa000700 0 client.15054 ms_handle_reset on v2:172.21.15.6:6838/2780306492 2024-05-09T04:54:43.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.018+0000 7f72d5e30b80 1 --2- 172.21.15.6:0/3490182575 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5639a1485550 0x5639a1484740 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:43.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.018+0000 7f72d5e30b80 1 -- 172.21.15.6:0/3490182575 >> 172.21.15.6:0/3490182575 conn(0x5639a1309740 msgr2=0x5639a148d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:43.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.019+0000 7f72d5e30b80 1 -- 172.21.15.6:0/3490182575 shutdown_connections 2024-05-09T04:54:43.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.019+0000 7f72d5e30b80 1 -- 172.21.15.6:0/3490182575 wait complete. 2024-05-09T04:54:43.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.019+0000 7f72d5e30b80 20 client.15054 trim_cache size 0 max 16384 2024-05-09T04:54:43.022 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-09T04:54:43.022 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "f74d813c-0dbd-11ef-bc97-c7b262605968", 2024-05-09T04:54:43.022 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 1, 2024-05-09T04:54:43.022 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24530, 2024-05-09T04:54:43.022 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-05-09T04:54:43.023 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-05-09T04:54:43.023 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-05-09T04:54:43.023 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 223.29197466700001, 2024-05-09T04:54:43.023 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 29, 2024-05-09T04:54:43.023 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 89, 2024-05-09T04:54:43.023 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 89, 2024-05-09T04:54:43.023 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 336.33334600900002 2024-05-09T04:54:43.023 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-09T04:54:43.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.019+0000 7f72c7200700 1 -- 172.21.15.6:0/1352833276 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f72a40b2880 msgr2=0x7f72a40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:43.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.019+0000 7f72c7200700 1 --2- 172.21.15.6:0/1352833276 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f72a40b2880 0x7f72a40b4d90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x5639a148a740 tx=0x7f72b8004b40 comp rx=0 tx=0).stop 2024-05-09T04:54:43.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.019+0000 7f72c7200700 1 -- 172.21.15.6:0/1352833276 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72c01054f0 msgr2=0x7f72c01031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:43.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.019+0000 7f72c7200700 1 --2- 172.21.15.6:0/1352833276 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72c01054f0 0x7f72c01031e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f72b00057f0 tx=0x7f72b000bfd0 comp rx=0 tx=0).stop 2024-05-09T04:54:43.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.020+0000 7f72c7200700 1 -- 172.21.15.6:0/1352833276 shutdown_connections 2024-05-09T04:54:43.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.020+0000 7f72c7200700 1 --2- 172.21.15.6:0/1352833276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72c010e920 0x7f72c0103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:43.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.020+0000 7f72c7200700 1 --2- 172.21.15.6:0/1352833276 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72c01054f0 0x7f72c01031e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:43.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.020+0000 7f72c7200700 1 --2- 172.21.15.6:0/1352833276 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f72a40b2880 0x7f72a40b4d90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:43.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.020+0000 7f72c7200700 1 --2- 172.21.15.6:0/1352833276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72c0105e40 0x7f72c0103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:43.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.020+0000 7f72c7200700 1 -- 172.21.15.6:0/1352833276 >> 172.21.15.6:0/1352833276 conn(0x7f72c0078e00 msgr2=0x7f72c01012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:43.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.020+0000 7f72c7200700 1 -- 172.21.15.6:0/1352833276 shutdown_connections 2024-05-09T04:54:43.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:43.020+0000 7f72c7200700 1 -- 172.21.15.6:0/1352833276 wait complete. 2024-05-09T04:54:43.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:42 smithi186 ceph-mon[34668]: pgmap v492: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:54:43.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:42 smithi169 ceph-mon[34075]: pgmap v492: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:54:43.419 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:42 smithi006 ceph-mon[29835]: pgmap v492: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:54:43.989 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-09T04:54:44.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:43 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:54:44.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:43 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:54:44.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:43 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:54:44.448 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:54:45.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:44 smithi006 ceph-mon[29835]: pgmap v493: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:45.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:44 smithi186 ceph-mon[34668]: pgmap v493: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:45.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:44 smithi169 ceph-mon[34075]: pgmap v493: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:46.177 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:45 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:54:46.178 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:45 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:54:46.178 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:45 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:54:46.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:45 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:54:46.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:45 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:54:46.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:45 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:54:46.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:45 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:54:46.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:45 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:54:46.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:45 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:54:46.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.845+0000 7fbbae400700 1 -- 172.21.15.6:0/2586783712 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbba81030c0 msgr2=0x7fbba81034d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:46.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.845+0000 7fbbae400700 1 --2- 172.21.15.6:0/2586783712 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbba81030c0 0x7fbba81034d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009990 tx=0x7fbb98009ca0 comp rx=0 tx=0).stop 2024-05-09T04:54:46.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.845+0000 7fbbae400700 1 -- 172.21.15.6:0/2586783712 shutdown_connections 2024-05-09T04:54:46.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.845+0000 7fbbae400700 1 --2- 172.21.15.6:0/2586783712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbba8110c70 0x7fbba8113090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:46.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.845+0000 7fbbae400700 1 --2- 172.21.15.6:0/2586783712 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbba81030c0 0x7fbba81034d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:46.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.845+0000 7fbbae400700 1 --2- 172.21.15.6:0/2586783712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbba8103a10 0x7fbba8110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:46.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.845+0000 7fbbae400700 1 -- 172.21.15.6:0/2586783712 >> 172.21.15.6:0/2586783712 conn(0x7fbba80fedc0 msgr2=0x7fbba81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:46.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.846+0000 7fbbae400700 1 -- 172.21.15.6:0/2586783712 shutdown_connections 2024-05-09T04:54:46.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.846+0000 7fbbae400700 1 -- 172.21.15.6:0/2586783712 wait complete. 2024-05-09T04:54:46.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.847+0000 7fbbae400700 1 Processor -- start 2024-05-09T04:54:46.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.847+0000 7fbbae400700 1 -- start start 2024-05-09T04:54:46.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.847+0000 7fbbae400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbba81030c0 0x7fbba819d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:46.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.847+0000 7fbbae400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbba8103a10 0x7fbba819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:46.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.847+0000 7fbbae400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbba8110c70 0x7fbba81a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:46.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.847+0000 7fbbae400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fbba81166b0 con 0x7fbba8110c70 2024-05-09T04:54:46.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.847+0000 7fbbae400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbba8116530 con 0x7fbba81030c0 2024-05-09T04:54:46.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.847+0000 7fbbae400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbba8116830 con 0x7fbba8103a10 2024-05-09T04:54:46.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.848+0000 7fbba7e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbba8110c70 0x7fbba81a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:46.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.848+0000 7fbba6a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbba8103a10 0x7fbba819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:46.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.848+0000 7fbba6a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbba8103a10 0x7fbba819d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:43266/0 (socket says 172.21.15.6:43266) 2024-05-09T04:54:46.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.848+0000 7fbba6a00700 1 -- 172.21.15.6:0/3529203407 learned_addr learned my addr 172.21.15.6:0/3529203407 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:46.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.848+0000 7fbba7400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbba81030c0 0x7fbba819d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:46.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.848+0000 7fbba6a00700 1 -- 172.21.15.6:0/3529203407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbba81030c0 msgr2=0x7fbba819d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:46.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.848+0000 7fbba6a00700 1 --2- 172.21.15.6:0/3529203407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbba81030c0 0x7fbba819d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:46.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.848+0000 7fbba6a00700 1 -- 172.21.15.6:0/3529203407 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbba8110c70 msgr2=0x7fbba81a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:46.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.848+0000 7fbba6a00700 1 --2- 172.21.15.6:0/3529203407 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbba8110c70 0x7fbba81a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:46.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.848+0000 7fbba6a00700 1 -- 172.21.15.6:0/3529203407 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba81a1f70 con 0x7fbba8103a10 2024-05-09T04:54:46.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.849+0000 7fbba6a00700 1 --2- 172.21.15.6:0/3529203407 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbba8103a10 0x7fbba819d640 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbb9400ea30 tx=0x7fbb9400ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:46.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.849+0000 7fbb9fe00700 1 -- 172.21.15.6:0/3529203407 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9400cbd0 con 0x7fbba8103a10 2024-05-09T04:54:46.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.849+0000 7fbbae400700 1 -- 172.21.15.6:0/3529203407 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbba81a2260 con 0x7fbba8103a10 2024-05-09T04:54:46.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.849+0000 7fbb9fe00700 1 -- 172.21.15.6:0/3529203407 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb940044d0 con 0x7fbba8103a10 2024-05-09T04:54:46.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.849+0000 7fbb9fe00700 1 -- 172.21.15.6:0/3529203407 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb94003dc0 con 0x7fbba8103a10 2024-05-09T04:54:46.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.849+0000 7fbbae400700 1 -- 172.21.15.6:0/3529203407 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbba81a9b80 con 0x7fbba8103a10 2024-05-09T04:54:46.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.854+0000 7fbbbcfc1b80 1 Processor -- start 2024-05-09T04:54:46.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.854+0000 7fbb9fe00700 1 -- 172.21.15.6:0/3529203407 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fbb94004000 con 0x7fbba8103a10 2024-05-09T04:54:46.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.854+0000 7fbbbcfc1b80 1 -- start start 2024-05-09T04:54:46.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.854+0000 7fbbbcfc1b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef35526450 0x55ef356a8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:46.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.854+0000 7fbbbcfc1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef356a9200 0x55ef356a3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:46.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.854+0000 7fbbbcfc1b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ef356a3550 0x55ef356a3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:46.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.854+0000 7fbbbcfc1b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55ef355c6200 con 0x55ef35526450 2024-05-09T04:54:46.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.854+0000 7fbbbcfc1b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ef35563fd0 con 0x55ef356a3550 2024-05-09T04:54:46.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.854+0000 7fbbbcfc1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55ef355c21c0 con 0x55ef356a9200 2024-05-09T04:54:46.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.855+0000 7fbba7e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef35526450 0x55ef356a8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:46.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.855+0000 7fbba7400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef356a9200 0x55ef356a3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:46.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.855+0000 7fbba6a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ef356a3550 0x55ef356a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:46.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.855+0000 7fbba7400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef356a9200 0x55ef356a3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:43270/0 (socket says 172.21.15.6:43270) 2024-05-09T04:54:46.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.855+0000 7fbba7400700 1 -- 172.21.15.6:0/3123302586 learned_addr learned my addr 172.21.15.6:0/3123302586 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:46.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.855+0000 7fbba7e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef35526450 0x55ef356a8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45808/0 (socket says 172.21.15.6:45808) 2024-05-09T04:54:46.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.855+0000 7fbba7400700 1 -- 172.21.15.6:0/3123302586 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ef356a3550 msgr2=0x55ef356a3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:46.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.855+0000 7fbba7400700 1 --2- 172.21.15.6:0/3123302586 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ef356a3550 0x55ef356a3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:46.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.855+0000 7fbba7400700 1 -- 172.21.15.6:0/3123302586 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef35526450 msgr2=0x55ef356a8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:46.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.855+0000 7fbba7400700 1 --2- 172.21.15.6:0/3123302586 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef35526450 0x55ef356a8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:46.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.855+0000 7fbba7400700 1 -- 172.21.15.6:0/3123302586 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef35441440 con 0x55ef356a9200 2024-05-09T04:54:46.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.856+0000 7fbba7400700 1 --2- 172.21.15.6:0/3123302586 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef356a9200 0x55ef356a3010 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbb980096d0 tx=0x7fbb98009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=565e52a13b8de660 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:46.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.856+0000 7fbb9fe00700 1 --2- 172.21.15.6:0/3529203407 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fbb8c0b2880 0x7fbb8c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:46.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.856+0000 7fbb9cc00700 1 -- 172.21.15.6:0/3123302586 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb98027070 con 0x55ef356a9200 2024-05-09T04:54:46.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.856+0000 7fbb9fe00700 1 -- 172.21.15.6:0/3529203407 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fbb940e2700 con 0x7fbba8103a10 2024-05-09T04:54:46.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.856+0000 7fbb9cc00700 1 -- 172.21.15.6:0/3123302586 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb9800bbd0 con 0x55ef356a9200 2024-05-09T04:54:46.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.856+0000 7fbba7e00700 1 --2- 172.21.15.6:0/3529203407 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fbb8c0b2880 0x7fbb8c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:46.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.856+0000 7fbb9cc00700 1 -- 172.21.15.6:0/3123302586 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb98003c70 con 0x55ef356a9200 2024-05-09T04:54:46.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.857+0000 7fbbbcfc1b80 1 -- 172.21.15.6:0/3123302586 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef356a9200 msgr2=0x55ef356a3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:46.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.857+0000 7fbba7e00700 1 --2- 172.21.15.6:0/3529203407 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fbb8c0b2880 0x7fbb8c0b4d90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x55ef356a8740 tx=0x7fbba000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:46.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.857+0000 7fbbbcfc1b80 1 --2- 172.21.15.6:0/3123302586 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef356a9200 0x55ef356a3010 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbb980096d0 tx=0x7fbb98009700 comp rx=0 tx=0).stop 2024-05-09T04:54:46.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.857+0000 7fbbbcfc1b80 1 -- 172.21.15.6:0/3123302586 shutdown_connections 2024-05-09T04:54:46.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.857+0000 7fbbbcfc1b80 1 --2- 172.21.15.6:0/3123302586 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ef356a3550 0x55ef356a3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:46.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.857+0000 7fbbbcfc1b80 1 --2- 172.21.15.6:0/3123302586 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef35526450 0x55ef356a8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:46.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.857+0000 7fbbbcfc1b80 1 --2- 172.21.15.6:0/3123302586 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef356a9200 0x55ef356a3010 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:46.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.857+0000 7fbbbcfc1b80 1 -- 172.21.15.6:0/3123302586 >> 172.21.15.6:0/3123302586 conn(0x55ef35527740 msgr2=0x55ef3569d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:46.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.858+0000 7fbbbcfc1b80 1 -- 172.21.15.6:0/3123302586 shutdown_connections 2024-05-09T04:54:46.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.858+0000 7fbbbcfc1b80 1 -- 172.21.15.6:0/3123302586 wait complete. 2024-05-09T04:54:46.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.858+0000 7fbbbcfc1b80 1 -- start start 2024-05-09T04:54:46.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.858+0000 7fbbbcfc1b80 1 Processor -- start 2024-05-09T04:54:46.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.859+0000 7fbbbcfc1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef35526450 0x55ef356ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:46.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.859+0000 7fbbbcfc1b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef356a3550 0x55ef356acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:46.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.859+0000 7fbbbcfc1b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ef356a9200 0x55ef356ad330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:46.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.859+0000 7fbbbcfc1b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55ef356afc80 con 0x55ef356a3550 2024-05-09T04:54:46.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.859+0000 7fbbbcfc1b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ef35560280 con 0x55ef356a9200 2024-05-09T04:54:46.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.859+0000 7fbbbcfc1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55ef356afe00 con 0x55ef35526450 2024-05-09T04:54:46.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.859+0000 7fbbbcfc1b80 10 client.0 _ll_register_callbacks cb 0x55ef3555d540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:46.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.859+0000 7fbba7400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef35526450 0x55ef356ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:46.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.859+0000 7fbba6a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef356a3550 0x55ef356acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:46.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.860+0000 7fbba6a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef356a3550 0x55ef356acdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45824/0 (socket says 172.21.15.6:45824) 2024-05-09T04:54:46.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.860+0000 7fbba7400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef35526450 0x55ef356ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:43276/0 (socket says 172.21.15.6:43276) 2024-05-09T04:54:46.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.860+0000 7fbba6a00700 1 -- 172.21.15.6:0/1663038844 learned_addr learned my addr 172.21.15.6:0/1663038844 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:46.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.860+0000 7fbba7e00700 1 --2- 172.21.15.6:0/1663038844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ef356a9200 0x55ef356ad330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:46.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.860+0000 7fbba6a00700 1 -- 172.21.15.6:0/1663038844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef35526450 msgr2=0x55ef356ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:46.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.860+0000 7fbba6a00700 1 --2- 172.21.15.6:0/1663038844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef35526450 0x55ef356ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:46.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.860+0000 7fbba6a00700 1 -- 172.21.15.6:0/1663038844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ef356a9200 msgr2=0x55ef356ad330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:46.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.860+0000 7fbba6a00700 1 --2- 172.21.15.6:0/1663038844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ef356a9200 0x55ef356ad330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:46.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.860+0000 7fbba6a00700 1 -- 172.21.15.6:0/1663038844 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef3558b800 con 0x55ef356a3550 2024-05-09T04:54:46.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.860+0000 7fbba7400700 1 --2- 172.21.15.6:0/1663038844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef35526450 0x55ef356ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:54:46.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.861+0000 7fbba6a00700 1 --2- 172.21.15.6:0/1663038844 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef356a3550 0x55ef356acdf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbb940a98d0 tx=0x7fbb940098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd0c572c965690d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:46.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.861+0000 7fbb91600700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-05-09T04:54:46.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.861+0000 7fbb91600700 1 -- 172.21.15.6:0/1663038844 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb940e8020 con 0x55ef356a3550 2024-05-09T04:54:46.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.861+0000 7fbbbcfc1b80 1 -- 172.21.15.6:0/1663038844 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb9800bbd0 con 0x55ef356a3550 2024-05-09T04:54:46.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.861+0000 7fbb91600700 1 -- 172.21.15.6:0/1663038844 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fbb940dff30 con 0x55ef356a3550 2024-05-09T04:54:46.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.861+0000 7fbb91600700 1 -- 172.21.15.6:0/1663038844 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbb940a9290 con 0x55ef356a3550 2024-05-09T04:54:46.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.861+0000 7fbbbcfc1b80 10 client.15066 fetch_fsmap learned FSMap version 29 2024-05-09T04:54:46.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.861+0000 7fbbbcfc1b80 1 -- 172.21.15.6:0/1663038844 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ef356ba880 con 0x55ef356a3550 2024-05-09T04:54:46.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.861+0000 7fbb91600700 1 -- 172.21.15.6:0/1663038844 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb9800bbd0 con 0x55ef356a3550 2024-05-09T04:54:46.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.862+0000 7fbb91600700 1 -- 172.21.15.6:0/1663038844 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbb940a9ba0 con 0x55ef356a3550 2024-05-09T04:54:46.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.862+0000 7fbbbcfc1b80 10 client.15066 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T04:54:46.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.862+0000 7fbbbcfc1b80 10 client.15066 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T04:54:46.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.862+0000 7fbbbcfc1b80 20 client.15066 populate_metadata read hostname 'smithi006' 2024-05-09T04:54:46.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.862+0000 7fbbbcfc1b80 1 --2- 172.21.15.6:0/1663038844 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x55ef356bce60 0x55ef356be2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:46.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.862+0000 7fbbbcfc1b80 4 client.15066 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:54:46.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.862+0000 7fbbbcfc1b80 1 -- 172.21.15.6:0/1663038844 --> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef355d3be0 con 0x55ef356bce60 2024-05-09T04:54:46.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.862+0000 7fbba7400700 1 --2- 172.21.15.6:0/1663038844 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x55ef356bce60 0x55ef356be2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:46.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.863+0000 7fbba7400700 1 --2- 172.21.15.6:0/1663038844 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x55ef356bce60 0x55ef356be2d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d66128b62fee94a5 server_cookie=1146fb34d1110fb2 in_seq=0 out_seq=0 2024-05-09T04:54:46.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.863+0000 7fbb91600700 10 client.15066 ms_handle_connect on v2:172.21.15.169:6838/614388771 2024-05-09T04:54:46.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.864+0000 7fbb91600700 1 -- 172.21.15.6:0/1663038844 <== mds.0 v2:172.21.15.169:6838/614388771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ef355d3be0 con 0x55ef356bce60 2024-05-09T04:54:46.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.864+0000 7fbb91600700 10 client.15066 handle_command_reply: tid=0 2024-05-09T04:54:46.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.864+0000 7fbbbcfc1b80 1 client.15066 shutdown 2024-05-09T04:54:46.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.864+0000 7fbbbcfc1b80 1 -- 172.21.15.6:0/1663038844 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef356a3550 msgr2=0x55ef356acdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:46.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.864+0000 7fbbbcfc1b80 1 --2- 172.21.15.6:0/1663038844 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef356a3550 0x55ef356acdf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbb940a98d0 tx=0x7fbb940098e0 comp rx=0 tx=0).stop 2024-05-09T04:54:46.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.864+0000 7fbbbcfc1b80 1 -- 172.21.15.6:0/1663038844 shutdown_connections 2024-05-09T04:54:46.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.864+0000 7fbbbcfc1b80 1 --2- 172.21.15.6:0/1663038844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ef356a9200 0x55ef356ad330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:46.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.864+0000 7fbbbcfc1b80 1 --2- 172.21.15.6:0/1663038844 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x55ef356bce60 0x55ef356be2d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:46.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.864+0000 7fbbbcfc1b80 1 --2- 172.21.15.6:0/1663038844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef35526450 0x55ef356ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:46.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.864+0000 7fbbbcfc1b80 1 --2- 172.21.15.6:0/1663038844 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef356a3550 0x55ef356acdf0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:46.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.864+0000 7fbbbcfc1b80 1 -- 172.21.15.6:0/1663038844 >> 172.21.15.6:0/1663038844 conn(0x55ef35527740 msgr2=0x55ef3569d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:46.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.864+0000 7fbb91600700 0 client.15066 ms_handle_reset on v2:172.21.15.169:6838/614388771 2024-05-09T04:54:46.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.865+0000 7fbbbcfc1b80 1 -- 172.21.15.6:0/1663038844 shutdown_connections 2024-05-09T04:54:46.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.865+0000 7fbbbcfc1b80 1 -- 172.21.15.6:0/1663038844 wait complete. 2024-05-09T04:54:46.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.865+0000 7fbbbcfc1b80 20 client.15066 trim_cache size 0 max 16384 2024-05-09T04:54:46.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.889+0000 7fbbbcfc1b80 1 Processor -- start 2024-05-09T04:54:46.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.889+0000 7fbbbcfc1b80 1 -- start start 2024-05-09T04:54:46.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.889+0000 7fbbbcfc1b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef35526450 0x55ef356a3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:46.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.889+0000 7fbbbcfc1b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ef356a3550 0x55ef356bc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:46.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.889+0000 7fbbbcfc1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef356a9200 0x55ef356ab810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:46.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.889+0000 7fbbbcfc1b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55ef356bc610 con 0x55ef35526450 2024-05-09T04:54:46.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.889+0000 7fbbbcfc1b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ef356bcce0 con 0x55ef356a3550 2024-05-09T04:54:46.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.889+0000 7fbbbcfc1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55ef356bd3b0 con 0x55ef356a9200 2024-05-09T04:54:46.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.890+0000 7fbba7400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef35526450 0x55ef356a3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:46.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.890+0000 7fbba7400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef35526450 0x55ef356a3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45832/0 (socket says 172.21.15.6:45832) 2024-05-09T04:54:46.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.890+0000 7fbba7400700 1 -- 172.21.15.6:0/4267957060 learned_addr learned my addr 172.21.15.6:0/4267957060 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:46.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.890+0000 7fbba7e00700 1 --2- 172.21.15.6:0/4267957060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef356a9200 0x55ef356ab810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:46.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.890+0000 7fbba6a00700 1 --2- 172.21.15.6:0/4267957060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ef356a3550 0x55ef356bc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:46.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.890+0000 7fbba7400700 1 -- 172.21.15.6:0/4267957060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef356a9200 msgr2=0x55ef356ab810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:46.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.890+0000 7fbba7400700 1 --2- 172.21.15.6:0/4267957060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef356a9200 0x55ef356ab810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:46.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.890+0000 7fbba7400700 1 -- 172.21.15.6:0/4267957060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ef356a3550 msgr2=0x55ef356bc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:46.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.890+0000 7fbba7400700 1 --2- 172.21.15.6:0/4267957060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ef356a3550 0x55ef356bc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:46.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.890+0000 7fbba7400700 1 -- 172.21.15.6:0/4267957060 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef3569d620 con 0x55ef35526450 2024-05-09T04:54:46.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.891+0000 7fbba7400700 1 --2- 172.21.15.6:0/4267957060 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef35526450 0x55ef356a3010 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009990 tx=0x7fbb98003830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d17637660a8c33f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:46.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.891+0000 7fbb93e00700 1 -- 172.21.15.6:0/4267957060 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb98027040 con 0x55ef35526450 2024-05-09T04:54:46.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.891+0000 7fbb93e00700 1 -- 172.21.15.6:0/4267957060 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb9800bbd0 con 0x55ef35526450 2024-05-09T04:54:46.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.891+0000 7fbb93e00700 1 -- 172.21.15.6:0/4267957060 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb98003c60 con 0x55ef35526450 2024-05-09T04:54:46.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.892+0000 7fbbbcfc1b80 1 -- 172.21.15.6:0/4267957060 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef35526450 msgr2=0x55ef356a3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:46.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.892+0000 7fbbbcfc1b80 1 --2- 172.21.15.6:0/4267957060 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef35526450 0x55ef356a3010 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009990 tx=0x7fbb98003830 comp rx=0 tx=0).stop 2024-05-09T04:54:46.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.892+0000 7fbbbcfc1b80 1 -- 172.21.15.6:0/4267957060 shutdown_connections 2024-05-09T04:54:46.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.892+0000 7fbbbcfc1b80 1 --2- 172.21.15.6:0/4267957060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef356a9200 0x55ef356ab810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:46.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.892+0000 7fbbbcfc1b80 1 --2- 172.21.15.6:0/4267957060 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef35526450 0x55ef356a3010 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:46.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.892+0000 7fbbbcfc1b80 1 --2- 172.21.15.6:0/4267957060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ef356a3550 0x55ef356bc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:46.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.892+0000 7fbbbcfc1b80 1 -- 172.21.15.6:0/4267957060 >> 172.21.15.6:0/4267957060 conn(0x55ef35527740 msgr2=0x55ef356a6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:46.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.892+0000 7fbbbcfc1b80 1 -- 172.21.15.6:0/4267957060 shutdown_connections 2024-05-09T04:54:46.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.892+0000 7fbbbcfc1b80 1 -- 172.21.15.6:0/4267957060 wait complete. 2024-05-09T04:54:46.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.893+0000 7fbbbcfc1b80 1 -- start start 2024-05-09T04:54:46.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.893+0000 7fbbbcfc1b80 1 Processor -- start 2024-05-09T04:54:46.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.894+0000 7fbbbcfc1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef35526450 0x55ef356c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:46.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.894+0000 7fbbbcfc1b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef356a3550 0x55ef356ac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:46.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.894+0000 7fbbbcfc1b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ef356a9200 0x55ef356ac610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:46.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.894+0000 7fbbbcfc1b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55ef356be150 con 0x55ef356a3550 2024-05-09T04:54:46.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.894+0000 7fbbbcfc1b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ef356bda80 con 0x55ef356a9200 2024-05-09T04:54:46.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.894+0000 7fbbbcfc1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55ef356be820 con 0x55ef35526450 2024-05-09T04:54:46.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.894+0000 7fbbbcfc1b80 10 client.0 _ll_register_callbacks cb 0x55ef3555d540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:46.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.894+0000 7fbba6a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef356a3550 0x55ef356ac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:46.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.894+0000 7fbba6a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef356a3550 0x55ef356ac0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45842/0 (socket says 172.21.15.6:45842) 2024-05-09T04:54:46.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.894+0000 7fbba6a00700 1 -- 172.21.15.6:0/1985683899 learned_addr learned my addr 172.21.15.6:0/1985683899 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:46.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.894+0000 7fbba7400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef35526450 0x55ef356c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:46.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.894+0000 7fbba7e00700 1 --2- 172.21.15.6:0/1985683899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ef356a9200 0x55ef356ac610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:46.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.895+0000 7fbba6a00700 1 -- 172.21.15.6:0/1985683899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef35526450 msgr2=0x55ef356c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:46.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.895+0000 7fbba6a00700 1 --2- 172.21.15.6:0/1985683899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef35526450 0x55ef356c79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:46.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.895+0000 7fbba6a00700 1 -- 172.21.15.6:0/1985683899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ef356a9200 msgr2=0x55ef356ac610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:46.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.895+0000 7fbba6a00700 1 --2- 172.21.15.6:0/1985683899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ef356a9200 0x55ef356ac610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:46.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.895+0000 7fbba6a00700 1 -- 172.21.15.6:0/1985683899 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef356acb50 con 0x55ef356a3550 2024-05-09T04:54:46.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.895+0000 7fbba7e00700 1 --2- 172.21.15.6:0/1985683899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ef356a9200 0x55ef356ac610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:54:46.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.895+0000 7fbba6a00700 1 --2- 172.21.15.6:0/1985683899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef356a3550 0x55ef356ac0d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbb94009230 tx=0x7fbb940a5930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb55e0a7b8eea69 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:46.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.895+0000 7fbb91600700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-05-09T04:54:46.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.895+0000 7fbb91600700 1 -- 172.21.15.6:0/1985683899 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb940a90b0 con 0x55ef356a3550 2024-05-09T04:54:46.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.895+0000 7fbbbcfc1b80 1 -- 172.21.15.6:0/1985683899 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb9800bbd0 con 0x55ef356a3550 2024-05-09T04:54:46.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.895+0000 7fbb91600700 1 -- 172.21.15.6:0/1985683899 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fbb940b2310 con 0x55ef356a3550 2024-05-09T04:54:46.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.896+0000 7fbb91600700 1 -- 172.21.15.6:0/1985683899 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbb9400ce00 con 0x55ef356a3550 2024-05-09T04:54:46.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.896+0000 7fbbbcfc1b80 10 client.15072 fetch_fsmap learned FSMap version 29 2024-05-09T04:54:46.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.896+0000 7fbbbcfc1b80 1 -- 172.21.15.6:0/1985683899 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ef356c7320 con 0x55ef356a3550 2024-05-09T04:54:46.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.896+0000 7fbb91600700 1 -- 172.21.15.6:0/1985683899 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb9800bbd0 con 0x55ef356a3550 2024-05-09T04:54:46.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.896+0000 7fbb91600700 1 -- 172.21.15.6:0/1985683899 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbb9406f250 con 0x55ef356a3550 2024-05-09T04:54:46.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.896+0000 7fbbbcfc1b80 10 client.15072 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T04:54:46.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.896+0000 7fbbbcfc1b80 10 client.15072 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T04:54:46.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.896+0000 7fbbbcfc1b80 20 client.15072 populate_metadata read hostname 'smithi006' 2024-05-09T04:54:46.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.896+0000 7fbbbcfc1b80 1 --2- 172.21.15.6:0/1985683899 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x55ef3572a3a0 0x55ef3572b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:46.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.896+0000 7fbbbcfc1b80 4 client.15072 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T04:54:46.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.896+0000 7fbbbcfc1b80 1 -- 172.21.15.6:0/1985683899 --> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] -- command(tid 0: {"prefix": "status"}) -- 0x55ef355cdc80 con 0x55ef3572a3a0 2024-05-09T04:54:46.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.897+0000 7fbba7400700 1 --2- 172.21.15.6:0/1985683899 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x55ef3572a3a0 0x55ef3572b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:46.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.897+0000 7fbba7400700 1 --2- 172.21.15.6:0/1985683899 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x55ef3572a3a0 0x55ef3572b810 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36e6a401e4992a31 server_cookie=c63f698b4159ff05 in_seq=0 out_seq=0 2024-05-09T04:54:46.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.897+0000 7fbb91600700 10 client.15072 ms_handle_connect on v2:172.21.15.169:6838/614388771 2024-05-09T04:54:46.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.897+0000 7fbb91600700 1 -- 172.21.15.6:0/1985683899 <== mds.0 v2:172.21.15.169:6838/614388771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ef355cdc80 con 0x55ef3572a3a0 2024-05-09T04:54:46.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.897+0000 7fbb91600700 10 client.15072 handle_command_reply: tid=0 2024-05-09T04:54:46.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.897+0000 7fbbbcfc1b80 1 client.15072 shutdown 2024-05-09T04:54:46.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.898+0000 7fbbbcfc1b80 1 -- 172.21.15.6:0/1985683899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef356a3550 msgr2=0x55ef356ac0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:46.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.898+0000 7fbbbcfc1b80 1 --2- 172.21.15.6:0/1985683899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef356a3550 0x55ef356ac0d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbb94009230 tx=0x7fbb940a5930 comp rx=0 tx=0).stop 2024-05-09T04:54:46.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.898+0000 7fbbbcfc1b80 1 -- 172.21.15.6:0/1985683899 shutdown_connections 2024-05-09T04:54:46.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.898+0000 7fbbbcfc1b80 1 --2- 172.21.15.6:0/1985683899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ef356a9200 0x55ef356ac610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:46.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.898+0000 7fbbbcfc1b80 1 --2- 172.21.15.6:0/1985683899 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x55ef3572a3a0 0x55ef3572b810 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:46.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.898+0000 7fbbbcfc1b80 1 --2- 172.21.15.6:0/1985683899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ef35526450 0x55ef356c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:46.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.898+0000 7fbbbcfc1b80 1 --2- 172.21.15.6:0/1985683899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ef356a3550 0x55ef356ac0d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:46.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.898+0000 7fbb91600700 0 client.15072 ms_handle_reset on v2:172.21.15.169:6838/614388771 2024-05-09T04:54:46.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.898+0000 7fbbbcfc1b80 1 -- 172.21.15.6:0/1985683899 >> 172.21.15.6:0/1985683899 conn(0x55ef35527740 msgr2=0x55ef356a6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:46.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.898+0000 7fbbbcfc1b80 1 -- 172.21.15.6:0/1985683899 shutdown_connections 2024-05-09T04:54:46.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.898+0000 7fbbbcfc1b80 1 -- 172.21.15.6:0/1985683899 wait complete. 2024-05-09T04:54:46.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.898+0000 7fbbbcfc1b80 20 client.15072 trim_cache size 0 max 16384 2024-05-09T04:54:46.902 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-09T04:54:46.902 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "f74d813c-0dbd-11ef-bc97-c7b262605968", 2024-05-09T04:54:46.902 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 0, 2024-05-09T04:54:46.902 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24554, 2024-05-09T04:54:46.902 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-05-09T04:54:46.902 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-05-09T04:54:46.902 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-05-09T04:54:46.902 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 309.74263148099999, 2024-05-09T04:54:46.902 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 29, 2024-05-09T04:54:46.902 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 89, 2024-05-09T04:54:46.902 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 89, 2024-05-09T04:54:46.902 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 331.04603445999999 2024-05-09T04:54:46.903 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-09T04:54:46.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.899+0000 7fbbae400700 1 -- 172.21.15.6:0/3529203407 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fbb8c0b2880 msgr2=0x7fbb8c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:46.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.899+0000 7fbbae400700 1 --2- 172.21.15.6:0/3529203407 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fbb8c0b2880 0x7fbb8c0b4d90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x55ef356a8740 tx=0x7fbba000b040 comp rx=0 tx=0).stop 2024-05-09T04:54:46.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.899+0000 7fbbae400700 1 -- 172.21.15.6:0/3529203407 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbba8103a10 msgr2=0x7fbba819d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:46.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.899+0000 7fbbae400700 1 --2- 172.21.15.6:0/3529203407 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbba8103a10 0x7fbba819d640 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbb9400ea30 tx=0x7fbb9400ed40 comp rx=0 tx=0).stop 2024-05-09T04:54:46.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.899+0000 7fbbae400700 1 -- 172.21.15.6:0/3529203407 shutdown_connections 2024-05-09T04:54:46.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.899+0000 7fbbae400700 1 --2- 172.21.15.6:0/3529203407 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbba8110c70 0x7fbba81a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:46.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.899+0000 7fbbae400700 1 --2- 172.21.15.6:0/3529203407 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fbb8c0b2880 0x7fbb8c0b4d90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:46.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.899+0000 7fbbae400700 1 --2- 172.21.15.6:0/3529203407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbba81030c0 0x7fbba819d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:46.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.899+0000 7fbbae400700 1 --2- 172.21.15.6:0/3529203407 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbba8103a10 0x7fbba819d640 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:46.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.899+0000 7fbbae400700 1 -- 172.21.15.6:0/3529203407 >> 172.21.15.6:0/3529203407 conn(0x7fbba80fedc0 msgr2=0x7fbba8112820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:46.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.899+0000 7fbbae400700 1 -- 172.21.15.6:0/3529203407 shutdown_connections 2024-05-09T04:54:46.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:46.899+0000 7fbbae400700 1 -- 172.21.15.6:0/3529203407 wait complete. 2024-05-09T04:54:47.342 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:46 smithi006 ceph-mon[29835]: pgmap v494: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:54:47.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:46 smithi186 ceph-mon[34668]: pgmap v494: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:54:47.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:46 smithi169 ceph-mon[34075]: pgmap v494: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:54:47.877 INFO:teuthology.run_tasks:Running task sequential... 2024-05-09T04:54:47.888 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-09T04:54:47.888 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-09T04:54:47.889 INFO:teuthology.task.print:Enabling mgr modules 2024-05-09T04:54:47.889 INFO:teuthology.run_tasks:Running task kclient... 2024-05-09T04:54:47.936 INFO:tasks.kclient:Mounting kernel clients... 2024-05-09T04:54:47.936 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-09T04:54:47.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:54:47.936 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2024-05-09T04:54:47.998 INFO:teuthology.orchestra.run.smithi006.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-09T04:54:47.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:54:48.000 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T04:54:48.074 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:48 smithi006 ceph-mon[29835]: pgmap v495: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:54:48.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:54:48.095 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link delete ceph-brx 2024-05-09T04:54:48.212 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-05-09T04:54:48.212 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T04:54:48.212 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T04:54:48.213 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-09T04:54:48.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:48 smithi186 ceph-mon[34668]: pgmap v495: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:54:48.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:48 smithi169 ceph-mon[34075]: pgmap v495: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:54:48.644 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:54:51.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.034+0000 7f5fe3200700 1 -- 172.21.15.6:0/2848034800 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5fe410b8c0 msgr2=0x7f5fe4113220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:51.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.034+0000 7f5fe3200700 1 --2- 172.21.15.6:0/2848034800 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5fe410b8c0 0x7f5fe4113220 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00b0d0 tx=0x7f5fdc00b3e0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.036 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:50 smithi006 ceph-mon[29835]: pgmap v496: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:51.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.034+0000 7f5fe3200700 1 -- 172.21.15.6:0/2848034800 shutdown_connections 2024-05-09T04:54:51.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.034+0000 7f5fe3200700 1 --2- 172.21.15.6:0/2848034800 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5fe410b8c0 0x7f5fe4113220 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.034+0000 7f5fe3200700 1 --2- 172.21.15.6:0/2848034800 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fe4106640 0x7f5fe4106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.034+0000 7f5fe3200700 1 --2- 172.21.15.6:0/2848034800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fe410af70 0x7f5fe410b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.034+0000 7f5fe3200700 1 -- 172.21.15.6:0/2848034800 >> 172.21.15.6:0/2848034800 conn(0x7f5fe4101ed0 msgr2=0x7f5fe4104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:51.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.034+0000 7f5fe3200700 1 -- 172.21.15.6:0/2848034800 shutdown_connections 2024-05-09T04:54:51.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.035+0000 7f5fe3200700 1 -- 172.21.15.6:0/2848034800 wait complete. 2024-05-09T04:54:51.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.036+0000 7f5fe3200700 1 Processor -- start 2024-05-09T04:54:51.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.036+0000 7f5fe3200700 1 -- start start 2024-05-09T04:54:51.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.037+0000 7f5fe3200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fe4106640 0x7f5fe419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:51.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.037+0000 7f5fe3200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fe410af70 0x7f5fe419d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:51.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.037+0000 7f5fe3200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5fe410b8c0 0x7f5fe41a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:51.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.037+0000 7f5fe3200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe41164f0 con 0x7f5fe410b8c0 2024-05-09T04:54:51.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.037+0000 7f5fe3200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe4116370 con 0x7f5fe4106640 2024-05-09T04:54:51.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.037+0000 7f5fe3200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe4116670 con 0x7f5fe410af70 2024-05-09T04:54:51.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.037+0000 7f5fe2800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5fe410b8c0 0x7f5fe41a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:51.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.037+0000 7f5fe1e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fe4106640 0x7f5fe419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:51.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.037+0000 7f5fe2800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5fe410b8c0 0x7f5fe41a15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45854/0 (socket says 172.21.15.6:45854) 2024-05-09T04:54:51.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.037+0000 7f5fe2800700 1 -- 172.21.15.6:0/2358350466 learned_addr learned my addr 172.21.15.6:0/2358350466 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:51.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.037+0000 7f5fe1400700 1 --2- 172.21.15.6:0/2358350466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fe410af70 0x7f5fe419d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:51.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.038+0000 7f5fe1e00700 1 -- 172.21.15.6:0/2358350466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fe410af70 msgr2=0x7f5fe419d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:51.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.038+0000 7f5fe1e00700 1 --2- 172.21.15.6:0/2358350466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fe410af70 0x7f5fe419d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.038+0000 7f5fe1e00700 1 -- 172.21.15.6:0/2358350466 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5fe410b8c0 msgr2=0x7f5fe41a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:51.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.038+0000 7f5fe1e00700 1 --2- 172.21.15.6:0/2358350466 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5fe410b8c0 0x7f5fe41a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.038+0000 7f5fe1e00700 1 -- 172.21.15.6:0/2358350466 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe41a1c50 con 0x7f5fe4106640 2024-05-09T04:54:51.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.038+0000 7f5fe1e00700 1 --2- 172.21.15.6:0/2358350466 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fe4106640 0x7f5fe419cd80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400e9f0 tx=0x7f5fd400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:51.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.038+0000 7f5fdaa00700 1 -- 172.21.15.6:0/2358350466 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd400c9c0 con 0x7f5fe4106640 2024-05-09T04:54:51.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.038+0000 7f5fdaa00700 1 -- 172.21.15.6:0/2358350466 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fd400cb60 con 0x7f5fe4106640 2024-05-09T04:54:51.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.038+0000 7f5fe3200700 1 -- 172.21.15.6:0/2358350466 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe41a1f40 con 0x7f5fe4106640 2024-05-09T04:54:51.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.038+0000 7f5fdaa00700 1 -- 172.21.15.6:0/2358350466 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4018680 con 0x7f5fe4106640 2024-05-09T04:54:51.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.039+0000 7f5fe3200700 1 -- 172.21.15.6:0/2358350466 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe41a9860 con 0x7f5fe4106640 2024-05-09T04:54:51.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.040+0000 7f5fe3200700 1 -- 172.21.15.6:0/2358350466 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fe40512c0 con 0x7f5fe4106640 2024-05-09T04:54:51.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.042+0000 7f5fdaa00700 1 -- 172.21.15.6:0/2358350466 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f5fd4010430 con 0x7f5fe4106640 2024-05-09T04:54:51.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.044+0000 7f5fdaa00700 1 --2- 172.21.15.6:0/2358350466 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f5fc80b2900 0x7f5fc80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:51.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.044+0000 7f5fdaa00700 1 -- 172.21.15.6:0/2358350466 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f5fd40e26d0 con 0x7f5fe4106640 2024-05-09T04:54:51.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.044+0000 7f5fe1400700 1 --2- 172.21.15.6:0/2358350466 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f5fc80b2900 0x7f5fc80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:51.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.045+0000 7f5fe1400700 1 --2- 172.21.15.6:0/2358350466 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f5fc80b2900 0x7f5fc80b4e10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4105990 tx=0x7f5fd0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:51.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.049+0000 7f5fdaa00700 1 -- 172.21.15.6:0/2358350466 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5fd40aa010 con 0x7f5fe4106640 2024-05-09T04:54:51.069 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:50 smithi169 ceph-mon[34075]: pgmap v496: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:51.102 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:50 smithi186 ceph-mon[34668]: pgmap v496: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:51.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.343+0000 7f5fe3200700 1 -- 172.21.15.6:0/2358350466 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5fe402e5f0 con 0x7f5fe4106640 2024-05-09T04:54:51.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.344+0000 7f5fdaa00700 1 -- 172.21.15.6:0/2358350466 <== mon.2 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) ==== 53+0+88 (secure 0 0 0) 0x7f5fd40b2090 con 0x7f5fe4106640 2024-05-09T04:54:51.345 INFO:teuthology.orchestra.run.smithi006.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-09T04:54:51.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.348+0000 7f5fe3200700 1 -- 172.21.15.6:0/2358350466 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f5fc80b2900 msgr2=0x7f5fc80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:51.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.348+0000 7f5fe3200700 1 --2- 172.21.15.6:0/2358350466 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f5fc80b2900 0x7f5fc80b4e10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4105990 tx=0x7f5fd0009260 comp rx=0 tx=0).stop 2024-05-09T04:54:51.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.348+0000 7f5fe3200700 1 -- 172.21.15.6:0/2358350466 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fe4106640 msgr2=0x7f5fe419cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:51.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.348+0000 7f5fe3200700 1 --2- 172.21.15.6:0/2358350466 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fe4106640 0x7f5fe419cd80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400e9f0 tx=0x7f5fd400ed00 comp rx=0 tx=0).stop 2024-05-09T04:54:51.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.348+0000 7f5fe3200700 1 -- 172.21.15.6:0/2358350466 shutdown_connections 2024-05-09T04:54:51.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.348+0000 7f5fe3200700 1 --2- 172.21.15.6:0/2358350466 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5fe410b8c0 0x7f5fe41a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.348+0000 7f5fe3200700 1 --2- 172.21.15.6:0/2358350466 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f5fc80b2900 0x7f5fc80b4e10 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.348+0000 7f5fe3200700 1 --2- 172.21.15.6:0/2358350466 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fe4106640 0x7f5fe419cd80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.348+0000 7f5fe3200700 1 --2- 172.21.15.6:0/2358350466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fe410af70 0x7f5fe419d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.348+0000 7f5fe3200700 1 -- 172.21.15.6:0/2358350466 >> 172.21.15.6:0/2358350466 conn(0x7f5fe4101ed0 msgr2=0x7f5fe410c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:51.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.349+0000 7f5fe3200700 1 -- 172.21.15.6:0/2358350466 shutdown_connections 2024-05-09T04:54:51.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:51.349+0000 7f5fe3200700 1 -- 172.21.15.6:0/2358350466 wait complete. 2024-05-09T04:54:52.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:51 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/2358350466' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T04:54:52.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:51 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/2358350466' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T04:54:52.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:51 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/2358350466' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T04:54:52.316 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T04:54:52.316 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T04:54:52.316 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi006.front.sepia.ceph.com 2024-05-09T04:54:52.316 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T04:54:52.316 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T04:54:52.316 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T04:54:52.316 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T04:54:52.316 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T04:54:52.317 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T04:54:52.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:54:52.317 DEBUG:teuthology.orchestra.run.smithi006:> ip addr 2024-05-09T04:54:52.377 INFO:teuthology.orchestra.run.smithi006.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T04:54:52.378 INFO:teuthology.orchestra.run.smithi006.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T04:54:52.378 INFO:teuthology.orchestra.run.smithi006.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T04:54:52.378 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:54:52.378 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 ::1/128 scope host 2024-05-09T04:54:52.378 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:54:52.378 INFO:teuthology.orchestra.run.smithi006.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T04:54:52.378 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e6 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:54:52.378 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp7s0f0 2024-05-09T04:54:52.378 INFO:teuthology.orchestra.run.smithi006.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T04:54:52.378 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e7 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:54:52.378 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp7s0f1 2024-05-09T04:54:52.378 INFO:teuthology.orchestra.run.smithi006.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T04:54:52.379 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e4 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:54:52.379 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp1s0f0 2024-05-09T04:54:52.379 INFO:teuthology.orchestra.run.smithi006.stdout: inet 172.21.15.6/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-09T04:54:52.379 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft 41673sec preferred_lft 41673sec 2024-05-09T04:54:52.379 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::ec4:7aff:febd:15e4/64 scope link noprefixroute 2024-05-09T04:54:52.379 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:54:52.379 INFO:teuthology.orchestra.run.smithi006.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T04:54:52.379 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e5 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:54:52.379 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp1s0f1 2024-05-09T04:54:52.380 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-09T04:54:52.380 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:54:52.380 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2024-05-09T04:54:52.380 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link add name ceph-brx type bridge 2024-05-09T04:54:52.380 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip addr flush dev ceph-brx 2024-05-09T04:54:52.381 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set ceph-brx up 2024-05-09T04:54:52.381 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-09T04:54:52.381 DEBUG:teuthology.orchestra.run.smithi006:> ') 2024-05-09T04:54:52.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:54:52.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:54:52.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:54:52.636 DEBUG:teuthology.orchestra.run.smithi006:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T04:54:52.706 INFO:teuthology.orchestra.run.smithi006.stdout:1 2024-05-09T04:54:52.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:54:52.712 DEBUG:teuthology.orchestra.run.smithi006:> route 2024-05-09T04:54:52.760 INFO:teuthology.orchestra.run.smithi006.stdout:Kernel IP routing table 2024-05-09T04:54:52.760 INFO:teuthology.orchestra.run.smithi006.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T04:54:52.760 INFO:teuthology.orchestra.run.smithi006.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T04:54:52.760 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T04:54:52.760 INFO:teuthology.orchestra.run.smithi006.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T04:54:52.761 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:54:52.761 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2024-05-09T04:54:52.761 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T04:54:52.761 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T04:54:52.761 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T04:54:52.761 DEBUG:teuthology.orchestra.run.smithi006:> ') 2024-05-09T04:54:52.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:54:52.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:54:52.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:54:52.975 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2024-05-09T04:54:53.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:54:53.023 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list-id 2024-05-09T04:54:53.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:52 smithi186 ceph-mon[34668]: pgmap v497: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:54:53.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:52 smithi169 ceph-mon[34075]: pgmap v497: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:54:53.122 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:54:53.122 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2024-05-09T04:54:53.122 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T04:54:53.122 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-09T04:54:53.122 DEBUG:teuthology.orchestra.run.smithi006:> ') 2024-05-09T04:54:53.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:52 smithi006 ceph-mon[29835]: pgmap v497: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:54:53.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:54:53.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:54:53.315 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-09T04:54:53.315 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:54:53.315 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2024-05-09T04:54:53.315 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-09T04:54:53.315 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-09T04:54:53.315 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-09T04:54:53.315 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-09T04:54:53.315 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-09T04:54:53.315 DEBUG:teuthology.orchestra.run.smithi006:> ') 2024-05-09T04:54:53.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:54:53.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:54:53.605 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:54:53.605 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2024-05-09T04:54:53.605 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set brx.0 up 2024-05-09T04:54:53.605 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set dev brx.0 master ceph-brx 2024-05-09T04:54:53.605 DEBUG:teuthology.orchestra.run.smithi006:> ') 2024-05-09T04:54:53.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:54:53.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:54:53.820 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T04:54:53.820 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T04:54:53.866 INFO:teuthology.orchestra.run.smithi006.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T04:54:53.866 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T04:54:53.866 DEBUG:teuthology.orchestra.run.smithi006:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T04:54:53.967 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-09T04:54:53.967 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-09T04:54:53.967 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/f303924b-50ca-4c64-949c-577236452acd 2024-05-09T04:54:53.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:54:53.967 DEBUG:teuthology.orchestra.run.smithi006:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/f303924b-50ca-4c64-949c-577236452acd /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-05-09T04:54:54.177 INFO:teuthology.orchestra.run.smithi006.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-05-09T04:54:54.177 INFO:teuthology.orchestra.run.smithi006.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-05-09T04:54:54.177 INFO:teuthology.orchestra.run.smithi006.stdout:invalid new device string format 2024-05-09T04:54:54.918 INFO:teuthology.orchestra.run.smithi006.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-05-09T04:54:54.919 INFO:teuthology.orchestra.run.smithi006.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-05-09T04:54:54.919 INFO:teuthology.orchestra.run.smithi006.stdout:invalid new device string format 2024-05-09T04:54:54.919 INFO:teuthology.orchestra.run.smithi006.stdout:mount.ceph: resolved to: "172.21.15.6:6789,172.21.15.169:6789,172.21.15.186:6789" 2024-05-09T04:54:54.919 INFO:teuthology.orchestra.run.smithi006.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.6:6789,172.21.15.169:6789,172.21.15.186:6789:/volumes/qa/sv_1/f303924b-50ca-4c64-949c-577236452acd 2024-05-09T04:54:54.919 INFO:teuthology.orchestra.run.smithi006.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,key=0,fsid=f74d813c-0dbd-11ef-bc97-c7b262605968" will pass to kernel 2024-05-09T04:54:54.925 INFO:teuthology.orchestra.run.smithi006.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-09T04:54:54.926 INFO:teuthology.orchestra.run.smithi006.stdout: You just mounted a file system that supports labels which does not 2024-05-09T04:54:54.926 INFO:teuthology.orchestra.run.smithi006.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-09T04:54:54.926 INFO:teuthology.orchestra.run.smithi006.stdout: applications will generate AVC messages and not be allowed access to 2024-05-09T04:54:54.926 INFO:teuthology.orchestra.run.smithi006.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-09T04:54:54.931 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-09T04:54:54.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:54:54.931 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T04:54:55.027 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T04:54:55.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:54 smithi186 ceph-mon[34668]: pgmap v498: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:55.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:54 smithi169 ceph-mon[34075]: pgmap v498: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:55.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T04:54:55.151 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:54:55.161 INFO:teuthology.orchestra.run.smithi006.stdout:client25216 2024-05-09T04:54:55.161 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T04:54:55.161 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:54:55.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:54 smithi006 ceph-mon[29835]: pgmap v498: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:54:55.216 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T04:54:55.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T04:54:55.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:54:55.389 INFO:teuthology.orchestra.run.smithi006.stdout:client25216 2024-05-09T04:54:55.390 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T04:54:55.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T04:54:55.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:54:55.520 INFO:teuthology.orchestra.run.smithi006.stdout:client25216 2024-05-09T04:54:55.520 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968.client25216/status') 2024-05-09T04:54:55.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:54:55.662 INFO:teuthology.orchestra.run.smithi006.stdout:instance: client.25216 (3)172.21.15.6:0/3355490830 2024-05-09T04:54:55.662 INFO:teuthology.orchestra.run.smithi006.stdout:blocklisted: false 2024-05-09T04:54:55.662 INFO:teuthology.orchestra.run.smithi006.stderr:0+1 records in 2024-05-09T04:54:55.663 INFO:teuthology.orchestra.run.smithi006.stderr:0+1 records out 2024-05-09T04:54:55.663 INFO:teuthology.orchestra.run.smithi006.stderr:70 bytes copied, 9.3239e-05 s, 751 kB/s 2024-05-09T04:54:55.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:54:55.676 INFO:teuthology.run_tasks:Running task exec... 2024-05-09T04:54:55.686 INFO:teuthology.task.exec:Executing custom commands... 2024-05-09T04:54:55.687 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi006.front.sepia.ceph.com 2024-05-09T04:54:55.687 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-05-09T04:54:55.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.971+0000 7f64e8c00640 1 -- 172.21.15.6:0/183853007 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f64e4109ad0 msgr2=0x7f64e410bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:55.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.971+0000 7f64e8c00640 1 --2- 172.21.15.6:0/183853007 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f64e4109ad0 0x7f64e410bef0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009ca0 tx=0x7f64dc02a8c0 comp rx=0 tx=0).stop 2024-05-09T04:54:55.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.972+0000 7f64e8c00640 1 -- 172.21.15.6:0/183853007 shutdown_connections 2024-05-09T04:54:55.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.972+0000 7f64e8c00640 1 --2- 172.21.15.6:0/183853007 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f64e4109ad0 0x7f64e410bef0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:55.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.972+0000 7f64e8c00640 1 --2- 172.21.15.6:0/183853007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64e4104dd0 0x7f64e41051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:55.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.972+0000 7f64e8c00640 1 --2- 172.21.15.6:0/183853007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64e4103b90 0x7f64e4104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:55.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.972+0000 7f64e8c00640 1 -- 172.21.15.6:0/183853007 >> 172.21.15.6:0/183853007 conn(0x7f64e40fed80 msgr2=0x7f64e4101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:55.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.972+0000 7f64e8c00640 1 -- 172.21.15.6:0/183853007 shutdown_connections 2024-05-09T04:54:55.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.972+0000 7f64e8c00640 1 -- 172.21.15.6:0/183853007 wait complete. 2024-05-09T04:54:55.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.973+0000 7f64e8c00640 1 Processor -- start 2024-05-09T04:54:55.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.973+0000 7f64e8c00640 1 -- start start 2024-05-09T04:54:55.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.973+0000 7f64e8c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64e4103b90 0x7f64e4077a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:55.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.973+0000 7f64e8c00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f64e4104dd0 0x7f64e4076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:55.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.973+0000 7f64e8c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64e4109ad0 0x7f64e4076670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:55.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.973+0000 7f64e8c00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f64e410f610 con 0x7f64e4104dd0 2024-05-09T04:54:55.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.973+0000 7f64e8c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f64e410f490 con 0x7f64e4109ad0 2024-05-09T04:54:55.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.973+0000 7f64e8c00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f64e410f790 con 0x7f64e4103b90 2024-05-09T04:54:55.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.974+0000 7f64e3400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64e4103b90 0x7f64e4077a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:55.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.974+0000 7f64e2a00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f64e4104dd0 0x7f64e4076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:55.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.974+0000 7f64e3400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64e4103b90 0x7f64e4077a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:43350/0 (socket says 172.21.15.6:43350) 2024-05-09T04:54:55.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.974+0000 7f64e3400640 1 -- 172.21.15.6:0/207324468 learned_addr learned my addr 172.21.15.6:0/207324468 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:55.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.974+0000 7f64e3e00640 1 --2- 172.21.15.6:0/207324468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64e4109ad0 0x7f64e4076670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:55.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.974+0000 7f64e3400640 1 -- 172.21.15.6:0/207324468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64e4109ad0 msgr2=0x7f64e4076670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:55.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.974+0000 7f64e3400640 1 --2- 172.21.15.6:0/207324468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64e4109ad0 0x7f64e4076670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:55.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.974+0000 7f64e3400640 1 -- 172.21.15.6:0/207324468 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f64e4104dd0 msgr2=0x7f64e4076130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:55.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.974+0000 7f64e3400640 1 --2- 172.21.15.6:0/207324468 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f64e4104dd0 0x7f64e4076130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:55.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.974+0000 7f64e3400640 1 -- 172.21.15.6:0/207324468 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e4076da0 con 0x7f64e4103b90 2024-05-09T04:54:55.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.975+0000 7f64e3400640 1 --2- 172.21.15.6:0/207324468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64e4103b90 0x7f64e4077a90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f64d4005fd0 tx=0x7f64d400dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:55.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.975+0000 7f64dbe00640 1 -- 172.21.15.6:0/207324468 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d4017070 con 0x7f64e4103b90 2024-05-09T04:54:55.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.975+0000 7f64e8c00640 1 -- 172.21.15.6:0/207324468 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64e41a41b0 con 0x7f64e4103b90 2024-05-09T04:54:55.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.975+0000 7f64dbe00640 1 -- 172.21.15.6:0/207324468 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64d4009c30 con 0x7f64e4103b90 2024-05-09T04:54:55.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.975+0000 7f64dbe00640 1 -- 172.21.15.6:0/207324468 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d4003d10 con 0x7f64e4103b90 2024-05-09T04:54:55.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.975+0000 7f64e8c00640 1 -- 172.21.15.6:0/207324468 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64e41a46a0 con 0x7f64e4103b90 2024-05-09T04:54:55.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.976+0000 7f64e8c00640 1 -- 172.21.15.6:0/207324468 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64e41088e0 con 0x7f64e4103b90 2024-05-09T04:54:55.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.979+0000 7f64dbe00640 1 -- 172.21.15.6:0/207324468 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f64d4003eb0 con 0x7f64e4103b90 2024-05-09T04:54:55.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.980+0000 7f64dbe00640 1 --2- 172.21.15.6:0/207324468 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f64c80b2860 0x7f64c80b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:55.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.980+0000 7f64dbe00640 1 -- 172.21.15.6:0/207324468 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f64d40e2150 con 0x7f64e4103b90 2024-05-09T04:54:55.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.980+0000 7f64e2a00640 1 --2- 172.21.15.6:0/207324468 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f64c80b2860 0x7f64c80b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:55.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.981+0000 7f64e2a00640 1 --2- 172.21.15.6:0/207324468 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f64c80b2860 0x7f64c80b4d70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f64d00095f0 tx=0x7f64d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:55.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:55.985+0000 7f64dbe00640 1 -- 172.21.15.6:0/207324468 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f64d4020030 con 0x7f64e4103b90 2024-05-09T04:54:56.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:56.244+0000 7f64e8c00640 1 -- 172.21.15.6:0/207324468 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f64e402dc30 con 0x7f64e4103b90 2024-05-09T04:54:56.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:56.806+0000 7f64dbe00640 1 -- 172.21.15.6:0/207324468 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v30) ==== 118+0+0 (secure 0 0 0) 0x7f64d4009de0 con 0x7f64e4103b90 2024-05-09T04:54:56.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:56.810+0000 7f64e8c00640 1 -- 172.21.15.6:0/207324468 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f64c80b2860 msgr2=0x7f64c80b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:56.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:56.810+0000 7f64e8c00640 1 --2- 172.21.15.6:0/207324468 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f64c80b2860 0x7f64c80b4d70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f64d00095f0 tx=0x7f64d0005f90 comp rx=0 tx=0).stop 2024-05-09T04:54:56.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:56.810+0000 7f64e8c00640 1 -- 172.21.15.6:0/207324468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64e4103b90 msgr2=0x7f64e4077a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:56.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:56.810+0000 7f64e8c00640 1 --2- 172.21.15.6:0/207324468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64e4103b90 0x7f64e4077a90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f64d4005fd0 tx=0x7f64d400dde0 comp rx=0 tx=0).stop 2024-05-09T04:54:56.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:56.810+0000 7f64e8c00640 1 -- 172.21.15.6:0/207324468 shutdown_connections 2024-05-09T04:54:56.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:56.810+0000 7f64e8c00640 1 --2- 172.21.15.6:0/207324468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64e4109ad0 0x7f64e4076670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:56.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:56.810+0000 7f64e8c00640 1 --2- 172.21.15.6:0/207324468 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f64e4104dd0 0x7f64e4076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:56.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:56.810+0000 7f64e8c00640 1 --2- 172.21.15.6:0/207324468 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f64c80b2860 0x7f64c80b4d70 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:56.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:56.810+0000 7f64e8c00640 1 --2- 172.21.15.6:0/207324468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64e4103b90 0x7f64e4077a90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:56.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:56.810+0000 7f64e8c00640 1 -- 172.21.15.6:0/207324468 >> 172.21.15.6:0/207324468 conn(0x7f64e40fed80 msgr2=0x7f64e4105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:56.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:56.811+0000 7f64e8c00640 1 -- 172.21.15.6:0/207324468 shutdown_connections 2024-05-09T04:54:56.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:56.811+0000 7f64e8c00640 1 -- 172.21.15.6:0/207324468 wait complete. 2024-05-09T04:54:56.842 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-05-09T04:54:57.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:56 smithi186 ceph-mon[34668]: pgmap v499: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T04:54:57.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:56 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/207324468' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-09T04:54:57.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:56 smithi186 ceph-mon[34668]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-09T04:54:57.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:56 smithi169 ceph-mon[34075]: pgmap v499: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T04:54:57.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:56 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/207324468' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-09T04:54:57.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:56 smithi169 ceph-mon[34075]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-09T04:54:57.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.127+0000 7fa7b4e00640 1 -- 172.21.15.6:0/3479981855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7b0104eb0 msgr2=0x7fa7b01052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:57.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.127+0000 7fa7b4e00640 1 --2- 172.21.15.6:0/3479981855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7b0104eb0 0x7fa7b01052c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa798009900 tx=0x7fa79802a410 comp rx=0 tx=0).stop 2024-05-09T04:54:57.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.127+0000 7fa7b4e00640 1 -- 172.21.15.6:0/3479981855 shutdown_connections 2024-05-09T04:54:57.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.127+0000 7fa7b4e00640 1 --2- 172.21.15.6:0/3479981855 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7b0109bb0 0x7fa7b010bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.127+0000 7fa7b4e00640 1 --2- 172.21.15.6:0/3479981855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7b0104eb0 0x7fa7b01052c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.127+0000 7fa7b4e00640 1 --2- 172.21.15.6:0/3479981855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7b0103c70 0x7fa7b0104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.127+0000 7fa7b4e00640 1 -- 172.21.15.6:0/3479981855 >> 172.21.15.6:0/3479981855 conn(0x7fa7b00fee60 msgr2=0x7fa7b0101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:57.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.127+0000 7fa7b4e00640 1 -- 172.21.15.6:0/3479981855 shutdown_connections 2024-05-09T04:54:57.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.127+0000 7fa7b4e00640 1 -- 172.21.15.6:0/3479981855 wait complete. 2024-05-09T04:54:57.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.128+0000 7fa7b4e00640 1 Processor -- start 2024-05-09T04:54:57.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.128+0000 7fa7b4e00640 1 -- start start 2024-05-09T04:54:57.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.129+0000 7fa7b4e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7b0103c70 0x7fa7b019c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:57.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.129+0000 7fa7b4e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7b0104eb0 0x7fa7b019ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:57.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.129+0000 7fa7b4e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7b0109bb0 0x7fa7b01a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:57.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.129+0000 7fa7b4e00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b010f7c0 con 0x7fa7b0109bb0 2024-05-09T04:54:57.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.129+0000 7fa7b4e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b010f640 con 0x7fa7b0103c70 2024-05-09T04:54:57.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.129+0000 7fa7b4e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b010f940 con 0x7fa7b0104eb0 2024-05-09T04:54:57.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.129+0000 7fa7ae800640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7b0109bb0 0x7fa7b01a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:57.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.129+0000 7fa7ade00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7b0103c70 0x7fa7b019c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:57.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.129+0000 7fa7a5400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7b0104eb0 0x7fa7b019ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:57.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.129+0000 7fa7ade00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7b0103c70 0x7fa7b019c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:59648/0 (socket says 172.21.15.6:59648) 2024-05-09T04:54:57.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.129+0000 7fa7ade00640 1 -- 172.21.15.6:0/1102364433 learned_addr learned my addr 172.21.15.6:0/1102364433 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:54:57.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.130+0000 7fa7ade00640 1 -- 172.21.15.6:0/1102364433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7b0104eb0 msgr2=0x7fa7b019ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:57.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.130+0000 7fa7ade00640 1 --2- 172.21.15.6:0/1102364433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7b0104eb0 0x7fa7b019ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.130+0000 7fa7ade00640 1 -- 172.21.15.6:0/1102364433 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7b0109bb0 msgr2=0x7fa7b01a3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:57.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.130+0000 7fa7ade00640 1 --2- 172.21.15.6:0/1102364433 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7b0109bb0 0x7fa7b01a3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.130+0000 7fa7ade00640 1 -- 172.21.15.6:0/1102364433 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b01a4090 con 0x7fa7b0103c70 2024-05-09T04:54:57.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.130+0000 7fa7a5400640 1 --2- 172.21.15.6:0/1102364433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7b0104eb0 0x7fa7b019ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:54:57.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.130+0000 7fa7ade00640 1 --2- 172.21.15.6:0/1102364433 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7b0103c70 0x7fa7b019c550 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa7a000e9f0 tx=0x7fa7a000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:57.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.131+0000 7fa7a7400640 1 -- 172.21.15.6:0/1102364433 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a000ccc0 con 0x7fa7b0103c70 2024-05-09T04:54:57.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.131+0000 7fa7a7400640 1 -- 172.21.15.6:0/1102364433 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7a00035f0 con 0x7fa7b0103c70 2024-05-09T04:54:57.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.131+0000 7fa7a7400640 1 -- 172.21.15.6:0/1102364433 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a000ce60 con 0x7fa7b0103c70 2024-05-09T04:54:57.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.131+0000 7fa7b4e00640 1 -- 172.21.15.6:0/1102364433 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b01a4380 con 0x7fa7b0103c70 2024-05-09T04:54:57.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.131+0000 7fa7b4e00640 1 -- 172.21.15.6:0/1102364433 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b01a4850 con 0x7fa7b0103c70 2024-05-09T04:54:57.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.132+0000 7fa7b4e00640 1 -- 172.21.15.6:0/1102364433 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7b01a4b10 con 0x7fa7b0103c70 2024-05-09T04:54:57.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.136+0000 7fa7a7400640 1 -- 172.21.15.6:0/1102364433 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fa7a0003aa0 con 0x7fa7b0103c70 2024-05-09T04:54:57.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.137+0000 7fa7a7400640 1 --2- 172.21.15.6:0/1102364433 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa7940b2860 0x7fa7940b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:57.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.138+0000 7fa7a7400640 1 -- 172.21.15.6:0/1102364433 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fa7a00e2920 con 0x7fa7b0103c70 2024-05-09T04:54:57.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.138+0000 7fa7a5400640 1 --2- 172.21.15.6:0/1102364433 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa7940b2860 0x7fa7940b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:57.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.139+0000 7fa7a5400640 1 --2- 172.21.15.6:0/1102364433 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa7940b2860 0x7fa7940b4d70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa798004770 tx=0x7fa7980314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:57.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.139+0000 7fa7a7400640 1 -- 172.21.15.6:0/1102364433 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa7a00aa260 con 0x7fa7b0103c70 2024-05-09T04:54:57.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:56 smithi006 ceph-mon[29835]: pgmap v499: 129 pgs: 129 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T04:54:57.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:56 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/207324468' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-09T04:54:57.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:56 smithi006 ceph-mon[29835]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-09T04:54:57.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.382+0000 7fa7b4e00640 1 -- 172.21.15.6:0/1102364433 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fa7b0100990 con 0x7fa7940b2860 2024-05-09T04:54:57.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.415+0000 7fa7a7400640 1 -- 172.21.15.6:0/1102364433 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fa7b0100990 con 0x7fa7940b2860 2024-05-09T04:54:57.416 INFO:teuthology.orchestra.run.smithi006.stdout:{} 2024-05-09T04:54:57.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.418+0000 7fa7b4e00640 1 -- 172.21.15.6:0/1102364433 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa7940b2860 msgr2=0x7fa7940b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:57.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.418+0000 7fa7b4e00640 1 --2- 172.21.15.6:0/1102364433 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa7940b2860 0x7fa7940b4d70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa798004770 tx=0x7fa7980314a0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.418+0000 7fa7b4e00640 1 -- 172.21.15.6:0/1102364433 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7b0103c70 msgr2=0x7fa7b019c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:57.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.418+0000 7fa7b4e00640 1 --2- 172.21.15.6:0/1102364433 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7b0103c70 0x7fa7b019c550 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa7a000e9f0 tx=0x7fa7a000eeb0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.418+0000 7fa7b4e00640 1 -- 172.21.15.6:0/1102364433 shutdown_connections 2024-05-09T04:54:57.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.418+0000 7fa7b4e00640 1 --2- 172.21.15.6:0/1102364433 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7b0109bb0 0x7fa7b01a3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.418+0000 7fa7b4e00640 1 --2- 172.21.15.6:0/1102364433 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa7940b2860 0x7fa7940b4d70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.418+0000 7fa7b4e00640 1 --2- 172.21.15.6:0/1102364433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7b0104eb0 0x7fa7b019ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.418+0000 7fa7b4e00640 1 --2- 172.21.15.6:0/1102364433 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7b0103c70 0x7fa7b019c550 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.418+0000 7fa7b4e00640 1 -- 172.21.15.6:0/1102364433 >> 172.21.15.6:0/1102364433 conn(0x7fa7b00fee60 msgr2=0x7fa7b00ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:57.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.418+0000 7fa7b4e00640 1 -- 172.21.15.6:0/1102364433 shutdown_connections 2024-05-09T04:54:57.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:54:57.418+0000 7fa7b4e00640 1 -- 172.21.15.6:0/1102364433 wait complete. 2024-05-09T04:54:57.446 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-09T04:54:57.457 INFO:teuthology.run_tasks:Running task quiescer... 2024-05-09T04:54:57.492 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-09T04:54:57.492 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T04:54:57.698 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:54:57 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:54:57.393+0000 7f9397400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T04:54:57.698 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:54:57 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:54:57.393+0000 7f9397400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T04:54:57.698 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:54:57 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:54:57.393+0000 7f9397400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T04:54:57.698 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:54:57 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:54:57.393+0000 7f9397400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T04:54:57.698 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:54:57 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:54:57.393+0000 7f9397400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T04:54:57.973 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:54:58.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:57 smithi186 ceph-mon[34668]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-09T04:54:58.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:57 smithi186 ceph-mon[34668]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T04:54:58.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:57 smithi169 ceph-mon[34075]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-09T04:54:58.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:57 smithi169 ceph-mon[34075]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T04:54:58.134 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:57 smithi006 ceph-mon[29835]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-09T04:54:58.135 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:57 smithi006 ceph-mon[29835]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T04:54:59.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:58 smithi186 ceph-mon[34668]: from='client.25238 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:54:59.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:58 smithi186 ceph-mon[34668]: pgmap v500: 129 pgs: 129 active+clean; 648 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-09T04:54:59.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:58 smithi169 ceph-mon[34075]: from='client.25238 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:54:59.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:58 smithi169 ceph-mon[34075]: pgmap v500: 129 pgs: 129 active+clean; 648 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-09T04:54:59.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:58 smithi006 ceph-mon[29835]: from='client.25238 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:54:59.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:58 smithi006 ceph-mon[29835]: pgmap v500: 129 pgs: 129 active+clean; 648 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-09T04:54:59.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:54:59 smithi006 ceph-mon[29835]: mgrmap e19: x(active, since 14m), standbys: z, y 2024-05-09T04:55:00.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:54:59 smithi169 ceph-mon[34075]: mgrmap e19: x(active, since 14m), standbys: z, y 2024-05-09T04:55:00.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:54:59 smithi186 ceph-mon[34668]: mgrmap e19: x(active, since 14m), standbys: z, y 2024-05-09T04:55:00.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.584+0000 7f86b2800700 1 -- 172.21.15.6:0/2905312228 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86ac1045f0 msgr2=0x7f86ac104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:00.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.584+0000 7f86b2800700 1 --2- 172.21.15.6:0/2905312228 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86ac1045f0 0x7f86ac104a00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f869c0099c0 tx=0x7f869c009cd0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.584+0000 7f86b2800700 1 -- 172.21.15.6:0/2905312228 shutdown_connections 2024-05-09T04:55:00.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.584+0000 7f86b2800700 1 --2- 172.21.15.6:0/2905312228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86ac109870 0x7f86ac1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.584+0000 7f86b2800700 1 --2- 172.21.15.6:0/2905312228 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86ac1045f0 0x7f86ac104a00 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.584+0000 7f86b2800700 1 --2- 172.21.15.6:0/2905312228 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86ac108f20 0x7f86ac109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.584+0000 7f86b2800700 1 -- 172.21.15.6:0/2905312228 >> 172.21.15.6:0/2905312228 conn(0x7f86ac100040 msgr2=0x7f86ac102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:00.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.584+0000 7f86b2800700 1 -- 172.21.15.6:0/2905312228 shutdown_connections 2024-05-09T04:55:00.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.584+0000 7f86b2800700 1 -- 172.21.15.6:0/2905312228 wait complete. 2024-05-09T04:55:00.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.586+0000 7f86b2800700 1 Processor -- start 2024-05-09T04:55:00.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.586+0000 7f86b2800700 1 -- start start 2024-05-09T04:55:00.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.586+0000 7f86b2800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86ac1045f0 0x7f86ac19d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:00.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.587+0000 7f86b2800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86ac108f20 0x7f86ac19d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:00.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.587+0000 7f86b2800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86ac109870 0x7f86ac1a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:00.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.587+0000 7f86b2800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f86ac1144a0 con 0x7f86ac108f20 2024-05-09T04:55:00.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.587+0000 7f86b2800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f86ac114320 con 0x7f86ac109870 2024-05-09T04:55:00.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.587+0000 7f86b2800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f86ac114620 con 0x7f86ac1045f0 2024-05-09T04:55:00.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.587+0000 7f86ab400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86ac1045f0 0x7f86ac19d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:00.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.587+0000 7f86aaa00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86ac108f20 0x7f86ac19d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:00.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.587+0000 7f86ab400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86ac1045f0 0x7f86ac19d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:45954/0 (socket says 172.21.15.6:45954) 2024-05-09T04:55:00.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.587+0000 7f86ab400700 1 -- 172.21.15.6:0/199047152 learned_addr learned my addr 172.21.15.6:0/199047152 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:55:00.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.587+0000 7f86abe00700 1 --2- 172.21.15.6:0/199047152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86ac109870 0x7f86ac1a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:00.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.587+0000 7f86ab400700 1 -- 172.21.15.6:0/199047152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86ac109870 msgr2=0x7f86ac1a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:00.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.587+0000 7f86ab400700 1 --2- 172.21.15.6:0/199047152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86ac109870 0x7f86ac1a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.587+0000 7f86ab400700 1 -- 172.21.15.6:0/199047152 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86ac108f20 msgr2=0x7f86ac19d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:00.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.587+0000 7f86ab400700 1 --2- 172.21.15.6:0/199047152 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86ac108f20 0x7f86ac19d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.587+0000 7f86ab400700 1 -- 172.21.15.6:0/199047152 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86ac1a1f00 con 0x7f86ac1045f0 2024-05-09T04:55:00.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.588+0000 7f86ab400700 1 --2- 172.21.15.6:0/199047152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86ac1045f0 0x7f86ac19d030 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f869c006010 tx=0x7f869c004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:00.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.588+0000 7f86a3e00700 1 -- 172.21.15.6:0/199047152 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c026070 con 0x7f86ac1045f0 2024-05-09T04:55:00.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.588+0000 7f86a3e00700 1 -- 172.21.15.6:0/199047152 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f869c004630 con 0x7f86ac1045f0 2024-05-09T04:55:00.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.588+0000 7f86b2800700 1 -- 172.21.15.6:0/199047152 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86ac1a2190 con 0x7f86ac1045f0 2024-05-09T04:55:00.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.588+0000 7f86a3e00700 1 -- 172.21.15.6:0/199047152 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c00fb20 con 0x7f86ac1045f0 2024-05-09T04:55:00.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.588+0000 7f86b2800700 1 -- 172.21.15.6:0/199047152 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86ac1a9b10 con 0x7f86ac1045f0 2024-05-09T04:55:00.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.590+0000 7f86b2800700 1 -- 172.21.15.6:0/199047152 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86ac105aa0 con 0x7f86ac1045f0 2024-05-09T04:55:00.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.592+0000 7f86a3e00700 1 -- 172.21.15.6:0/199047152 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 19) ==== 149702+0+0 (secure 0 0 0) 0x7f869c029070 con 0x7f86ac1045f0 2024-05-09T04:55:00.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.594+0000 7f86a3e00700 1 --2- 172.21.15.6:0/199047152 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f86900b2990 0x7f86900b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:00.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.594+0000 7f86a3e00700 1 -- 172.21.15.6:0/199047152 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f869c0e33e0 con 0x7f86ac1045f0 2024-05-09T04:55:00.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.594+0000 7f86aaa00700 1 --2- 172.21.15.6:0/199047152 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f86900b2990 0x7f86900b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:00.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.595+0000 7f86aaa00700 1 --2- 172.21.15.6:0/199047152 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f86900b2990 0x7f86900b4ea0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f86ac10a810 tx=0x7f8694009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:00.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.596+0000 7f86a3e00700 1 -- 172.21.15.6:0/199047152 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f869c013070 con 0x7f86ac1045f0 2024-05-09T04:55:00.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.835+0000 7f86b2800700 1 -- 172.21.15.6:0/199047152 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f86ac110540 con 0x7f86900b2990 2024-05-09T04:55:00.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.869+0000 7f86a3e00700 1 -- 172.21.15.6:0/199047152 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340556 (secure 0 0 0) 0x7f86ac110540 con 0x7f86900b2990 2024-05-09T04:55:00.878 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:55:00.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.879+0000 7f86b2800700 1 -- 172.21.15.6:0/199047152 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f86900b2990 msgr2=0x7f86900b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:00.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.879+0000 7f86b2800700 1 --2- 172.21.15.6:0/199047152 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f86900b2990 0x7f86900b4ea0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f86ac10a810 tx=0x7f8694009260 comp rx=0 tx=0).stop 2024-05-09T04:55:00.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.879+0000 7f86b2800700 1 -- 172.21.15.6:0/199047152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86ac1045f0 msgr2=0x7f86ac19d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:00.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.879+0000 7f86b2800700 1 --2- 172.21.15.6:0/199047152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86ac1045f0 0x7f86ac19d030 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f869c006010 tx=0x7f869c004270 comp rx=0 tx=0).stop 2024-05-09T04:55:00.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.879+0000 7f86b2800700 1 -- 172.21.15.6:0/199047152 shutdown_connections 2024-05-09T04:55:00.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.879+0000 7f86b2800700 1 --2- 172.21.15.6:0/199047152 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f86900b2990 0x7f86900b4ea0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.879+0000 7f86b2800700 1 --2- 172.21.15.6:0/199047152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86ac109870 0x7f86ac1a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.879+0000 7f86b2800700 1 --2- 172.21.15.6:0/199047152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86ac1045f0 0x7f86ac19d030 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.879+0000 7f86b2800700 1 --2- 172.21.15.6:0/199047152 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86ac108f20 0x7f86ac19d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.879+0000 7f86b2800700 1 -- 172.21.15.6:0/199047152 >> 172.21.15.6:0/199047152 conn(0x7f86ac100040 msgr2=0x7f86ac108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:00.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.879+0000 7f86b2800700 1 -- 172.21.15.6:0/199047152 shutdown_connections 2024-05-09T04:55:00.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:00.879+0000 7f86b2800700 1 -- 172.21.15.6:0/199047152 wait complete. 2024-05-09T04:55:00.882 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-05-09T04:55:01.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:00 smithi169 ceph-mon[34075]: pgmap v501: 129 pgs: 129 active+clean; 648 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-09T04:55:01.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:00 smithi006 ceph-mon[29835]: pgmap v501: 129 pgs: 129 active+clean; 648 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-09T04:55:01.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:00 smithi186 ceph-mon[34668]: pgmap v501: 129 pgs: 129 active+clean; 648 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-09T04:55:01.837 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":501,"stamp":"2024-05-09T04:54:59.712219+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663197,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":2454,"num_read_kb":1334,"num_write":390,"num_write_kb":2308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":823,"ondisk_log_size":823,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":367468,"kb_used_data":14156,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124574356,"statfs":{"total":1151940427776,"available":1151564140544,"internally_reserved":0,"allocated":14495744,"data_stored":8091087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19137,"internal_metadata":361674047},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":620,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":62,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006709"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.559597+0000","last_change":"2024-05-09T04:49:34.144921+0000","last_active":"2024-05-09T04:52:55.559597+0000","last_peered":"2024-05-09T04:52:55.559597+0000","last_clean":"2024-05-09T04:52:55.559597+0000","last_became_active":"2024-05-09T04:49:34.144275+0000","last_became_peered":"2024-05-09T04:49:34.144275+0000","last_unstale":"2024-05-09T04:52:55.559597+0000","last_undegraded":"2024-05-09T04:52:55.559597+0000","last_fullsized":"2024-05-09T04:52:55.559597+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:00:31.377738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.559444+0000","last_change":"2024-05-09T04:49:34.158057+0000","last_active":"2024-05-09T04:52:55.559444+0000","last_peered":"2024-05-09T04:52:55.559444+0000","last_clean":"2024-05-09T04:52:55.559444+0000","last_became_active":"2024-05-09T04:49:34.157129+0000","last_became_peered":"2024-05-09T04:49:34.157129+0000","last_unstale":"2024-05-09T04:52:55.559444+0000","last_undegraded":"2024-05-09T04:52:55.559444+0000","last_fullsized":"2024-05-09T04:52:55.559444+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:49.360309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.554527+0000","last_change":"2024-05-09T04:49:34.145519+0000","last_active":"2024-05-09T04:52:55.554527+0000","last_peered":"2024-05-09T04:52:55.554527+0000","last_clean":"2024-05-09T04:52:55.554527+0000","last_became_active":"2024-05-09T04:49:34.144501+0000","last_became_peered":"2024-05-09T04:49:34.144501+0000","last_unstale":"2024-05-09T04:52:55.554527+0000","last_undegraded":"2024-05-09T04:52:55.554527+0000","last_fullsized":"2024-05-09T04:52:55.554527+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:05:53.727520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.521578+0000","last_change":"2024-05-09T04:49:34.134383+0000","last_active":"2024-05-09T04:52:55.521578+0000","last_peered":"2024-05-09T04:52:55.521578+0000","last_clean":"2024-05-09T04:52:55.521578+0000","last_became_active":"2024-05-09T04:49:34.134049+0000","last_became_peered":"2024-05-09T04:49:34.134049+0000","last_unstale":"2024-05-09T04:52:55.521578+0000","last_undegraded":"2024-05-09T04:52:55.521578+0000","last_fullsized":"2024-05-09T04:52:55.521578+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:36:02.548188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.535797+0000","last_change":"2024-05-09T04:49:34.157897+0000","last_active":"2024-05-09T04:52:55.535797+0000","last_peered":"2024-05-09T04:52:55.535797+0000","last_clean":"2024-05-09T04:52:55.535797+0000","last_became_active":"2024-05-09T04:49:34.157516+0000","last_became_peered":"2024-05-09T04:49:34.157516+0000","last_unstale":"2024-05-09T04:52:55.535797+0000","last_undegraded":"2024-05-09T04:52:55.535797+0000","last_fullsized":"2024-05-09T04:52:55.535797+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:33:25.444155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.535225+0000","last_change":"2024-05-09T04:49:34.146389+0000","last_active":"2024-05-09T04:52:55.535225+0000","last_peered":"2024-05-09T04:52:55.535225+0000","last_clean":"2024-05-09T04:52:55.535225+0000","last_became_active":"2024-05-09T04:49:34.145994+0000","last_became_peered":"2024-05-09T04:49:34.145994+0000","last_unstale":"2024-05-09T04:52:55.535225+0000","last_undegraded":"2024-05-09T04:52:55.535225+0000","last_fullsized":"2024-05-09T04:52:55.535225+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:59.094826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.537694+0000","last_change":"2024-05-09T04:49:34.146428+0000","last_active":"2024-05-09T04:52:55.537694+0000","last_peered":"2024-05-09T04:52:55.537694+0000","last_clean":"2024-05-09T04:52:55.537694+0000","last_became_active":"2024-05-09T04:49:34.146045+0000","last_became_peered":"2024-05-09T04:49:34.146045+0000","last_unstale":"2024-05-09T04:52:55.537694+0000","last_undegraded":"2024-05-09T04:52:55.537694+0000","last_fullsized":"2024-05-09T04:52:55.537694+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:13.129474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.559750+0000","last_change":"2024-05-09T04:49:34.135494+0000","last_active":"2024-05-09T04:52:55.559750+0000","last_peered":"2024-05-09T04:52:55.559750+0000","last_clean":"2024-05-09T04:52:55.559750+0000","last_became_active":"2024-05-09T04:49:34.135085+0000","last_became_peered":"2024-05-09T04:49:34.135085+0000","last_unstale":"2024-05-09T04:52:55.559750+0000","last_undegraded":"2024-05-09T04:52:55.559750+0000","last_fullsized":"2024-05-09T04:52:55.559750+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:57.023344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.541612+0000","last_change":"2024-05-09T04:49:34.131353+0000","last_active":"2024-05-09T04:52:55.541612+0000","last_peered":"2024-05-09T04:52:55.541612+0000","last_clean":"2024-05-09T04:52:55.541612+0000","last_became_active":"2024-05-09T04:49:34.130954+0000","last_became_peered":"2024-05-09T04:49:34.130954+0000","last_unstale":"2024-05-09T04:52:55.541612+0000","last_undegraded":"2024-05-09T04:52:55.541612+0000","last_fullsized":"2024-05-09T04:52:55.541612+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:04:46.947861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.526988+0000","last_change":"2024-05-09T04:49:34.134733+0000","last_active":"2024-05-09T04:52:55.526988+0000","last_peered":"2024-05-09T04:52:55.526988+0000","last_clean":"2024-05-09T04:52:55.526988+0000","last_became_active":"2024-05-09T04:49:34.134337+0000","last_became_peered":"2024-05-09T04:49:34.134337+0000","last_unstale":"2024-05-09T04:52:55.526988+0000","last_undegraded":"2024-05-09T04:52:55.526988+0000","last_fullsized":"2024-05-09T04:52:55.526988+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:05.452488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.540863+0000","last_change":"2024-05-09T04:49:34.139143+0000","last_active":"2024-05-09T04:52:55.540863+0000","last_peered":"2024-05-09T04:52:55.540863+0000","last_clean":"2024-05-09T04:52:55.540863+0000","last_became_active":"2024-05-09T04:49:34.138749+0000","last_became_peered":"2024-05-09T04:49:34.138749+0000","last_unstale":"2024-05-09T04:52:55.540863+0000","last_undegraded":"2024-05-09T04:52:55.540863+0000","last_fullsized":"2024-05-09T04:52:55.540863+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:11.552063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.524374+0000","last_change":"2024-05-09T04:49:34.134687+0000","last_active":"2024-05-09T04:52:55.524374+0000","last_peered":"2024-05-09T04:52:55.524374+0000","last_clean":"2024-05-09T04:52:55.524374+0000","last_became_active":"2024-05-09T04:49:34.134324+0000","last_became_peered":"2024-05-09T04:49:34.134324+0000","last_unstale":"2024-05-09T04:52:55.524374+0000","last_undegraded":"2024-05-09T04:52:55.524374+0000","last_fullsized":"2024-05-09T04:52:55.524374+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:11.667883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.546963+0000","last_change":"2024-05-09T04:49:34.153592+0000","last_active":"2024-05-09T04:52:55.546963+0000","last_peered":"2024-05-09T04:52:55.546963+0000","last_clean":"2024-05-09T04:52:55.546963+0000","last_became_active":"2024-05-09T04:49:34.153184+0000","last_became_peered":"2024-05-09T04:49:34.153184+0000","last_unstale":"2024-05-09T04:52:55.546963+0000","last_undegraded":"2024-05-09T04:52:55.546963+0000","last_fullsized":"2024-05-09T04:52:55.546963+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:36:56.784781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.553328+0000","last_change":"2024-05-09T04:49:34.144046+0000","last_active":"2024-05-09T04:52:55.553328+0000","last_peered":"2024-05-09T04:52:55.553328+0000","last_clean":"2024-05-09T04:52:55.553328+0000","last_became_active":"2024-05-09T04:49:34.141004+0000","last_became_peered":"2024-05-09T04:49:34.141004+0000","last_unstale":"2024-05-09T04:52:55.553328+0000","last_undegraded":"2024-05-09T04:52:55.553328+0000","last_fullsized":"2024-05-09T04:52:55.553328+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:09:12.274750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.554621+0000","last_change":"2024-05-09T04:49:34.147063+0000","last_active":"2024-05-09T04:52:55.554621+0000","last_peered":"2024-05-09T04:52:55.554621+0000","last_clean":"2024-05-09T04:52:55.554621+0000","last_became_active":"2024-05-09T04:49:34.144934+0000","last_became_peered":"2024-05-09T04:49:34.144934+0000","last_unstale":"2024-05-09T04:52:55.554621+0000","last_undegraded":"2024-05-09T04:52:55.554621+0000","last_fullsized":"2024-05-09T04:52:55.554621+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:47:59.125451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.540971+0000","last_change":"2024-05-09T04:49:34.141070+0000","last_active":"2024-05-09T04:52:55.540971+0000","last_peered":"2024-05-09T04:52:55.540971+0000","last_clean":"2024-05-09T04:52:55.540971+0000","last_became_active":"2024-05-09T04:49:34.140618+0000","last_became_peered":"2024-05-09T04:49:34.140618+0000","last_unstale":"2024-05-09T04:52:55.540971+0000","last_undegraded":"2024-05-09T04:52:55.540971+0000","last_fullsized":"2024-05-09T04:52:55.540971+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:02.435768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.559709+0000","last_change":"2024-05-09T04:49:34.144434+0000","last_active":"2024-05-09T04:52:55.559709+0000","last_peered":"2024-05-09T04:52:55.559709+0000","last_clean":"2024-05-09T04:52:55.559709+0000","last_became_active":"2024-05-09T04:49:34.144052+0000","last_became_peered":"2024-05-09T04:49:34.144052+0000","last_unstale":"2024-05-09T04:52:55.559709+0000","last_undegraded":"2024-05-09T04:52:55.559709+0000","last_fullsized":"2024-05-09T04:52:55.559709+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:05:26.141627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.548013+0000","last_change":"2024-05-09T04:49:34.148681+0000","last_active":"2024-05-09T04:52:55.548013+0000","last_peered":"2024-05-09T04:52:55.548013+0000","last_clean":"2024-05-09T04:52:55.548013+0000","last_became_active":"2024-05-09T04:49:34.145277+0000","last_became_peered":"2024-05-09T04:49:34.145277+0000","last_unstale":"2024-05-09T04:52:55.548013+0000","last_undegraded":"2024-05-09T04:52:55.548013+0000","last_fullsized":"2024-05-09T04:52:55.548013+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:09.546998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.548620+0000","last_change":"2024-05-09T04:49:34.153229+0000","last_active":"2024-05-09T04:52:55.548620+0000","last_peered":"2024-05-09T04:52:55.548620+0000","last_clean":"2024-05-09T04:52:55.548620+0000","last_became_active":"2024-05-09T04:49:34.152869+0000","last_became_peered":"2024-05-09T04:49:34.152869+0000","last_unstale":"2024-05-09T04:52:55.548620+0000","last_undegraded":"2024-05-09T04:52:55.548620+0000","last_fullsized":"2024-05-09T04:52:55.548620+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:54.350842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:12.136825+0000","last_change":"2024-05-09T04:49:56.710411+0000","last_active":"2024-05-09T04:52:12.136825+0000","last_peered":"2024-05-09T04:52:12.136825+0000","last_clean":"2024-05-09T04:52:12.136825+0000","last_became_active":"2024-05-09T04:49:56.709903+0000","last_became_peered":"2024-05-09T04:49:56.709903+0000","last_unstale":"2024-05-09T04:52:12.136825+0000","last_undegraded":"2024-05-09T04:52:12.136825+0000","last_fullsized":"2024-05-09T04:52:12.136825+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:53:44.216648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,10,0],"acting":[2,3,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":291,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:54:56.317312+0000","last_change":"2024-05-09T04:49:30.061766+0000","last_active":"2024-05-09T04:54:56.317312+0000","last_peered":"2024-05-09T04:54:56.317312+0000","last_clean":"2024-05-09T04:54:56.317312+0000","last_became_active":"2024-05-09T04:49:30.060545+0000","last_became_peered":"2024-05-09T04:49:30.060545+0000","last_unstale":"2024-05-09T04:54:56.317312+0000","last_undegraded":"2024-05-09T04:54:56.317312+0000","last_fullsized":"2024-05-09T04:54:56.317312+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:15:03.857107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":235,"num_read_kb":235,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.555879+0000","last_change":"2024-05-09T04:49:34.140706+0000","last_active":"2024-05-09T04:52:55.555879+0000","last_peered":"2024-05-09T04:52:55.555879+0000","last_clean":"2024-05-09T04:52:55.555879+0000","last_became_active":"2024-05-09T04:49:34.139032+0000","last_became_peered":"2024-05-09T04:49:34.139032+0000","last_unstale":"2024-05-09T04:52:55.555879+0000","last_undegraded":"2024-05-09T04:52:55.555879+0000","last_fullsized":"2024-05-09T04:52:55.555879+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:19.878523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:16.237850+0000","last_change":"2024-05-09T04:49:47.674254+0000","last_active":"2024-05-09T04:52:16.237850+0000","last_peered":"2024-05-09T04:52:16.237850+0000","last_clean":"2024-05-09T04:52:16.237850+0000","last_became_active":"2024-05-09T04:49:47.673476+0000","last_became_peered":"2024-05-09T04:49:47.673476+0000","last_unstale":"2024-05-09T04:52:16.237850+0000","last_undegraded":"2024-05-09T04:52:16.237850+0000","last_fullsized":"2024-05-09T04:52:16.237850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:41:31.709210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:33.128358+0000","last_change":"2024-05-09T04:49:30.154162+0000","last_active":"2024-05-09T04:51:33.128358+0000","last_peered":"2024-05-09T04:51:33.128358+0000","last_clean":"2024-05-09T04:51:33.128358+0000","last_became_active":"2024-05-09T04:49:30.152869+0000","last_became_peered":"2024-05-09T04:49:30.152869+0000","last_unstale":"2024-05-09T04:51:33.128358+0000","last_undegraded":"2024-05-09T04:51:33.128358+0000","last_fullsized":"2024-05-09T04:51:33.128358+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:43.433033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.563473+0000","last_change":"2024-05-09T04:49:34.157729+0000","last_active":"2024-05-09T04:52:55.563473+0000","last_peered":"2024-05-09T04:52:55.563473+0000","last_clean":"2024-05-09T04:52:55.563473+0000","last_became_active":"2024-05-09T04:49:34.155040+0000","last_became_peered":"2024-05-09T04:49:34.155040+0000","last_unstale":"2024-05-09T04:52:55.563473+0000","last_undegraded":"2024-05-09T04:52:55.563473+0000","last_fullsized":"2024-05-09T04:52:55.563473+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:43:30.104394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:21.238400+0000","last_change":"2024-05-09T04:49:47.679665+0000","last_active":"2024-05-09T04:52:21.238400+0000","last_peered":"2024-05-09T04:52:21.238400+0000","last_clean":"2024-05-09T04:52:21.238400+0000","last_became_active":"2024-05-09T04:49:47.679257+0000","last_became_peered":"2024-05-09T04:49:47.679257+0000","last_unstale":"2024-05-09T04:52:21.238400+0000","last_undegraded":"2024-05-09T04:52:21.238400+0000","last_fullsized":"2024-05-09T04:52:21.238400+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:53.111205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":57,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:00.713890+0000","last_change":"2024-05-09T04:49:30.047386+0000","last_active":"2024-05-09T04:52:00.713890+0000","last_peered":"2024-05-09T04:52:00.713890+0000","last_clean":"2024-05-09T04:52:00.713890+0000","last_became_active":"2024-05-09T04:49:30.046947+0000","last_became_peered":"2024-05-09T04:49:30.046947+0000","last_unstale":"2024-05-09T04:52:00.713890+0000","last_undegraded":"2024-05-09T04:52:00.713890+0000","last_fullsized":"2024-05-09T04:52:00.713890+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:46.861354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.548062+0000","last_change":"2024-05-09T04:49:34.153320+0000","last_active":"2024-05-09T04:52:55.548062+0000","last_peered":"2024-05-09T04:52:55.548062+0000","last_clean":"2024-05-09T04:52:55.548062+0000","last_became_active":"2024-05-09T04:49:34.152854+0000","last_became_peered":"2024-05-09T04:49:34.152854+0000","last_unstale":"2024-05-09T04:52:55.548062+0000","last_undegraded":"2024-05-09T04:52:55.548062+0000","last_fullsized":"2024-05-09T04:52:55.548062+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:40.801340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:21.238377+0000","last_change":"2024-05-09T04:49:47.668475+0000","last_active":"2024-05-09T04:52:21.238377+0000","last_peered":"2024-05-09T04:52:21.238377+0000","last_clean":"2024-05-09T04:52:21.238377+0000","last_became_active":"2024-05-09T04:49:47.667909+0000","last_became_peered":"2024-05-09T04:49:47.667909+0000","last_unstale":"2024-05-09T04:52:21.238377+0000","last_undegraded":"2024-05-09T04:52:21.238377+0000","last_fullsized":"2024-05-09T04:52:21.238377+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:39:27.687468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:00.713826+0000","last_change":"2024-05-09T04:49:30.048659+0000","last_active":"2024-05-09T04:52:00.713826+0000","last_peered":"2024-05-09T04:52:00.713826+0000","last_clean":"2024-05-09T04:52:00.713826+0000","last_became_active":"2024-05-09T04:49:30.048259+0000","last_became_peered":"2024-05-09T04:49:30.048259+0000","last_unstale":"2024-05-09T04:52:00.713826+0000","last_undegraded":"2024-05-09T04:52:00.713826+0000","last_fullsized":"2024-05-09T04:52:00.713826+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:37:46.012594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.539385+0000","last_change":"2024-05-09T04:49:34.131416+0000","last_active":"2024-05-09T04:52:55.539385+0000","last_peered":"2024-05-09T04:52:55.539385+0000","last_clean":"2024-05-09T04:52:55.539385+0000","last_became_active":"2024-05-09T04:49:34.130985+0000","last_became_peered":"2024-05-09T04:49:34.130985+0000","last_unstale":"2024-05-09T04:52:55.539385+0000","last_undegraded":"2024-05-09T04:52:55.539385+0000","last_fullsized":"2024-05-09T04:52:55.539385+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:02:43.228142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:16.222557+0000","last_change":"2024-05-09T04:49:47.652247+0000","last_active":"2024-05-09T04:52:16.222557+0000","last_peered":"2024-05-09T04:52:16.222557+0000","last_clean":"2024-05-09T04:52:16.222557+0000","last_became_active":"2024-05-09T04:49:47.651747+0000","last_became_peered":"2024-05-09T04:49:47.651747+0000","last_unstale":"2024-05-09T04:52:16.222557+0000","last_undegraded":"2024-05-09T04:52:16.222557+0000","last_fullsized":"2024-05-09T04:52:16.222557+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:45.353027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":287,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:54:54.132816+0000","last_change":"2024-05-09T04:49:30.047421+0000","last_active":"2024-05-09T04:54:54.132816+0000","last_peered":"2024-05-09T04:54:54.132816+0000","last_clean":"2024-05-09T04:54:54.132816+0000","last_became_active":"2024-05-09T04:49:30.047007+0000","last_became_peered":"2024-05-09T04:49:30.047007+0000","last_unstale":"2024-05-09T04:54:54.132816+0000","last_undegraded":"2024-05-09T04:54:54.132816+0000","last_fullsized":"2024-05-09T04:54:54.132816+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:23.930326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":236,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.533509+0000","last_change":"2024-05-09T04:49:34.146605+0000","last_active":"2024-05-09T04:52:55.533509+0000","last_peered":"2024-05-09T04:52:55.533509+0000","last_clean":"2024-05-09T04:52:55.533509+0000","last_became_active":"2024-05-09T04:49:34.146205+0000","last_became_peered":"2024-05-09T04:49:34.146205+0000","last_unstale":"2024-05-09T04:52:55.533509+0000","last_undegraded":"2024-05-09T04:52:55.533509+0000","last_fullsized":"2024-05-09T04:52:55.533509+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:13:52.234951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:16.222527+0000","last_change":"2024-05-09T04:49:47.673652+0000","last_active":"2024-05-09T04:52:16.222527+0000","last_peered":"2024-05-09T04:52:16.222527+0000","last_clean":"2024-05-09T04:52:16.222527+0000","last_became_active":"2024-05-09T04:49:47.673255+0000","last_became_peered":"2024-05-09T04:49:47.673255+0000","last_unstale":"2024-05-09T04:52:16.222527+0000","last_undegraded":"2024-05-09T04:52:16.222527+0000","last_fullsized":"2024-05-09T04:52:16.222527+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:12:50.672212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:00.294775+0000","last_change":"2024-05-09T04:49:30.056089+0000","last_active":"2024-05-09T04:52:00.294775+0000","last_peered":"2024-05-09T04:52:00.294775+0000","last_clean":"2024-05-09T04:52:00.294775+0000","last_became_active":"2024-05-09T04:49:30.055635+0000","last_became_peered":"2024-05-09T04:49:30.055635+0000","last_unstale":"2024-05-09T04:52:00.294775+0000","last_undegraded":"2024-05-09T04:52:00.294775+0000","last_fullsized":"2024-05-09T04:52:00.294775+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:35:06.529058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.562131+0000","last_change":"2024-05-09T04:49:34.147628+0000","last_active":"2024-05-09T04:52:55.562131+0000","last_peered":"2024-05-09T04:52:55.562131+0000","last_clean":"2024-05-09T04:52:55.562131+0000","last_became_active":"2024-05-09T04:49:34.147208+0000","last_became_peered":"2024-05-09T04:49:34.147208+0000","last_unstale":"2024-05-09T04:52:55.562131+0000","last_undegraded":"2024-05-09T04:52:55.562131+0000","last_fullsized":"2024-05-09T04:52:55.562131+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:09:22.520468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:51.957724+0000","last_change":"2024-05-09T04:49:56.758366+0000","last_active":"2024-05-09T04:51:51.957724+0000","last_peered":"2024-05-09T04:51:51.957724+0000","last_clean":"2024-05-09T04:51:51.957724+0000","last_became_active":"2024-05-09T04:49:56.757763+0000","last_became_peered":"2024-05-09T04:49:56.757763+0000","last_unstale":"2024-05-09T04:51:51.957724+0000","last_undegraded":"2024-05-09T04:51:51.957724+0000","last_fullsized":"2024-05-09T04:51:51.957724+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:58.711764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:57.852993+0000","last_change":"2024-05-09T04:49:30.058805+0000","last_active":"2024-05-09T04:51:57.852993+0000","last_peered":"2024-05-09T04:51:57.852993+0000","last_clean":"2024-05-09T04:51:57.852993+0000","last_became_active":"2024-05-09T04:49:30.057278+0000","last_became_peered":"2024-05-09T04:49:30.057278+0000","last_unstale":"2024-05-09T04:51:57.852993+0000","last_undegraded":"2024-05-09T04:51:57.852993+0000","last_fullsized":"2024-05-09T04:51:57.852993+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:34.727781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.559587+0000","last_change":"2024-05-09T04:49:34.148753+0000","last_active":"2024-05-09T04:52:55.559587+0000","last_peered":"2024-05-09T04:52:55.559587+0000","last_clean":"2024-05-09T04:52:55.559587+0000","last_became_active":"2024-05-09T04:49:34.148407+0000","last_became_peered":"2024-05-09T04:49:34.148407+0000","last_unstale":"2024-05-09T04:52:55.559587+0000","last_undegraded":"2024-05-09T04:52:55.559587+0000","last_fullsized":"2024-05-09T04:52:55.559587+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:39.602934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:46.807773+0000","last_change":"2024-05-09T04:49:47.672521+0000","last_active":"2024-05-09T04:51:46.807773+0000","last_peered":"2024-05-09T04:51:46.807773+0000","last_clean":"2024-05-09T04:51:46.807773+0000","last_became_active":"2024-05-09T04:49:47.671353+0000","last_became_peered":"2024-05-09T04:49:47.671353+0000","last_unstale":"2024-05-09T04:51:46.807773+0000","last_undegraded":"2024-05-09T04:51:46.807773+0000","last_fullsized":"2024-05-09T04:51:46.807773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:56:34.832651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:55.722708+0000","last_change":"2024-05-09T04:49:30.048589+0000","last_active":"2024-05-09T04:51:55.722708+0000","last_peered":"2024-05-09T04:51:55.722708+0000","last_clean":"2024-05-09T04:51:55.722708+0000","last_became_active":"2024-05-09T04:49:30.048241+0000","last_became_peered":"2024-05-09T04:49:30.048241+0000","last_unstale":"2024-05-09T04:51:55.722708+0000","last_undegraded":"2024-05-09T04:51:55.722708+0000","last_fullsized":"2024-05-09T04:51:55.722708+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:38:36.204539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.562269+0000","last_change":"2024-05-09T04:49:34.151201+0000","last_active":"2024-05-09T04:52:55.562269+0000","last_peered":"2024-05-09T04:52:55.562269+0000","last_clean":"2024-05-09T04:52:55.562269+0000","last_became_active":"2024-05-09T04:49:34.150505+0000","last_became_peered":"2024-05-09T04:49:34.150505+0000","last_unstale":"2024-05-09T04:52:55.562269+0000","last_undegraded":"2024-05-09T04:52:55.562269+0000","last_fullsized":"2024-05-09T04:52:55.562269+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:40.011516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:08.135251+0000","last_change":"2024-05-09T04:49:47.653827+0000","last_active":"2024-05-09T04:52:08.135251+0000","last_peered":"2024-05-09T04:52:08.135251+0000","last_clean":"2024-05-09T04:52:08.135251+0000","last_became_active":"2024-05-09T04:49:47.652423+0000","last_became_peered":"2024-05-09T04:49:47.652423+0000","last_unstale":"2024-05-09T04:52:08.135251+0000","last_undegraded":"2024-05-09T04:52:08.135251+0000","last_fullsized":"2024-05-09T04:52:08.135251+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:07:07.602659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'6","reported_seq":514,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:54:55.133597+0000","last_change":"2024-05-09T04:49:30.062849+0000","last_active":"2024-05-09T04:54:55.133597+0000","last_peered":"2024-05-09T04:54:55.133597+0000","last_clean":"2024-05-09T04:54:55.133597+0000","last_became_active":"2024-05-09T04:49:30.062426+0000","last_became_peered":"2024-05-09T04:49:30.062426+0000","last_unstale":"2024-05-09T04:54:55.133597+0000","last_undegraded":"2024-05-09T04:54:55.133597+0000","last_fullsized":"2024-05-09T04:54:55.133597+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:25:11.996208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":464,"num_read_kb":464,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.544151+0000","last_change":"2024-05-09T04:49:34.151145+0000","last_active":"2024-05-09T04:52:55.544151+0000","last_peered":"2024-05-09T04:52:55.544151+0000","last_clean":"2024-05-09T04:52:55.544151+0000","last_became_active":"2024-05-09T04:49:34.150745+0000","last_became_peered":"2024-05-09T04:49:34.150745+0000","last_unstale":"2024-05-09T04:52:55.544151+0000","last_undegraded":"2024-05-09T04:52:55.544151+0000","last_fullsized":"2024-05-09T04:52:55.544151+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:40.647454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:07.844209+0000","last_change":"2024-05-09T04:49:47.672796+0000","last_active":"2024-05-09T04:52:07.844209+0000","last_peered":"2024-05-09T04:52:07.844209+0000","last_clean":"2024-05-09T04:52:07.844209+0000","last_became_active":"2024-05-09T04:49:47.667807+0000","last_became_peered":"2024-05-09T04:49:47.667807+0000","last_unstale":"2024-05-09T04:52:07.844209+0000","last_undegraded":"2024-05-09T04:52:07.844209+0000","last_fullsized":"2024-05-09T04:52:07.844209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:58.542973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:54.594836+0000","last_change":"2024-05-09T04:49:30.047265+0000","last_active":"2024-05-09T04:51:54.594836+0000","last_peered":"2024-05-09T04:51:54.594836+0000","last_clean":"2024-05-09T04:51:54.594836+0000","last_became_active":"2024-05-09T04:49:30.046888+0000","last_became_peered":"2024-05-09T04:49:30.046888+0000","last_unstale":"2024-05-09T04:51:54.594836+0000","last_undegraded":"2024-05-09T04:51:54.594836+0000","last_fullsized":"2024-05-09T04:51:54.594836+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:38:41.878678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.554268+0000","last_change":"2024-05-09T04:49:34.144447+0000","last_active":"2024-05-09T04:52:55.554268+0000","last_peered":"2024-05-09T04:52:55.554268+0000","last_clean":"2024-05-09T04:52:55.554268+0000","last_became_active":"2024-05-09T04:49:34.144073+0000","last_became_peered":"2024-05-09T04:49:34.144073+0000","last_unstale":"2024-05-09T04:52:55.554268+0000","last_undegraded":"2024-05-09T04:52:55.554268+0000","last_fullsized":"2024-05-09T04:52:55.554268+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:53:42.619552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:11.237300+0000","last_change":"2024-05-09T04:49:47.673364+0000","last_active":"2024-05-09T04:52:11.237300+0000","last_peered":"2024-05-09T04:52:11.237300+0000","last_clean":"2024-05-09T04:52:11.237300+0000","last_became_active":"2024-05-09T04:49:47.672712+0000","last_became_peered":"2024-05-09T04:49:47.672712+0000","last_unstale":"2024-05-09T04:52:11.237300+0000","last_undegraded":"2024-05-09T04:52:11.237300+0000","last_fullsized":"2024-05-09T04:52:11.237300+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:25:57.165777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:55.722783+0000","last_change":"2024-05-09T04:49:30.160924+0000","last_active":"2024-05-09T04:51:55.722783+0000","last_peered":"2024-05-09T04:51:55.722783+0000","last_clean":"2024-05-09T04:51:55.722783+0000","last_became_active":"2024-05-09T04:49:30.160479+0000","last_became_peered":"2024-05-09T04:49:30.160479+0000","last_unstale":"2024-05-09T04:51:55.722783+0000","last_undegraded":"2024-05-09T04:51:55.722783+0000","last_fullsized":"2024-05-09T04:51:55.722783+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:44:57.171441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.545843+0000","last_change":"2024-05-09T04:49:34.136514+0000","last_active":"2024-05-09T04:52:55.545843+0000","last_peered":"2024-05-09T04:52:55.545843+0000","last_clean":"2024-05-09T04:52:55.545843+0000","last_became_active":"2024-05-09T04:49:34.136121+0000","last_became_peered":"2024-05-09T04:49:34.136121+0000","last_unstale":"2024-05-09T04:52:55.545843+0000","last_undegraded":"2024-05-09T04:52:55.545843+0000","last_fullsized":"2024-05-09T04:52:55.545843+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:43:27.781720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:07.844267+0000","last_change":"2024-05-09T04:49:47.673371+0000","last_active":"2024-05-09T04:52:07.844267+0000","last_peered":"2024-05-09T04:52:07.844267+0000","last_clean":"2024-05-09T04:52:07.844267+0000","last_became_active":"2024-05-09T04:49:47.669727+0000","last_became_peered":"2024-05-09T04:49:47.669727+0000","last_unstale":"2024-05-09T04:52:07.844267+0000","last_undegraded":"2024-05-09T04:52:07.844267+0000","last_fullsized":"2024-05-09T04:52:07.844267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:34:45.510934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'42","reported_seq":563,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:54:55.134003+0000","last_change":"2024-05-09T04:49:30.048359+0000","last_active":"2024-05-09T04:54:55.134003+0000","last_peered":"2024-05-09T04:54:55.134003+0000","last_clean":"2024-05-09T04:54:55.134003+0000","last_became_active":"2024-05-09T04:49:30.047949+0000","last_became_peered":"2024-05-09T04:49:30.047949+0000","last_unstale":"2024-05-09T04:54:55.134003+0000","last_undegraded":"2024-05-09T04:54:55.134003+0000","last_fullsized":"2024-05-09T04:54:55.134003+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:01:10.498997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64799,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":477,"num_read_kb":303,"num_write":44,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.521477+0000","last_change":"2024-05-09T04:49:34.132589+0000","last_active":"2024-05-09T04:52:55.521477+0000","last_peered":"2024-05-09T04:52:55.521477+0000","last_clean":"2024-05-09T04:52:55.521477+0000","last_became_active":"2024-05-09T04:49:34.132227+0000","last_became_peered":"2024-05-09T04:49:34.132227+0000","last_unstale":"2024-05-09T04:52:55.521477+0000","last_undegraded":"2024-05-09T04:52:55.521477+0000","last_fullsized":"2024-05-09T04:52:55.521477+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:50.003878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:15.725082+0000","last_change":"2024-05-09T04:49:47.668236+0000","last_active":"2024-05-09T04:52:15.725082+0000","last_peered":"2024-05-09T04:52:15.725082+0000","last_clean":"2024-05-09T04:52:15.725082+0000","last_became_active":"2024-05-09T04:49:47.667779+0000","last_became_peered":"2024-05-09T04:49:47.667779+0000","last_unstale":"2024-05-09T04:52:15.725082+0000","last_undegraded":"2024-05-09T04:52:15.725082+0000","last_fullsized":"2024-05-09T04:52:15.725082+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:16.177888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:52.134520+0000","last_change":"2024-05-09T04:49:30.046145+0000","last_active":"2024-05-09T04:51:52.134520+0000","last_peered":"2024-05-09T04:51:52.134520+0000","last_clean":"2024-05-09T04:51:52.134520+0000","last_became_active":"2024-05-09T04:49:30.045478+0000","last_became_peered":"2024-05-09T04:49:30.045478+0000","last_unstale":"2024-05-09T04:51:52.134520+0000","last_undegraded":"2024-05-09T04:51:52.134520+0000","last_fullsized":"2024-05-09T04:51:52.134520+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:52:08.313984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.550418+0000","last_change":"2024-05-09T04:49:34.154349+0000","last_active":"2024-05-09T04:52:55.550418+0000","last_peered":"2024-05-09T04:52:55.550418+0000","last_clean":"2024-05-09T04:52:55.550418+0000","last_became_active":"2024-05-09T04:49:34.154011+0000","last_became_peered":"2024-05-09T04:49:34.154011+0000","last_unstale":"2024-05-09T04:52:55.550418+0000","last_undegraded":"2024-05-09T04:52:55.550418+0000","last_fullsized":"2024-05-09T04:52:55.550418+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:50:32.345863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:06.959550+0000","last_change":"2024-05-09T04:49:47.670170+0000","last_active":"2024-05-09T04:52:06.959550+0000","last_peered":"2024-05-09T04:52:06.959550+0000","last_clean":"2024-05-09T04:52:06.959550+0000","last_became_active":"2024-05-09T04:49:47.669705+0000","last_became_peered":"2024-05-09T04:49:47.669705+0000","last_unstale":"2024-05-09T04:52:06.959550+0000","last_undegraded":"2024-05-09T04:52:06.959550+0000","last_fullsized":"2024-05-09T04:52:06.959550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:59:27.339811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:50.722218+0000","last_change":"2024-05-09T04:49:30.051159+0000","last_active":"2024-05-09T04:51:50.722218+0000","last_peered":"2024-05-09T04:51:50.722218+0000","last_clean":"2024-05-09T04:51:50.722218+0000","last_became_active":"2024-05-09T04:49:30.050108+0000","last_became_peered":"2024-05-09T04:49:30.050108+0000","last_unstale":"2024-05-09T04:51:50.722218+0000","last_undegraded":"2024-05-09T04:51:50.722218+0000","last_fullsized":"2024-05-09T04:51:50.722218+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:32:56.778567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.549234+0000","last_change":"2024-05-09T04:49:34.162751+0000","last_active":"2024-05-09T04:52:55.549234+0000","last_peered":"2024-05-09T04:52:55.549234+0000","last_clean":"2024-05-09T04:52:55.549234+0000","last_became_active":"2024-05-09T04:49:34.162336+0000","last_became_peered":"2024-05-09T04:49:34.162336+0000","last_unstale":"2024-05-09T04:52:55.549234+0000","last_undegraded":"2024-05-09T04:52:55.549234+0000","last_fullsized":"2024-05-09T04:52:55.549234+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:41.758074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:12.845173+0000","last_change":"2024-05-09T04:49:47.680955+0000","last_active":"2024-05-09T04:52:12.845173+0000","last_peered":"2024-05-09T04:52:12.845173+0000","last_clean":"2024-05-09T04:52:12.845173+0000","last_became_active":"2024-05-09T04:49:47.680041+0000","last_became_peered":"2024-05-09T04:49:47.680041+0000","last_unstale":"2024-05-09T04:52:12.845173+0000","last_undegraded":"2024-05-09T04:52:12.845173+0000","last_fullsized":"2024-05-09T04:52:12.845173+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:46.297611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'7","reported_seq":286,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:54:54.358529+0000","last_change":"2024-05-09T04:49:30.045857+0000","last_active":"2024-05-09T04:54:54.358529+0000","last_peered":"2024-05-09T04:54:54.358529+0000","last_clean":"2024-05-09T04:54:54.358529+0000","last_became_active":"2024-05-09T04:49:30.045404+0000","last_became_peered":"2024-05-09T04:49:30.045404+0000","last_unstale":"2024-05-09T04:54:54.358529+0000","last_undegraded":"2024-05-09T04:54:54.358529+0000","last_fullsized":"2024-05-09T04:54:54.358529+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:44:19.473449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":237,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.539356+0000","last_change":"2024-05-09T04:49:34.131734+0000","last_active":"2024-05-09T04:52:55.539356+0000","last_peered":"2024-05-09T04:52:55.539356+0000","last_clean":"2024-05-09T04:52:55.539356+0000","last_became_active":"2024-05-09T04:49:34.131396+0000","last_became_peered":"2024-05-09T04:49:34.131396+0000","last_unstale":"2024-05-09T04:52:55.539356+0000","last_undegraded":"2024-05-09T04:52:55.539356+0000","last_fullsized":"2024-05-09T04:52:55.539356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:21.080651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:12.854733+0000","last_change":"2024-05-09T04:49:56.710429+0000","last_active":"2024-05-09T04:52:12.854733+0000","last_peered":"2024-05-09T04:52:12.854733+0000","last_clean":"2024-05-09T04:52:12.854733+0000","last_became_active":"2024-05-09T04:49:56.708497+0000","last_became_peered":"2024-05-09T04:49:56.708497+0000","last_unstale":"2024-05-09T04:52:12.854733+0000","last_undegraded":"2024-05-09T04:52:12.854733+0000","last_fullsized":"2024-05-09T04:52:12.854733+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:36.281300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:16.960685+0000","last_change":"2024-05-09T04:49:56.718731+0000","last_active":"2024-05-09T04:52:16.960685+0000","last_peered":"2024-05-09T04:52:16.960685+0000","last_clean":"2024-05-09T04:52:16.960685+0000","last_became_active":"2024-05-09T04:49:56.709958+0000","last_became_peered":"2024-05-09T04:49:56.709958+0000","last_unstale":"2024-05-09T04:52:16.960685+0000","last_undegraded":"2024-05-09T04:52:16.960685+0000","last_fullsized":"2024-05-09T04:52:16.960685+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:11.003906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.539006+0000","last_change":"2024-05-09T04:49:34.131705+0000","last_active":"2024-05-09T04:52:55.539006+0000","last_peered":"2024-05-09T04:52:55.539006+0000","last_clean":"2024-05-09T04:52:55.539006+0000","last_became_active":"2024-05-09T04:49:34.131287+0000","last_became_peered":"2024-05-09T04:49:34.131287+0000","last_unstale":"2024-05-09T04:52:55.539006+0000","last_undegraded":"2024-05-09T04:52:55.539006+0000","last_fullsized":"2024-05-09T04:52:55.539006+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:10:24.914832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:00.723219+0000","last_change":"2024-05-09T04:49:47.663861+0000","last_active":"2024-05-09T04:52:00.723219+0000","last_peered":"2024-05-09T04:52:00.723219+0000","last_clean":"2024-05-09T04:52:00.723219+0000","last_became_active":"2024-05-09T04:49:47.663374+0000","last_became_peered":"2024-05-09T04:49:47.663374+0000","last_unstale":"2024-05-09T04:52:00.723219+0000","last_undegraded":"2024-05-09T04:52:00.723219+0000","last_fullsized":"2024-05-09T04:52:00.723219+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:08.063040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:45.712130+0000","last_change":"2024-05-09T04:49:30.044183+0000","last_active":"2024-05-09T04:51:45.712130+0000","last_peered":"2024-05-09T04:51:45.712130+0000","last_clean":"2024-05-09T04:51:45.712130+0000","last_became_active":"2024-05-09T04:49:30.043608+0000","last_became_peered":"2024-05-09T04:49:30.043608+0000","last_unstale":"2024-05-09T04:51:45.712130+0000","last_undegraded":"2024-05-09T04:51:45.712130+0000","last_fullsized":"2024-05-09T04:51:45.712130+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:48:57.788970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.528008+0000","last_change":"2024-05-09T04:49:34.133941+0000","last_active":"2024-05-09T04:52:55.528008+0000","last_peered":"2024-05-09T04:52:55.528008+0000","last_clean":"2024-05-09T04:52:55.528008+0000","last_became_active":"2024-05-09T04:49:34.133550+0000","last_became_peered":"2024-05-09T04:49:34.133550+0000","last_unstale":"2024-05-09T04:52:55.528008+0000","last_undegraded":"2024-05-09T04:52:55.528008+0000","last_fullsized":"2024-05-09T04:52:55.528008+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:45:14.703565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:55.713155+0000","last_change":"2024-05-09T04:49:47.901085+0000","last_active":"2024-05-09T04:51:55.713155+0000","last_peered":"2024-05-09T04:51:55.713155+0000","last_clean":"2024-05-09T04:51:55.713155+0000","last_became_active":"2024-05-09T04:49:47.900444+0000","last_became_peered":"2024-05-09T04:49:47.900444+0000","last_unstale":"2024-05-09T04:51:55.713155+0000","last_undegraded":"2024-05-09T04:51:55.713155+0000","last_fullsized":"2024-05-09T04:51:55.713155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:44.195683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:37.839364+0000","last_change":"2024-05-09T04:49:30.154060+0000","last_active":"2024-05-09T04:51:37.839364+0000","last_peered":"2024-05-09T04:51:37.839364+0000","last_clean":"2024-05-09T04:51:37.839364+0000","last_became_active":"2024-05-09T04:49:30.152556+0000","last_became_peered":"2024-05-09T04:49:30.152556+0000","last_unstale":"2024-05-09T04:51:37.839364+0000","last_undegraded":"2024-05-09T04:51:37.839364+0000","last_fullsized":"2024-05-09T04:51:37.839364+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:27.056251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.531546+0000","last_change":"2024-05-09T04:49:34.153128+0000","last_active":"2024-05-09T04:52:55.531546+0000","last_peered":"2024-05-09T04:52:55.531546+0000","last_clean":"2024-05-09T04:52:55.531546+0000","last_became_active":"2024-05-09T04:49:34.152734+0000","last_became_peered":"2024-05-09T04:49:34.152734+0000","last_unstale":"2024-05-09T04:52:55.531546+0000","last_undegraded":"2024-05-09T04:52:55.531546+0000","last_fullsized":"2024-05-09T04:52:55.531546+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:54:06.920877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:52.852082+0000","last_change":"2024-05-09T04:49:47.670263+0000","last_active":"2024-05-09T04:51:52.852082+0000","last_peered":"2024-05-09T04:51:52.852082+0000","last_clean":"2024-05-09T04:51:52.852082+0000","last_became_active":"2024-05-09T04:49:47.665743+0000","last_became_peered":"2024-05-09T04:49:47.665743+0000","last_unstale":"2024-05-09T04:51:52.852082+0000","last_undegraded":"2024-05-09T04:51:52.852082+0000","last_fullsized":"2024-05-09T04:51:52.852082+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:27.980226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":286,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:54:58.323116+0000","last_change":"2024-05-09T04:49:30.056119+0000","last_active":"2024-05-09T04:54:58.323116+0000","last_peered":"2024-05-09T04:54:58.323116+0000","last_clean":"2024-05-09T04:54:58.323116+0000","last_became_active":"2024-05-09T04:49:30.055660+0000","last_became_peered":"2024-05-09T04:49:30.055660+0000","last_unstale":"2024-05-09T04:54:58.323116+0000","last_undegraded":"2024-05-09T04:54:58.323116+0000","last_fullsized":"2024-05-09T04:54:58.323116+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:21.513879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.524834+0000","last_change":"2024-05-09T04:49:56.764303+0000","last_active":"2024-05-09T04:52:55.524834+0000","last_peered":"2024-05-09T04:52:55.524834+0000","last_clean":"2024-05-09T04:52:55.524834+0000","last_became_active":"2024-05-09T04:49:56.763854+0000","last_became_peered":"2024-05-09T04:49:56.763854+0000","last_unstale":"2024-05-09T04:52:55.524834+0000","last_undegraded":"2024-05-09T04:52:55.524834+0000","last_fullsized":"2024-05-09T04:52:55.524834+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:06.086602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:57.853087+0000","last_change":"2024-05-09T04:49:47.900866+0000","last_active":"2024-05-09T04:51:57.853087+0000","last_peered":"2024-05-09T04:51:57.853087+0000","last_clean":"2024-05-09T04:51:57.853087+0000","last_became_active":"2024-05-09T04:49:47.900332+0000","last_became_peered":"2024-05-09T04:49:47.900332+0000","last_unstale":"2024-05-09T04:51:57.853087+0000","last_undegraded":"2024-05-09T04:51:57.853087+0000","last_fullsized":"2024-05-09T04:51:57.853087+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:06:41.448262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.527180+0000","last_change":"2024-05-09T04:49:56.702559+0000","last_active":"2024-05-09T04:52:55.527180+0000","last_peered":"2024-05-09T04:52:55.527180+0000","last_clean":"2024-05-09T04:52:55.527180+0000","last_became_active":"2024-05-09T04:49:56.702135+0000","last_became_peered":"2024-05-09T04:49:56.702135+0000","last_unstale":"2024-05-09T04:52:55.527180+0000","last_undegraded":"2024-05-09T04:52:55.527180+0000","last_fullsized":"2024-05-09T04:52:55.527180+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:43:05.848239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":285,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:54:56.362399+0000","last_change":"2024-05-09T04:49:30.158050+0000","last_active":"2024-05-09T04:54:56.362399+0000","last_peered":"2024-05-09T04:54:56.362399+0000","last_clean":"2024-05-09T04:54:56.362399+0000","last_became_active":"2024-05-09T04:49:30.156719+0000","last_became_peered":"2024-05-09T04:49:30.156719+0000","last_unstale":"2024-05-09T04:54:56.362399+0000","last_undegraded":"2024-05-09T04:54:56.362399+0000","last_fullsized":"2024-05-09T04:54:56.362399+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:47.963111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":235,"num_read_kb":235,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:54.594819+0000","last_change":"2024-05-09T04:49:47.673594+0000","last_active":"2024-05-09T04:51:54.594819+0000","last_peered":"2024-05-09T04:51:54.594819+0000","last_clean":"2024-05-09T04:51:54.594819+0000","last_became_active":"2024-05-09T04:49:47.672671+0000","last_became_peered":"2024-05-09T04:49:47.672671+0000","last_unstale":"2024-05-09T04:51:54.594819+0000","last_undegraded":"2024-05-09T04:51:54.594819+0000","last_fullsized":"2024-05-09T04:51:54.594819+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:58:56.078180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.547961+0000","last_change":"2024-05-09T04:50:58.035054+0000","last_active":"2024-05-09T04:52:55.547961+0000","last_peered":"2024-05-09T04:52:55.547961+0000","last_clean":"2024-05-09T04:52:55.547961+0000","last_became_active":"2024-05-09T04:50:58.034297+0000","last_became_peered":"2024-05-09T04:50:58.034297+0000","last_unstale":"2024-05-09T04:52:55.547961+0000","last_undegraded":"2024-05-09T04:52:55.547961+0000","last_fullsized":"2024-05-09T04:52:55.547961+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:10:56.827950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:36.232917+0000","last_change":"2024-05-09T04:49:30.060224+0000","last_active":"2024-05-09T04:51:36.232917+0000","last_peered":"2024-05-09T04:51:36.232917+0000","last_clean":"2024-05-09T04:51:36.232917+0000","last_became_active":"2024-05-09T04:49:30.059830+0000","last_became_peered":"2024-05-09T04:49:30.059830+0000","last_unstale":"2024-05-09T04:51:36.232917+0000","last_undegraded":"2024-05-09T04:51:36.232917+0000","last_fullsized":"2024-05-09T04:51:36.232917+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:02.925226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":118,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:24.598674+0000","last_change":"2024-05-09T04:47:53.050380+0000","last_active":"2024-05-09T04:52:24.598674+0000","last_peered":"2024-05-09T04:52:24.598674+0000","last_clean":"2024-05-09T04:52:24.598674+0000","last_became_active":"2024-05-09T04:47:53.044058+0000","last_became_peered":"2024-05-09T04:47:53.044058+0000","last_unstale":"2024-05-09T04:52:24.598674+0000","last_undegraded":"2024-05-09T04:52:24.598674+0000","last_fullsized":"2024-05-09T04:52:24.598674+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:43:48.067159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:43:48.067159+0000","last_clean_scrub_stamp":"2024-05-09T04:43:48.067159+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:17:18.714607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:00.723281+0000","last_change":"2024-05-09T04:49:47.901114+0000","last_active":"2024-05-09T04:52:00.723281+0000","last_peered":"2024-05-09T04:52:00.723281+0000","last_clean":"2024-05-09T04:52:00.723281+0000","last_became_active":"2024-05-09T04:49:47.900450+0000","last_became_peered":"2024-05-09T04:49:47.900450+0000","last_unstale":"2024-05-09T04:52:00.723281+0000","last_undegraded":"2024-05-09T04:52:00.723281+0000","last_fullsized":"2024-05-09T04:52:00.723281+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:25:44.517789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.538748+0000","last_change":"2024-05-09T04:49:34.147027+0000","last_active":"2024-05-09T04:52:55.538748+0000","last_peered":"2024-05-09T04:52:55.538748+0000","last_clean":"2024-05-09T04:52:55.538748+0000","last_became_active":"2024-05-09T04:49:34.146672+0000","last_became_peered":"2024-05-09T04:49:34.146672+0000","last_unstale":"2024-05-09T04:52:55.538748+0000","last_undegraded":"2024-05-09T04:52:55.538748+0000","last_fullsized":"2024-05-09T04:52:55.538748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:15:44.847304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:29.591646+0000","last_change":"2024-05-09T04:49:30.043877+0000","last_active":"2024-05-09T04:51:29.591646+0000","last_peered":"2024-05-09T04:51:29.591646+0000","last_clean":"2024-05-09T04:51:29.591646+0000","last_became_active":"2024-05-09T04:49:30.043513+0000","last_became_peered":"2024-05-09T04:49:30.043513+0000","last_unstale":"2024-05-09T04:51:29.591646+0000","last_undegraded":"2024-05-09T04:51:29.591646+0000","last_fullsized":"2024-05-09T04:51:29.591646+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:16.501692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:51.957672+0000","last_change":"2024-05-09T04:49:47.668605+0000","last_active":"2024-05-09T04:51:51.957672+0000","last_peered":"2024-05-09T04:51:51.957672+0000","last_clean":"2024-05-09T04:51:51.957672+0000","last_became_active":"2024-05-09T04:49:47.668095+0000","last_became_peered":"2024-05-09T04:49:47.668095+0000","last_unstale":"2024-05-09T04:51:51.957672+0000","last_undegraded":"2024-05-09T04:51:51.957672+0000","last_fullsized":"2024-05-09T04:51:51.957672+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:39:30.508618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:35.710826+0000","last_change":"2024-05-09T04:49:30.159554+0000","last_active":"2024-05-09T04:51:35.710826+0000","last_peered":"2024-05-09T04:51:35.710826+0000","last_clean":"2024-05-09T04:51:35.710826+0000","last_became_active":"2024-05-09T04:49:30.159266+0000","last_became_peered":"2024-05-09T04:49:30.159266+0000","last_unstale":"2024-05-09T04:51:35.710826+0000","last_undegraded":"2024-05-09T04:51:35.710826+0000","last_fullsized":"2024-05-09T04:51:35.710826+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:42:04.252862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.542103+0000","last_change":"2024-05-09T04:49:34.150688+0000","last_active":"2024-05-09T04:52:55.542103+0000","last_peered":"2024-05-09T04:52:55.542103+0000","last_clean":"2024-05-09T04:52:55.542103+0000","last_became_active":"2024-05-09T04:49:34.150319+0000","last_became_peered":"2024-05-09T04:49:34.150319+0000","last_unstale":"2024-05-09T04:52:55.542103+0000","last_undegraded":"2024-05-09T04:52:55.542103+0000","last_fullsized":"2024-05-09T04:52:55.542103+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:35:04.832761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:48.131063+0000","last_change":"2024-05-09T04:49:47.902323+0000","last_active":"2024-05-09T04:51:48.131063+0000","last_peered":"2024-05-09T04:51:48.131063+0000","last_clean":"2024-05-09T04:51:48.131063+0000","last_became_active":"2024-05-09T04:49:47.901389+0000","last_became_peered":"2024-05-09T04:49:47.901389+0000","last_unstale":"2024-05-09T04:51:48.131063+0000","last_undegraded":"2024-05-09T04:51:48.131063+0000","last_fullsized":"2024-05-09T04:51:48.131063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:32.615944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'17","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:53:52.864012+0000","last_change":"2024-05-09T04:49:30.061172+0000","last_active":"2024-05-09T04:53:52.864012+0000","last_peered":"2024-05-09T04:53:52.864012+0000","last_clean":"2024-05-09T04:53:52.864012+0000","last_became_active":"2024-05-09T04:49:30.060680+0000","last_became_peered":"2024-05-09T04:49:30.060680+0000","last_unstale":"2024-05-09T04:53:52.864012+0000","last_undegraded":"2024-05-09T04:53:52.864012+0000","last_fullsized":"2024-05-09T04:53:52.864012+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:45:44.649124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.551743+0000","last_change":"2024-05-09T04:49:34.130319+0000","last_active":"2024-05-09T04:52:55.551743+0000","last_peered":"2024-05-09T04:52:55.551743+0000","last_clean":"2024-05-09T04:52:55.551743+0000","last_became_active":"2024-05-09T04:49:34.129610+0000","last_became_peered":"2024-05-09T04:49:34.129610+0000","last_unstale":"2024-05-09T04:52:55.551743+0000","last_undegraded":"2024-05-09T04:52:55.551743+0000","last_fullsized":"2024-05-09T04:52:55.551743+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:45.505221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:52.134484+0000","last_change":"2024-05-09T04:49:47.647513+0000","last_active":"2024-05-09T04:51:52.134484+0000","last_peered":"2024-05-09T04:51:52.134484+0000","last_clean":"2024-05-09T04:51:52.134484+0000","last_became_active":"2024-05-09T04:49:47.647069+0000","last_became_peered":"2024-05-09T04:49:47.647069+0000","last_unstale":"2024-05-09T04:51:52.134484+0000","last_undegraded":"2024-05-09T04:51:52.134484+0000","last_fullsized":"2024-05-09T04:51:52.134484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:05:27.658919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:41.233581+0000","last_change":"2024-05-09T04:49:30.350850+0000","last_active":"2024-05-09T04:51:41.233581+0000","last_peered":"2024-05-09T04:51:41.233581+0000","last_clean":"2024-05-09T04:51:41.233581+0000","last_became_active":"2024-05-09T04:49:30.350431+0000","last_became_peered":"2024-05-09T04:49:30.350431+0000","last_unstale":"2024-05-09T04:51:41.233581+0000","last_undegraded":"2024-05-09T04:51:41.233581+0000","last_fullsized":"2024-05-09T04:51:41.233581+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:53:05.388338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.562514+0000","last_change":"2024-05-09T04:49:34.153630+0000","last_active":"2024-05-09T04:52:55.562514+0000","last_peered":"2024-05-09T04:52:55.562514+0000","last_clean":"2024-05-09T04:52:55.562514+0000","last_became_active":"2024-05-09T04:49:34.152045+0000","last_became_peered":"2024-05-09T04:49:34.152045+0000","last_unstale":"2024-05-09T04:52:55.562514+0000","last_undegraded":"2024-05-09T04:52:55.562514+0000","last_fullsized":"2024-05-09T04:52:55.562514+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:32:35.991731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:06.236624+0000","last_change":"2024-05-09T04:49:47.901256+0000","last_active":"2024-05-09T04:52:06.236624+0000","last_peered":"2024-05-09T04:52:06.236624+0000","last_clean":"2024-05-09T04:52:06.236624+0000","last_became_active":"2024-05-09T04:49:47.900413+0000","last_became_peered":"2024-05-09T04:49:47.900413+0000","last_unstale":"2024-05-09T04:52:06.236624+0000","last_undegraded":"2024-05-09T04:52:06.236624+0000","last_fullsized":"2024-05-09T04:52:06.236624+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:46:11.845220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:42.647230+0000","last_change":"2024-05-09T04:49:30.152273+0000","last_active":"2024-05-09T04:51:42.647230+0000","last_peered":"2024-05-09T04:51:42.647230+0000","last_clean":"2024-05-09T04:51:42.647230+0000","last_became_active":"2024-05-09T04:49:30.151973+0000","last_became_peered":"2024-05-09T04:49:30.151973+0000","last_unstale":"2024-05-09T04:51:42.647230+0000","last_undegraded":"2024-05-09T04:51:42.647230+0000","last_fullsized":"2024-05-09T04:51:42.647230+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:14:52.912588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.517045+0000","last_change":"2024-05-09T04:49:34.126829+0000","last_active":"2024-05-09T04:52:55.517045+0000","last_peered":"2024-05-09T04:52:55.517045+0000","last_clean":"2024-05-09T04:52:55.517045+0000","last_became_active":"2024-05-09T04:49:34.126430+0000","last_became_peered":"2024-05-09T04:49:34.126430+0000","last_unstale":"2024-05-09T04:52:55.517045+0000","last_undegraded":"2024-05-09T04:52:55.517045+0000","last_fullsized":"2024-05-09T04:52:55.517045+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:38.247148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:02.843444+0000","last_change":"2024-05-09T04:49:47.675292+0000","last_active":"2024-05-09T04:52:02.843444+0000","last_peered":"2024-05-09T04:52:02.843444+0000","last_clean":"2024-05-09T04:52:02.843444+0000","last_became_active":"2024-05-09T04:49:47.674802+0000","last_became_peered":"2024-05-09T04:49:47.674802+0000","last_unstale":"2024-05-09T04:52:02.843444+0000","last_undegraded":"2024-05-09T04:52:02.843444+0000","last_fullsized":"2024-05-09T04:52:02.843444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:47:21.988054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:43.130050+0000","last_change":"2024-05-09T04:49:30.057854+0000","last_active":"2024-05-09T04:51:43.130050+0000","last_peered":"2024-05-09T04:51:43.130050+0000","last_clean":"2024-05-09T04:51:43.130050+0000","last_became_active":"2024-05-09T04:49:30.056625+0000","last_became_peered":"2024-05-09T04:49:30.056625+0000","last_unstale":"2024-05-09T04:51:43.130050+0000","last_undegraded":"2024-05-09T04:51:43.130050+0000","last_fullsized":"2024-05-09T04:51:43.130050+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:51.272533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.559538+0000","last_change":"2024-05-09T04:49:34.144414+0000","last_active":"2024-05-09T04:52:55.559538+0000","last_peered":"2024-05-09T04:52:55.559538+0000","last_clean":"2024-05-09T04:52:55.559538+0000","last_became_active":"2024-05-09T04:49:34.144064+0000","last_became_peered":"2024-05-09T04:49:34.144064+0000","last_unstale":"2024-05-09T04:52:55.559538+0000","last_undegraded":"2024-05-09T04:52:55.559538+0000","last_fullsized":"2024-05-09T04:52:55.559538+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:44:40.991576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:06.236580+0000","last_change":"2024-05-09T04:49:47.672956+0000","last_active":"2024-05-09T04:52:06.236580+0000","last_peered":"2024-05-09T04:52:06.236580+0000","last_clean":"2024-05-09T04:52:06.236580+0000","last_became_active":"2024-05-09T04:49:47.672513+0000","last_became_peered":"2024-05-09T04:49:47.672513+0000","last_unstale":"2024-05-09T04:52:06.236580+0000","last_undegraded":"2024-05-09T04:52:06.236580+0000","last_fullsized":"2024-05-09T04:52:06.236580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:32.250039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:06.959528+0000","last_change":"2024-05-09T04:49:56.704058+0000","last_active":"2024-05-09T04:52:06.959528+0000","last_peered":"2024-05-09T04:52:06.959528+0000","last_clean":"2024-05-09T04:52:06.959528+0000","last_became_active":"2024-05-09T04:49:56.703589+0000","last_became_peered":"2024-05-09T04:49:56.703589+0000","last_unstale":"2024-05-09T04:52:06.959528+0000","last_undegraded":"2024-05-09T04:52:06.959528+0000","last_fullsized":"2024-05-09T04:52:06.959528+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:54.596200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,1],"acting":[11,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.555701+0000","last_change":"2024-05-09T04:49:34.159321+0000","last_active":"2024-05-09T04:52:55.555701+0000","last_peered":"2024-05-09T04:52:55.555701+0000","last_clean":"2024-05-09T04:52:55.555701+0000","last_became_active":"2024-05-09T04:49:34.157918+0000","last_became_peered":"2024-05-09T04:49:34.157918+0000","last_unstale":"2024-05-09T04:52:55.555701+0000","last_undegraded":"2024-05-09T04:52:55.555701+0000","last_fullsized":"2024-05-09T04:52:55.555701+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:33:59.659905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:06.236605+0000","last_change":"2024-05-09T04:49:47.900692+0000","last_active":"2024-05-09T04:52:06.236605+0000","last_peered":"2024-05-09T04:52:06.236605+0000","last_clean":"2024-05-09T04:52:06.236605+0000","last_became_active":"2024-05-09T04:49:47.900095+0000","last_became_peered":"2024-05-09T04:49:47.900095+0000","last_unstale":"2024-05-09T04:52:06.236605+0000","last_undegraded":"2024-05-09T04:52:06.236605+0000","last_fullsized":"2024-05-09T04:52:06.236605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:08.632408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:45.712091+0000","last_change":"2024-05-09T04:49:30.048768+0000","last_active":"2024-05-09T04:51:45.712091+0000","last_peered":"2024-05-09T04:51:45.712091+0000","last_clean":"2024-05-09T04:51:45.712091+0000","last_became_active":"2024-05-09T04:49:30.048235+0000","last_became_peered":"2024-05-09T04:49:30.048235+0000","last_unstale":"2024-05-09T04:51:45.712091+0000","last_undegraded":"2024-05-09T04:51:45.712091+0000","last_fullsized":"2024-05-09T04:51:45.712091+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:51:16.988726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.540136+0000","last_change":"2024-05-09T04:49:34.142110+0000","last_active":"2024-05-09T04:52:55.540136+0000","last_peered":"2024-05-09T04:52:55.540136+0000","last_clean":"2024-05-09T04:52:55.540136+0000","last_became_active":"2024-05-09T04:49:34.141709+0000","last_became_peered":"2024-05-09T04:49:34.141709+0000","last_unstale":"2024-05-09T04:52:55.540136+0000","last_undegraded":"2024-05-09T04:52:55.540136+0000","last_fullsized":"2024-05-09T04:52:55.540136+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:16.365915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:57.649137+0000","last_change":"2024-05-09T04:49:47.667784+0000","last_active":"2024-05-09T04:51:57.649137+0000","last_peered":"2024-05-09T04:51:57.649137+0000","last_clean":"2024-05-09T04:51:57.649137+0000","last_became_active":"2024-05-09T04:49:47.667316+0000","last_became_peered":"2024-05-09T04:49:47.667316+0000","last_unstale":"2024-05-09T04:51:57.649137+0000","last_undegraded":"2024-05-09T04:51:57.649137+0000","last_fullsized":"2024-05-09T04:51:57.649137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:04:20.233287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'6","reported_seq":287,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:54:56.136712+0000","last_change":"2024-05-09T04:49:30.351231+0000","last_active":"2024-05-09T04:54:56.136712+0000","last_peered":"2024-05-09T04:54:56.136712+0000","last_clean":"2024-05-09T04:54:56.136712+0000","last_became_active":"2024-05-09T04:49:30.350817+0000","last_became_peered":"2024-05-09T04:49:30.350817+0000","last_unstale":"2024-05-09T04:54:56.136712+0000","last_undegraded":"2024-05-09T04:54:56.136712+0000","last_fullsized":"2024-05-09T04:54:56.136712+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:51.234300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1480,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":237,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.540495+0000","last_change":"2024-05-09T04:49:34.129625+0000","last_active":"2024-05-09T04:52:55.540495+0000","last_peered":"2024-05-09T04:52:55.540495+0000","last_clean":"2024-05-09T04:52:55.540495+0000","last_became_active":"2024-05-09T04:49:34.129215+0000","last_became_peered":"2024-05-09T04:49:34.129215+0000","last_unstale":"2024-05-09T04:52:55.540495+0000","last_undegraded":"2024-05-09T04:52:55.540495+0000","last_fullsized":"2024-05-09T04:52:55.540495+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:26.247116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:01.811085+0000","last_change":"2024-05-09T04:49:47.682819+0000","last_active":"2024-05-09T04:52:01.811085+0000","last_peered":"2024-05-09T04:52:01.811085+0000","last_clean":"2024-05-09T04:52:01.811085+0000","last_became_active":"2024-05-09T04:49:47.682331+0000","last_became_peered":"2024-05-09T04:49:47.682331+0000","last_unstale":"2024-05-09T04:52:01.811085+0000","last_undegraded":"2024-05-09T04:52:01.811085+0000","last_fullsized":"2024-05-09T04:52:01.811085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:03.852975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:45.721497+0000","last_change":"2024-05-09T04:49:30.050247+0000","last_active":"2024-05-09T04:51:45.721497+0000","last_peered":"2024-05-09T04:51:45.721497+0000","last_clean":"2024-05-09T04:51:45.721497+0000","last_became_active":"2024-05-09T04:49:30.049835+0000","last_became_peered":"2024-05-09T04:49:30.049835+0000","last_unstale":"2024-05-09T04:51:45.721497+0000","last_undegraded":"2024-05-09T04:51:45.721497+0000","last_fullsized":"2024-05-09T04:51:45.721497+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:52.591186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.559764+0000","last_change":"2024-05-09T04:49:34.145406+0000","last_active":"2024-05-09T04:52:55.559764+0000","last_peered":"2024-05-09T04:52:55.559764+0000","last_clean":"2024-05-09T04:52:55.559764+0000","last_became_active":"2024-05-09T04:49:34.144946+0000","last_became_peered":"2024-05-09T04:49:34.144946+0000","last_unstale":"2024-05-09T04:52:55.559764+0000","last_undegraded":"2024-05-09T04:52:55.559764+0000","last_fullsized":"2024-05-09T04:52:55.559764+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:43:04.704215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:05.714387+0000","last_change":"2024-05-09T04:49:56.758844+0000","last_active":"2024-05-09T04:52:05.714387+0000","last_peered":"2024-05-09T04:52:05.714387+0000","last_clean":"2024-05-09T04:52:05.714387+0000","last_became_active":"2024-05-09T04:49:56.758335+0000","last_became_peered":"2024-05-09T04:49:56.758335+0000","last_unstale":"2024-05-09T04:52:05.714387+0000","last_undegraded":"2024-05-09T04:52:05.714387+0000","last_fullsized":"2024-05-09T04:52:05.714387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:39:33.344225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:47.850941+0000","last_change":"2024-05-09T04:49:56.766262+0000","last_active":"2024-05-09T04:51:47.850941+0000","last_peered":"2024-05-09T04:51:47.850941+0000","last_clean":"2024-05-09T04:51:47.850941+0000","last_became_active":"2024-05-09T04:49:56.758765+0000","last_became_peered":"2024-05-09T04:49:56.758765+0000","last_unstale":"2024-05-09T04:51:47.850941+0000","last_undegraded":"2024-05-09T04:51:47.850941+0000","last_fullsized":"2024-05-09T04:51:47.850941+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:44:18.271729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.540919+0000","last_change":"2024-05-09T04:49:34.134957+0000","last_active":"2024-05-09T04:52:55.540919+0000","last_peered":"2024-05-09T04:52:55.540919+0000","last_clean":"2024-05-09T04:52:55.540919+0000","last_became_active":"2024-05-09T04:49:34.134544+0000","last_became_peered":"2024-05-09T04:49:34.134544+0000","last_unstale":"2024-05-09T04:52:55.540919+0000","last_undegraded":"2024-05-09T04:52:55.540919+0000","last_fullsized":"2024-05-09T04:52:55.540919+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:21:54.855905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.525564+0000","last_change":"2024-05-09T04:49:34.138112+0000","last_active":"2024-05-09T04:52:55.525564+0000","last_peered":"2024-05-09T04:52:55.525564+0000","last_clean":"2024-05-09T04:52:55.525564+0000","last_became_active":"2024-05-09T04:49:34.137716+0000","last_became_peered":"2024-05-09T04:49:34.137716+0000","last_unstale":"2024-05-09T04:52:55.525564+0000","last_undegraded":"2024-05-09T04:52:55.525564+0000","last_fullsized":"2024-05-09T04:52:55.525564+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:17.028262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.534478+0000","last_change":"2024-05-09T04:50:58.204287+0000","last_active":"2024-05-09T04:52:55.534478+0000","last_peered":"2024-05-09T04:52:55.534478+0000","last_clean":"2024-05-09T04:52:55.534478+0000","last_became_active":"2024-05-09T04:50:58.203894+0000","last_became_peered":"2024-05-09T04:50:58.203894+0000","last_unstale":"2024-05-09T04:52:55.534478+0000","last_undegraded":"2024-05-09T04:52:55.534478+0000","last_fullsized":"2024-05-09T04:52:55.534478+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:02.214347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.558687+0000","last_change":"2024-05-09T04:49:34.147742+0000","last_active":"2024-05-09T04:52:55.558687+0000","last_peered":"2024-05-09T04:52:55.558687+0000","last_clean":"2024-05-09T04:52:55.558687+0000","last_became_active":"2024-05-09T04:49:34.147301+0000","last_became_peered":"2024-05-09T04:49:34.147301+0000","last_unstale":"2024-05-09T04:52:55.558687+0000","last_undegraded":"2024-05-09T04:52:55.558687+0000","last_fullsized":"2024-05-09T04:52:55.558687+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:36:55.371212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.552970+0000","last_change":"2024-05-09T04:49:34.163599+0000","last_active":"2024-05-09T04:52:55.552970+0000","last_peered":"2024-05-09T04:52:55.552970+0000","last_clean":"2024-05-09T04:52:55.552970+0000","last_became_active":"2024-05-09T04:49:34.162898+0000","last_became_peered":"2024-05-09T04:49:34.162898+0000","last_unstale":"2024-05-09T04:52:55.552970+0000","last_undegraded":"2024-05-09T04:52:55.552970+0000","last_fullsized":"2024-05-09T04:52:55.552970+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:55:57.213901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.552433+0000","last_change":"2024-05-09T04:49:34.151648+0000","last_active":"2024-05-09T04:52:55.552433+0000","last_peered":"2024-05-09T04:52:55.552433+0000","last_clean":"2024-05-09T04:52:55.552433+0000","last_became_active":"2024-05-09T04:49:34.148701+0000","last_became_peered":"2024-05-09T04:49:34.148701+0000","last_unstale":"2024-05-09T04:52:55.552433+0000","last_undegraded":"2024-05-09T04:52:55.552433+0000","last_fullsized":"2024-05-09T04:52:55.552433+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:54:30.122252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.521268+0000","last_change":"2024-05-09T04:49:34.136679+0000","last_active":"2024-05-09T04:52:55.521268+0000","last_peered":"2024-05-09T04:52:55.521268+0000","last_clean":"2024-05-09T04:52:55.521268+0000","last_became_active":"2024-05-09T04:49:34.136306+0000","last_became_peered":"2024-05-09T04:49:34.136306+0000","last_unstale":"2024-05-09T04:52:55.521268+0000","last_undegraded":"2024-05-09T04:52:55.521268+0000","last_fullsized":"2024-05-09T04:52:55.521268+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:51:39.476564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.539107+0000","last_change":"2024-05-09T04:49:34.130769+0000","last_active":"2024-05-09T04:52:55.539107+0000","last_peered":"2024-05-09T04:52:55.539107+0000","last_clean":"2024-05-09T04:52:55.539107+0000","last_became_active":"2024-05-09T04:49:34.130362+0000","last_became_peered":"2024-05-09T04:49:34.130362+0000","last_unstale":"2024-05-09T04:52:55.539107+0000","last_undegraded":"2024-05-09T04:52:55.539107+0000","last_fullsized":"2024-05-09T04:52:55.539107+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:54.942259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.552538+0000","last_change":"2024-05-09T04:49:34.138054+0000","last_active":"2024-05-09T04:52:55.552538+0000","last_peered":"2024-05-09T04:52:55.552538+0000","last_clean":"2024-05-09T04:52:55.552538+0000","last_became_active":"2024-05-09T04:49:34.136111+0000","last_became_peered":"2024-05-09T04:49:34.136111+0000","last_unstale":"2024-05-09T04:52:55.552538+0000","last_undegraded":"2024-05-09T04:52:55.552538+0000","last_fullsized":"2024-05-09T04:52:55.552538+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:15:27.870874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.562318+0000","last_change":"2024-05-09T04:49:34.146370+0000","last_active":"2024-05-09T04:52:55.562318+0000","last_peered":"2024-05-09T04:52:55.562318+0000","last_clean":"2024-05-09T04:52:55.562318+0000","last_became_active":"2024-05-09T04:49:34.145948+0000","last_became_peered":"2024-05-09T04:49:34.145948+0000","last_unstale":"2024-05-09T04:52:55.562318+0000","last_undegraded":"2024-05-09T04:52:55.562318+0000","last_fullsized":"2024-05-09T04:52:55.562318+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:14.826168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.555403+0000","last_change":"2024-05-09T04:49:34.138028+0000","last_active":"2024-05-09T04:52:55.555403+0000","last_peered":"2024-05-09T04:52:55.555403+0000","last_clean":"2024-05-09T04:52:55.555403+0000","last_became_active":"2024-05-09T04:49:34.137558+0000","last_became_peered":"2024-05-09T04:49:34.137558+0000","last_unstale":"2024-05-09T04:52:55.555403+0000","last_undegraded":"2024-05-09T04:52:55.555403+0000","last_fullsized":"2024-05-09T04:52:55.555403+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:50:16.258945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.556779+0000","last_change":"2024-05-09T04:49:34.137946+0000","last_active":"2024-05-09T04:52:55.556779+0000","last_peered":"2024-05-09T04:52:55.556779+0000","last_clean":"2024-05-09T04:52:55.556779+0000","last_became_active":"2024-05-09T04:49:34.137498+0000","last_became_peered":"2024-05-09T04:49:34.137498+0000","last_unstale":"2024-05-09T04:52:55.556779+0000","last_undegraded":"2024-05-09T04:52:55.556779+0000","last_fullsized":"2024-05-09T04:52:55.556779+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:36:54.473689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.552098+0000","last_change":"2024-05-09T04:49:34.162103+0000","last_active":"2024-05-09T04:52:55.552098+0000","last_peered":"2024-05-09T04:52:55.552098+0000","last_clean":"2024-05-09T04:52:55.552098+0000","last_became_active":"2024-05-09T04:49:34.161681+0000","last_became_peered":"2024-05-09T04:49:34.161681+0000","last_unstale":"2024-05-09T04:52:55.552098+0000","last_undegraded":"2024-05-09T04:52:55.552098+0000","last_fullsized":"2024-05-09T04:52:55.552098+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:36:10.460401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.529070+0000","last_change":"2024-05-09T04:49:34.136538+0000","last_active":"2024-05-09T04:52:55.529070+0000","last_peered":"2024-05-09T04:52:55.529070+0000","last_clean":"2024-05-09T04:52:55.529070+0000","last_became_active":"2024-05-09T04:49:34.134903+0000","last_became_peered":"2024-05-09T04:49:34.134903+0000","last_unstale":"2024-05-09T04:52:55.529070+0000","last_undegraded":"2024-05-09T04:52:55.529070+0000","last_fullsized":"2024-05-09T04:52:55.529070+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:44:48.673600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72595,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":2378,"num_read_kb":1270,"num_write":265,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":229313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678098,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":510561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874328,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37016,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708136,"statfs":{"total":95995035648,"available":95957131264,"internally_reserved":0,"allocated":1720320,"data_stored":1161976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037853,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":514165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234085,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":510335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430314,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1642496,"data_stored":1108008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593840,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":505573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574990,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":577166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738514,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":507339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1118208,"data_stored":577837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149826,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":505281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953597,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":510179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790070,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1650688,"data_stored":1102667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":72315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":67067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T04:55:01.843 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T04:55:02.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:01 smithi006 ceph-mon[29835]: from='client.25240 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:55:02.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:01 smithi186 ceph-mon[34668]: from='client.25240 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:55:02.357 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:55:02.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:01 smithi169 ceph-mon[34075]: from='client.25240 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:55:03.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:02 smithi006 ceph-mon[29835]: pgmap v502: 129 pgs: 129 active+clean; 654 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-09T04:55:03.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:02 smithi186 ceph-mon[34668]: pgmap v502: 129 pgs: 129 active+clean; 654 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-09T04:55:03.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:02 smithi169 ceph-mon[34075]: pgmap v502: 129 pgs: 129 active+clean; 654 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-09T04:55:04.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.730+0000 7f528fe00700 1 -- 172.21.15.6:0/1503503782 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5290075b60 msgr2=0x7f5290075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:04.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.730+0000 7f528fe00700 1 --2- 172.21.15.6:0/1503503782 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5290075b60 0x7f5290075f70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5280009990 tx=0x7f5280009ca0 comp rx=0 tx=0).stop 2024-05-09T04:55:04.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.731+0000 7f528fe00700 1 -- 172.21.15.6:0/1503503782 shutdown_connections 2024-05-09T04:55:04.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.731+0000 7f528fe00700 1 --2- 172.21.15.6:0/1503503782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5290115180 0x7f52901175a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:04.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.731+0000 7f528fe00700 1 --2- 172.21.15.6:0/1503503782 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5290075b60 0x7f5290075f70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:04.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.731+0000 7f528fe00700 1 --2- 172.21.15.6:0/1503503782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52900764b0 0x7f5290114c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:04.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.731+0000 7f528fe00700 1 -- 172.21.15.6:0/1503503782 >> 172.21.15.6:0/1503503782 conn(0x7f5290079d10 msgr2=0x7f529007a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:04.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.731+0000 7f528fe00700 1 -- 172.21.15.6:0/1503503782 shutdown_connections 2024-05-09T04:55:04.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.731+0000 7f528fe00700 1 -- 172.21.15.6:0/1503503782 wait complete. 2024-05-09T04:55:04.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.732+0000 7f528fe00700 1 Processor -- start 2024-05-09T04:55:04.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.733+0000 7f528fe00700 1 -- start start 2024-05-09T04:55:04.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.733+0000 7f528fe00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5290075b60 0x7f52901a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:04.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.733+0000 7f528fe00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52900764b0 0x7f52901a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:04.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.733+0000 7f528fe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5290115180 0x7f52901a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:04.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.733+0000 7f528fe00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f529011a8e0 con 0x7f5290115180 2024-05-09T04:55:04.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.733+0000 7f528fe00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f529011a760 con 0x7f52900764b0 2024-05-09T04:55:04.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.733+0000 7f528fe00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f529011aa60 con 0x7f5290075b60 2024-05-09T04:55:04.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.734+0000 7f528f400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5290115180 0x7f52901a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:04.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.734+0000 7f528e000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52900764b0 0x7f52901a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:04.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.734+0000 7f528f400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5290115180 0x7f52901a5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36908/0 (socket says 172.21.15.6:36908) 2024-05-09T04:55:04.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.734+0000 7f528f400700 1 -- 172.21.15.6:0/2856848499 learned_addr learned my addr 172.21.15.6:0/2856848499 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:55:04.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.734+0000 7f528e000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52900764b0 0x7f52901a18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:59714/0 (socket says 172.21.15.6:59714) 2024-05-09T04:55:04.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.734+0000 7f528ea00700 1 --2- 172.21.15.6:0/2856848499 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5290075b60 0x7f52901a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:04.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.734+0000 7f528e000700 1 -- 172.21.15.6:0/2856848499 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5290075b60 msgr2=0x7f52901a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:04.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.734+0000 7f528e000700 1 --2- 172.21.15.6:0/2856848499 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5290075b60 0x7f52901a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:04.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.734+0000 7f528e000700 1 -- 172.21.15.6:0/2856848499 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5290115180 msgr2=0x7f52901a5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:04.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.734+0000 7f528e000700 1 --2- 172.21.15.6:0/2856848499 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5290115180 0x7f52901a5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:04.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.734+0000 7f528e000700 1 -- 172.21.15.6:0/2856848499 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52901a61e0 con 0x7f52900764b0 2024-05-09T04:55:04.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.734+0000 7f528f400700 1 --2- 172.21.15.6:0/2856848499 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5290115180 0x7f52901a5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:55:04.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.735+0000 7f528e000700 1 --2- 172.21.15.6:0/2856848499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52900764b0 0x7f52901a18b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f527c00e9f0 tx=0x7f527c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:04.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.735+0000 7f5294a00700 1 -- 172.21.15.6:0/2856848499 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c00c9c0 con 0x7f52900764b0 2024-05-09T04:55:04.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.735+0000 7f5294a00700 1 -- 172.21.15.6:0/2856848499 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f527c00cb60 con 0x7f52900764b0 2024-05-09T04:55:04.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.735+0000 7f528fe00700 1 -- 172.21.15.6:0/2856848499 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52901a64d0 con 0x7f52900764b0 2024-05-09T04:55:04.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.735+0000 7f5294a00700 1 -- 172.21.15.6:0/2856848499 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c018680 con 0x7f52900764b0 2024-05-09T04:55:04.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.735+0000 7f528fe00700 1 -- 172.21.15.6:0/2856848499 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52901addf0 con 0x7f52900764b0 2024-05-09T04:55:04.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.736+0000 7f528fe00700 1 -- 172.21.15.6:0/2856848499 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52900772a0 con 0x7f52900764b0 2024-05-09T04:55:04.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.739+0000 7f5294a00700 1 -- 172.21.15.6:0/2856848499 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 19) ==== 149702+0+0 (secure 0 0 0) 0x7f527c04aae0 con 0x7f52900764b0 2024-05-09T04:55:04.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.741+0000 7f5294a00700 1 --2- 172.21.15.6:0/2856848499 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f52740b2910 0x7f52740b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:04.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.741+0000 7f5294a00700 1 -- 172.21.15.6:0/2856848499 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f527c0e2a60 con 0x7f52900764b0 2024-05-09T04:55:04.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.741+0000 7f528ea00700 1 --2- 172.21.15.6:0/2856848499 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f52740b2910 0x7f52740b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:04.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.742+0000 7f528ea00700 1 --2- 172.21.15.6:0/2856848499 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f52740b2910 0x7f52740b4e20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5280006010 tx=0x7f528000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:04.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.743+0000 7f5294a00700 1 -- 172.21.15.6:0/2856848499 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f527c0aa240 con 0x7f52900764b0 2024-05-09T04:55:04.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.976+0000 7f528fe00700 1 -- 172.21.15.6:0/2856848499 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f52901a2670 con 0x7f52740b2910 2024-05-09T04:55:04.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:04.993+0000 7f5294a00700 1 -- 172.21.15.6:0/2856848499 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340571 (secure 0 0 0) 0x7f528007b270 con 0x7f52740b2910 2024-05-09T04:55:04.994 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:04 smithi006 ceph-mon[29835]: pgmap v503: 129 pgs: 129 active+clean; 654 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-05-09T04:55:04.995 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:55:05.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:05.002+0000 7f528fe00700 1 -- 172.21.15.6:0/2856848499 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f52740b2910 msgr2=0x7f52740b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:05.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:05.002+0000 7f528fe00700 1 --2- 172.21.15.6:0/2856848499 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f52740b2910 0x7f52740b4e20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5280006010 tx=0x7f528000b260 comp rx=0 tx=0).stop 2024-05-09T04:55:05.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:05.002+0000 7f528fe00700 1 -- 172.21.15.6:0/2856848499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52900764b0 msgr2=0x7f52901a18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:05.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:05.002+0000 7f528fe00700 1 --2- 172.21.15.6:0/2856848499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52900764b0 0x7f52901a18b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f527c00e9f0 tx=0x7f527c00ed00 comp rx=0 tx=0).stop 2024-05-09T04:55:05.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:05.003+0000 7f528fe00700 1 -- 172.21.15.6:0/2856848499 shutdown_connections 2024-05-09T04:55:05.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:05.003+0000 7f528fe00700 1 --2- 172.21.15.6:0/2856848499 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5290115180 0x7f52901a5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:05.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:05.003+0000 7f528fe00700 1 --2- 172.21.15.6:0/2856848499 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5290075b60 0x7f52901a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:05.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:05.003+0000 7f528fe00700 1 --2- 172.21.15.6:0/2856848499 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f52740b2910 0x7f52740b4e20 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:05.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:05.003+0000 7f528fe00700 1 --2- 172.21.15.6:0/2856848499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52900764b0 0x7f52901a18b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:05.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:05.003+0000 7f528fe00700 1 -- 172.21.15.6:0/2856848499 >> 172.21.15.6:0/2856848499 conn(0x7f5290079d10 msgr2=0x7f5290102270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:05.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:05.003+0000 7f528fe00700 1 -- 172.21.15.6:0/2856848499 shutdown_connections 2024-05-09T04:55:05.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:05.003+0000 7f528fe00700 1 -- 172.21.15.6:0/2856848499 wait complete. 2024-05-09T04:55:05.005 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-05-09T04:55:05.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:04 smithi186 ceph-mon[34668]: pgmap v503: 129 pgs: 129 active+clean; 654 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-05-09T04:55:05.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:04 smithi169 ceph-mon[34075]: pgmap v503: 129 pgs: 129 active+clean; 654 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-05-09T04:55:05.895 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":503,"stamp":"2024-05-09T04:55:03.714221+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":669743,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":2503,"num_read_kb":1368,"num_write":399,"num_write_kb":2320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":830,"ondisk_log_size":830,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371640,"kb_used_data":14168,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570184,"statfs":{"total":1151940427776,"available":1151559868416,"internally_reserved":0,"allocated":14508032,"data_stored":8104547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19137,"internal_metadata":365933887},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7166,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":121,"num_read_kb":71,"num_write":15,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005964"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.559597+0000","last_change":"2024-05-09T04:49:34.144921+0000","last_active":"2024-05-09T04:52:55.559597+0000","last_peered":"2024-05-09T04:52:55.559597+0000","last_clean":"2024-05-09T04:52:55.559597+0000","last_became_active":"2024-05-09T04:49:34.144275+0000","last_became_peered":"2024-05-09T04:49:34.144275+0000","last_unstale":"2024-05-09T04:52:55.559597+0000","last_undegraded":"2024-05-09T04:52:55.559597+0000","last_fullsized":"2024-05-09T04:52:55.559597+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:00:31.377738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.559444+0000","last_change":"2024-05-09T04:49:34.158057+0000","last_active":"2024-05-09T04:52:55.559444+0000","last_peered":"2024-05-09T04:52:55.559444+0000","last_clean":"2024-05-09T04:52:55.559444+0000","last_became_active":"2024-05-09T04:49:34.157129+0000","last_became_peered":"2024-05-09T04:49:34.157129+0000","last_unstale":"2024-05-09T04:52:55.559444+0000","last_undegraded":"2024-05-09T04:52:55.559444+0000","last_fullsized":"2024-05-09T04:52:55.559444+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:49.360309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.554527+0000","last_change":"2024-05-09T04:49:34.145519+0000","last_active":"2024-05-09T04:52:55.554527+0000","last_peered":"2024-05-09T04:52:55.554527+0000","last_clean":"2024-05-09T04:52:55.554527+0000","last_became_active":"2024-05-09T04:49:34.144501+0000","last_became_peered":"2024-05-09T04:49:34.144501+0000","last_unstale":"2024-05-09T04:52:55.554527+0000","last_undegraded":"2024-05-09T04:52:55.554527+0000","last_fullsized":"2024-05-09T04:52:55.554527+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:05:53.727520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.521578+0000","last_change":"2024-05-09T04:49:34.134383+0000","last_active":"2024-05-09T04:52:55.521578+0000","last_peered":"2024-05-09T04:52:55.521578+0000","last_clean":"2024-05-09T04:52:55.521578+0000","last_became_active":"2024-05-09T04:49:34.134049+0000","last_became_peered":"2024-05-09T04:49:34.134049+0000","last_unstale":"2024-05-09T04:52:55.521578+0000","last_undegraded":"2024-05-09T04:52:55.521578+0000","last_fullsized":"2024-05-09T04:52:55.521578+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:36:02.548188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.535797+0000","last_change":"2024-05-09T04:49:34.157897+0000","last_active":"2024-05-09T04:52:55.535797+0000","last_peered":"2024-05-09T04:52:55.535797+0000","last_clean":"2024-05-09T04:52:55.535797+0000","last_became_active":"2024-05-09T04:49:34.157516+0000","last_became_peered":"2024-05-09T04:49:34.157516+0000","last_unstale":"2024-05-09T04:52:55.535797+0000","last_undegraded":"2024-05-09T04:52:55.535797+0000","last_fullsized":"2024-05-09T04:52:55.535797+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:33:25.444155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.535225+0000","last_change":"2024-05-09T04:49:34.146389+0000","last_active":"2024-05-09T04:52:55.535225+0000","last_peered":"2024-05-09T04:52:55.535225+0000","last_clean":"2024-05-09T04:52:55.535225+0000","last_became_active":"2024-05-09T04:49:34.145994+0000","last_became_peered":"2024-05-09T04:49:34.145994+0000","last_unstale":"2024-05-09T04:52:55.535225+0000","last_undegraded":"2024-05-09T04:52:55.535225+0000","last_fullsized":"2024-05-09T04:52:55.535225+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:59.094826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.537694+0000","last_change":"2024-05-09T04:49:34.146428+0000","last_active":"2024-05-09T04:52:55.537694+0000","last_peered":"2024-05-09T04:52:55.537694+0000","last_clean":"2024-05-09T04:52:55.537694+0000","last_became_active":"2024-05-09T04:49:34.146045+0000","last_became_peered":"2024-05-09T04:49:34.146045+0000","last_unstale":"2024-05-09T04:52:55.537694+0000","last_undegraded":"2024-05-09T04:52:55.537694+0000","last_fullsized":"2024-05-09T04:52:55.537694+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:13.129474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.559750+0000","last_change":"2024-05-09T04:49:34.135494+0000","last_active":"2024-05-09T04:52:55.559750+0000","last_peered":"2024-05-09T04:52:55.559750+0000","last_clean":"2024-05-09T04:52:55.559750+0000","last_became_active":"2024-05-09T04:49:34.135085+0000","last_became_peered":"2024-05-09T04:49:34.135085+0000","last_unstale":"2024-05-09T04:52:55.559750+0000","last_undegraded":"2024-05-09T04:52:55.559750+0000","last_fullsized":"2024-05-09T04:52:55.559750+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:57.023344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.541612+0000","last_change":"2024-05-09T04:49:34.131353+0000","last_active":"2024-05-09T04:52:55.541612+0000","last_peered":"2024-05-09T04:52:55.541612+0000","last_clean":"2024-05-09T04:52:55.541612+0000","last_became_active":"2024-05-09T04:49:34.130954+0000","last_became_peered":"2024-05-09T04:49:34.130954+0000","last_unstale":"2024-05-09T04:52:55.541612+0000","last_undegraded":"2024-05-09T04:52:55.541612+0000","last_fullsized":"2024-05-09T04:52:55.541612+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:04:46.947861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.526988+0000","last_change":"2024-05-09T04:49:34.134733+0000","last_active":"2024-05-09T04:52:55.526988+0000","last_peered":"2024-05-09T04:52:55.526988+0000","last_clean":"2024-05-09T04:52:55.526988+0000","last_became_active":"2024-05-09T04:49:34.134337+0000","last_became_peered":"2024-05-09T04:49:34.134337+0000","last_unstale":"2024-05-09T04:52:55.526988+0000","last_undegraded":"2024-05-09T04:52:55.526988+0000","last_fullsized":"2024-05-09T04:52:55.526988+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:05.452488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.540863+0000","last_change":"2024-05-09T04:49:34.139143+0000","last_active":"2024-05-09T04:52:55.540863+0000","last_peered":"2024-05-09T04:52:55.540863+0000","last_clean":"2024-05-09T04:52:55.540863+0000","last_became_active":"2024-05-09T04:49:34.138749+0000","last_became_peered":"2024-05-09T04:49:34.138749+0000","last_unstale":"2024-05-09T04:52:55.540863+0000","last_undegraded":"2024-05-09T04:52:55.540863+0000","last_fullsized":"2024-05-09T04:52:55.540863+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:11.552063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.524374+0000","last_change":"2024-05-09T04:49:34.134687+0000","last_active":"2024-05-09T04:52:55.524374+0000","last_peered":"2024-05-09T04:52:55.524374+0000","last_clean":"2024-05-09T04:52:55.524374+0000","last_became_active":"2024-05-09T04:49:34.134324+0000","last_became_peered":"2024-05-09T04:49:34.134324+0000","last_unstale":"2024-05-09T04:52:55.524374+0000","last_undegraded":"2024-05-09T04:52:55.524374+0000","last_fullsized":"2024-05-09T04:52:55.524374+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:11.667883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.546963+0000","last_change":"2024-05-09T04:49:34.153592+0000","last_active":"2024-05-09T04:52:55.546963+0000","last_peered":"2024-05-09T04:52:55.546963+0000","last_clean":"2024-05-09T04:52:55.546963+0000","last_became_active":"2024-05-09T04:49:34.153184+0000","last_became_peered":"2024-05-09T04:49:34.153184+0000","last_unstale":"2024-05-09T04:52:55.546963+0000","last_undegraded":"2024-05-09T04:52:55.546963+0000","last_fullsized":"2024-05-09T04:52:55.546963+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:36:56.784781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.553328+0000","last_change":"2024-05-09T04:49:34.144046+0000","last_active":"2024-05-09T04:52:55.553328+0000","last_peered":"2024-05-09T04:52:55.553328+0000","last_clean":"2024-05-09T04:52:55.553328+0000","last_became_active":"2024-05-09T04:49:34.141004+0000","last_became_peered":"2024-05-09T04:49:34.141004+0000","last_unstale":"2024-05-09T04:52:55.553328+0000","last_undegraded":"2024-05-09T04:52:55.553328+0000","last_fullsized":"2024-05-09T04:52:55.553328+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:09:12.274750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.554621+0000","last_change":"2024-05-09T04:49:34.147063+0000","last_active":"2024-05-09T04:52:55.554621+0000","last_peered":"2024-05-09T04:52:55.554621+0000","last_clean":"2024-05-09T04:52:55.554621+0000","last_became_active":"2024-05-09T04:49:34.144934+0000","last_became_peered":"2024-05-09T04:49:34.144934+0000","last_unstale":"2024-05-09T04:52:55.554621+0000","last_undegraded":"2024-05-09T04:52:55.554621+0000","last_fullsized":"2024-05-09T04:52:55.554621+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:47:59.125451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.540971+0000","last_change":"2024-05-09T04:49:34.141070+0000","last_active":"2024-05-09T04:52:55.540971+0000","last_peered":"2024-05-09T04:52:55.540971+0000","last_clean":"2024-05-09T04:52:55.540971+0000","last_became_active":"2024-05-09T04:49:34.140618+0000","last_became_peered":"2024-05-09T04:49:34.140618+0000","last_unstale":"2024-05-09T04:52:55.540971+0000","last_undegraded":"2024-05-09T04:52:55.540971+0000","last_fullsized":"2024-05-09T04:52:55.540971+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:02.435768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.559709+0000","last_change":"2024-05-09T04:49:34.144434+0000","last_active":"2024-05-09T04:52:55.559709+0000","last_peered":"2024-05-09T04:52:55.559709+0000","last_clean":"2024-05-09T04:52:55.559709+0000","last_became_active":"2024-05-09T04:49:34.144052+0000","last_became_peered":"2024-05-09T04:49:34.144052+0000","last_unstale":"2024-05-09T04:52:55.559709+0000","last_undegraded":"2024-05-09T04:52:55.559709+0000","last_fullsized":"2024-05-09T04:52:55.559709+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:05:26.141627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.548013+0000","last_change":"2024-05-09T04:49:34.148681+0000","last_active":"2024-05-09T04:52:55.548013+0000","last_peered":"2024-05-09T04:52:55.548013+0000","last_clean":"2024-05-09T04:52:55.548013+0000","last_became_active":"2024-05-09T04:49:34.145277+0000","last_became_peered":"2024-05-09T04:49:34.145277+0000","last_unstale":"2024-05-09T04:52:55.548013+0000","last_undegraded":"2024-05-09T04:52:55.548013+0000","last_fullsized":"2024-05-09T04:52:55.548013+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:09.546998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.548620+0000","last_change":"2024-05-09T04:49:34.153229+0000","last_active":"2024-05-09T04:52:55.548620+0000","last_peered":"2024-05-09T04:52:55.548620+0000","last_clean":"2024-05-09T04:52:55.548620+0000","last_became_active":"2024-05-09T04:49:34.152869+0000","last_became_peered":"2024-05-09T04:49:34.152869+0000","last_unstale":"2024-05-09T04:52:55.548620+0000","last_undegraded":"2024-05-09T04:52:55.548620+0000","last_fullsized":"2024-05-09T04:52:55.548620+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:54.350842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:12.136825+0000","last_change":"2024-05-09T04:49:56.710411+0000","last_active":"2024-05-09T04:52:12.136825+0000","last_peered":"2024-05-09T04:52:12.136825+0000","last_clean":"2024-05-09T04:52:12.136825+0000","last_became_active":"2024-05-09T04:49:56.709903+0000","last_became_peered":"2024-05-09T04:49:56.709903+0000","last_unstale":"2024-05-09T04:52:12.136825+0000","last_undegraded":"2024-05-09T04:52:12.136825+0000","last_fullsized":"2024-05-09T04:52:12.136825+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:53:44.216648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,10,0],"acting":[2,3,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":296,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:55:01.335513+0000","last_change":"2024-05-09T04:49:30.061766+0000","last_active":"2024-05-09T04:55:01.335513+0000","last_peered":"2024-05-09T04:55:01.335513+0000","last_clean":"2024-05-09T04:55:01.335513+0000","last_became_active":"2024-05-09T04:49:30.060545+0000","last_became_peered":"2024-05-09T04:49:30.060545+0000","last_unstale":"2024-05-09T04:55:01.335513+0000","last_undegraded":"2024-05-09T04:55:01.335513+0000","last_fullsized":"2024-05-09T04:55:01.335513+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:15:03.857107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":240,"num_read_kb":240,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.555879+0000","last_change":"2024-05-09T04:49:34.140706+0000","last_active":"2024-05-09T04:52:55.555879+0000","last_peered":"2024-05-09T04:52:55.555879+0000","last_clean":"2024-05-09T04:52:55.555879+0000","last_became_active":"2024-05-09T04:49:34.139032+0000","last_became_peered":"2024-05-09T04:49:34.139032+0000","last_unstale":"2024-05-09T04:52:55.555879+0000","last_undegraded":"2024-05-09T04:52:55.555879+0000","last_fullsized":"2024-05-09T04:52:55.555879+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:19.878523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:16.237850+0000","last_change":"2024-05-09T04:49:47.674254+0000","last_active":"2024-05-09T04:52:16.237850+0000","last_peered":"2024-05-09T04:52:16.237850+0000","last_clean":"2024-05-09T04:52:16.237850+0000","last_became_active":"2024-05-09T04:49:47.673476+0000","last_became_peered":"2024-05-09T04:49:47.673476+0000","last_unstale":"2024-05-09T04:52:16.237850+0000","last_undegraded":"2024-05-09T04:52:16.237850+0000","last_fullsized":"2024-05-09T04:52:16.237850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:41:31.709210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:33.128358+0000","last_change":"2024-05-09T04:49:30.154162+0000","last_active":"2024-05-09T04:51:33.128358+0000","last_peered":"2024-05-09T04:51:33.128358+0000","last_clean":"2024-05-09T04:51:33.128358+0000","last_became_active":"2024-05-09T04:49:30.152869+0000","last_became_peered":"2024-05-09T04:49:30.152869+0000","last_unstale":"2024-05-09T04:51:33.128358+0000","last_undegraded":"2024-05-09T04:51:33.128358+0000","last_fullsized":"2024-05-09T04:51:33.128358+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:43.433033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.563473+0000","last_change":"2024-05-09T04:49:34.157729+0000","last_active":"2024-05-09T04:52:55.563473+0000","last_peered":"2024-05-09T04:52:55.563473+0000","last_clean":"2024-05-09T04:52:55.563473+0000","last_became_active":"2024-05-09T04:49:34.155040+0000","last_became_peered":"2024-05-09T04:49:34.155040+0000","last_unstale":"2024-05-09T04:52:55.563473+0000","last_undegraded":"2024-05-09T04:52:55.563473+0000","last_fullsized":"2024-05-09T04:52:55.563473+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:43:30.104394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:21.238400+0000","last_change":"2024-05-09T04:49:47.679665+0000","last_active":"2024-05-09T04:52:21.238400+0000","last_peered":"2024-05-09T04:52:21.238400+0000","last_clean":"2024-05-09T04:52:21.238400+0000","last_became_active":"2024-05-09T04:49:47.679257+0000","last_became_peered":"2024-05-09T04:49:47.679257+0000","last_unstale":"2024-05-09T04:52:21.238400+0000","last_undegraded":"2024-05-09T04:52:21.238400+0000","last_fullsized":"2024-05-09T04:52:21.238400+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:53.111205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":57,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:00.713890+0000","last_change":"2024-05-09T04:49:30.047386+0000","last_active":"2024-05-09T04:52:00.713890+0000","last_peered":"2024-05-09T04:52:00.713890+0000","last_clean":"2024-05-09T04:52:00.713890+0000","last_became_active":"2024-05-09T04:49:30.046947+0000","last_became_peered":"2024-05-09T04:49:30.046947+0000","last_unstale":"2024-05-09T04:52:00.713890+0000","last_undegraded":"2024-05-09T04:52:00.713890+0000","last_fullsized":"2024-05-09T04:52:00.713890+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:46.861354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.548062+0000","last_change":"2024-05-09T04:49:34.153320+0000","last_active":"2024-05-09T04:52:55.548062+0000","last_peered":"2024-05-09T04:52:55.548062+0000","last_clean":"2024-05-09T04:52:55.548062+0000","last_became_active":"2024-05-09T04:49:34.152854+0000","last_became_peered":"2024-05-09T04:49:34.152854+0000","last_unstale":"2024-05-09T04:52:55.548062+0000","last_undegraded":"2024-05-09T04:52:55.548062+0000","last_fullsized":"2024-05-09T04:52:55.548062+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:40.801340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:21.238377+0000","last_change":"2024-05-09T04:49:47.668475+0000","last_active":"2024-05-09T04:52:21.238377+0000","last_peered":"2024-05-09T04:52:21.238377+0000","last_clean":"2024-05-09T04:52:21.238377+0000","last_became_active":"2024-05-09T04:49:47.667909+0000","last_became_peered":"2024-05-09T04:49:47.667909+0000","last_unstale":"2024-05-09T04:52:21.238377+0000","last_undegraded":"2024-05-09T04:52:21.238377+0000","last_fullsized":"2024-05-09T04:52:21.238377+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:39:27.687468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:00.713826+0000","last_change":"2024-05-09T04:49:30.048659+0000","last_active":"2024-05-09T04:52:00.713826+0000","last_peered":"2024-05-09T04:52:00.713826+0000","last_clean":"2024-05-09T04:52:00.713826+0000","last_became_active":"2024-05-09T04:49:30.048259+0000","last_became_peered":"2024-05-09T04:49:30.048259+0000","last_unstale":"2024-05-09T04:52:00.713826+0000","last_undegraded":"2024-05-09T04:52:00.713826+0000","last_fullsized":"2024-05-09T04:52:00.713826+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:37:46.012594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.539385+0000","last_change":"2024-05-09T04:49:34.131416+0000","last_active":"2024-05-09T04:52:55.539385+0000","last_peered":"2024-05-09T04:52:55.539385+0000","last_clean":"2024-05-09T04:52:55.539385+0000","last_became_active":"2024-05-09T04:49:34.130985+0000","last_became_peered":"2024-05-09T04:49:34.130985+0000","last_unstale":"2024-05-09T04:52:55.539385+0000","last_undegraded":"2024-05-09T04:52:55.539385+0000","last_fullsized":"2024-05-09T04:52:55.539385+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:02:43.228142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:16.222557+0000","last_change":"2024-05-09T04:49:47.652247+0000","last_active":"2024-05-09T04:52:16.222557+0000","last_peered":"2024-05-09T04:52:16.222557+0000","last_clean":"2024-05-09T04:52:16.222557+0000","last_became_active":"2024-05-09T04:49:47.651747+0000","last_became_peered":"2024-05-09T04:49:47.651747+0000","last_unstale":"2024-05-09T04:52:16.222557+0000","last_undegraded":"2024-05-09T04:52:16.222557+0000","last_fullsized":"2024-05-09T04:52:16.222557+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:45.353027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'8","reported_seq":294,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:54:59.141877+0000","last_change":"2024-05-09T04:49:30.047421+0000","last_active":"2024-05-09T04:54:59.141877+0000","last_peered":"2024-05-09T04:54:59.141877+0000","last_clean":"2024-05-09T04:54:59.141877+0000","last_became_active":"2024-05-09T04:49:30.047007+0000","last_became_peered":"2024-05-09T04:49:30.047007+0000","last_unstale":"2024-05-09T04:54:59.141877+0000","last_undegraded":"2024-05-09T04:54:59.141877+0000","last_fullsized":"2024-05-09T04:54:59.141877+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:23.930326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":242,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.533509+0000","last_change":"2024-05-09T04:49:34.146605+0000","last_active":"2024-05-09T04:52:55.533509+0000","last_peered":"2024-05-09T04:52:55.533509+0000","last_clean":"2024-05-09T04:52:55.533509+0000","last_became_active":"2024-05-09T04:49:34.146205+0000","last_became_peered":"2024-05-09T04:49:34.146205+0000","last_unstale":"2024-05-09T04:52:55.533509+0000","last_undegraded":"2024-05-09T04:52:55.533509+0000","last_fullsized":"2024-05-09T04:52:55.533509+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:13:52.234951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:16.222527+0000","last_change":"2024-05-09T04:49:47.673652+0000","last_active":"2024-05-09T04:52:16.222527+0000","last_peered":"2024-05-09T04:52:16.222527+0000","last_clean":"2024-05-09T04:52:16.222527+0000","last_became_active":"2024-05-09T04:49:47.673255+0000","last_became_peered":"2024-05-09T04:49:47.673255+0000","last_unstale":"2024-05-09T04:52:16.222527+0000","last_undegraded":"2024-05-09T04:52:16.222527+0000","last_fullsized":"2024-05-09T04:52:16.222527+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:12:50.672212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:00.294775+0000","last_change":"2024-05-09T04:49:30.056089+0000","last_active":"2024-05-09T04:52:00.294775+0000","last_peered":"2024-05-09T04:52:00.294775+0000","last_clean":"2024-05-09T04:52:00.294775+0000","last_became_active":"2024-05-09T04:49:30.055635+0000","last_became_peered":"2024-05-09T04:49:30.055635+0000","last_unstale":"2024-05-09T04:52:00.294775+0000","last_undegraded":"2024-05-09T04:52:00.294775+0000","last_fullsized":"2024-05-09T04:52:00.294775+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:35:06.529058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.562131+0000","last_change":"2024-05-09T04:49:34.147628+0000","last_active":"2024-05-09T04:52:55.562131+0000","last_peered":"2024-05-09T04:52:55.562131+0000","last_clean":"2024-05-09T04:52:55.562131+0000","last_became_active":"2024-05-09T04:49:34.147208+0000","last_became_peered":"2024-05-09T04:49:34.147208+0000","last_unstale":"2024-05-09T04:52:55.562131+0000","last_undegraded":"2024-05-09T04:52:55.562131+0000","last_fullsized":"2024-05-09T04:52:55.562131+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:09:22.520468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:51.957724+0000","last_change":"2024-05-09T04:49:56.758366+0000","last_active":"2024-05-09T04:51:51.957724+0000","last_peered":"2024-05-09T04:51:51.957724+0000","last_clean":"2024-05-09T04:51:51.957724+0000","last_became_active":"2024-05-09T04:49:56.757763+0000","last_became_peered":"2024-05-09T04:49:56.757763+0000","last_unstale":"2024-05-09T04:51:51.957724+0000","last_undegraded":"2024-05-09T04:51:51.957724+0000","last_fullsized":"2024-05-09T04:51:51.957724+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:58.711764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:57.852993+0000","last_change":"2024-05-09T04:49:30.058805+0000","last_active":"2024-05-09T04:51:57.852993+0000","last_peered":"2024-05-09T04:51:57.852993+0000","last_clean":"2024-05-09T04:51:57.852993+0000","last_became_active":"2024-05-09T04:49:30.057278+0000","last_became_peered":"2024-05-09T04:49:30.057278+0000","last_unstale":"2024-05-09T04:51:57.852993+0000","last_undegraded":"2024-05-09T04:51:57.852993+0000","last_fullsized":"2024-05-09T04:51:57.852993+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:34.727781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.559587+0000","last_change":"2024-05-09T04:49:34.148753+0000","last_active":"2024-05-09T04:52:55.559587+0000","last_peered":"2024-05-09T04:52:55.559587+0000","last_clean":"2024-05-09T04:52:55.559587+0000","last_became_active":"2024-05-09T04:49:34.148407+0000","last_became_peered":"2024-05-09T04:49:34.148407+0000","last_unstale":"2024-05-09T04:52:55.559587+0000","last_undegraded":"2024-05-09T04:52:55.559587+0000","last_fullsized":"2024-05-09T04:52:55.559587+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:39.602934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:46.807773+0000","last_change":"2024-05-09T04:49:47.672521+0000","last_active":"2024-05-09T04:51:46.807773+0000","last_peered":"2024-05-09T04:51:46.807773+0000","last_clean":"2024-05-09T04:51:46.807773+0000","last_became_active":"2024-05-09T04:49:47.671353+0000","last_became_peered":"2024-05-09T04:49:47.671353+0000","last_unstale":"2024-05-09T04:51:46.807773+0000","last_undegraded":"2024-05-09T04:51:46.807773+0000","last_fullsized":"2024-05-09T04:51:46.807773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:56:34.832651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:55.722708+0000","last_change":"2024-05-09T04:49:30.048589+0000","last_active":"2024-05-09T04:51:55.722708+0000","last_peered":"2024-05-09T04:51:55.722708+0000","last_clean":"2024-05-09T04:51:55.722708+0000","last_became_active":"2024-05-09T04:49:30.048241+0000","last_became_peered":"2024-05-09T04:49:30.048241+0000","last_unstale":"2024-05-09T04:51:55.722708+0000","last_undegraded":"2024-05-09T04:51:55.722708+0000","last_fullsized":"2024-05-09T04:51:55.722708+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:38:36.204539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.562269+0000","last_change":"2024-05-09T04:49:34.151201+0000","last_active":"2024-05-09T04:52:55.562269+0000","last_peered":"2024-05-09T04:52:55.562269+0000","last_clean":"2024-05-09T04:52:55.562269+0000","last_became_active":"2024-05-09T04:49:34.150505+0000","last_became_peered":"2024-05-09T04:49:34.150505+0000","last_unstale":"2024-05-09T04:52:55.562269+0000","last_undegraded":"2024-05-09T04:52:55.562269+0000","last_fullsized":"2024-05-09T04:52:55.562269+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:40.011516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:08.135251+0000","last_change":"2024-05-09T04:49:47.653827+0000","last_active":"2024-05-09T04:52:08.135251+0000","last_peered":"2024-05-09T04:52:08.135251+0000","last_clean":"2024-05-09T04:52:08.135251+0000","last_became_active":"2024-05-09T04:49:47.652423+0000","last_became_peered":"2024-05-09T04:49:47.652423+0000","last_unstale":"2024-05-09T04:52:08.135251+0000","last_undegraded":"2024-05-09T04:52:08.135251+0000","last_fullsized":"2024-05-09T04:52:08.135251+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:07:07.602659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'7","reported_seq":525,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:55:00.142553+0000","last_change":"2024-05-09T04:49:30.062849+0000","last_active":"2024-05-09T04:55:00.142553+0000","last_peered":"2024-05-09T04:55:00.142553+0000","last_clean":"2024-05-09T04:55:00.142553+0000","last_became_active":"2024-05-09T04:49:30.062426+0000","last_became_peered":"2024-05-09T04:49:30.062426+0000","last_unstale":"2024-05-09T04:55:00.142553+0000","last_undegraded":"2024-05-09T04:55:00.142553+0000","last_fullsized":"2024-05-09T04:55:00.142553+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:25:11.996208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":474,"num_read_kb":474,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.544151+0000","last_change":"2024-05-09T04:49:34.151145+0000","last_active":"2024-05-09T04:52:55.544151+0000","last_peered":"2024-05-09T04:52:55.544151+0000","last_clean":"2024-05-09T04:52:55.544151+0000","last_became_active":"2024-05-09T04:49:34.150745+0000","last_became_peered":"2024-05-09T04:49:34.150745+0000","last_unstale":"2024-05-09T04:52:55.544151+0000","last_undegraded":"2024-05-09T04:52:55.544151+0000","last_fullsized":"2024-05-09T04:52:55.544151+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:40.647454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:07.844209+0000","last_change":"2024-05-09T04:49:47.672796+0000","last_active":"2024-05-09T04:52:07.844209+0000","last_peered":"2024-05-09T04:52:07.844209+0000","last_clean":"2024-05-09T04:52:07.844209+0000","last_became_active":"2024-05-09T04:49:47.667807+0000","last_became_peered":"2024-05-09T04:49:47.667807+0000","last_unstale":"2024-05-09T04:52:07.844209+0000","last_undegraded":"2024-05-09T04:52:07.844209+0000","last_fullsized":"2024-05-09T04:52:07.844209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:58.542973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:54.594836+0000","last_change":"2024-05-09T04:49:30.047265+0000","last_active":"2024-05-09T04:51:54.594836+0000","last_peered":"2024-05-09T04:51:54.594836+0000","last_clean":"2024-05-09T04:51:54.594836+0000","last_became_active":"2024-05-09T04:49:30.046888+0000","last_became_peered":"2024-05-09T04:49:30.046888+0000","last_unstale":"2024-05-09T04:51:54.594836+0000","last_undegraded":"2024-05-09T04:51:54.594836+0000","last_fullsized":"2024-05-09T04:51:54.594836+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:38:41.878678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.554268+0000","last_change":"2024-05-09T04:49:34.144447+0000","last_active":"2024-05-09T04:52:55.554268+0000","last_peered":"2024-05-09T04:52:55.554268+0000","last_clean":"2024-05-09T04:52:55.554268+0000","last_became_active":"2024-05-09T04:49:34.144073+0000","last_became_peered":"2024-05-09T04:49:34.144073+0000","last_unstale":"2024-05-09T04:52:55.554268+0000","last_undegraded":"2024-05-09T04:52:55.554268+0000","last_fullsized":"2024-05-09T04:52:55.554268+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:53:42.619552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:11.237300+0000","last_change":"2024-05-09T04:49:47.673364+0000","last_active":"2024-05-09T04:52:11.237300+0000","last_peered":"2024-05-09T04:52:11.237300+0000","last_clean":"2024-05-09T04:52:11.237300+0000","last_became_active":"2024-05-09T04:49:47.672712+0000","last_became_peered":"2024-05-09T04:49:47.672712+0000","last_unstale":"2024-05-09T04:52:11.237300+0000","last_undegraded":"2024-05-09T04:52:11.237300+0000","last_fullsized":"2024-05-09T04:52:11.237300+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:25:57.165777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:55.722783+0000","last_change":"2024-05-09T04:49:30.160924+0000","last_active":"2024-05-09T04:51:55.722783+0000","last_peered":"2024-05-09T04:51:55.722783+0000","last_clean":"2024-05-09T04:51:55.722783+0000","last_became_active":"2024-05-09T04:49:30.160479+0000","last_became_peered":"2024-05-09T04:49:30.160479+0000","last_unstale":"2024-05-09T04:51:55.722783+0000","last_undegraded":"2024-05-09T04:51:55.722783+0000","last_fullsized":"2024-05-09T04:51:55.722783+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:44:57.171441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.545843+0000","last_change":"2024-05-09T04:49:34.136514+0000","last_active":"2024-05-09T04:52:55.545843+0000","last_peered":"2024-05-09T04:52:55.545843+0000","last_clean":"2024-05-09T04:52:55.545843+0000","last_became_active":"2024-05-09T04:49:34.136121+0000","last_became_peered":"2024-05-09T04:49:34.136121+0000","last_unstale":"2024-05-09T04:52:55.545843+0000","last_undegraded":"2024-05-09T04:52:55.545843+0000","last_fullsized":"2024-05-09T04:52:55.545843+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:43:27.781720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:07.844267+0000","last_change":"2024-05-09T04:49:47.673371+0000","last_active":"2024-05-09T04:52:07.844267+0000","last_peered":"2024-05-09T04:52:07.844267+0000","last_clean":"2024-05-09T04:52:07.844267+0000","last_became_active":"2024-05-09T04:49:47.669727+0000","last_became_peered":"2024-05-09T04:49:47.669727+0000","last_unstale":"2024-05-09T04:52:07.844267+0000","last_undegraded":"2024-05-09T04:52:07.844267+0000","last_fullsized":"2024-05-09T04:52:07.844267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:34:45.510934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'44","reported_seq":577,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:55:00.142596+0000","last_change":"2024-05-09T04:49:30.048359+0000","last_active":"2024-05-09T04:55:00.142596+0000","last_peered":"2024-05-09T04:55:00.142596+0000","last_clean":"2024-05-09T04:55:00.142596+0000","last_became_active":"2024-05-09T04:49:30.047949+0000","last_became_peered":"2024-05-09T04:49:30.047949+0000","last_unstale":"2024-05-09T04:55:00.142596+0000","last_undegraded":"2024-05-09T04:55:00.142596+0000","last_fullsized":"2024-05-09T04:55:00.142596+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:01:10.498997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70563,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":489,"num_read_kb":315,"num_write":46,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.521477+0000","last_change":"2024-05-09T04:49:34.132589+0000","last_active":"2024-05-09T04:52:55.521477+0000","last_peered":"2024-05-09T04:52:55.521477+0000","last_clean":"2024-05-09T04:52:55.521477+0000","last_became_active":"2024-05-09T04:49:34.132227+0000","last_became_peered":"2024-05-09T04:49:34.132227+0000","last_unstale":"2024-05-09T04:52:55.521477+0000","last_undegraded":"2024-05-09T04:52:55.521477+0000","last_fullsized":"2024-05-09T04:52:55.521477+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:50.003878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:15.725082+0000","last_change":"2024-05-09T04:49:47.668236+0000","last_active":"2024-05-09T04:52:15.725082+0000","last_peered":"2024-05-09T04:52:15.725082+0000","last_clean":"2024-05-09T04:52:15.725082+0000","last_became_active":"2024-05-09T04:49:47.667779+0000","last_became_peered":"2024-05-09T04:49:47.667779+0000","last_unstale":"2024-05-09T04:52:15.725082+0000","last_undegraded":"2024-05-09T04:52:15.725082+0000","last_fullsized":"2024-05-09T04:52:15.725082+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:16.177888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:52.134520+0000","last_change":"2024-05-09T04:49:30.046145+0000","last_active":"2024-05-09T04:51:52.134520+0000","last_peered":"2024-05-09T04:51:52.134520+0000","last_clean":"2024-05-09T04:51:52.134520+0000","last_became_active":"2024-05-09T04:49:30.045478+0000","last_became_peered":"2024-05-09T04:49:30.045478+0000","last_unstale":"2024-05-09T04:51:52.134520+0000","last_undegraded":"2024-05-09T04:51:52.134520+0000","last_fullsized":"2024-05-09T04:51:52.134520+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:52:08.313984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.550418+0000","last_change":"2024-05-09T04:49:34.154349+0000","last_active":"2024-05-09T04:52:55.550418+0000","last_peered":"2024-05-09T04:52:55.550418+0000","last_clean":"2024-05-09T04:52:55.550418+0000","last_became_active":"2024-05-09T04:49:34.154011+0000","last_became_peered":"2024-05-09T04:49:34.154011+0000","last_unstale":"2024-05-09T04:52:55.550418+0000","last_undegraded":"2024-05-09T04:52:55.550418+0000","last_fullsized":"2024-05-09T04:52:55.550418+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:50:32.345863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:06.959550+0000","last_change":"2024-05-09T04:49:47.670170+0000","last_active":"2024-05-09T04:52:06.959550+0000","last_peered":"2024-05-09T04:52:06.959550+0000","last_clean":"2024-05-09T04:52:06.959550+0000","last_became_active":"2024-05-09T04:49:47.669705+0000","last_became_peered":"2024-05-09T04:49:47.669705+0000","last_unstale":"2024-05-09T04:52:06.959550+0000","last_undegraded":"2024-05-09T04:52:06.959550+0000","last_fullsized":"2024-05-09T04:52:06.959550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:59:27.339811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:50.722218+0000","last_change":"2024-05-09T04:49:30.051159+0000","last_active":"2024-05-09T04:51:50.722218+0000","last_peered":"2024-05-09T04:51:50.722218+0000","last_clean":"2024-05-09T04:51:50.722218+0000","last_became_active":"2024-05-09T04:49:30.050108+0000","last_became_peered":"2024-05-09T04:49:30.050108+0000","last_unstale":"2024-05-09T04:51:50.722218+0000","last_undegraded":"2024-05-09T04:51:50.722218+0000","last_fullsized":"2024-05-09T04:51:50.722218+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:32:56.778567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.549234+0000","last_change":"2024-05-09T04:49:34.162751+0000","last_active":"2024-05-09T04:52:55.549234+0000","last_peered":"2024-05-09T04:52:55.549234+0000","last_clean":"2024-05-09T04:52:55.549234+0000","last_became_active":"2024-05-09T04:49:34.162336+0000","last_became_peered":"2024-05-09T04:49:34.162336+0000","last_unstale":"2024-05-09T04:52:55.549234+0000","last_undegraded":"2024-05-09T04:52:55.549234+0000","last_fullsized":"2024-05-09T04:52:55.549234+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:41.758074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:12.845173+0000","last_change":"2024-05-09T04:49:47.680955+0000","last_active":"2024-05-09T04:52:12.845173+0000","last_peered":"2024-05-09T04:52:12.845173+0000","last_clean":"2024-05-09T04:52:12.845173+0000","last_became_active":"2024-05-09T04:49:47.680041+0000","last_became_peered":"2024-05-09T04:49:47.680041+0000","last_unstale":"2024-05-09T04:52:12.845173+0000","last_undegraded":"2024-05-09T04:52:12.845173+0000","last_fullsized":"2024-05-09T04:52:12.845173+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:46.297611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'7","reported_seq":291,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:54:59.371562+0000","last_change":"2024-05-09T04:49:30.045857+0000","last_active":"2024-05-09T04:54:59.371562+0000","last_peered":"2024-05-09T04:54:59.371562+0000","last_clean":"2024-05-09T04:54:59.371562+0000","last_became_active":"2024-05-09T04:49:30.045404+0000","last_became_peered":"2024-05-09T04:49:30.045404+0000","last_unstale":"2024-05-09T04:54:59.371562+0000","last_undegraded":"2024-05-09T04:54:59.371562+0000","last_fullsized":"2024-05-09T04:54:59.371562+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:44:19.473449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":242,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.539356+0000","last_change":"2024-05-09T04:49:34.131734+0000","last_active":"2024-05-09T04:52:55.539356+0000","last_peered":"2024-05-09T04:52:55.539356+0000","last_clean":"2024-05-09T04:52:55.539356+0000","last_became_active":"2024-05-09T04:49:34.131396+0000","last_became_peered":"2024-05-09T04:49:34.131396+0000","last_unstale":"2024-05-09T04:52:55.539356+0000","last_undegraded":"2024-05-09T04:52:55.539356+0000","last_fullsized":"2024-05-09T04:52:55.539356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:21.080651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:12.854733+0000","last_change":"2024-05-09T04:49:56.710429+0000","last_active":"2024-05-09T04:52:12.854733+0000","last_peered":"2024-05-09T04:52:12.854733+0000","last_clean":"2024-05-09T04:52:12.854733+0000","last_became_active":"2024-05-09T04:49:56.708497+0000","last_became_peered":"2024-05-09T04:49:56.708497+0000","last_unstale":"2024-05-09T04:52:12.854733+0000","last_undegraded":"2024-05-09T04:52:12.854733+0000","last_fullsized":"2024-05-09T04:52:12.854733+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:36.281300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:16.960685+0000","last_change":"2024-05-09T04:49:56.718731+0000","last_active":"2024-05-09T04:52:16.960685+0000","last_peered":"2024-05-09T04:52:16.960685+0000","last_clean":"2024-05-09T04:52:16.960685+0000","last_became_active":"2024-05-09T04:49:56.709958+0000","last_became_peered":"2024-05-09T04:49:56.709958+0000","last_unstale":"2024-05-09T04:52:16.960685+0000","last_undegraded":"2024-05-09T04:52:16.960685+0000","last_fullsized":"2024-05-09T04:52:16.960685+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:11.003906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.539006+0000","last_change":"2024-05-09T04:49:34.131705+0000","last_active":"2024-05-09T04:52:55.539006+0000","last_peered":"2024-05-09T04:52:55.539006+0000","last_clean":"2024-05-09T04:52:55.539006+0000","last_became_active":"2024-05-09T04:49:34.131287+0000","last_became_peered":"2024-05-09T04:49:34.131287+0000","last_unstale":"2024-05-09T04:52:55.539006+0000","last_undegraded":"2024-05-09T04:52:55.539006+0000","last_fullsized":"2024-05-09T04:52:55.539006+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:10:24.914832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:00.723219+0000","last_change":"2024-05-09T04:49:47.663861+0000","last_active":"2024-05-09T04:52:00.723219+0000","last_peered":"2024-05-09T04:52:00.723219+0000","last_clean":"2024-05-09T04:52:00.723219+0000","last_became_active":"2024-05-09T04:49:47.663374+0000","last_became_peered":"2024-05-09T04:49:47.663374+0000","last_unstale":"2024-05-09T04:52:00.723219+0000","last_undegraded":"2024-05-09T04:52:00.723219+0000","last_fullsized":"2024-05-09T04:52:00.723219+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:08.063040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:45.712130+0000","last_change":"2024-05-09T04:49:30.044183+0000","last_active":"2024-05-09T04:51:45.712130+0000","last_peered":"2024-05-09T04:51:45.712130+0000","last_clean":"2024-05-09T04:51:45.712130+0000","last_became_active":"2024-05-09T04:49:30.043608+0000","last_became_peered":"2024-05-09T04:49:30.043608+0000","last_unstale":"2024-05-09T04:51:45.712130+0000","last_undegraded":"2024-05-09T04:51:45.712130+0000","last_fullsized":"2024-05-09T04:51:45.712130+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:48:57.788970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.528008+0000","last_change":"2024-05-09T04:49:34.133941+0000","last_active":"2024-05-09T04:52:55.528008+0000","last_peered":"2024-05-09T04:52:55.528008+0000","last_clean":"2024-05-09T04:52:55.528008+0000","last_became_active":"2024-05-09T04:49:34.133550+0000","last_became_peered":"2024-05-09T04:49:34.133550+0000","last_unstale":"2024-05-09T04:52:55.528008+0000","last_undegraded":"2024-05-09T04:52:55.528008+0000","last_fullsized":"2024-05-09T04:52:55.528008+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:45:14.703565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:55.713155+0000","last_change":"2024-05-09T04:49:47.901085+0000","last_active":"2024-05-09T04:51:55.713155+0000","last_peered":"2024-05-09T04:51:55.713155+0000","last_clean":"2024-05-09T04:51:55.713155+0000","last_became_active":"2024-05-09T04:49:47.900444+0000","last_became_peered":"2024-05-09T04:49:47.900444+0000","last_unstale":"2024-05-09T04:51:55.713155+0000","last_undegraded":"2024-05-09T04:51:55.713155+0000","last_fullsized":"2024-05-09T04:51:55.713155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:44.195683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:37.839364+0000","last_change":"2024-05-09T04:49:30.154060+0000","last_active":"2024-05-09T04:51:37.839364+0000","last_peered":"2024-05-09T04:51:37.839364+0000","last_clean":"2024-05-09T04:51:37.839364+0000","last_became_active":"2024-05-09T04:49:30.152556+0000","last_became_peered":"2024-05-09T04:49:30.152556+0000","last_unstale":"2024-05-09T04:51:37.839364+0000","last_undegraded":"2024-05-09T04:51:37.839364+0000","last_fullsized":"2024-05-09T04:51:37.839364+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:27.056251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.531546+0000","last_change":"2024-05-09T04:49:34.153128+0000","last_active":"2024-05-09T04:52:55.531546+0000","last_peered":"2024-05-09T04:52:55.531546+0000","last_clean":"2024-05-09T04:52:55.531546+0000","last_became_active":"2024-05-09T04:49:34.152734+0000","last_became_peered":"2024-05-09T04:49:34.152734+0000","last_unstale":"2024-05-09T04:52:55.531546+0000","last_undegraded":"2024-05-09T04:52:55.531546+0000","last_fullsized":"2024-05-09T04:52:55.531546+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:54:06.920877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:52.852082+0000","last_change":"2024-05-09T04:49:47.670263+0000","last_active":"2024-05-09T04:51:52.852082+0000","last_peered":"2024-05-09T04:51:52.852082+0000","last_clean":"2024-05-09T04:51:52.852082+0000","last_became_active":"2024-05-09T04:49:47.665743+0000","last_became_peered":"2024-05-09T04:49:47.665743+0000","last_unstale":"2024-05-09T04:51:52.852082+0000","last_undegraded":"2024-05-09T04:51:52.852082+0000","last_fullsized":"2024-05-09T04:51:52.852082+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:27.980226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":286,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:54:58.323116+0000","last_change":"2024-05-09T04:49:30.056119+0000","last_active":"2024-05-09T04:54:58.323116+0000","last_peered":"2024-05-09T04:54:58.323116+0000","last_clean":"2024-05-09T04:54:58.323116+0000","last_became_active":"2024-05-09T04:49:30.055660+0000","last_became_peered":"2024-05-09T04:49:30.055660+0000","last_unstale":"2024-05-09T04:54:58.323116+0000","last_undegraded":"2024-05-09T04:54:58.323116+0000","last_fullsized":"2024-05-09T04:54:58.323116+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:21.513879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.524834+0000","last_change":"2024-05-09T04:49:56.764303+0000","last_active":"2024-05-09T04:52:55.524834+0000","last_peered":"2024-05-09T04:52:55.524834+0000","last_clean":"2024-05-09T04:52:55.524834+0000","last_became_active":"2024-05-09T04:49:56.763854+0000","last_became_peered":"2024-05-09T04:49:56.763854+0000","last_unstale":"2024-05-09T04:52:55.524834+0000","last_undegraded":"2024-05-09T04:52:55.524834+0000","last_fullsized":"2024-05-09T04:52:55.524834+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:06.086602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:57.853087+0000","last_change":"2024-05-09T04:49:47.900866+0000","last_active":"2024-05-09T04:51:57.853087+0000","last_peered":"2024-05-09T04:51:57.853087+0000","last_clean":"2024-05-09T04:51:57.853087+0000","last_became_active":"2024-05-09T04:49:47.900332+0000","last_became_peered":"2024-05-09T04:49:47.900332+0000","last_unstale":"2024-05-09T04:51:57.853087+0000","last_undegraded":"2024-05-09T04:51:57.853087+0000","last_fullsized":"2024-05-09T04:51:57.853087+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:06:41.448262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.527180+0000","last_change":"2024-05-09T04:49:56.702559+0000","last_active":"2024-05-09T04:52:55.527180+0000","last_peered":"2024-05-09T04:52:55.527180+0000","last_clean":"2024-05-09T04:52:55.527180+0000","last_became_active":"2024-05-09T04:49:56.702135+0000","last_became_peered":"2024-05-09T04:49:56.702135+0000","last_unstale":"2024-05-09T04:52:55.527180+0000","last_undegraded":"2024-05-09T04:52:55.527180+0000","last_fullsized":"2024-05-09T04:52:55.527180+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:43:05.848239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":290,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:55:01.376068+0000","last_change":"2024-05-09T04:49:30.158050+0000","last_active":"2024-05-09T04:55:01.376068+0000","last_peered":"2024-05-09T04:55:01.376068+0000","last_clean":"2024-05-09T04:55:01.376068+0000","last_became_active":"2024-05-09T04:49:30.156719+0000","last_became_peered":"2024-05-09T04:49:30.156719+0000","last_unstale":"2024-05-09T04:55:01.376068+0000","last_undegraded":"2024-05-09T04:55:01.376068+0000","last_fullsized":"2024-05-09T04:55:01.376068+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:47.963111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":240,"num_read_kb":240,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:54.594819+0000","last_change":"2024-05-09T04:49:47.673594+0000","last_active":"2024-05-09T04:51:54.594819+0000","last_peered":"2024-05-09T04:51:54.594819+0000","last_clean":"2024-05-09T04:51:54.594819+0000","last_became_active":"2024-05-09T04:49:47.672671+0000","last_became_peered":"2024-05-09T04:49:47.672671+0000","last_unstale":"2024-05-09T04:51:54.594819+0000","last_undegraded":"2024-05-09T04:51:54.594819+0000","last_fullsized":"2024-05-09T04:51:54.594819+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:58:56.078180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.547961+0000","last_change":"2024-05-09T04:50:58.035054+0000","last_active":"2024-05-09T04:52:55.547961+0000","last_peered":"2024-05-09T04:52:55.547961+0000","last_clean":"2024-05-09T04:52:55.547961+0000","last_became_active":"2024-05-09T04:50:58.034297+0000","last_became_peered":"2024-05-09T04:50:58.034297+0000","last_unstale":"2024-05-09T04:52:55.547961+0000","last_undegraded":"2024-05-09T04:52:55.547961+0000","last_fullsized":"2024-05-09T04:52:55.547961+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:10:56.827950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:36.232917+0000","last_change":"2024-05-09T04:49:30.060224+0000","last_active":"2024-05-09T04:51:36.232917+0000","last_peered":"2024-05-09T04:51:36.232917+0000","last_clean":"2024-05-09T04:51:36.232917+0000","last_became_active":"2024-05-09T04:49:30.059830+0000","last_became_peered":"2024-05-09T04:49:30.059830+0000","last_unstale":"2024-05-09T04:51:36.232917+0000","last_undegraded":"2024-05-09T04:51:36.232917+0000","last_fullsized":"2024-05-09T04:51:36.232917+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:02.925226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":118,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:24.598674+0000","last_change":"2024-05-09T04:47:53.050380+0000","last_active":"2024-05-09T04:52:24.598674+0000","last_peered":"2024-05-09T04:52:24.598674+0000","last_clean":"2024-05-09T04:52:24.598674+0000","last_became_active":"2024-05-09T04:47:53.044058+0000","last_became_peered":"2024-05-09T04:47:53.044058+0000","last_unstale":"2024-05-09T04:52:24.598674+0000","last_undegraded":"2024-05-09T04:52:24.598674+0000","last_fullsized":"2024-05-09T04:52:24.598674+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:43:48.067159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:43:48.067159+0000","last_clean_scrub_stamp":"2024-05-09T04:43:48.067159+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:17:18.714607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:00.723281+0000","last_change":"2024-05-09T04:49:47.901114+0000","last_active":"2024-05-09T04:52:00.723281+0000","last_peered":"2024-05-09T04:52:00.723281+0000","last_clean":"2024-05-09T04:52:00.723281+0000","last_became_active":"2024-05-09T04:49:47.900450+0000","last_became_peered":"2024-05-09T04:49:47.900450+0000","last_unstale":"2024-05-09T04:52:00.723281+0000","last_undegraded":"2024-05-09T04:52:00.723281+0000","last_fullsized":"2024-05-09T04:52:00.723281+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:25:44.517789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.538748+0000","last_change":"2024-05-09T04:49:34.147027+0000","last_active":"2024-05-09T04:52:55.538748+0000","last_peered":"2024-05-09T04:52:55.538748+0000","last_clean":"2024-05-09T04:52:55.538748+0000","last_became_active":"2024-05-09T04:49:34.146672+0000","last_became_peered":"2024-05-09T04:49:34.146672+0000","last_unstale":"2024-05-09T04:52:55.538748+0000","last_undegraded":"2024-05-09T04:52:55.538748+0000","last_fullsized":"2024-05-09T04:52:55.538748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:15:44.847304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:29.591646+0000","last_change":"2024-05-09T04:49:30.043877+0000","last_active":"2024-05-09T04:51:29.591646+0000","last_peered":"2024-05-09T04:51:29.591646+0000","last_clean":"2024-05-09T04:51:29.591646+0000","last_became_active":"2024-05-09T04:49:30.043513+0000","last_became_peered":"2024-05-09T04:49:30.043513+0000","last_unstale":"2024-05-09T04:51:29.591646+0000","last_undegraded":"2024-05-09T04:51:29.591646+0000","last_fullsized":"2024-05-09T04:51:29.591646+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:16.501692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:51.957672+0000","last_change":"2024-05-09T04:49:47.668605+0000","last_active":"2024-05-09T04:51:51.957672+0000","last_peered":"2024-05-09T04:51:51.957672+0000","last_clean":"2024-05-09T04:51:51.957672+0000","last_became_active":"2024-05-09T04:49:47.668095+0000","last_became_peered":"2024-05-09T04:49:47.668095+0000","last_unstale":"2024-05-09T04:51:51.957672+0000","last_undegraded":"2024-05-09T04:51:51.957672+0000","last_fullsized":"2024-05-09T04:51:51.957672+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:39:30.508618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:35.710826+0000","last_change":"2024-05-09T04:49:30.159554+0000","last_active":"2024-05-09T04:51:35.710826+0000","last_peered":"2024-05-09T04:51:35.710826+0000","last_clean":"2024-05-09T04:51:35.710826+0000","last_became_active":"2024-05-09T04:49:30.159266+0000","last_became_peered":"2024-05-09T04:49:30.159266+0000","last_unstale":"2024-05-09T04:51:35.710826+0000","last_undegraded":"2024-05-09T04:51:35.710826+0000","last_fullsized":"2024-05-09T04:51:35.710826+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:42:04.252862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.542103+0000","last_change":"2024-05-09T04:49:34.150688+0000","last_active":"2024-05-09T04:52:55.542103+0000","last_peered":"2024-05-09T04:52:55.542103+0000","last_clean":"2024-05-09T04:52:55.542103+0000","last_became_active":"2024-05-09T04:49:34.150319+0000","last_became_peered":"2024-05-09T04:49:34.150319+0000","last_unstale":"2024-05-09T04:52:55.542103+0000","last_undegraded":"2024-05-09T04:52:55.542103+0000","last_fullsized":"2024-05-09T04:52:55.542103+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:35:04.832761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:48.131063+0000","last_change":"2024-05-09T04:49:47.902323+0000","last_active":"2024-05-09T04:51:48.131063+0000","last_peered":"2024-05-09T04:51:48.131063+0000","last_clean":"2024-05-09T04:51:48.131063+0000","last_became_active":"2024-05-09T04:49:47.901389+0000","last_became_peered":"2024-05-09T04:49:47.901389+0000","last_unstale":"2024-05-09T04:51:48.131063+0000","last_undegraded":"2024-05-09T04:51:48.131063+0000","last_fullsized":"2024-05-09T04:51:48.131063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:32.615944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'18","reported_seq":61,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:54:55.206777+0000","last_change":"2024-05-09T04:49:30.061172+0000","last_active":"2024-05-09T04:54:55.206777+0000","last_peered":"2024-05-09T04:54:55.206777+0000","last_clean":"2024-05-09T04:54:55.206777+0000","last_became_active":"2024-05-09T04:49:30.060680+0000","last_became_peered":"2024-05-09T04:49:30.060680+0000","last_unstale":"2024-05-09T04:54:55.206777+0000","last_undegraded":"2024-05-09T04:54:55.206777+0000","last_fullsized":"2024-05-09T04:54:55.206777+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:45:44.649124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.551743+0000","last_change":"2024-05-09T04:49:34.130319+0000","last_active":"2024-05-09T04:52:55.551743+0000","last_peered":"2024-05-09T04:52:55.551743+0000","last_clean":"2024-05-09T04:52:55.551743+0000","last_became_active":"2024-05-09T04:49:34.129610+0000","last_became_peered":"2024-05-09T04:49:34.129610+0000","last_unstale":"2024-05-09T04:52:55.551743+0000","last_undegraded":"2024-05-09T04:52:55.551743+0000","last_fullsized":"2024-05-09T04:52:55.551743+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:45.505221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:52.134484+0000","last_change":"2024-05-09T04:49:47.647513+0000","last_active":"2024-05-09T04:51:52.134484+0000","last_peered":"2024-05-09T04:51:52.134484+0000","last_clean":"2024-05-09T04:51:52.134484+0000","last_became_active":"2024-05-09T04:49:47.647069+0000","last_became_peered":"2024-05-09T04:49:47.647069+0000","last_unstale":"2024-05-09T04:51:52.134484+0000","last_undegraded":"2024-05-09T04:51:52.134484+0000","last_fullsized":"2024-05-09T04:51:52.134484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:05:27.658919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:41.233581+0000","last_change":"2024-05-09T04:49:30.350850+0000","last_active":"2024-05-09T04:51:41.233581+0000","last_peered":"2024-05-09T04:51:41.233581+0000","last_clean":"2024-05-09T04:51:41.233581+0000","last_became_active":"2024-05-09T04:49:30.350431+0000","last_became_peered":"2024-05-09T04:49:30.350431+0000","last_unstale":"2024-05-09T04:51:41.233581+0000","last_undegraded":"2024-05-09T04:51:41.233581+0000","last_fullsized":"2024-05-09T04:51:41.233581+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:53:05.388338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.562514+0000","last_change":"2024-05-09T04:49:34.153630+0000","last_active":"2024-05-09T04:52:55.562514+0000","last_peered":"2024-05-09T04:52:55.562514+0000","last_clean":"2024-05-09T04:52:55.562514+0000","last_became_active":"2024-05-09T04:49:34.152045+0000","last_became_peered":"2024-05-09T04:49:34.152045+0000","last_unstale":"2024-05-09T04:52:55.562514+0000","last_undegraded":"2024-05-09T04:52:55.562514+0000","last_fullsized":"2024-05-09T04:52:55.562514+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:32:35.991731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:06.236624+0000","last_change":"2024-05-09T04:49:47.901256+0000","last_active":"2024-05-09T04:52:06.236624+0000","last_peered":"2024-05-09T04:52:06.236624+0000","last_clean":"2024-05-09T04:52:06.236624+0000","last_became_active":"2024-05-09T04:49:47.900413+0000","last_became_peered":"2024-05-09T04:49:47.900413+0000","last_unstale":"2024-05-09T04:52:06.236624+0000","last_undegraded":"2024-05-09T04:52:06.236624+0000","last_fullsized":"2024-05-09T04:52:06.236624+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:46:11.845220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:42.647230+0000","last_change":"2024-05-09T04:49:30.152273+0000","last_active":"2024-05-09T04:51:42.647230+0000","last_peered":"2024-05-09T04:51:42.647230+0000","last_clean":"2024-05-09T04:51:42.647230+0000","last_became_active":"2024-05-09T04:49:30.151973+0000","last_became_peered":"2024-05-09T04:49:30.151973+0000","last_unstale":"2024-05-09T04:51:42.647230+0000","last_undegraded":"2024-05-09T04:51:42.647230+0000","last_fullsized":"2024-05-09T04:51:42.647230+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:14:52.912588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.517045+0000","last_change":"2024-05-09T04:49:34.126829+0000","last_active":"2024-05-09T04:52:55.517045+0000","last_peered":"2024-05-09T04:52:55.517045+0000","last_clean":"2024-05-09T04:52:55.517045+0000","last_became_active":"2024-05-09T04:49:34.126430+0000","last_became_peered":"2024-05-09T04:49:34.126430+0000","last_unstale":"2024-05-09T04:52:55.517045+0000","last_undegraded":"2024-05-09T04:52:55.517045+0000","last_fullsized":"2024-05-09T04:52:55.517045+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:38.247148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:02.843444+0000","last_change":"2024-05-09T04:49:47.675292+0000","last_active":"2024-05-09T04:52:02.843444+0000","last_peered":"2024-05-09T04:52:02.843444+0000","last_clean":"2024-05-09T04:52:02.843444+0000","last_became_active":"2024-05-09T04:49:47.674802+0000","last_became_peered":"2024-05-09T04:49:47.674802+0000","last_unstale":"2024-05-09T04:52:02.843444+0000","last_undegraded":"2024-05-09T04:52:02.843444+0000","last_fullsized":"2024-05-09T04:52:02.843444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:47:21.988054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:43.130050+0000","last_change":"2024-05-09T04:49:30.057854+0000","last_active":"2024-05-09T04:51:43.130050+0000","last_peered":"2024-05-09T04:51:43.130050+0000","last_clean":"2024-05-09T04:51:43.130050+0000","last_became_active":"2024-05-09T04:49:30.056625+0000","last_became_peered":"2024-05-09T04:49:30.056625+0000","last_unstale":"2024-05-09T04:51:43.130050+0000","last_undegraded":"2024-05-09T04:51:43.130050+0000","last_fullsized":"2024-05-09T04:51:43.130050+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:51.272533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.559538+0000","last_change":"2024-05-09T04:49:34.144414+0000","last_active":"2024-05-09T04:52:55.559538+0000","last_peered":"2024-05-09T04:52:55.559538+0000","last_clean":"2024-05-09T04:52:55.559538+0000","last_became_active":"2024-05-09T04:49:34.144064+0000","last_became_peered":"2024-05-09T04:49:34.144064+0000","last_unstale":"2024-05-09T04:52:55.559538+0000","last_undegraded":"2024-05-09T04:52:55.559538+0000","last_fullsized":"2024-05-09T04:52:55.559538+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:44:40.991576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:06.236580+0000","last_change":"2024-05-09T04:49:47.672956+0000","last_active":"2024-05-09T04:52:06.236580+0000","last_peered":"2024-05-09T04:52:06.236580+0000","last_clean":"2024-05-09T04:52:06.236580+0000","last_became_active":"2024-05-09T04:49:47.672513+0000","last_became_peered":"2024-05-09T04:49:47.672513+0000","last_unstale":"2024-05-09T04:52:06.236580+0000","last_undegraded":"2024-05-09T04:52:06.236580+0000","last_fullsized":"2024-05-09T04:52:06.236580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:32.250039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'4","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:54:16.975631+0000","last_change":"2024-05-09T04:49:56.704058+0000","last_active":"2024-05-09T04:54:16.975631+0000","last_peered":"2024-05-09T04:54:16.975631+0000","last_clean":"2024-05-09T04:54:16.975631+0000","last_became_active":"2024-05-09T04:49:56.703589+0000","last_became_peered":"2024-05-09T04:49:56.703589+0000","last_unstale":"2024-05-09T04:54:16.975631+0000","last_undegraded":"2024-05-09T04:54:16.975631+0000","last_fullsized":"2024-05-09T04:54:16.975631+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:54.596200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,1],"acting":[11,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.555701+0000","last_change":"2024-05-09T04:49:34.159321+0000","last_active":"2024-05-09T04:52:55.555701+0000","last_peered":"2024-05-09T04:52:55.555701+0000","last_clean":"2024-05-09T04:52:55.555701+0000","last_became_active":"2024-05-09T04:49:34.157918+0000","last_became_peered":"2024-05-09T04:49:34.157918+0000","last_unstale":"2024-05-09T04:52:55.555701+0000","last_undegraded":"2024-05-09T04:52:55.555701+0000","last_fullsized":"2024-05-09T04:52:55.555701+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:33:59.659905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:06.236605+0000","last_change":"2024-05-09T04:49:47.900692+0000","last_active":"2024-05-09T04:52:06.236605+0000","last_peered":"2024-05-09T04:52:06.236605+0000","last_clean":"2024-05-09T04:52:06.236605+0000","last_became_active":"2024-05-09T04:49:47.900095+0000","last_became_peered":"2024-05-09T04:49:47.900095+0000","last_unstale":"2024-05-09T04:52:06.236605+0000","last_undegraded":"2024-05-09T04:52:06.236605+0000","last_fullsized":"2024-05-09T04:52:06.236605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:08.632408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:45.712091+0000","last_change":"2024-05-09T04:49:30.048768+0000","last_active":"2024-05-09T04:51:45.712091+0000","last_peered":"2024-05-09T04:51:45.712091+0000","last_clean":"2024-05-09T04:51:45.712091+0000","last_became_active":"2024-05-09T04:49:30.048235+0000","last_became_peered":"2024-05-09T04:49:30.048235+0000","last_unstale":"2024-05-09T04:51:45.712091+0000","last_undegraded":"2024-05-09T04:51:45.712091+0000","last_fullsized":"2024-05-09T04:51:45.712091+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:51:16.988726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.540136+0000","last_change":"2024-05-09T04:49:34.142110+0000","last_active":"2024-05-09T04:52:55.540136+0000","last_peered":"2024-05-09T04:52:55.540136+0000","last_clean":"2024-05-09T04:52:55.540136+0000","last_became_active":"2024-05-09T04:49:34.141709+0000","last_became_peered":"2024-05-09T04:49:34.141709+0000","last_unstale":"2024-05-09T04:52:55.540136+0000","last_undegraded":"2024-05-09T04:52:55.540136+0000","last_fullsized":"2024-05-09T04:52:55.540136+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:16.365915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:57.649137+0000","last_change":"2024-05-09T04:49:47.667784+0000","last_active":"2024-05-09T04:51:57.649137+0000","last_peered":"2024-05-09T04:51:57.649137+0000","last_clean":"2024-05-09T04:51:57.649137+0000","last_became_active":"2024-05-09T04:49:47.667316+0000","last_became_peered":"2024-05-09T04:49:47.667316+0000","last_unstale":"2024-05-09T04:51:57.649137+0000","last_undegraded":"2024-05-09T04:51:57.649137+0000","last_fullsized":"2024-05-09T04:51:57.649137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:04:20.233287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'7","reported_seq":294,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:55:01.144856+0000","last_change":"2024-05-09T04:49:30.351231+0000","last_active":"2024-05-09T04:55:01.144856+0000","last_peered":"2024-05-09T04:55:01.144856+0000","last_clean":"2024-05-09T04:55:01.144856+0000","last_became_active":"2024-05-09T04:49:30.350817+0000","last_became_peered":"2024-05-09T04:49:30.350817+0000","last_unstale":"2024-05-09T04:55:01.144856+0000","last_undegraded":"2024-05-09T04:55:01.144856+0000","last_fullsized":"2024-05-09T04:55:01.144856+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:51.234300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1871,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":243,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.540495+0000","last_change":"2024-05-09T04:49:34.129625+0000","last_active":"2024-05-09T04:52:55.540495+0000","last_peered":"2024-05-09T04:52:55.540495+0000","last_clean":"2024-05-09T04:52:55.540495+0000","last_became_active":"2024-05-09T04:49:34.129215+0000","last_became_peered":"2024-05-09T04:49:34.129215+0000","last_unstale":"2024-05-09T04:52:55.540495+0000","last_undegraded":"2024-05-09T04:52:55.540495+0000","last_fullsized":"2024-05-09T04:52:55.540495+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:26.247116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:01.811085+0000","last_change":"2024-05-09T04:49:47.682819+0000","last_active":"2024-05-09T04:52:01.811085+0000","last_peered":"2024-05-09T04:52:01.811085+0000","last_clean":"2024-05-09T04:52:01.811085+0000","last_became_active":"2024-05-09T04:49:47.682331+0000","last_became_peered":"2024-05-09T04:49:47.682331+0000","last_unstale":"2024-05-09T04:52:01.811085+0000","last_undegraded":"2024-05-09T04:52:01.811085+0000","last_fullsized":"2024-05-09T04:52:01.811085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:03.852975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:45.721497+0000","last_change":"2024-05-09T04:49:30.050247+0000","last_active":"2024-05-09T04:51:45.721497+0000","last_peered":"2024-05-09T04:51:45.721497+0000","last_clean":"2024-05-09T04:51:45.721497+0000","last_became_active":"2024-05-09T04:49:30.049835+0000","last_became_peered":"2024-05-09T04:49:30.049835+0000","last_unstale":"2024-05-09T04:51:45.721497+0000","last_undegraded":"2024-05-09T04:51:45.721497+0000","last_fullsized":"2024-05-09T04:51:45.721497+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:52.591186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.559764+0000","last_change":"2024-05-09T04:49:34.145406+0000","last_active":"2024-05-09T04:52:55.559764+0000","last_peered":"2024-05-09T04:52:55.559764+0000","last_clean":"2024-05-09T04:52:55.559764+0000","last_became_active":"2024-05-09T04:49:34.144946+0000","last_became_peered":"2024-05-09T04:49:34.144946+0000","last_unstale":"2024-05-09T04:52:55.559764+0000","last_undegraded":"2024-05-09T04:52:55.559764+0000","last_fullsized":"2024-05-09T04:52:55.559764+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:43:04.704215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:05.714387+0000","last_change":"2024-05-09T04:49:56.758844+0000","last_active":"2024-05-09T04:52:05.714387+0000","last_peered":"2024-05-09T04:52:05.714387+0000","last_clean":"2024-05-09T04:52:05.714387+0000","last_became_active":"2024-05-09T04:49:56.758335+0000","last_became_peered":"2024-05-09T04:49:56.758335+0000","last_unstale":"2024-05-09T04:52:05.714387+0000","last_undegraded":"2024-05-09T04:52:05.714387+0000","last_fullsized":"2024-05-09T04:52:05.714387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:39:33.344225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:47.850941+0000","last_change":"2024-05-09T04:49:56.766262+0000","last_active":"2024-05-09T04:51:47.850941+0000","last_peered":"2024-05-09T04:51:47.850941+0000","last_clean":"2024-05-09T04:51:47.850941+0000","last_became_active":"2024-05-09T04:49:56.758765+0000","last_became_peered":"2024-05-09T04:49:56.758765+0000","last_unstale":"2024-05-09T04:51:47.850941+0000","last_undegraded":"2024-05-09T04:51:47.850941+0000","last_fullsized":"2024-05-09T04:51:47.850941+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:44:18.271729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.540919+0000","last_change":"2024-05-09T04:49:34.134957+0000","last_active":"2024-05-09T04:52:55.540919+0000","last_peered":"2024-05-09T04:52:55.540919+0000","last_clean":"2024-05-09T04:52:55.540919+0000","last_became_active":"2024-05-09T04:49:34.134544+0000","last_became_peered":"2024-05-09T04:49:34.134544+0000","last_unstale":"2024-05-09T04:52:55.540919+0000","last_undegraded":"2024-05-09T04:52:55.540919+0000","last_fullsized":"2024-05-09T04:52:55.540919+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:21:54.855905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.525564+0000","last_change":"2024-05-09T04:49:34.138112+0000","last_active":"2024-05-09T04:52:55.525564+0000","last_peered":"2024-05-09T04:52:55.525564+0000","last_clean":"2024-05-09T04:52:55.525564+0000","last_became_active":"2024-05-09T04:49:34.137716+0000","last_became_peered":"2024-05-09T04:49:34.137716+0000","last_unstale":"2024-05-09T04:52:55.525564+0000","last_undegraded":"2024-05-09T04:52:55.525564+0000","last_fullsized":"2024-05-09T04:52:55.525564+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:17.028262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.534478+0000","last_change":"2024-05-09T04:50:58.204287+0000","last_active":"2024-05-09T04:52:55.534478+0000","last_peered":"2024-05-09T04:52:55.534478+0000","last_clean":"2024-05-09T04:52:55.534478+0000","last_became_active":"2024-05-09T04:50:58.203894+0000","last_became_peered":"2024-05-09T04:50:58.203894+0000","last_unstale":"2024-05-09T04:52:55.534478+0000","last_undegraded":"2024-05-09T04:52:55.534478+0000","last_fullsized":"2024-05-09T04:52:55.534478+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:02.214347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.558687+0000","last_change":"2024-05-09T04:49:34.147742+0000","last_active":"2024-05-09T04:52:55.558687+0000","last_peered":"2024-05-09T04:52:55.558687+0000","last_clean":"2024-05-09T04:52:55.558687+0000","last_became_active":"2024-05-09T04:49:34.147301+0000","last_became_peered":"2024-05-09T04:49:34.147301+0000","last_unstale":"2024-05-09T04:52:55.558687+0000","last_undegraded":"2024-05-09T04:52:55.558687+0000","last_fullsized":"2024-05-09T04:52:55.558687+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:36:55.371212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.552970+0000","last_change":"2024-05-09T04:49:34.163599+0000","last_active":"2024-05-09T04:52:55.552970+0000","last_peered":"2024-05-09T04:52:55.552970+0000","last_clean":"2024-05-09T04:52:55.552970+0000","last_became_active":"2024-05-09T04:49:34.162898+0000","last_became_peered":"2024-05-09T04:49:34.162898+0000","last_unstale":"2024-05-09T04:52:55.552970+0000","last_undegraded":"2024-05-09T04:52:55.552970+0000","last_fullsized":"2024-05-09T04:52:55.552970+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:55:57.213901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.552433+0000","last_change":"2024-05-09T04:49:34.151648+0000","last_active":"2024-05-09T04:52:55.552433+0000","last_peered":"2024-05-09T04:52:55.552433+0000","last_clean":"2024-05-09T04:52:55.552433+0000","last_became_active":"2024-05-09T04:49:34.148701+0000","last_became_peered":"2024-05-09T04:49:34.148701+0000","last_unstale":"2024-05-09T04:52:55.552433+0000","last_undegraded":"2024-05-09T04:52:55.552433+0000","last_fullsized":"2024-05-09T04:52:55.552433+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:54:30.122252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.521268+0000","last_change":"2024-05-09T04:49:34.136679+0000","last_active":"2024-05-09T04:52:55.521268+0000","last_peered":"2024-05-09T04:52:55.521268+0000","last_clean":"2024-05-09T04:52:55.521268+0000","last_became_active":"2024-05-09T04:49:34.136306+0000","last_became_peered":"2024-05-09T04:49:34.136306+0000","last_unstale":"2024-05-09T04:52:55.521268+0000","last_undegraded":"2024-05-09T04:52:55.521268+0000","last_fullsized":"2024-05-09T04:52:55.521268+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:51:39.476564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.539107+0000","last_change":"2024-05-09T04:49:34.130769+0000","last_active":"2024-05-09T04:52:55.539107+0000","last_peered":"2024-05-09T04:52:55.539107+0000","last_clean":"2024-05-09T04:52:55.539107+0000","last_became_active":"2024-05-09T04:49:34.130362+0000","last_became_peered":"2024-05-09T04:49:34.130362+0000","last_unstale":"2024-05-09T04:52:55.539107+0000","last_undegraded":"2024-05-09T04:52:55.539107+0000","last_fullsized":"2024-05-09T04:52:55.539107+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:54.942259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.552538+0000","last_change":"2024-05-09T04:49:34.138054+0000","last_active":"2024-05-09T04:52:55.552538+0000","last_peered":"2024-05-09T04:52:55.552538+0000","last_clean":"2024-05-09T04:52:55.552538+0000","last_became_active":"2024-05-09T04:49:34.136111+0000","last_became_peered":"2024-05-09T04:49:34.136111+0000","last_unstale":"2024-05-09T04:52:55.552538+0000","last_undegraded":"2024-05-09T04:52:55.552538+0000","last_fullsized":"2024-05-09T04:52:55.552538+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:15:27.870874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.562318+0000","last_change":"2024-05-09T04:49:34.146370+0000","last_active":"2024-05-09T04:52:55.562318+0000","last_peered":"2024-05-09T04:52:55.562318+0000","last_clean":"2024-05-09T04:52:55.562318+0000","last_became_active":"2024-05-09T04:49:34.145948+0000","last_became_peered":"2024-05-09T04:49:34.145948+0000","last_unstale":"2024-05-09T04:52:55.562318+0000","last_undegraded":"2024-05-09T04:52:55.562318+0000","last_fullsized":"2024-05-09T04:52:55.562318+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:14.826168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.555403+0000","last_change":"2024-05-09T04:49:34.138028+0000","last_active":"2024-05-09T04:52:55.555403+0000","last_peered":"2024-05-09T04:52:55.555403+0000","last_clean":"2024-05-09T04:52:55.555403+0000","last_became_active":"2024-05-09T04:49:34.137558+0000","last_became_peered":"2024-05-09T04:49:34.137558+0000","last_unstale":"2024-05-09T04:52:55.555403+0000","last_undegraded":"2024-05-09T04:52:55.555403+0000","last_fullsized":"2024-05-09T04:52:55.555403+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:50:16.258945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.556779+0000","last_change":"2024-05-09T04:49:34.137946+0000","last_active":"2024-05-09T04:52:55.556779+0000","last_peered":"2024-05-09T04:52:55.556779+0000","last_clean":"2024-05-09T04:52:55.556779+0000","last_became_active":"2024-05-09T04:49:34.137498+0000","last_became_peered":"2024-05-09T04:49:34.137498+0000","last_unstale":"2024-05-09T04:52:55.556779+0000","last_undegraded":"2024-05-09T04:52:55.556779+0000","last_fullsized":"2024-05-09T04:52:55.556779+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:36:54.473689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.552098+0000","last_change":"2024-05-09T04:49:34.162103+0000","last_active":"2024-05-09T04:52:55.552098+0000","last_peered":"2024-05-09T04:52:55.552098+0000","last_clean":"2024-05-09T04:52:55.552098+0000","last_became_active":"2024-05-09T04:49:34.161681+0000","last_became_peered":"2024-05-09T04:49:34.161681+0000","last_unstale":"2024-05-09T04:52:55.552098+0000","last_undegraded":"2024-05-09T04:52:55.552098+0000","last_fullsized":"2024-05-09T04:52:55.552098+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:36:10.460401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.529070+0000","last_change":"2024-05-09T04:49:34.136538+0000","last_active":"2024-05-09T04:52:55.529070+0000","last_peered":"2024-05-09T04:52:55.529070+0000","last_clean":"2024-05-09T04:52:55.529070+0000","last_became_active":"2024-05-09T04:49:34.134903+0000","last_became_peered":"2024-05-09T04:49:34.134903+0000","last_unstale":"2024-05-09T04:52:55.529070+0000","last_undegraded":"2024-05-09T04:52:55.529070+0000","last_fullsized":"2024-05-09T04:52:55.529070+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:44:48.673600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":79141,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":2427,"num_read_kb":1304,"num_write":274,"num_write_kb":168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":602112,"data_stored":242463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678099,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":510952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874329,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37024,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708128,"statfs":{"total":95995035648,"available":95957123072,"internally_reserved":0,"allocated":1728512,"data_stored":1168050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037854,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":514165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234086,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":510335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430314,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1642496,"data_stored":1108008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593841,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":505573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574991,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":579686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738515,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":507730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1122304,"data_stored":581530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149827,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":505281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953598,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":510179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790071,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1650688,"data_stored":1103058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":74835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":76679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":72831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T04:55:05.900 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-09T04:55:05.900 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T04:55:06.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:05 smithi006 ceph-mon[29835]: from='client.25256 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:55:06.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:05 smithi186 ceph-mon[34668]: from='client.25256 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:55:06.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:05 smithi169 ceph-mon[34075]: from='client.25256 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:55:06.407 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:55:07.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:06 smithi006 ceph-mon[29835]: pgmap v504: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-09T04:55:07.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:06 smithi186 ceph-mon[34668]: pgmap v504: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-09T04:55:07.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:06 smithi169 ceph-mon[34075]: pgmap v504: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-09T04:55:08.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.747+0000 7f30c8e00700 1 -- 172.21.15.6:0/3767732500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30c410c5b0 msgr2=0x7f30c410c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:08.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.747+0000 7f30c8e00700 1 --2- 172.21.15.6:0/3767732500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30c410c5b0 0x7f30c410c9c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f30b4009990 tx=0x7f30b4009ca0 comp rx=0 tx=0).stop 2024-05-09T04:55:08.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.747+0000 7f30c8e00700 1 -- 172.21.15.6:0/3767732500 shutdown_connections 2024-05-09T04:55:08.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.747+0000 7f30c8e00700 1 --2- 172.21.15.6:0/3767732500 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30c4106ec0 0x7f30c4113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:08.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.747+0000 7f30c8e00700 1 --2- 172.21.15.6:0/3767732500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30c410c5b0 0x7f30c410c9c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:08.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.747+0000 7f30c8e00700 1 --2- 172.21.15.6:0/3767732500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30c4106570 0x7f30c4106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:08.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.747+0000 7f30c8e00700 1 -- 172.21.15.6:0/3767732500 >> 172.21.15.6:0/3767732500 conn(0x7f30c4102050 msgr2=0x7f30c41044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:08.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.748+0000 7f30c8e00700 1 -- 172.21.15.6:0/3767732500 shutdown_connections 2024-05-09T04:55:08.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.748+0000 7f30c8e00700 1 -- 172.21.15.6:0/3767732500 wait complete. 2024-05-09T04:55:08.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.749+0000 7f30c8e00700 1 Processor -- start 2024-05-09T04:55:08.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.749+0000 7f30c8e00700 1 -- start start 2024-05-09T04:55:08.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.749+0000 7f30c8e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30c4106570 0x7f30c419d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:08.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.749+0000 7f30c8e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30c4106ec0 0x7f30c419d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:08.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.749+0000 7f30c8e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30c410c5b0 0x7f30c41a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:08.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.749+0000 7f30c8e00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f30c41166c0 con 0x7f30c4106570 2024-05-09T04:55:08.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.749+0000 7f30c8e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f30c4116540 con 0x7f30c410c5b0 2024-05-09T04:55:08.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.749+0000 7f30c8e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f30c4116840 con 0x7f30c4106ec0 2024-05-09T04:55:08.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.750+0000 7f30c1e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30c4106570 0x7f30c419d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:08.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.750+0000 7f30c1e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30c4106570 0x7f30c419d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45390/0 (socket says 172.21.15.6:45390) 2024-05-09T04:55:08.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.750+0000 7f30c1e00700 1 -- 172.21.15.6:0/1736820509 learned_addr learned my addr 172.21.15.6:0/1736820509 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:55:08.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.750+0000 7f30c2800700 1 --2- 172.21.15.6:0/1736820509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30c410c5b0 0x7f30c41a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:08.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.750+0000 7f30c1400700 1 --2- 172.21.15.6:0/1736820509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30c4106ec0 0x7f30c419d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:08.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.750+0000 7f30c1e00700 1 -- 172.21.15.6:0/1736820509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30c4106ec0 msgr2=0x7f30c419d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:08.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.750+0000 7f30c1e00700 1 --2- 172.21.15.6:0/1736820509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30c4106ec0 0x7f30c419d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:08.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.750+0000 7f30c1e00700 1 -- 172.21.15.6:0/1736820509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30c410c5b0 msgr2=0x7f30c41a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:08.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.750+0000 7f30c1e00700 1 --2- 172.21.15.6:0/1736820509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30c410c5b0 0x7f30c41a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:08.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.750+0000 7f30c1e00700 1 -- 172.21.15.6:0/1736820509 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c41a2080 con 0x7f30c4106570 2024-05-09T04:55:08.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.750+0000 7f30c1400700 1 --2- 172.21.15.6:0/1736820509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30c4106ec0 0x7f30c419d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:55:08.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.751+0000 7f30c1e00700 1 --2- 172.21.15.6:0/1736820509 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30c4106570 0x7f30c419d1b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f30b4000c00 tx=0x7f30b400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:08.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.751+0000 7f30baa00700 1 -- 172.21.15.6:0/1736820509 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b4027070 con 0x7f30c4106570 2024-05-09T04:55:08.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.751+0000 7f30baa00700 1 -- 172.21.15.6:0/1736820509 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30b4022470 con 0x7f30c4106570 2024-05-09T04:55:08.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.751+0000 7f30baa00700 1 -- 172.21.15.6:0/1736820509 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b4017650 con 0x7f30c4106570 2024-05-09T04:55:08.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.751+0000 7f30c8e00700 1 -- 172.21.15.6:0/1736820509 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30c41a2310 con 0x7f30c4106570 2024-05-09T04:55:08.754 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.751+0000 7f30c8e00700 1 -- 172.21.15.6:0/1736820509 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30c41a27f0 con 0x7f30c4106570 2024-05-09T04:55:08.754 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.753+0000 7f30c8e00700 1 -- 172.21.15.6:0/1736820509 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30c40512c0 con 0x7f30c4106570 2024-05-09T04:55:08.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.755+0000 7f30baa00700 1 -- 172.21.15.6:0/1736820509 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) ==== 149702+0+0 (secure 0 0 0) 0x7f30b402a070 con 0x7f30c4106570 2024-05-09T04:55:08.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.756+0000 7f30baa00700 1 --2- 172.21.15.6:0/1736820509 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f30a80b2910 0x7f30a80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:08.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.757+0000 7f30baa00700 1 -- 172.21.15.6:0/1736820509 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f30b40e34b0 con 0x7f30c4106570 2024-05-09T04:55:08.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.757+0000 7f30c1400700 1 --2- 172.21.15.6:0/1736820509 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f30a80b2910 0x7f30a80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:08.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.758+0000 7f30c1400700 1 --2- 172.21.15.6:0/1736820509 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f30a80b2910 0x7f30a80b4e20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f30c4105580 tx=0x7f30b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:08.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.762+0000 7f30baa00700 1 -- 172.21.15.6:0/1736820509 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f30b4013070 con 0x7f30c4106570 2024-05-09T04:55:08.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:08.995+0000 7f30c8e00700 1 -- 172.21.15.6:0/1736820509 --> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f30c4063c00 con 0x7f30a80b2910 2024-05-09T04:55:09.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:09.023+0000 7f30baa00700 1 -- 172.21.15.6:0/1736820509 <== mgr.14152 v2:172.21.15.6:6800/2997423326 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340571 (secure 0 0 0) 0x7f30c4063c00 con 0x7f30a80b2910 2024-05-09T04:55:09.024 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:08 smithi006 ceph-mon[29835]: pgmap v505: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-05-09T04:55:09.026 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:55:09.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:09.032+0000 7f30c8e00700 1 -- 172.21.15.6:0/1736820509 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f30a80b2910 msgr2=0x7f30a80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:09.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:09.032+0000 7f30c8e00700 1 --2- 172.21.15.6:0/1736820509 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f30a80b2910 0x7f30a80b4e20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f30c4105580 tx=0x7f30b0009260 comp rx=0 tx=0).stop 2024-05-09T04:55:09.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:09.032+0000 7f30c8e00700 1 -- 172.21.15.6:0/1736820509 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30c4106570 msgr2=0x7f30c419d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:09.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:09.032+0000 7f30c8e00700 1 --2- 172.21.15.6:0/1736820509 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30c4106570 0x7f30c419d1b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f30b4000c00 tx=0x7f30b400bfd0 comp rx=0 tx=0).stop 2024-05-09T04:55:09.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:09.033+0000 7f30c8e00700 1 -- 172.21.15.6:0/1736820509 shutdown_connections 2024-05-09T04:55:09.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:09.033+0000 7f30c8e00700 1 --2- 172.21.15.6:0/1736820509 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f30a80b2910 0x7f30a80b4e20 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:09.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:09.033+0000 7f30c8e00700 1 --2- 172.21.15.6:0/1736820509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30c410c5b0 0x7f30c41a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:09.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:09.033+0000 7f30c8e00700 1 --2- 172.21.15.6:0/1736820509 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30c4106570 0x7f30c419d1b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:09.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:09.033+0000 7f30c8e00700 1 --2- 172.21.15.6:0/1736820509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30c4106ec0 0x7f30c419d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:09.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:09.033+0000 7f30c8e00700 1 -- 172.21.15.6:0/1736820509 >> 172.21.15.6:0/1736820509 conn(0x7f30c4102050 msgr2=0x7f30c41037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:09.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:09.033+0000 7f30c8e00700 1 -- 172.21.15.6:0/1736820509 shutdown_connections 2024-05-09T04:55:09.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:09.033+0000 7f30c8e00700 1 -- 172.21.15.6:0/1736820509 wait complete. 2024-05-09T04:55:09.035 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-05-09T04:55:09.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:08 smithi186 ceph-mon[34668]: pgmap v505: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-05-09T04:55:09.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:08 smithi169 ceph-mon[34075]: pgmap v505: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-05-09T04:55:09.955 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":505,"stamp":"2024-05-09T04:55:07.716773+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":672263,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":2544,"num_read_kb":1386,"num_write":400,"num_write_kb":2323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":831,"ondisk_log_size":831,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371640,"kb_used_data":14168,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570184,"statfs":{"total":1151940427776,"available":1151559868416,"internally_reserved":0,"allocated":14508032,"data_stored":8107067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19137,"internal_metadata":365933887},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9376,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":121,"num_read_kb":63,"num_write":13,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006645"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.559597+0000","last_change":"2024-05-09T04:49:34.144921+0000","last_active":"2024-05-09T04:52:55.559597+0000","last_peered":"2024-05-09T04:52:55.559597+0000","last_clean":"2024-05-09T04:52:55.559597+0000","last_became_active":"2024-05-09T04:49:34.144275+0000","last_became_peered":"2024-05-09T04:49:34.144275+0000","last_unstale":"2024-05-09T04:52:55.559597+0000","last_undegraded":"2024-05-09T04:52:55.559597+0000","last_fullsized":"2024-05-09T04:52:55.559597+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:00:31.377738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.559444+0000","last_change":"2024-05-09T04:49:34.158057+0000","last_active":"2024-05-09T04:52:55.559444+0000","last_peered":"2024-05-09T04:52:55.559444+0000","last_clean":"2024-05-09T04:52:55.559444+0000","last_became_active":"2024-05-09T04:49:34.157129+0000","last_became_peered":"2024-05-09T04:49:34.157129+0000","last_unstale":"2024-05-09T04:52:55.559444+0000","last_undegraded":"2024-05-09T04:52:55.559444+0000","last_fullsized":"2024-05-09T04:52:55.559444+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:49.360309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.554527+0000","last_change":"2024-05-09T04:49:34.145519+0000","last_active":"2024-05-09T04:52:55.554527+0000","last_peered":"2024-05-09T04:52:55.554527+0000","last_clean":"2024-05-09T04:52:55.554527+0000","last_became_active":"2024-05-09T04:49:34.144501+0000","last_became_peered":"2024-05-09T04:49:34.144501+0000","last_unstale":"2024-05-09T04:52:55.554527+0000","last_undegraded":"2024-05-09T04:52:55.554527+0000","last_fullsized":"2024-05-09T04:52:55.554527+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:05:53.727520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.521578+0000","last_change":"2024-05-09T04:49:34.134383+0000","last_active":"2024-05-09T04:52:55.521578+0000","last_peered":"2024-05-09T04:52:55.521578+0000","last_clean":"2024-05-09T04:52:55.521578+0000","last_became_active":"2024-05-09T04:49:34.134049+0000","last_became_peered":"2024-05-09T04:49:34.134049+0000","last_unstale":"2024-05-09T04:52:55.521578+0000","last_undegraded":"2024-05-09T04:52:55.521578+0000","last_fullsized":"2024-05-09T04:52:55.521578+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:36:02.548188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.535797+0000","last_change":"2024-05-09T04:49:34.157897+0000","last_active":"2024-05-09T04:52:55.535797+0000","last_peered":"2024-05-09T04:52:55.535797+0000","last_clean":"2024-05-09T04:52:55.535797+0000","last_became_active":"2024-05-09T04:49:34.157516+0000","last_became_peered":"2024-05-09T04:49:34.157516+0000","last_unstale":"2024-05-09T04:52:55.535797+0000","last_undegraded":"2024-05-09T04:52:55.535797+0000","last_fullsized":"2024-05-09T04:52:55.535797+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:33:25.444155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.535225+0000","last_change":"2024-05-09T04:49:34.146389+0000","last_active":"2024-05-09T04:52:55.535225+0000","last_peered":"2024-05-09T04:52:55.535225+0000","last_clean":"2024-05-09T04:52:55.535225+0000","last_became_active":"2024-05-09T04:49:34.145994+0000","last_became_peered":"2024-05-09T04:49:34.145994+0000","last_unstale":"2024-05-09T04:52:55.535225+0000","last_undegraded":"2024-05-09T04:52:55.535225+0000","last_fullsized":"2024-05-09T04:52:55.535225+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:59.094826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.537694+0000","last_change":"2024-05-09T04:49:34.146428+0000","last_active":"2024-05-09T04:52:55.537694+0000","last_peered":"2024-05-09T04:52:55.537694+0000","last_clean":"2024-05-09T04:52:55.537694+0000","last_became_active":"2024-05-09T04:49:34.146045+0000","last_became_peered":"2024-05-09T04:49:34.146045+0000","last_unstale":"2024-05-09T04:52:55.537694+0000","last_undegraded":"2024-05-09T04:52:55.537694+0000","last_fullsized":"2024-05-09T04:52:55.537694+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:13.129474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.559750+0000","last_change":"2024-05-09T04:49:34.135494+0000","last_active":"2024-05-09T04:52:55.559750+0000","last_peered":"2024-05-09T04:52:55.559750+0000","last_clean":"2024-05-09T04:52:55.559750+0000","last_became_active":"2024-05-09T04:49:34.135085+0000","last_became_peered":"2024-05-09T04:49:34.135085+0000","last_unstale":"2024-05-09T04:52:55.559750+0000","last_undegraded":"2024-05-09T04:52:55.559750+0000","last_fullsized":"2024-05-09T04:52:55.559750+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:57.023344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.541612+0000","last_change":"2024-05-09T04:49:34.131353+0000","last_active":"2024-05-09T04:52:55.541612+0000","last_peered":"2024-05-09T04:52:55.541612+0000","last_clean":"2024-05-09T04:52:55.541612+0000","last_became_active":"2024-05-09T04:49:34.130954+0000","last_became_peered":"2024-05-09T04:49:34.130954+0000","last_unstale":"2024-05-09T04:52:55.541612+0000","last_undegraded":"2024-05-09T04:52:55.541612+0000","last_fullsized":"2024-05-09T04:52:55.541612+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:04:46.947861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.526988+0000","last_change":"2024-05-09T04:49:34.134733+0000","last_active":"2024-05-09T04:52:55.526988+0000","last_peered":"2024-05-09T04:52:55.526988+0000","last_clean":"2024-05-09T04:52:55.526988+0000","last_became_active":"2024-05-09T04:49:34.134337+0000","last_became_peered":"2024-05-09T04:49:34.134337+0000","last_unstale":"2024-05-09T04:52:55.526988+0000","last_undegraded":"2024-05-09T04:52:55.526988+0000","last_fullsized":"2024-05-09T04:52:55.526988+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:05.452488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.540863+0000","last_change":"2024-05-09T04:49:34.139143+0000","last_active":"2024-05-09T04:52:55.540863+0000","last_peered":"2024-05-09T04:52:55.540863+0000","last_clean":"2024-05-09T04:52:55.540863+0000","last_became_active":"2024-05-09T04:49:34.138749+0000","last_became_peered":"2024-05-09T04:49:34.138749+0000","last_unstale":"2024-05-09T04:52:55.540863+0000","last_undegraded":"2024-05-09T04:52:55.540863+0000","last_fullsized":"2024-05-09T04:52:55.540863+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:11.552063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.524374+0000","last_change":"2024-05-09T04:49:34.134687+0000","last_active":"2024-05-09T04:52:55.524374+0000","last_peered":"2024-05-09T04:52:55.524374+0000","last_clean":"2024-05-09T04:52:55.524374+0000","last_became_active":"2024-05-09T04:49:34.134324+0000","last_became_peered":"2024-05-09T04:49:34.134324+0000","last_unstale":"2024-05-09T04:52:55.524374+0000","last_undegraded":"2024-05-09T04:52:55.524374+0000","last_fullsized":"2024-05-09T04:52:55.524374+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:11.667883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.546963+0000","last_change":"2024-05-09T04:49:34.153592+0000","last_active":"2024-05-09T04:52:55.546963+0000","last_peered":"2024-05-09T04:52:55.546963+0000","last_clean":"2024-05-09T04:52:55.546963+0000","last_became_active":"2024-05-09T04:49:34.153184+0000","last_became_peered":"2024-05-09T04:49:34.153184+0000","last_unstale":"2024-05-09T04:52:55.546963+0000","last_undegraded":"2024-05-09T04:52:55.546963+0000","last_fullsized":"2024-05-09T04:52:55.546963+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:36:56.784781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.553328+0000","last_change":"2024-05-09T04:49:34.144046+0000","last_active":"2024-05-09T04:52:55.553328+0000","last_peered":"2024-05-09T04:52:55.553328+0000","last_clean":"2024-05-09T04:52:55.553328+0000","last_became_active":"2024-05-09T04:49:34.141004+0000","last_became_peered":"2024-05-09T04:49:34.141004+0000","last_unstale":"2024-05-09T04:52:55.553328+0000","last_undegraded":"2024-05-09T04:52:55.553328+0000","last_fullsized":"2024-05-09T04:52:55.553328+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:09:12.274750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.554621+0000","last_change":"2024-05-09T04:49:34.147063+0000","last_active":"2024-05-09T04:52:55.554621+0000","last_peered":"2024-05-09T04:52:55.554621+0000","last_clean":"2024-05-09T04:52:55.554621+0000","last_became_active":"2024-05-09T04:49:34.144934+0000","last_became_peered":"2024-05-09T04:49:34.144934+0000","last_unstale":"2024-05-09T04:52:55.554621+0000","last_undegraded":"2024-05-09T04:52:55.554621+0000","last_fullsized":"2024-05-09T04:52:55.554621+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:47:59.125451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.540971+0000","last_change":"2024-05-09T04:49:34.141070+0000","last_active":"2024-05-09T04:52:55.540971+0000","last_peered":"2024-05-09T04:52:55.540971+0000","last_clean":"2024-05-09T04:52:55.540971+0000","last_became_active":"2024-05-09T04:49:34.140618+0000","last_became_peered":"2024-05-09T04:49:34.140618+0000","last_unstale":"2024-05-09T04:52:55.540971+0000","last_undegraded":"2024-05-09T04:52:55.540971+0000","last_fullsized":"2024-05-09T04:52:55.540971+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:02.435768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.559709+0000","last_change":"2024-05-09T04:49:34.144434+0000","last_active":"2024-05-09T04:52:55.559709+0000","last_peered":"2024-05-09T04:52:55.559709+0000","last_clean":"2024-05-09T04:52:55.559709+0000","last_became_active":"2024-05-09T04:49:34.144052+0000","last_became_peered":"2024-05-09T04:49:34.144052+0000","last_unstale":"2024-05-09T04:52:55.559709+0000","last_undegraded":"2024-05-09T04:52:55.559709+0000","last_fullsized":"2024-05-09T04:52:55.559709+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:05:26.141627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.548013+0000","last_change":"2024-05-09T04:49:34.148681+0000","last_active":"2024-05-09T04:52:55.548013+0000","last_peered":"2024-05-09T04:52:55.548013+0000","last_clean":"2024-05-09T04:52:55.548013+0000","last_became_active":"2024-05-09T04:49:34.145277+0000","last_became_peered":"2024-05-09T04:49:34.145277+0000","last_unstale":"2024-05-09T04:52:55.548013+0000","last_undegraded":"2024-05-09T04:52:55.548013+0000","last_fullsized":"2024-05-09T04:52:55.548013+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:09.546998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.548620+0000","last_change":"2024-05-09T04:49:34.153229+0000","last_active":"2024-05-09T04:52:55.548620+0000","last_peered":"2024-05-09T04:52:55.548620+0000","last_clean":"2024-05-09T04:52:55.548620+0000","last_became_active":"2024-05-09T04:49:34.152869+0000","last_became_peered":"2024-05-09T04:49:34.152869+0000","last_unstale":"2024-05-09T04:52:55.548620+0000","last_undegraded":"2024-05-09T04:52:55.548620+0000","last_fullsized":"2024-05-09T04:52:55.548620+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:54.350842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:12.136825+0000","last_change":"2024-05-09T04:49:56.710411+0000","last_active":"2024-05-09T04:52:12.136825+0000","last_peered":"2024-05-09T04:52:12.136825+0000","last_clean":"2024-05-09T04:52:12.136825+0000","last_became_active":"2024-05-09T04:49:56.709903+0000","last_became_peered":"2024-05-09T04:49:56.709903+0000","last_unstale":"2024-05-09T04:52:12.136825+0000","last_undegraded":"2024-05-09T04:52:12.136825+0000","last_fullsized":"2024-05-09T04:52:12.136825+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:53:44.216648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,10,0],"acting":[2,3,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":296,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:55:01.335513+0000","last_change":"2024-05-09T04:49:30.061766+0000","last_active":"2024-05-09T04:55:01.335513+0000","last_peered":"2024-05-09T04:55:01.335513+0000","last_clean":"2024-05-09T04:55:01.335513+0000","last_became_active":"2024-05-09T04:49:30.060545+0000","last_became_peered":"2024-05-09T04:49:30.060545+0000","last_unstale":"2024-05-09T04:55:01.335513+0000","last_undegraded":"2024-05-09T04:55:01.335513+0000","last_fullsized":"2024-05-09T04:55:01.335513+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:15:03.857107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":240,"num_read_kb":240,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.555879+0000","last_change":"2024-05-09T04:49:34.140706+0000","last_active":"2024-05-09T04:52:55.555879+0000","last_peered":"2024-05-09T04:52:55.555879+0000","last_clean":"2024-05-09T04:52:55.555879+0000","last_became_active":"2024-05-09T04:49:34.139032+0000","last_became_peered":"2024-05-09T04:49:34.139032+0000","last_unstale":"2024-05-09T04:52:55.555879+0000","last_undegraded":"2024-05-09T04:52:55.555879+0000","last_fullsized":"2024-05-09T04:52:55.555879+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:19.878523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:16.237850+0000","last_change":"2024-05-09T04:49:47.674254+0000","last_active":"2024-05-09T04:52:16.237850+0000","last_peered":"2024-05-09T04:52:16.237850+0000","last_clean":"2024-05-09T04:52:16.237850+0000","last_became_active":"2024-05-09T04:49:47.673476+0000","last_became_peered":"2024-05-09T04:49:47.673476+0000","last_unstale":"2024-05-09T04:52:16.237850+0000","last_undegraded":"2024-05-09T04:52:16.237850+0000","last_fullsized":"2024-05-09T04:52:16.237850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:41:31.709210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:33.128358+0000","last_change":"2024-05-09T04:49:30.154162+0000","last_active":"2024-05-09T04:51:33.128358+0000","last_peered":"2024-05-09T04:51:33.128358+0000","last_clean":"2024-05-09T04:51:33.128358+0000","last_became_active":"2024-05-09T04:49:30.152869+0000","last_became_peered":"2024-05-09T04:49:30.152869+0000","last_unstale":"2024-05-09T04:51:33.128358+0000","last_undegraded":"2024-05-09T04:51:33.128358+0000","last_fullsized":"2024-05-09T04:51:33.128358+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:43.433033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.563473+0000","last_change":"2024-05-09T04:49:34.157729+0000","last_active":"2024-05-09T04:52:55.563473+0000","last_peered":"2024-05-09T04:52:55.563473+0000","last_clean":"2024-05-09T04:52:55.563473+0000","last_became_active":"2024-05-09T04:49:34.155040+0000","last_became_peered":"2024-05-09T04:49:34.155040+0000","last_unstale":"2024-05-09T04:52:55.563473+0000","last_undegraded":"2024-05-09T04:52:55.563473+0000","last_fullsized":"2024-05-09T04:52:55.563473+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:43:30.104394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:21.238400+0000","last_change":"2024-05-09T04:49:47.679665+0000","last_active":"2024-05-09T04:52:21.238400+0000","last_peered":"2024-05-09T04:52:21.238400+0000","last_clean":"2024-05-09T04:52:21.238400+0000","last_became_active":"2024-05-09T04:49:47.679257+0000","last_became_peered":"2024-05-09T04:49:47.679257+0000","last_unstale":"2024-05-09T04:52:21.238400+0000","last_undegraded":"2024-05-09T04:52:21.238400+0000","last_fullsized":"2024-05-09T04:52:21.238400+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:53.111205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":57,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:00.713890+0000","last_change":"2024-05-09T04:49:30.047386+0000","last_active":"2024-05-09T04:52:00.713890+0000","last_peered":"2024-05-09T04:52:00.713890+0000","last_clean":"2024-05-09T04:52:00.713890+0000","last_became_active":"2024-05-09T04:49:30.046947+0000","last_became_peered":"2024-05-09T04:49:30.046947+0000","last_unstale":"2024-05-09T04:52:00.713890+0000","last_undegraded":"2024-05-09T04:52:00.713890+0000","last_fullsized":"2024-05-09T04:52:00.713890+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:46.861354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.548062+0000","last_change":"2024-05-09T04:49:34.153320+0000","last_active":"2024-05-09T04:52:55.548062+0000","last_peered":"2024-05-09T04:52:55.548062+0000","last_clean":"2024-05-09T04:52:55.548062+0000","last_became_active":"2024-05-09T04:49:34.152854+0000","last_became_peered":"2024-05-09T04:49:34.152854+0000","last_unstale":"2024-05-09T04:52:55.548062+0000","last_undegraded":"2024-05-09T04:52:55.548062+0000","last_fullsized":"2024-05-09T04:52:55.548062+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:40.801340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:21.238377+0000","last_change":"2024-05-09T04:49:47.668475+0000","last_active":"2024-05-09T04:52:21.238377+0000","last_peered":"2024-05-09T04:52:21.238377+0000","last_clean":"2024-05-09T04:52:21.238377+0000","last_became_active":"2024-05-09T04:49:47.667909+0000","last_became_peered":"2024-05-09T04:49:47.667909+0000","last_unstale":"2024-05-09T04:52:21.238377+0000","last_undegraded":"2024-05-09T04:52:21.238377+0000","last_fullsized":"2024-05-09T04:52:21.238377+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:39:27.687468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:00.713826+0000","last_change":"2024-05-09T04:49:30.048659+0000","last_active":"2024-05-09T04:52:00.713826+0000","last_peered":"2024-05-09T04:52:00.713826+0000","last_clean":"2024-05-09T04:52:00.713826+0000","last_became_active":"2024-05-09T04:49:30.048259+0000","last_became_peered":"2024-05-09T04:49:30.048259+0000","last_unstale":"2024-05-09T04:52:00.713826+0000","last_undegraded":"2024-05-09T04:52:00.713826+0000","last_fullsized":"2024-05-09T04:52:00.713826+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:37:46.012594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.539385+0000","last_change":"2024-05-09T04:49:34.131416+0000","last_active":"2024-05-09T04:52:55.539385+0000","last_peered":"2024-05-09T04:52:55.539385+0000","last_clean":"2024-05-09T04:52:55.539385+0000","last_became_active":"2024-05-09T04:49:34.130985+0000","last_became_peered":"2024-05-09T04:49:34.130985+0000","last_unstale":"2024-05-09T04:52:55.539385+0000","last_undegraded":"2024-05-09T04:52:55.539385+0000","last_fullsized":"2024-05-09T04:52:55.539385+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:02:43.228142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:16.222557+0000","last_change":"2024-05-09T04:49:47.652247+0000","last_active":"2024-05-09T04:52:16.222557+0000","last_peered":"2024-05-09T04:52:16.222557+0000","last_clean":"2024-05-09T04:52:16.222557+0000","last_became_active":"2024-05-09T04:49:47.651747+0000","last_became_peered":"2024-05-09T04:49:47.651747+0000","last_unstale":"2024-05-09T04:52:16.222557+0000","last_undegraded":"2024-05-09T04:52:16.222557+0000","last_fullsized":"2024-05-09T04:52:16.222557+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:45.353027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'8","reported_seq":299,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:55:04.149512+0000","last_change":"2024-05-09T04:49:30.047421+0000","last_active":"2024-05-09T04:55:04.149512+0000","last_peered":"2024-05-09T04:55:04.149512+0000","last_clean":"2024-05-09T04:55:04.149512+0000","last_became_active":"2024-05-09T04:49:30.047007+0000","last_became_peered":"2024-05-09T04:49:30.047007+0000","last_unstale":"2024-05-09T04:55:04.149512+0000","last_undegraded":"2024-05-09T04:55:04.149512+0000","last_fullsized":"2024-05-09T04:55:04.149512+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:23.930326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":247,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.533509+0000","last_change":"2024-05-09T04:49:34.146605+0000","last_active":"2024-05-09T04:52:55.533509+0000","last_peered":"2024-05-09T04:52:55.533509+0000","last_clean":"2024-05-09T04:52:55.533509+0000","last_became_active":"2024-05-09T04:49:34.146205+0000","last_became_peered":"2024-05-09T04:49:34.146205+0000","last_unstale":"2024-05-09T04:52:55.533509+0000","last_undegraded":"2024-05-09T04:52:55.533509+0000","last_fullsized":"2024-05-09T04:52:55.533509+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:13:52.234951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:16.222527+0000","last_change":"2024-05-09T04:49:47.673652+0000","last_active":"2024-05-09T04:52:16.222527+0000","last_peered":"2024-05-09T04:52:16.222527+0000","last_clean":"2024-05-09T04:52:16.222527+0000","last_became_active":"2024-05-09T04:49:47.673255+0000","last_became_peered":"2024-05-09T04:49:47.673255+0000","last_unstale":"2024-05-09T04:52:16.222527+0000","last_undegraded":"2024-05-09T04:52:16.222527+0000","last_fullsized":"2024-05-09T04:52:16.222527+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:12:50.672212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:00.294775+0000","last_change":"2024-05-09T04:49:30.056089+0000","last_active":"2024-05-09T04:52:00.294775+0000","last_peered":"2024-05-09T04:52:00.294775+0000","last_clean":"2024-05-09T04:52:00.294775+0000","last_became_active":"2024-05-09T04:49:30.055635+0000","last_became_peered":"2024-05-09T04:49:30.055635+0000","last_unstale":"2024-05-09T04:52:00.294775+0000","last_undegraded":"2024-05-09T04:52:00.294775+0000","last_fullsized":"2024-05-09T04:52:00.294775+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:35:06.529058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.562131+0000","last_change":"2024-05-09T04:49:34.147628+0000","last_active":"2024-05-09T04:52:55.562131+0000","last_peered":"2024-05-09T04:52:55.562131+0000","last_clean":"2024-05-09T04:52:55.562131+0000","last_became_active":"2024-05-09T04:49:34.147208+0000","last_became_peered":"2024-05-09T04:49:34.147208+0000","last_unstale":"2024-05-09T04:52:55.562131+0000","last_undegraded":"2024-05-09T04:52:55.562131+0000","last_fullsized":"2024-05-09T04:52:55.562131+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:09:22.520468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:51.957724+0000","last_change":"2024-05-09T04:49:56.758366+0000","last_active":"2024-05-09T04:51:51.957724+0000","last_peered":"2024-05-09T04:51:51.957724+0000","last_clean":"2024-05-09T04:51:51.957724+0000","last_became_active":"2024-05-09T04:49:56.757763+0000","last_became_peered":"2024-05-09T04:49:56.757763+0000","last_unstale":"2024-05-09T04:51:51.957724+0000","last_undegraded":"2024-05-09T04:51:51.957724+0000","last_fullsized":"2024-05-09T04:51:51.957724+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:58.711764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:57.852993+0000","last_change":"2024-05-09T04:49:30.058805+0000","last_active":"2024-05-09T04:51:57.852993+0000","last_peered":"2024-05-09T04:51:57.852993+0000","last_clean":"2024-05-09T04:51:57.852993+0000","last_became_active":"2024-05-09T04:49:30.057278+0000","last_became_peered":"2024-05-09T04:49:30.057278+0000","last_unstale":"2024-05-09T04:51:57.852993+0000","last_undegraded":"2024-05-09T04:51:57.852993+0000","last_fullsized":"2024-05-09T04:51:57.852993+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:34.727781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.559587+0000","last_change":"2024-05-09T04:49:34.148753+0000","last_active":"2024-05-09T04:52:55.559587+0000","last_peered":"2024-05-09T04:52:55.559587+0000","last_clean":"2024-05-09T04:52:55.559587+0000","last_became_active":"2024-05-09T04:49:34.148407+0000","last_became_peered":"2024-05-09T04:49:34.148407+0000","last_unstale":"2024-05-09T04:52:55.559587+0000","last_undegraded":"2024-05-09T04:52:55.559587+0000","last_fullsized":"2024-05-09T04:52:55.559587+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:39.602934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:46.807773+0000","last_change":"2024-05-09T04:49:47.672521+0000","last_active":"2024-05-09T04:51:46.807773+0000","last_peered":"2024-05-09T04:51:46.807773+0000","last_clean":"2024-05-09T04:51:46.807773+0000","last_became_active":"2024-05-09T04:49:47.671353+0000","last_became_peered":"2024-05-09T04:49:47.671353+0000","last_unstale":"2024-05-09T04:51:46.807773+0000","last_undegraded":"2024-05-09T04:51:46.807773+0000","last_fullsized":"2024-05-09T04:51:46.807773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:56:34.832651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:55.722708+0000","last_change":"2024-05-09T04:49:30.048589+0000","last_active":"2024-05-09T04:51:55.722708+0000","last_peered":"2024-05-09T04:51:55.722708+0000","last_clean":"2024-05-09T04:51:55.722708+0000","last_became_active":"2024-05-09T04:49:30.048241+0000","last_became_peered":"2024-05-09T04:49:30.048241+0000","last_unstale":"2024-05-09T04:51:55.722708+0000","last_undegraded":"2024-05-09T04:51:55.722708+0000","last_fullsized":"2024-05-09T04:51:55.722708+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:38:36.204539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.562269+0000","last_change":"2024-05-09T04:49:34.151201+0000","last_active":"2024-05-09T04:52:55.562269+0000","last_peered":"2024-05-09T04:52:55.562269+0000","last_clean":"2024-05-09T04:52:55.562269+0000","last_became_active":"2024-05-09T04:49:34.150505+0000","last_became_peered":"2024-05-09T04:49:34.150505+0000","last_unstale":"2024-05-09T04:52:55.562269+0000","last_undegraded":"2024-05-09T04:52:55.562269+0000","last_fullsized":"2024-05-09T04:52:55.562269+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:40.011516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:08.135251+0000","last_change":"2024-05-09T04:49:47.653827+0000","last_active":"2024-05-09T04:52:08.135251+0000","last_peered":"2024-05-09T04:52:08.135251+0000","last_clean":"2024-05-09T04:52:08.135251+0000","last_became_active":"2024-05-09T04:49:47.652423+0000","last_became_peered":"2024-05-09T04:49:47.652423+0000","last_unstale":"2024-05-09T04:52:08.135251+0000","last_undegraded":"2024-05-09T04:52:08.135251+0000","last_fullsized":"2024-05-09T04:52:08.135251+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:07:07.602659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'7","reported_seq":535,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:55:05.150118+0000","last_change":"2024-05-09T04:49:30.062849+0000","last_active":"2024-05-09T04:55:05.150118+0000","last_peered":"2024-05-09T04:55:05.150118+0000","last_clean":"2024-05-09T04:55:05.150118+0000","last_became_active":"2024-05-09T04:49:30.062426+0000","last_became_peered":"2024-05-09T04:49:30.062426+0000","last_unstale":"2024-05-09T04:55:05.150118+0000","last_undegraded":"2024-05-09T04:55:05.150118+0000","last_fullsized":"2024-05-09T04:55:05.150118+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:25:11.996208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":484,"num_read_kb":484,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.544151+0000","last_change":"2024-05-09T04:49:34.151145+0000","last_active":"2024-05-09T04:52:55.544151+0000","last_peered":"2024-05-09T04:52:55.544151+0000","last_clean":"2024-05-09T04:52:55.544151+0000","last_became_active":"2024-05-09T04:49:34.150745+0000","last_became_peered":"2024-05-09T04:49:34.150745+0000","last_unstale":"2024-05-09T04:52:55.544151+0000","last_undegraded":"2024-05-09T04:52:55.544151+0000","last_fullsized":"2024-05-09T04:52:55.544151+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:40.647454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:07.844209+0000","last_change":"2024-05-09T04:49:47.672796+0000","last_active":"2024-05-09T04:52:07.844209+0000","last_peered":"2024-05-09T04:52:07.844209+0000","last_clean":"2024-05-09T04:52:07.844209+0000","last_became_active":"2024-05-09T04:49:47.667807+0000","last_became_peered":"2024-05-09T04:49:47.667807+0000","last_unstale":"2024-05-09T04:52:07.844209+0000","last_undegraded":"2024-05-09T04:52:07.844209+0000","last_fullsized":"2024-05-09T04:52:07.844209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:58.542973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:54.594836+0000","last_change":"2024-05-09T04:49:30.047265+0000","last_active":"2024-05-09T04:51:54.594836+0000","last_peered":"2024-05-09T04:51:54.594836+0000","last_clean":"2024-05-09T04:51:54.594836+0000","last_became_active":"2024-05-09T04:49:30.046888+0000","last_became_peered":"2024-05-09T04:49:30.046888+0000","last_unstale":"2024-05-09T04:51:54.594836+0000","last_undegraded":"2024-05-09T04:51:54.594836+0000","last_fullsized":"2024-05-09T04:51:54.594836+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:38:41.878678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.554268+0000","last_change":"2024-05-09T04:49:34.144447+0000","last_active":"2024-05-09T04:52:55.554268+0000","last_peered":"2024-05-09T04:52:55.554268+0000","last_clean":"2024-05-09T04:52:55.554268+0000","last_became_active":"2024-05-09T04:49:34.144073+0000","last_became_peered":"2024-05-09T04:49:34.144073+0000","last_unstale":"2024-05-09T04:52:55.554268+0000","last_undegraded":"2024-05-09T04:52:55.554268+0000","last_fullsized":"2024-05-09T04:52:55.554268+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:53:42.619552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:11.237300+0000","last_change":"2024-05-09T04:49:47.673364+0000","last_active":"2024-05-09T04:52:11.237300+0000","last_peered":"2024-05-09T04:52:11.237300+0000","last_clean":"2024-05-09T04:52:11.237300+0000","last_became_active":"2024-05-09T04:49:47.672712+0000","last_became_peered":"2024-05-09T04:49:47.672712+0000","last_unstale":"2024-05-09T04:52:11.237300+0000","last_undegraded":"2024-05-09T04:52:11.237300+0000","last_fullsized":"2024-05-09T04:52:11.237300+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:25:57.165777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:55.722783+0000","last_change":"2024-05-09T04:49:30.160924+0000","last_active":"2024-05-09T04:51:55.722783+0000","last_peered":"2024-05-09T04:51:55.722783+0000","last_clean":"2024-05-09T04:51:55.722783+0000","last_became_active":"2024-05-09T04:49:30.160479+0000","last_became_peered":"2024-05-09T04:49:30.160479+0000","last_unstale":"2024-05-09T04:51:55.722783+0000","last_undegraded":"2024-05-09T04:51:55.722783+0000","last_fullsized":"2024-05-09T04:51:55.722783+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:44:57.171441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.545843+0000","last_change":"2024-05-09T04:49:34.136514+0000","last_active":"2024-05-09T04:52:55.545843+0000","last_peered":"2024-05-09T04:52:55.545843+0000","last_clean":"2024-05-09T04:52:55.545843+0000","last_became_active":"2024-05-09T04:49:34.136121+0000","last_became_peered":"2024-05-09T04:49:34.136121+0000","last_unstale":"2024-05-09T04:52:55.545843+0000","last_undegraded":"2024-05-09T04:52:55.545843+0000","last_fullsized":"2024-05-09T04:52:55.545843+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:43:27.781720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:07.844267+0000","last_change":"2024-05-09T04:49:47.673371+0000","last_active":"2024-05-09T04:52:07.844267+0000","last_peered":"2024-05-09T04:52:07.844267+0000","last_clean":"2024-05-09T04:52:07.844267+0000","last_became_active":"2024-05-09T04:49:47.669727+0000","last_became_peered":"2024-05-09T04:49:47.669727+0000","last_unstale":"2024-05-09T04:52:07.844267+0000","last_undegraded":"2024-05-09T04:52:07.844267+0000","last_fullsized":"2024-05-09T04:52:07.844267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:34:45.510934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'45","reported_seq":589,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:55:05.150113+0000","last_change":"2024-05-09T04:49:30.048359+0000","last_active":"2024-05-09T04:55:05.150113+0000","last_peered":"2024-05-09T04:55:05.150113+0000","last_clean":"2024-05-09T04:55:05.150113+0000","last_became_active":"2024-05-09T04:49:30.047949+0000","last_became_peered":"2024-05-09T04:49:30.047949+0000","last_unstale":"2024-05-09T04:55:05.150113+0000","last_undegraded":"2024-05-09T04:55:05.150113+0000","last_fullsized":"2024-05-09T04:55:05.150113+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:01:10.498997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73083,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":500,"num_read_kb":323,"num_write":47,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.521477+0000","last_change":"2024-05-09T04:49:34.132589+0000","last_active":"2024-05-09T04:52:55.521477+0000","last_peered":"2024-05-09T04:52:55.521477+0000","last_clean":"2024-05-09T04:52:55.521477+0000","last_became_active":"2024-05-09T04:49:34.132227+0000","last_became_peered":"2024-05-09T04:49:34.132227+0000","last_unstale":"2024-05-09T04:52:55.521477+0000","last_undegraded":"2024-05-09T04:52:55.521477+0000","last_fullsized":"2024-05-09T04:52:55.521477+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:50.003878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:15.725082+0000","last_change":"2024-05-09T04:49:47.668236+0000","last_active":"2024-05-09T04:52:15.725082+0000","last_peered":"2024-05-09T04:52:15.725082+0000","last_clean":"2024-05-09T04:52:15.725082+0000","last_became_active":"2024-05-09T04:49:47.667779+0000","last_became_peered":"2024-05-09T04:49:47.667779+0000","last_unstale":"2024-05-09T04:52:15.725082+0000","last_undegraded":"2024-05-09T04:52:15.725082+0000","last_fullsized":"2024-05-09T04:52:15.725082+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:16.177888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:52.134520+0000","last_change":"2024-05-09T04:49:30.046145+0000","last_active":"2024-05-09T04:51:52.134520+0000","last_peered":"2024-05-09T04:51:52.134520+0000","last_clean":"2024-05-09T04:51:52.134520+0000","last_became_active":"2024-05-09T04:49:30.045478+0000","last_became_peered":"2024-05-09T04:49:30.045478+0000","last_unstale":"2024-05-09T04:51:52.134520+0000","last_undegraded":"2024-05-09T04:51:52.134520+0000","last_fullsized":"2024-05-09T04:51:52.134520+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:52:08.313984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.550418+0000","last_change":"2024-05-09T04:49:34.154349+0000","last_active":"2024-05-09T04:52:55.550418+0000","last_peered":"2024-05-09T04:52:55.550418+0000","last_clean":"2024-05-09T04:52:55.550418+0000","last_became_active":"2024-05-09T04:49:34.154011+0000","last_became_peered":"2024-05-09T04:49:34.154011+0000","last_unstale":"2024-05-09T04:52:55.550418+0000","last_undegraded":"2024-05-09T04:52:55.550418+0000","last_fullsized":"2024-05-09T04:52:55.550418+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:50:32.345863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:06.959550+0000","last_change":"2024-05-09T04:49:47.670170+0000","last_active":"2024-05-09T04:52:06.959550+0000","last_peered":"2024-05-09T04:52:06.959550+0000","last_clean":"2024-05-09T04:52:06.959550+0000","last_became_active":"2024-05-09T04:49:47.669705+0000","last_became_peered":"2024-05-09T04:49:47.669705+0000","last_unstale":"2024-05-09T04:52:06.959550+0000","last_undegraded":"2024-05-09T04:52:06.959550+0000","last_fullsized":"2024-05-09T04:52:06.959550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:59:27.339811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:50.722218+0000","last_change":"2024-05-09T04:49:30.051159+0000","last_active":"2024-05-09T04:51:50.722218+0000","last_peered":"2024-05-09T04:51:50.722218+0000","last_clean":"2024-05-09T04:51:50.722218+0000","last_became_active":"2024-05-09T04:49:30.050108+0000","last_became_peered":"2024-05-09T04:49:30.050108+0000","last_unstale":"2024-05-09T04:51:50.722218+0000","last_undegraded":"2024-05-09T04:51:50.722218+0000","last_fullsized":"2024-05-09T04:51:50.722218+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:32:56.778567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.549234+0000","last_change":"2024-05-09T04:49:34.162751+0000","last_active":"2024-05-09T04:52:55.549234+0000","last_peered":"2024-05-09T04:52:55.549234+0000","last_clean":"2024-05-09T04:52:55.549234+0000","last_became_active":"2024-05-09T04:49:34.162336+0000","last_became_peered":"2024-05-09T04:49:34.162336+0000","last_unstale":"2024-05-09T04:52:55.549234+0000","last_undegraded":"2024-05-09T04:52:55.549234+0000","last_fullsized":"2024-05-09T04:52:55.549234+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:41.758074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:12.845173+0000","last_change":"2024-05-09T04:49:47.680955+0000","last_active":"2024-05-09T04:52:12.845173+0000","last_peered":"2024-05-09T04:52:12.845173+0000","last_clean":"2024-05-09T04:52:12.845173+0000","last_became_active":"2024-05-09T04:49:47.680041+0000","last_became_peered":"2024-05-09T04:49:47.680041+0000","last_unstale":"2024-05-09T04:52:12.845173+0000","last_undegraded":"2024-05-09T04:52:12.845173+0000","last_fullsized":"2024-05-09T04:52:12.845173+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:46.297611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'7","reported_seq":296,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:55:04.385086+0000","last_change":"2024-05-09T04:49:30.045857+0000","last_active":"2024-05-09T04:55:04.385086+0000","last_peered":"2024-05-09T04:55:04.385086+0000","last_clean":"2024-05-09T04:55:04.385086+0000","last_became_active":"2024-05-09T04:49:30.045404+0000","last_became_peered":"2024-05-09T04:49:30.045404+0000","last_unstale":"2024-05-09T04:55:04.385086+0000","last_undegraded":"2024-05-09T04:55:04.385086+0000","last_fullsized":"2024-05-09T04:55:04.385086+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:44:19.473449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":247,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.539356+0000","last_change":"2024-05-09T04:49:34.131734+0000","last_active":"2024-05-09T04:52:55.539356+0000","last_peered":"2024-05-09T04:52:55.539356+0000","last_clean":"2024-05-09T04:52:55.539356+0000","last_became_active":"2024-05-09T04:49:34.131396+0000","last_became_peered":"2024-05-09T04:49:34.131396+0000","last_unstale":"2024-05-09T04:52:55.539356+0000","last_undegraded":"2024-05-09T04:52:55.539356+0000","last_fullsized":"2024-05-09T04:52:55.539356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:21.080651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:12.854733+0000","last_change":"2024-05-09T04:49:56.710429+0000","last_active":"2024-05-09T04:52:12.854733+0000","last_peered":"2024-05-09T04:52:12.854733+0000","last_clean":"2024-05-09T04:52:12.854733+0000","last_became_active":"2024-05-09T04:49:56.708497+0000","last_became_peered":"2024-05-09T04:49:56.708497+0000","last_unstale":"2024-05-09T04:52:12.854733+0000","last_undegraded":"2024-05-09T04:52:12.854733+0000","last_fullsized":"2024-05-09T04:52:12.854733+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:36.281300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:16.960685+0000","last_change":"2024-05-09T04:49:56.718731+0000","last_active":"2024-05-09T04:52:16.960685+0000","last_peered":"2024-05-09T04:52:16.960685+0000","last_clean":"2024-05-09T04:52:16.960685+0000","last_became_active":"2024-05-09T04:49:56.709958+0000","last_became_peered":"2024-05-09T04:49:56.709958+0000","last_unstale":"2024-05-09T04:52:16.960685+0000","last_undegraded":"2024-05-09T04:52:16.960685+0000","last_fullsized":"2024-05-09T04:52:16.960685+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:11.003906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.539006+0000","last_change":"2024-05-09T04:49:34.131705+0000","last_active":"2024-05-09T04:52:55.539006+0000","last_peered":"2024-05-09T04:52:55.539006+0000","last_clean":"2024-05-09T04:52:55.539006+0000","last_became_active":"2024-05-09T04:49:34.131287+0000","last_became_peered":"2024-05-09T04:49:34.131287+0000","last_unstale":"2024-05-09T04:52:55.539006+0000","last_undegraded":"2024-05-09T04:52:55.539006+0000","last_fullsized":"2024-05-09T04:52:55.539006+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:10:24.914832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:00.723219+0000","last_change":"2024-05-09T04:49:47.663861+0000","last_active":"2024-05-09T04:52:00.723219+0000","last_peered":"2024-05-09T04:52:00.723219+0000","last_clean":"2024-05-09T04:52:00.723219+0000","last_became_active":"2024-05-09T04:49:47.663374+0000","last_became_peered":"2024-05-09T04:49:47.663374+0000","last_unstale":"2024-05-09T04:52:00.723219+0000","last_undegraded":"2024-05-09T04:52:00.723219+0000","last_fullsized":"2024-05-09T04:52:00.723219+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:08.063040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:45.712130+0000","last_change":"2024-05-09T04:49:30.044183+0000","last_active":"2024-05-09T04:51:45.712130+0000","last_peered":"2024-05-09T04:51:45.712130+0000","last_clean":"2024-05-09T04:51:45.712130+0000","last_became_active":"2024-05-09T04:49:30.043608+0000","last_became_peered":"2024-05-09T04:49:30.043608+0000","last_unstale":"2024-05-09T04:51:45.712130+0000","last_undegraded":"2024-05-09T04:51:45.712130+0000","last_fullsized":"2024-05-09T04:51:45.712130+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:48:57.788970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.528008+0000","last_change":"2024-05-09T04:49:34.133941+0000","last_active":"2024-05-09T04:52:55.528008+0000","last_peered":"2024-05-09T04:52:55.528008+0000","last_clean":"2024-05-09T04:52:55.528008+0000","last_became_active":"2024-05-09T04:49:34.133550+0000","last_became_peered":"2024-05-09T04:49:34.133550+0000","last_unstale":"2024-05-09T04:52:55.528008+0000","last_undegraded":"2024-05-09T04:52:55.528008+0000","last_fullsized":"2024-05-09T04:52:55.528008+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:45:14.703565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:55.713155+0000","last_change":"2024-05-09T04:49:47.901085+0000","last_active":"2024-05-09T04:51:55.713155+0000","last_peered":"2024-05-09T04:51:55.713155+0000","last_clean":"2024-05-09T04:51:55.713155+0000","last_became_active":"2024-05-09T04:49:47.900444+0000","last_became_peered":"2024-05-09T04:49:47.900444+0000","last_unstale":"2024-05-09T04:51:55.713155+0000","last_undegraded":"2024-05-09T04:51:55.713155+0000","last_fullsized":"2024-05-09T04:51:55.713155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:44.195683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:37.839364+0000","last_change":"2024-05-09T04:49:30.154060+0000","last_active":"2024-05-09T04:51:37.839364+0000","last_peered":"2024-05-09T04:51:37.839364+0000","last_clean":"2024-05-09T04:51:37.839364+0000","last_became_active":"2024-05-09T04:49:30.152556+0000","last_became_peered":"2024-05-09T04:49:30.152556+0000","last_unstale":"2024-05-09T04:51:37.839364+0000","last_undegraded":"2024-05-09T04:51:37.839364+0000","last_fullsized":"2024-05-09T04:51:37.839364+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:27.056251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.531546+0000","last_change":"2024-05-09T04:49:34.153128+0000","last_active":"2024-05-09T04:52:55.531546+0000","last_peered":"2024-05-09T04:52:55.531546+0000","last_clean":"2024-05-09T04:52:55.531546+0000","last_became_active":"2024-05-09T04:49:34.152734+0000","last_became_peered":"2024-05-09T04:49:34.152734+0000","last_unstale":"2024-05-09T04:52:55.531546+0000","last_undegraded":"2024-05-09T04:52:55.531546+0000","last_fullsized":"2024-05-09T04:52:55.531546+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:54:06.920877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:52.852082+0000","last_change":"2024-05-09T04:49:47.670263+0000","last_active":"2024-05-09T04:51:52.852082+0000","last_peered":"2024-05-09T04:51:52.852082+0000","last_clean":"2024-05-09T04:51:52.852082+0000","last_became_active":"2024-05-09T04:49:47.665743+0000","last_became_peered":"2024-05-09T04:49:47.665743+0000","last_unstale":"2024-05-09T04:51:52.852082+0000","last_undegraded":"2024-05-09T04:51:52.852082+0000","last_fullsized":"2024-05-09T04:51:52.852082+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:27.980226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":291,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:55:03.341559+0000","last_change":"2024-05-09T04:49:30.056119+0000","last_active":"2024-05-09T04:55:03.341559+0000","last_peered":"2024-05-09T04:55:03.341559+0000","last_clean":"2024-05-09T04:55:03.341559+0000","last_became_active":"2024-05-09T04:49:30.055660+0000","last_became_peered":"2024-05-09T04:49:30.055660+0000","last_unstale":"2024-05-09T04:55:03.341559+0000","last_undegraded":"2024-05-09T04:55:03.341559+0000","last_fullsized":"2024-05-09T04:55:03.341559+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:21.513879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":242,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.524834+0000","last_change":"2024-05-09T04:49:56.764303+0000","last_active":"2024-05-09T04:52:55.524834+0000","last_peered":"2024-05-09T04:52:55.524834+0000","last_clean":"2024-05-09T04:52:55.524834+0000","last_became_active":"2024-05-09T04:49:56.763854+0000","last_became_peered":"2024-05-09T04:49:56.763854+0000","last_unstale":"2024-05-09T04:52:55.524834+0000","last_undegraded":"2024-05-09T04:52:55.524834+0000","last_fullsized":"2024-05-09T04:52:55.524834+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:06.086602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:57.853087+0000","last_change":"2024-05-09T04:49:47.900866+0000","last_active":"2024-05-09T04:51:57.853087+0000","last_peered":"2024-05-09T04:51:57.853087+0000","last_clean":"2024-05-09T04:51:57.853087+0000","last_became_active":"2024-05-09T04:49:47.900332+0000","last_became_peered":"2024-05-09T04:49:47.900332+0000","last_unstale":"2024-05-09T04:51:57.853087+0000","last_undegraded":"2024-05-09T04:51:57.853087+0000","last_fullsized":"2024-05-09T04:51:57.853087+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:06:41.448262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.527180+0000","last_change":"2024-05-09T04:49:56.702559+0000","last_active":"2024-05-09T04:52:55.527180+0000","last_peered":"2024-05-09T04:52:55.527180+0000","last_clean":"2024-05-09T04:52:55.527180+0000","last_became_active":"2024-05-09T04:49:56.702135+0000","last_became_peered":"2024-05-09T04:49:56.702135+0000","last_unstale":"2024-05-09T04:52:55.527180+0000","last_undegraded":"2024-05-09T04:52:55.527180+0000","last_fullsized":"2024-05-09T04:52:55.527180+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:43:05.848239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":290,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:55:01.376068+0000","last_change":"2024-05-09T04:49:30.158050+0000","last_active":"2024-05-09T04:55:01.376068+0000","last_peered":"2024-05-09T04:55:01.376068+0000","last_clean":"2024-05-09T04:55:01.376068+0000","last_became_active":"2024-05-09T04:49:30.156719+0000","last_became_peered":"2024-05-09T04:49:30.156719+0000","last_unstale":"2024-05-09T04:55:01.376068+0000","last_undegraded":"2024-05-09T04:55:01.376068+0000","last_fullsized":"2024-05-09T04:55:01.376068+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:47.963111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":240,"num_read_kb":240,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:54.594819+0000","last_change":"2024-05-09T04:49:47.673594+0000","last_active":"2024-05-09T04:51:54.594819+0000","last_peered":"2024-05-09T04:51:54.594819+0000","last_clean":"2024-05-09T04:51:54.594819+0000","last_became_active":"2024-05-09T04:49:47.672671+0000","last_became_peered":"2024-05-09T04:49:47.672671+0000","last_unstale":"2024-05-09T04:51:54.594819+0000","last_undegraded":"2024-05-09T04:51:54.594819+0000","last_fullsized":"2024-05-09T04:51:54.594819+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:58:56.078180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.547961+0000","last_change":"2024-05-09T04:50:58.035054+0000","last_active":"2024-05-09T04:52:55.547961+0000","last_peered":"2024-05-09T04:52:55.547961+0000","last_clean":"2024-05-09T04:52:55.547961+0000","last_became_active":"2024-05-09T04:50:58.034297+0000","last_became_peered":"2024-05-09T04:50:58.034297+0000","last_unstale":"2024-05-09T04:52:55.547961+0000","last_undegraded":"2024-05-09T04:52:55.547961+0000","last_fullsized":"2024-05-09T04:52:55.547961+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:10:56.827950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:36.232917+0000","last_change":"2024-05-09T04:49:30.060224+0000","last_active":"2024-05-09T04:51:36.232917+0000","last_peered":"2024-05-09T04:51:36.232917+0000","last_clean":"2024-05-09T04:51:36.232917+0000","last_became_active":"2024-05-09T04:49:30.059830+0000","last_became_peered":"2024-05-09T04:49:30.059830+0000","last_unstale":"2024-05-09T04:51:36.232917+0000","last_undegraded":"2024-05-09T04:51:36.232917+0000","last_fullsized":"2024-05-09T04:51:36.232917+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:02.925226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":118,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:24.598674+0000","last_change":"2024-05-09T04:47:53.050380+0000","last_active":"2024-05-09T04:52:24.598674+0000","last_peered":"2024-05-09T04:52:24.598674+0000","last_clean":"2024-05-09T04:52:24.598674+0000","last_became_active":"2024-05-09T04:47:53.044058+0000","last_became_peered":"2024-05-09T04:47:53.044058+0000","last_unstale":"2024-05-09T04:52:24.598674+0000","last_undegraded":"2024-05-09T04:52:24.598674+0000","last_fullsized":"2024-05-09T04:52:24.598674+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:43:48.067159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:43:48.067159+0000","last_clean_scrub_stamp":"2024-05-09T04:43:48.067159+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:17:18.714607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:00.723281+0000","last_change":"2024-05-09T04:49:47.901114+0000","last_active":"2024-05-09T04:52:00.723281+0000","last_peered":"2024-05-09T04:52:00.723281+0000","last_clean":"2024-05-09T04:52:00.723281+0000","last_became_active":"2024-05-09T04:49:47.900450+0000","last_became_peered":"2024-05-09T04:49:47.900450+0000","last_unstale":"2024-05-09T04:52:00.723281+0000","last_undegraded":"2024-05-09T04:52:00.723281+0000","last_fullsized":"2024-05-09T04:52:00.723281+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:25:44.517789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.538748+0000","last_change":"2024-05-09T04:49:34.147027+0000","last_active":"2024-05-09T04:52:55.538748+0000","last_peered":"2024-05-09T04:52:55.538748+0000","last_clean":"2024-05-09T04:52:55.538748+0000","last_became_active":"2024-05-09T04:49:34.146672+0000","last_became_peered":"2024-05-09T04:49:34.146672+0000","last_unstale":"2024-05-09T04:52:55.538748+0000","last_undegraded":"2024-05-09T04:52:55.538748+0000","last_fullsized":"2024-05-09T04:52:55.538748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:15:44.847304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:29.591646+0000","last_change":"2024-05-09T04:49:30.043877+0000","last_active":"2024-05-09T04:51:29.591646+0000","last_peered":"2024-05-09T04:51:29.591646+0000","last_clean":"2024-05-09T04:51:29.591646+0000","last_became_active":"2024-05-09T04:49:30.043513+0000","last_became_peered":"2024-05-09T04:49:30.043513+0000","last_unstale":"2024-05-09T04:51:29.591646+0000","last_undegraded":"2024-05-09T04:51:29.591646+0000","last_fullsized":"2024-05-09T04:51:29.591646+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:16.501692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:51.957672+0000","last_change":"2024-05-09T04:49:47.668605+0000","last_active":"2024-05-09T04:51:51.957672+0000","last_peered":"2024-05-09T04:51:51.957672+0000","last_clean":"2024-05-09T04:51:51.957672+0000","last_became_active":"2024-05-09T04:49:47.668095+0000","last_became_peered":"2024-05-09T04:49:47.668095+0000","last_unstale":"2024-05-09T04:51:51.957672+0000","last_undegraded":"2024-05-09T04:51:51.957672+0000","last_fullsized":"2024-05-09T04:51:51.957672+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:39:30.508618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:35.710826+0000","last_change":"2024-05-09T04:49:30.159554+0000","last_active":"2024-05-09T04:51:35.710826+0000","last_peered":"2024-05-09T04:51:35.710826+0000","last_clean":"2024-05-09T04:51:35.710826+0000","last_became_active":"2024-05-09T04:49:30.159266+0000","last_became_peered":"2024-05-09T04:49:30.159266+0000","last_unstale":"2024-05-09T04:51:35.710826+0000","last_undegraded":"2024-05-09T04:51:35.710826+0000","last_fullsized":"2024-05-09T04:51:35.710826+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:42:04.252862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.542103+0000","last_change":"2024-05-09T04:49:34.150688+0000","last_active":"2024-05-09T04:52:55.542103+0000","last_peered":"2024-05-09T04:52:55.542103+0000","last_clean":"2024-05-09T04:52:55.542103+0000","last_became_active":"2024-05-09T04:49:34.150319+0000","last_became_peered":"2024-05-09T04:49:34.150319+0000","last_unstale":"2024-05-09T04:52:55.542103+0000","last_undegraded":"2024-05-09T04:52:55.542103+0000","last_fullsized":"2024-05-09T04:52:55.542103+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:35:04.832761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:48.131063+0000","last_change":"2024-05-09T04:49:47.902323+0000","last_active":"2024-05-09T04:51:48.131063+0000","last_peered":"2024-05-09T04:51:48.131063+0000","last_clean":"2024-05-09T04:51:48.131063+0000","last_became_active":"2024-05-09T04:49:47.901389+0000","last_became_peered":"2024-05-09T04:49:47.901389+0000","last_unstale":"2024-05-09T04:51:48.131063+0000","last_undegraded":"2024-05-09T04:51:48.131063+0000","last_fullsized":"2024-05-09T04:51:48.131063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:32.615944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'18","reported_seq":61,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:54:55.206777+0000","last_change":"2024-05-09T04:49:30.061172+0000","last_active":"2024-05-09T04:54:55.206777+0000","last_peered":"2024-05-09T04:54:55.206777+0000","last_clean":"2024-05-09T04:54:55.206777+0000","last_became_active":"2024-05-09T04:49:30.060680+0000","last_became_peered":"2024-05-09T04:49:30.060680+0000","last_unstale":"2024-05-09T04:54:55.206777+0000","last_undegraded":"2024-05-09T04:54:55.206777+0000","last_fullsized":"2024-05-09T04:54:55.206777+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:45:44.649124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.551743+0000","last_change":"2024-05-09T04:49:34.130319+0000","last_active":"2024-05-09T04:52:55.551743+0000","last_peered":"2024-05-09T04:52:55.551743+0000","last_clean":"2024-05-09T04:52:55.551743+0000","last_became_active":"2024-05-09T04:49:34.129610+0000","last_became_peered":"2024-05-09T04:49:34.129610+0000","last_unstale":"2024-05-09T04:52:55.551743+0000","last_undegraded":"2024-05-09T04:52:55.551743+0000","last_fullsized":"2024-05-09T04:52:55.551743+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:45.505221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:52.134484+0000","last_change":"2024-05-09T04:49:47.647513+0000","last_active":"2024-05-09T04:51:52.134484+0000","last_peered":"2024-05-09T04:51:52.134484+0000","last_clean":"2024-05-09T04:51:52.134484+0000","last_became_active":"2024-05-09T04:49:47.647069+0000","last_became_peered":"2024-05-09T04:49:47.647069+0000","last_unstale":"2024-05-09T04:51:52.134484+0000","last_undegraded":"2024-05-09T04:51:52.134484+0000","last_fullsized":"2024-05-09T04:51:52.134484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:05:27.658919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:41.233581+0000","last_change":"2024-05-09T04:49:30.350850+0000","last_active":"2024-05-09T04:51:41.233581+0000","last_peered":"2024-05-09T04:51:41.233581+0000","last_clean":"2024-05-09T04:51:41.233581+0000","last_became_active":"2024-05-09T04:49:30.350431+0000","last_became_peered":"2024-05-09T04:49:30.350431+0000","last_unstale":"2024-05-09T04:51:41.233581+0000","last_undegraded":"2024-05-09T04:51:41.233581+0000","last_fullsized":"2024-05-09T04:51:41.233581+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:53:05.388338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.562514+0000","last_change":"2024-05-09T04:49:34.153630+0000","last_active":"2024-05-09T04:52:55.562514+0000","last_peered":"2024-05-09T04:52:55.562514+0000","last_clean":"2024-05-09T04:52:55.562514+0000","last_became_active":"2024-05-09T04:49:34.152045+0000","last_became_peered":"2024-05-09T04:49:34.152045+0000","last_unstale":"2024-05-09T04:52:55.562514+0000","last_undegraded":"2024-05-09T04:52:55.562514+0000","last_fullsized":"2024-05-09T04:52:55.562514+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:32:35.991731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:06.236624+0000","last_change":"2024-05-09T04:49:47.901256+0000","last_active":"2024-05-09T04:52:06.236624+0000","last_peered":"2024-05-09T04:52:06.236624+0000","last_clean":"2024-05-09T04:52:06.236624+0000","last_became_active":"2024-05-09T04:49:47.900413+0000","last_became_peered":"2024-05-09T04:49:47.900413+0000","last_unstale":"2024-05-09T04:52:06.236624+0000","last_undegraded":"2024-05-09T04:52:06.236624+0000","last_fullsized":"2024-05-09T04:52:06.236624+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:46:11.845220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:42.647230+0000","last_change":"2024-05-09T04:49:30.152273+0000","last_active":"2024-05-09T04:51:42.647230+0000","last_peered":"2024-05-09T04:51:42.647230+0000","last_clean":"2024-05-09T04:51:42.647230+0000","last_became_active":"2024-05-09T04:49:30.151973+0000","last_became_peered":"2024-05-09T04:49:30.151973+0000","last_unstale":"2024-05-09T04:51:42.647230+0000","last_undegraded":"2024-05-09T04:51:42.647230+0000","last_fullsized":"2024-05-09T04:51:42.647230+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:14:52.912588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.517045+0000","last_change":"2024-05-09T04:49:34.126829+0000","last_active":"2024-05-09T04:52:55.517045+0000","last_peered":"2024-05-09T04:52:55.517045+0000","last_clean":"2024-05-09T04:52:55.517045+0000","last_became_active":"2024-05-09T04:49:34.126430+0000","last_became_peered":"2024-05-09T04:49:34.126430+0000","last_unstale":"2024-05-09T04:52:55.517045+0000","last_undegraded":"2024-05-09T04:52:55.517045+0000","last_fullsized":"2024-05-09T04:52:55.517045+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:38.247148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:02.843444+0000","last_change":"2024-05-09T04:49:47.675292+0000","last_active":"2024-05-09T04:52:02.843444+0000","last_peered":"2024-05-09T04:52:02.843444+0000","last_clean":"2024-05-09T04:52:02.843444+0000","last_became_active":"2024-05-09T04:49:47.674802+0000","last_became_peered":"2024-05-09T04:49:47.674802+0000","last_unstale":"2024-05-09T04:52:02.843444+0000","last_undegraded":"2024-05-09T04:52:02.843444+0000","last_fullsized":"2024-05-09T04:52:02.843444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:47:21.988054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:43.130050+0000","last_change":"2024-05-09T04:49:30.057854+0000","last_active":"2024-05-09T04:51:43.130050+0000","last_peered":"2024-05-09T04:51:43.130050+0000","last_clean":"2024-05-09T04:51:43.130050+0000","last_became_active":"2024-05-09T04:49:30.056625+0000","last_became_peered":"2024-05-09T04:49:30.056625+0000","last_unstale":"2024-05-09T04:51:43.130050+0000","last_undegraded":"2024-05-09T04:51:43.130050+0000","last_fullsized":"2024-05-09T04:51:43.130050+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:51.272533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.559538+0000","last_change":"2024-05-09T04:49:34.144414+0000","last_active":"2024-05-09T04:52:55.559538+0000","last_peered":"2024-05-09T04:52:55.559538+0000","last_clean":"2024-05-09T04:52:55.559538+0000","last_became_active":"2024-05-09T04:49:34.144064+0000","last_became_peered":"2024-05-09T04:49:34.144064+0000","last_unstale":"2024-05-09T04:52:55.559538+0000","last_undegraded":"2024-05-09T04:52:55.559538+0000","last_fullsized":"2024-05-09T04:52:55.559538+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:44:40.991576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:06.236580+0000","last_change":"2024-05-09T04:49:47.672956+0000","last_active":"2024-05-09T04:52:06.236580+0000","last_peered":"2024-05-09T04:52:06.236580+0000","last_clean":"2024-05-09T04:52:06.236580+0000","last_became_active":"2024-05-09T04:49:47.672513+0000","last_became_peered":"2024-05-09T04:49:47.672513+0000","last_unstale":"2024-05-09T04:52:06.236580+0000","last_undegraded":"2024-05-09T04:52:06.236580+0000","last_fullsized":"2024-05-09T04:52:06.236580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:32.250039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'4","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:54:16.975631+0000","last_change":"2024-05-09T04:49:56.704058+0000","last_active":"2024-05-09T04:54:16.975631+0000","last_peered":"2024-05-09T04:54:16.975631+0000","last_clean":"2024-05-09T04:54:16.975631+0000","last_became_active":"2024-05-09T04:49:56.703589+0000","last_became_peered":"2024-05-09T04:49:56.703589+0000","last_unstale":"2024-05-09T04:54:16.975631+0000","last_undegraded":"2024-05-09T04:54:16.975631+0000","last_fullsized":"2024-05-09T04:54:16.975631+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:54.596200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,1],"acting":[11,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.555701+0000","last_change":"2024-05-09T04:49:34.159321+0000","last_active":"2024-05-09T04:52:55.555701+0000","last_peered":"2024-05-09T04:52:55.555701+0000","last_clean":"2024-05-09T04:52:55.555701+0000","last_became_active":"2024-05-09T04:49:34.157918+0000","last_became_peered":"2024-05-09T04:49:34.157918+0000","last_unstale":"2024-05-09T04:52:55.555701+0000","last_undegraded":"2024-05-09T04:52:55.555701+0000","last_fullsized":"2024-05-09T04:52:55.555701+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:33:59.659905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:06.236605+0000","last_change":"2024-05-09T04:49:47.900692+0000","last_active":"2024-05-09T04:52:06.236605+0000","last_peered":"2024-05-09T04:52:06.236605+0000","last_clean":"2024-05-09T04:52:06.236605+0000","last_became_active":"2024-05-09T04:49:47.900095+0000","last_became_peered":"2024-05-09T04:49:47.900095+0000","last_unstale":"2024-05-09T04:52:06.236605+0000","last_undegraded":"2024-05-09T04:52:06.236605+0000","last_fullsized":"2024-05-09T04:52:06.236605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:08.632408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:45.712091+0000","last_change":"2024-05-09T04:49:30.048768+0000","last_active":"2024-05-09T04:51:45.712091+0000","last_peered":"2024-05-09T04:51:45.712091+0000","last_clean":"2024-05-09T04:51:45.712091+0000","last_became_active":"2024-05-09T04:49:30.048235+0000","last_became_peered":"2024-05-09T04:49:30.048235+0000","last_unstale":"2024-05-09T04:51:45.712091+0000","last_undegraded":"2024-05-09T04:51:45.712091+0000","last_fullsized":"2024-05-09T04:51:45.712091+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:51:16.988726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.540136+0000","last_change":"2024-05-09T04:49:34.142110+0000","last_active":"2024-05-09T04:52:55.540136+0000","last_peered":"2024-05-09T04:52:55.540136+0000","last_clean":"2024-05-09T04:52:55.540136+0000","last_became_active":"2024-05-09T04:49:34.141709+0000","last_became_peered":"2024-05-09T04:49:34.141709+0000","last_unstale":"2024-05-09T04:52:55.540136+0000","last_undegraded":"2024-05-09T04:52:55.540136+0000","last_fullsized":"2024-05-09T04:52:55.540136+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:16.365915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:57.649137+0000","last_change":"2024-05-09T04:49:47.667784+0000","last_active":"2024-05-09T04:51:57.649137+0000","last_peered":"2024-05-09T04:51:57.649137+0000","last_clean":"2024-05-09T04:51:57.649137+0000","last_became_active":"2024-05-09T04:49:47.667316+0000","last_became_peered":"2024-05-09T04:49:47.667316+0000","last_unstale":"2024-05-09T04:51:57.649137+0000","last_undegraded":"2024-05-09T04:51:57.649137+0000","last_fullsized":"2024-05-09T04:51:57.649137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:04:20.233287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'7","reported_seq":299,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:55:06.152188+0000","last_change":"2024-05-09T04:49:30.351231+0000","last_active":"2024-05-09T04:55:06.152188+0000","last_peered":"2024-05-09T04:55:06.152188+0000","last_clean":"2024-05-09T04:55:06.152188+0000","last_became_active":"2024-05-09T04:49:30.350817+0000","last_became_peered":"2024-05-09T04:49:30.350817+0000","last_unstale":"2024-05-09T04:55:06.152188+0000","last_undegraded":"2024-05-09T04:55:06.152188+0000","last_fullsized":"2024-05-09T04:55:06.152188+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:51.234300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1871,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":248,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.540495+0000","last_change":"2024-05-09T04:49:34.129625+0000","last_active":"2024-05-09T04:52:55.540495+0000","last_peered":"2024-05-09T04:52:55.540495+0000","last_clean":"2024-05-09T04:52:55.540495+0000","last_became_active":"2024-05-09T04:49:34.129215+0000","last_became_peered":"2024-05-09T04:49:34.129215+0000","last_unstale":"2024-05-09T04:52:55.540495+0000","last_undegraded":"2024-05-09T04:52:55.540495+0000","last_fullsized":"2024-05-09T04:52:55.540495+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:26.247116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:01.811085+0000","last_change":"2024-05-09T04:49:47.682819+0000","last_active":"2024-05-09T04:52:01.811085+0000","last_peered":"2024-05-09T04:52:01.811085+0000","last_clean":"2024-05-09T04:52:01.811085+0000","last_became_active":"2024-05-09T04:49:47.682331+0000","last_became_peered":"2024-05-09T04:49:47.682331+0000","last_unstale":"2024-05-09T04:52:01.811085+0000","last_undegraded":"2024-05-09T04:52:01.811085+0000","last_fullsized":"2024-05-09T04:52:01.811085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:03.852975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:45.721497+0000","last_change":"2024-05-09T04:49:30.050247+0000","last_active":"2024-05-09T04:51:45.721497+0000","last_peered":"2024-05-09T04:51:45.721497+0000","last_clean":"2024-05-09T04:51:45.721497+0000","last_became_active":"2024-05-09T04:49:30.049835+0000","last_became_peered":"2024-05-09T04:49:30.049835+0000","last_unstale":"2024-05-09T04:51:45.721497+0000","last_undegraded":"2024-05-09T04:51:45.721497+0000","last_fullsized":"2024-05-09T04:51:45.721497+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:52.591186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.559764+0000","last_change":"2024-05-09T04:49:34.145406+0000","last_active":"2024-05-09T04:52:55.559764+0000","last_peered":"2024-05-09T04:52:55.559764+0000","last_clean":"2024-05-09T04:52:55.559764+0000","last_became_active":"2024-05-09T04:49:34.144946+0000","last_became_peered":"2024-05-09T04:49:34.144946+0000","last_unstale":"2024-05-09T04:52:55.559764+0000","last_undegraded":"2024-05-09T04:52:55.559764+0000","last_fullsized":"2024-05-09T04:52:55.559764+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:43:04.704215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:05.714387+0000","last_change":"2024-05-09T04:49:56.758844+0000","last_active":"2024-05-09T04:52:05.714387+0000","last_peered":"2024-05-09T04:52:05.714387+0000","last_clean":"2024-05-09T04:52:05.714387+0000","last_became_active":"2024-05-09T04:49:56.758335+0000","last_became_peered":"2024-05-09T04:49:56.758335+0000","last_unstale":"2024-05-09T04:52:05.714387+0000","last_undegraded":"2024-05-09T04:52:05.714387+0000","last_fullsized":"2024-05-09T04:52:05.714387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:46.606035+0000","last_clean_scrub_stamp":"2024-05-09T04:49:46.606035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:39:33.344225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:51:47.850941+0000","last_change":"2024-05-09T04:49:56.766262+0000","last_active":"2024-05-09T04:51:47.850941+0000","last_peered":"2024-05-09T04:51:47.850941+0000","last_clean":"2024-05-09T04:51:47.850941+0000","last_became_active":"2024-05-09T04:49:56.758765+0000","last_became_peered":"2024-05-09T04:49:56.758765+0000","last_unstale":"2024-05-09T04:51:47.850941+0000","last_undegraded":"2024-05-09T04:51:47.850941+0000","last_fullsized":"2024-05-09T04:51:47.850941+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:29.005969+0000","last_clean_scrub_stamp":"2024-05-09T04:49:29.005969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:44:18.271729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.540919+0000","last_change":"2024-05-09T04:49:34.134957+0000","last_active":"2024-05-09T04:52:55.540919+0000","last_peered":"2024-05-09T04:52:55.540919+0000","last_clean":"2024-05-09T04:52:55.540919+0000","last_became_active":"2024-05-09T04:49:34.134544+0000","last_became_peered":"2024-05-09T04:49:34.134544+0000","last_unstale":"2024-05-09T04:52:55.540919+0000","last_undegraded":"2024-05-09T04:52:55.540919+0000","last_fullsized":"2024-05-09T04:52:55.540919+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:21:54.855905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.525564+0000","last_change":"2024-05-09T04:49:34.138112+0000","last_active":"2024-05-09T04:52:55.525564+0000","last_peered":"2024-05-09T04:52:55.525564+0000","last_clean":"2024-05-09T04:52:55.525564+0000","last_became_active":"2024-05-09T04:49:34.137716+0000","last_became_peered":"2024-05-09T04:49:34.137716+0000","last_unstale":"2024-05-09T04:52:55.525564+0000","last_undegraded":"2024-05-09T04:52:55.525564+0000","last_fullsized":"2024-05-09T04:52:55.525564+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:17.028262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.534478+0000","last_change":"2024-05-09T04:50:58.204287+0000","last_active":"2024-05-09T04:52:55.534478+0000","last_peered":"2024-05-09T04:52:55.534478+0000","last_clean":"2024-05-09T04:52:55.534478+0000","last_became_active":"2024-05-09T04:50:58.203894+0000","last_became_peered":"2024-05-09T04:50:58.203894+0000","last_unstale":"2024-05-09T04:52:55.534478+0000","last_undegraded":"2024-05-09T04:52:55.534478+0000","last_fullsized":"2024-05-09T04:52:55.534478+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:02.214347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.558687+0000","last_change":"2024-05-09T04:49:34.147742+0000","last_active":"2024-05-09T04:52:55.558687+0000","last_peered":"2024-05-09T04:52:55.558687+0000","last_clean":"2024-05-09T04:52:55.558687+0000","last_became_active":"2024-05-09T04:49:34.147301+0000","last_became_peered":"2024-05-09T04:49:34.147301+0000","last_unstale":"2024-05-09T04:52:55.558687+0000","last_undegraded":"2024-05-09T04:52:55.558687+0000","last_fullsized":"2024-05-09T04:52:55.558687+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:36:55.371212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.552970+0000","last_change":"2024-05-09T04:49:34.163599+0000","last_active":"2024-05-09T04:52:55.552970+0000","last_peered":"2024-05-09T04:52:55.552970+0000","last_clean":"2024-05-09T04:52:55.552970+0000","last_became_active":"2024-05-09T04:49:34.162898+0000","last_became_peered":"2024-05-09T04:49:34.162898+0000","last_unstale":"2024-05-09T04:52:55.552970+0000","last_undegraded":"2024-05-09T04:52:55.552970+0000","last_fullsized":"2024-05-09T04:52:55.552970+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:55:57.213901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.552433+0000","last_change":"2024-05-09T04:49:34.151648+0000","last_active":"2024-05-09T04:52:55.552433+0000","last_peered":"2024-05-09T04:52:55.552433+0000","last_clean":"2024-05-09T04:52:55.552433+0000","last_became_active":"2024-05-09T04:49:34.148701+0000","last_became_peered":"2024-05-09T04:49:34.148701+0000","last_unstale":"2024-05-09T04:52:55.552433+0000","last_undegraded":"2024-05-09T04:52:55.552433+0000","last_fullsized":"2024-05-09T04:52:55.552433+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:54:30.122252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.521268+0000","last_change":"2024-05-09T04:49:34.136679+0000","last_active":"2024-05-09T04:52:55.521268+0000","last_peered":"2024-05-09T04:52:55.521268+0000","last_clean":"2024-05-09T04:52:55.521268+0000","last_became_active":"2024-05-09T04:49:34.136306+0000","last_became_peered":"2024-05-09T04:49:34.136306+0000","last_unstale":"2024-05-09T04:52:55.521268+0000","last_undegraded":"2024-05-09T04:52:55.521268+0000","last_fullsized":"2024-05-09T04:52:55.521268+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:51:39.476564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.539107+0000","last_change":"2024-05-09T04:49:34.130769+0000","last_active":"2024-05-09T04:52:55.539107+0000","last_peered":"2024-05-09T04:52:55.539107+0000","last_clean":"2024-05-09T04:52:55.539107+0000","last_became_active":"2024-05-09T04:49:34.130362+0000","last_became_peered":"2024-05-09T04:49:34.130362+0000","last_unstale":"2024-05-09T04:52:55.539107+0000","last_undegraded":"2024-05-09T04:52:55.539107+0000","last_fullsized":"2024-05-09T04:52:55.539107+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:54.942259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.552538+0000","last_change":"2024-05-09T04:49:34.138054+0000","last_active":"2024-05-09T04:52:55.552538+0000","last_peered":"2024-05-09T04:52:55.552538+0000","last_clean":"2024-05-09T04:52:55.552538+0000","last_became_active":"2024-05-09T04:49:34.136111+0000","last_became_peered":"2024-05-09T04:49:34.136111+0000","last_unstale":"2024-05-09T04:52:55.552538+0000","last_undegraded":"2024-05-09T04:52:55.552538+0000","last_fullsized":"2024-05-09T04:52:55.552538+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:15:27.870874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.562318+0000","last_change":"2024-05-09T04:49:34.146370+0000","last_active":"2024-05-09T04:52:55.562318+0000","last_peered":"2024-05-09T04:52:55.562318+0000","last_clean":"2024-05-09T04:52:55.562318+0000","last_became_active":"2024-05-09T04:49:34.145948+0000","last_became_peered":"2024-05-09T04:49:34.145948+0000","last_unstale":"2024-05-09T04:52:55.562318+0000","last_undegraded":"2024-05-09T04:52:55.562318+0000","last_fullsized":"2024-05-09T04:52:55.562318+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:14.826168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.555403+0000","last_change":"2024-05-09T04:49:34.138028+0000","last_active":"2024-05-09T04:52:55.555403+0000","last_peered":"2024-05-09T04:52:55.555403+0000","last_clean":"2024-05-09T04:52:55.555403+0000","last_became_active":"2024-05-09T04:49:34.137558+0000","last_became_peered":"2024-05-09T04:49:34.137558+0000","last_unstale":"2024-05-09T04:52:55.555403+0000","last_undegraded":"2024-05-09T04:52:55.555403+0000","last_fullsized":"2024-05-09T04:52:55.555403+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:50:16.258945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.556779+0000","last_change":"2024-05-09T04:49:34.137946+0000","last_active":"2024-05-09T04:52:55.556779+0000","last_peered":"2024-05-09T04:52:55.556779+0000","last_clean":"2024-05-09T04:52:55.556779+0000","last_became_active":"2024-05-09T04:49:34.137498+0000","last_became_peered":"2024-05-09T04:49:34.137498+0000","last_unstale":"2024-05-09T04:52:55.556779+0000","last_undegraded":"2024-05-09T04:52:55.556779+0000","last_fullsized":"2024-05-09T04:52:55.556779+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:36:54.473689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.552098+0000","last_change":"2024-05-09T04:49:34.162103+0000","last_active":"2024-05-09T04:52:55.552098+0000","last_peered":"2024-05-09T04:52:55.552098+0000","last_clean":"2024-05-09T04:52:55.552098+0000","last_became_active":"2024-05-09T04:49:34.161681+0000","last_became_peered":"2024-05-09T04:49:34.161681+0000","last_unstale":"2024-05-09T04:52:55.552098+0000","last_undegraded":"2024-05-09T04:52:55.552098+0000","last_fullsized":"2024-05-09T04:52:55.552098+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:36:10.460401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-09T04:52:55.529070+0000","last_change":"2024-05-09T04:49:34.136538+0000","last_active":"2024-05-09T04:52:55.529070+0000","last_peered":"2024-05-09T04:52:55.529070+0000","last_clean":"2024-05-09T04:52:55.529070+0000","last_became_active":"2024-05-09T04:49:34.134903+0000","last_became_peered":"2024-05-09T04:49:34.134903+0000","last_unstale":"2024-05-09T04:52:55.529070+0000","last_undegraded":"2024-05-09T04:52:55.529070+0000","last_fullsized":"2024-05-09T04:52:55.529070+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:49:33.069858+0000","last_clean_scrub_stamp":"2024-05-09T04:49:33.069858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:44:48.673600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":81661,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":2468,"num_read_kb":1322,"num_write":275,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":602112,"data_stored":244983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":237,"ondisk_log_size":237,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678100,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":510952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874330,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37024,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708128,"statfs":{"total":95995035648,"available":95957123072,"internally_reserved":0,"allocated":1728512,"data_stored":1170570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037854,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":514165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234087,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":510335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430315,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1642496,"data_stored":1108008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593841,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":505573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574992,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":579686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738516,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":507730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411468,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1122304,"data_stored":581530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149827,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":505281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953599,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":510179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790072,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1650688,"data_stored":1103058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":74835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":76679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":75351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T04:55:09.960 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:55:10.478 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:55:11.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:10 smithi006 ceph-mon[29835]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:55:11.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:10 smithi006 ceph-mon[29835]: pgmap v506: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-09T04:55:11.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:10 smithi186 ceph-mon[34668]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:55:11.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:10 smithi186 ceph-mon[34668]: pgmap v506: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-09T04:55:11.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:10 smithi169 ceph-mon[34075]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:55:11.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:10 smithi169 ceph-mon[34075]: pgmap v506: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-09T04:55:12.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.810+0000 7f89d9800700 1 -- 172.21.15.6:0/3475261892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89d41039d0 msgr2=0x7f89d4107f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:12.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.810+0000 7f89d9800700 1 --2- 172.21.15.6:0/3475261892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89d41039d0 0x7f89d4107f00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009990 tx=0x7f89c0009ca0 comp rx=0 tx=0).stop 2024-05-09T04:55:12.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.810+0000 7f89d9800700 1 -- 172.21.15.6:0/3475261892 shutdown_connections 2024-05-09T04:55:12.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.810+0000 7f89d9800700 1 --2- 172.21.15.6:0/3475261892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89d41084d0 0x7f89d41089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:12.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.810+0000 7f89d9800700 1 --2- 172.21.15.6:0/3475261892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89d4103080 0x7f89d4103490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:12.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.810+0000 7f89d9800700 1 --2- 172.21.15.6:0/3475261892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89d41039d0 0x7f89d4107f00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:12.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.810+0000 7f89d9800700 1 -- 172.21.15.6:0/3475261892 >> 172.21.15.6:0/3475261892 conn(0x7f89d4078ac0 msgr2=0x7f89d4078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:12.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.810+0000 7f89d9800700 1 -- 172.21.15.6:0/3475261892 shutdown_connections 2024-05-09T04:55:12.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.810+0000 7f89d9800700 1 -- 172.21.15.6:0/3475261892 wait complete. 2024-05-09T04:55:12.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.812+0000 7f89d9800700 1 Processor -- start 2024-05-09T04:55:12.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.813+0000 7f89d9800700 1 -- start start 2024-05-09T04:55:12.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.814+0000 7f89d9800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89d4103080 0x7f89d41a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:12.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.814+0000 7f89d9800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89d41039d0 0x7f89d41a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:12.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.814+0000 7f89d9800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89d41084d0 0x7f89d41a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:12.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.814+0000 7f89d9800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f89d411a860 con 0x7f89d41084d0 2024-05-09T04:55:12.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.814+0000 7f89d9800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f89d411a6e0 con 0x7f89d41039d0 2024-05-09T04:55:12.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.814+0000 7f89d9800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f89d411a9e0 con 0x7f89d4103080 2024-05-09T04:55:12.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.814+0000 7f89d8e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89d41084d0 0x7f89d41a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:12.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.814+0000 7f89d3e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89d4103080 0x7f89d41a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:12.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.814+0000 7f89d3e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89d4103080 0x7f89d41a1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:40714/0 (socket says 172.21.15.6:40714) 2024-05-09T04:55:12.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.814+0000 7f89d8e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89d41084d0 0x7f89d41a5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45408/0 (socket says 172.21.15.6:45408) 2024-05-09T04:55:12.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.814+0000 7f89d3400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89d41039d0 0x7f89d41a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:12.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.814+0000 7f89d3e00700 1 -- 172.21.15.6:0/1131929214 learned_addr learned my addr 172.21.15.6:0/1131929214 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:55:12.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.815+0000 7f89d3e00700 1 -- 172.21.15.6:0/1131929214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89d41039d0 msgr2=0x7f89d41a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:12.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.815+0000 7f89d3e00700 1 --2- 172.21.15.6:0/1131929214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89d41039d0 0x7f89d41a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:12.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.815+0000 7f89d3e00700 1 -- 172.21.15.6:0/1131929214 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89d41084d0 msgr2=0x7f89d41a5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:12.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.815+0000 7f89d3e00700 1 --2- 172.21.15.6:0/1131929214 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89d41084d0 0x7f89d41a5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:12.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.815+0000 7f89d3e00700 1 -- 172.21.15.6:0/1131929214 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d41a6100 con 0x7f89d4103080 2024-05-09T04:55:12.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.815+0000 7f89d8e00700 1 --2- 172.21.15.6:0/1131929214 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89d41084d0 0x7f89d41a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:55:12.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.815+0000 7f89d3e00700 1 --2- 172.21.15.6:0/1131929214 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89d4103080 0x7f89d41a1230 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f89c400e9a0 tx=0x7f89c400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:12.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.815+0000 7f89d0c00700 1 -- 172.21.15.6:0/1131929214 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c400ca10 con 0x7f89d4103080 2024-05-09T04:55:12.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.816+0000 7f89d9800700 1 -- 172.21.15.6:0/1131929214 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89d41a63f0 con 0x7f89d4103080 2024-05-09T04:55:12.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.816+0000 7f89d0c00700 1 -- 172.21.15.6:0/1131929214 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89c400cbb0 con 0x7f89d4103080 2024-05-09T04:55:12.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.816+0000 7f89d0c00700 1 -- 172.21.15.6:0/1131929214 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c4018680 con 0x7f89d4103080 2024-05-09T04:55:12.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.816+0000 7f89d9800700 1 -- 172.21.15.6:0/1131929214 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89d41add10 con 0x7f89d4103080 2024-05-09T04:55:12.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.817+0000 7f89d9800700 1 -- 172.21.15.6:0/1131929214 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89d40512c0 con 0x7f89d4103080 2024-05-09T04:55:12.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.819+0000 7f89d0c00700 1 -- 172.21.15.6:0/1131929214 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 19) ==== 149702+0+0 (secure 0 0 0) 0x7f89c4010430 con 0x7f89d4103080 2024-05-09T04:55:12.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.821+0000 7f89d0c00700 1 --2- 172.21.15.6:0/1131929214 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f89b80b2910 0x7f89b80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:12.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.821+0000 7f89d0c00700 1 -- 172.21.15.6:0/1131929214 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f89c40e28d0 con 0x7f89d4103080 2024-05-09T04:55:12.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.821+0000 7f89d3400700 1 --2- 172.21.15.6:0/1131929214 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f89b80b2910 0x7f89b80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:12.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.822+0000 7f89d3400700 1 --2- 172.21.15.6:0/1131929214 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f89b80b2910 0x7f89b80b4e20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f89c00098e0 tx=0x7f89c0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:12.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:12.826+0000 7f89d0c00700 1 -- 172.21.15.6:0/1131929214 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f89c40aa0b0 con 0x7f89d4103080 2024-05-09T04:55:13.116 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:12 smithi006 ceph-mon[29835]: pgmap v507: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-09T04:55:13.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:13.115+0000 7f89d9800700 1 -- 172.21.15.6:0/1131929214 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89d41adee0 con 0x7f89d4103080 2024-05-09T04:55:13.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:13.117+0000 7f89d0c00700 1 -- 172.21.15.6:0/1131929214 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11927 (secure 0 0 0) 0x7f89c40b2130 con 0x7f89d4103080 2024-05-09T04:55:13.117 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:55:13.117 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/162094599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":162094599},{"type":"v1","addr":"172.21.15.6:6835","nonce":162094599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/589294363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":589294363},{"type":"v1","addr":"172.21.15.6:6841","nonce":589294363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:49:36.920376+0000","modified":"2024-05-09T04:54:56.247431+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24554,"mds_1":24530,"mds_2":24512,"mds_3":24511,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24442":{"gid":24442,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6833/2977145206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2977145206},{"type":"v1","addr":"172.21.15.186:6833","nonce":2977145206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6833/133345948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":133345948},{"type":"v1","addr":"172.21.15.169:6833","nonce":133345948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.6:6837/1281521088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":1281521088},{"type":"v1","addr":"172.21.15.6:6837","nonce":1281521088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2268322007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2268322007},{"type":"v1","addr":"172.21.15.186:6835","nonce":2268322007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/3811005052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3811005052},{"type":"v1","addr":"172.21.15.169:6835","nonce":3811005052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.186:6837/1647607118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1647607118},{"type":"v1","addr":"172.21.15.186:6837","nonce":1647607118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":32,"addr":"172.21.15.186:6839/3954883105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":3954883105},{"type":"v1","addr":"172.21.15.186:6839","nonce":3954883105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.169:6837/1845715294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1845715294},{"type":"v1","addr":"172.21.15.169:6837","nonce":1845715294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.6:6839/2780306492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":2780306492},{"type":"v1","addr":"172.21.15.6:6839","nonce":2780306492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.169:6839/614388771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":614388771},{"type":"v1","addr":"172.21.15.169:6839","nonce":614388771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24512,24554,24530,24493,24511]},"id":1}]} 2024-05-09T04:55:13.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:13.121+0000 7f89d9800700 1 -- 172.21.15.6:0/1131929214 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f89b80b2910 msgr2=0x7f89b80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:13.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:13.121+0000 7f89d9800700 1 --2- 172.21.15.6:0/1131929214 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f89b80b2910 0x7f89b80b4e20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f89c00098e0 tx=0x7f89c0005960 comp rx=0 tx=0).stop 2024-05-09T04:55:13.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:13.121+0000 7f89d9800700 1 -- 172.21.15.6:0/1131929214 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89d4103080 msgr2=0x7f89d41a1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:13.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:13.121+0000 7f89d9800700 1 --2- 172.21.15.6:0/1131929214 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89d4103080 0x7f89d41a1230 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f89c400e9a0 tx=0x7f89c400ecb0 comp rx=0 tx=0).stop 2024-05-09T04:55:13.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:13.121+0000 7f89d9800700 1 -- 172.21.15.6:0/1131929214 shutdown_connections 2024-05-09T04:55:13.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:13.121+0000 7f89d9800700 1 --2- 172.21.15.6:0/1131929214 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89d41084d0 0x7f89d41a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:13.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:13.121+0000 7f89d9800700 1 --2- 172.21.15.6:0/1131929214 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89d4103080 0x7f89d41a1230 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:13.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:13.121+0000 7f89d9800700 1 --2- 172.21.15.6:0/1131929214 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f89b80b2910 0x7f89b80b4e20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:13.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:13.121+0000 7f89d9800700 1 --2- 172.21.15.6:0/1131929214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89d41039d0 0x7f89d41a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:13.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:13.121+0000 7f89d9800700 1 -- 172.21.15.6:0/1131929214 >> 172.21.15.6:0/1131929214 conn(0x7f89d4078ac0 msgr2=0x7f89d4105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:13.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:13.121+0000 7f89d9800700 1 -- 172.21.15.6:0/1131929214 shutdown_connections 2024-05-09T04:55:13.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:13.121+0000 7f89d9800700 1 -- 172.21.15.6:0/1131929214 wait complete. 2024-05-09T04:55:13.123 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 30 2024-05-09T04:55:13.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:12 smithi186 ceph-mon[34668]: pgmap v507: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-09T04:55:13.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:12 smithi169 ceph-mon[34075]: pgmap v507: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-09T04:55:14.148 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:55:14.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:13 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1131929214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:55:14.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:13 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/1131929214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:55:14.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:13 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/1131929214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:55:14.605 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:55:15.066 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:14 smithi006 ceph-mon[29835]: pgmap v508: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-05-09T04:55:15.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:14 smithi186 ceph-mon[34668]: pgmap v508: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-05-09T04:55:15.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:14 smithi169 ceph-mon[34075]: pgmap v508: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-05-09T04:55:16.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.898+0000 7f3e43e00700 1 -- 172.21.15.6:0/193649205 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e44115180 msgr2=0x7f3e441175a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:16.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.898+0000 7f3e43e00700 1 --2- 172.21.15.6:0/193649205 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e44115180 0x7f3e441175a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c00b0a0 tx=0x7f3e3c00b3b0 comp rx=0 tx=0).stop 2024-05-09T04:55:16.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.898+0000 7f3e43e00700 1 -- 172.21.15.6:0/193649205 shutdown_connections 2024-05-09T04:55:16.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.898+0000 7f3e43e00700 1 --2- 172.21.15.6:0/193649205 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e44115180 0x7f3e441175a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:16.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.898+0000 7f3e43e00700 1 --2- 172.21.15.6:0/193649205 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e44075b60 0x7f3e44075f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:16.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.898+0000 7f3e43e00700 1 --2- 172.21.15.6:0/193649205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e440764b0 0x7f3e44114c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:16.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.898+0000 7f3e43e00700 1 -- 172.21.15.6:0/193649205 >> 172.21.15.6:0/193649205 conn(0x7f3e44079d10 msgr2=0x7f3e4407a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:16.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.898+0000 7f3e43e00700 1 -- 172.21.15.6:0/193649205 shutdown_connections 2024-05-09T04:55:16.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.898+0000 7f3e43e00700 1 -- 172.21.15.6:0/193649205 wait complete. 2024-05-09T04:55:16.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.900+0000 7f3e43e00700 1 Processor -- start 2024-05-09T04:55:16.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.900+0000 7f3e43e00700 1 -- start start 2024-05-09T04:55:16.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.901+0000 7f3e43e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e44075b60 0x7f3e441a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:16.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.901+0000 7f3e43e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e440764b0 0x7f3e441a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:16.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.901+0000 7f3e43e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e44115180 0x7f3e441a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:16.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.901+0000 7f3e43e00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4411a8e0 con 0x7f3e44075b60 2024-05-09T04:55:16.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.901+0000 7f3e43e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4411a760 con 0x7f3e44115180 2024-05-09T04:55:16.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.901+0000 7f3e43e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4411aa60 con 0x7f3e440764b0 2024-05-09T04:55:16.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.901+0000 7f3e42a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e44075b60 0x7f3e441a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:16.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.901+0000 7f3e42000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e440764b0 0x7f3e441a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:16.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.901+0000 7f3e42000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e440764b0 0x7f3e441a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:37370/0 (socket says 172.21.15.6:37370) 2024-05-09T04:55:16.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.901+0000 7f3e42000700 1 -- 172.21.15.6:0/292893448 learned_addr learned my addr 172.21.15.6:0/292893448 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:55:16.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.901+0000 7f3e42a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e44075b60 0x7f3e441a11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39354/0 (socket says 172.21.15.6:39354) 2024-05-09T04:55:16.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.901+0000 7f3e43400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e44115180 0x7f3e441a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:16.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.902+0000 7f3e42000700 1 -- 172.21.15.6:0/292893448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e44115180 msgr2=0x7f3e441a59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:16.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.902+0000 7f3e42000700 1 --2- 172.21.15.6:0/292893448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e44115180 0x7f3e441a59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:16.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.902+0000 7f3e42000700 1 -- 172.21.15.6:0/292893448 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e44075b60 msgr2=0x7f3e441a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:16.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.902+0000 7f3e42000700 1 --2- 172.21.15.6:0/292893448 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e44075b60 0x7f3e441a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:16.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.902+0000 7f3e42000700 1 -- 172.21.15.6:0/292893448 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e441a6080 con 0x7f3e440764b0 2024-05-09T04:55:16.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.902+0000 7f3e42000700 1 --2- 172.21.15.6:0/292893448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e440764b0 0x7f3e441a1750 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000e9a0 tx=0x7f3e3000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:16.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.902+0000 7f3e3b400700 1 -- 172.21.15.6:0/292893448 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3000ca10 con 0x7f3e440764b0 2024-05-09T04:55:16.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.903+0000 7f3e43e00700 1 -- 172.21.15.6:0/292893448 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e441a6370 con 0x7f3e440764b0 2024-05-09T04:55:16.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.903+0000 7f3e3b400700 1 -- 172.21.15.6:0/292893448 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e3000cbb0 con 0x7f3e440764b0 2024-05-09T04:55:16.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.903+0000 7f3e3b400700 1 -- 172.21.15.6:0/292893448 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e30018680 con 0x7f3e440764b0 2024-05-09T04:55:16.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.903+0000 7f3e43e00700 1 -- 172.21.15.6:0/292893448 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e441adc90 con 0x7f3e440764b0 2024-05-09T04:55:16.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.904+0000 7f3e43e00700 1 -- 172.21.15.6:0/292893448 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e440512c0 con 0x7f3e440764b0 2024-05-09T04:55:16.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.907+0000 7f3e3b400700 1 -- 172.21.15.6:0/292893448 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 19) ==== 149702+0+0 (secure 0 0 0) 0x7f3e30010430 con 0x7f3e440764b0 2024-05-09T04:55:16.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.909+0000 7f3e3b400700 1 --2- 172.21.15.6:0/292893448 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f3e280b2990 0x7f3e280b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:16.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.909+0000 7f3e3b400700 1 -- 172.21.15.6:0/292893448 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f3e300e2b70 con 0x7f3e440764b0 2024-05-09T04:55:16.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.909+0000 7f3e42a00700 1 --2- 172.21.15.6:0/292893448 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f3e280b2990 0x7f3e280b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:16.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.910+0000 7f3e3b400700 1 -- 172.21.15.6:0/292893448 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3e300aa350 con 0x7f3e440764b0 2024-05-09T04:55:16.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:16.910+0000 7f3e42a00700 1 --2- 172.21.15.6:0/292893448 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f3e280b2990 0x7f3e280b4ea0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3e34009c60 tx=0x7f3e34009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:17.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:16 smithi006 ceph-mon[29835]: pgmap v509: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T04:55:17.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:17.200+0000 7f3e43e00700 1 -- 172.21.15.6:0/292893448 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e4402e5f0 con 0x7f3e440764b0 2024-05-09T04:55:17.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:17.201+0000 7f3e3b400700 1 -- 172.21.15.6:0/292893448 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11927 (secure 0 0 0) 0x7f3e300b23d0 con 0x7f3e440764b0 2024-05-09T04:55:17.201 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:55:17.201 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/162094599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":162094599},{"type":"v1","addr":"172.21.15.6:6835","nonce":162094599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/589294363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":589294363},{"type":"v1","addr":"172.21.15.6:6841","nonce":589294363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:49:36.920376+0000","modified":"2024-05-09T04:54:56.247431+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24554,"mds_1":24530,"mds_2":24512,"mds_3":24511,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24442":{"gid":24442,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6833/2977145206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2977145206},{"type":"v1","addr":"172.21.15.186:6833","nonce":2977145206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6833/133345948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":133345948},{"type":"v1","addr":"172.21.15.169:6833","nonce":133345948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.6:6837/1281521088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":1281521088},{"type":"v1","addr":"172.21.15.6:6837","nonce":1281521088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2268322007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2268322007},{"type":"v1","addr":"172.21.15.186:6835","nonce":2268322007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/3811005052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3811005052},{"type":"v1","addr":"172.21.15.169:6835","nonce":3811005052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.186:6837/1647607118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1647607118},{"type":"v1","addr":"172.21.15.186:6837","nonce":1647607118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":32,"addr":"172.21.15.186:6839/3954883105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":3954883105},{"type":"v1","addr":"172.21.15.186:6839","nonce":3954883105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.169:6837/1845715294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1845715294},{"type":"v1","addr":"172.21.15.169:6837","nonce":1845715294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.6:6839/2780306492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":2780306492},{"type":"v1","addr":"172.21.15.6:6839","nonce":2780306492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.169:6839/614388771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":614388771},{"type":"v1","addr":"172.21.15.169:6839","nonce":614388771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24512,24554,24530,24493,24511]},"id":1}]} 2024-05-09T04:55:17.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:17.204+0000 7f3e43e00700 1 -- 172.21.15.6:0/292893448 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f3e280b2990 msgr2=0x7f3e280b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:17.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:17.204+0000 7f3e43e00700 1 --2- 172.21.15.6:0/292893448 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f3e280b2990 0x7f3e280b4ea0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3e34009c60 tx=0x7f3e34009260 comp rx=0 tx=0).stop 2024-05-09T04:55:17.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:17.204+0000 7f3e43e00700 1 -- 172.21.15.6:0/292893448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e440764b0 msgr2=0x7f3e441a1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:17.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:17.204+0000 7f3e43e00700 1 --2- 172.21.15.6:0/292893448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e440764b0 0x7f3e441a1750 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000e9a0 tx=0x7f3e3000ecb0 comp rx=0 tx=0).stop 2024-05-09T04:55:17.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:17.204+0000 7f3e43e00700 1 -- 172.21.15.6:0/292893448 shutdown_connections 2024-05-09T04:55:17.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:17.204+0000 7f3e43e00700 1 --2- 172.21.15.6:0/292893448 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f3e280b2990 0x7f3e280b4ea0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:17.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:17.204+0000 7f3e43e00700 1 --2- 172.21.15.6:0/292893448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e44115180 0x7f3e441a59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:17.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:17.204+0000 7f3e43e00700 1 --2- 172.21.15.6:0/292893448 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3e44075b60 0x7f3e441a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:17.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:17.204+0000 7f3e43e00700 1 --2- 172.21.15.6:0/292893448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e440764b0 0x7f3e441a1750 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:17.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:17.204+0000 7f3e43e00700 1 -- 172.21.15.6:0/292893448 >> 172.21.15.6:0/292893448 conn(0x7f3e44079d10 msgr2=0x7f3e44100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:17.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:17.205+0000 7f3e43e00700 1 -- 172.21.15.6:0/292893448 shutdown_connections 2024-05-09T04:55:17.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:17.205+0000 7f3e43e00700 1 -- 172.21.15.6:0/292893448 wait complete. 2024-05-09T04:55:17.206 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 30 2024-05-09T04:55:17.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:16 smithi186 ceph-mon[34668]: pgmap v509: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T04:55:17.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:16 smithi169 ceph-mon[34075]: pgmap v509: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T04:55:18.130 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T04:55:18.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:17 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/292893448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:55:18.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:17 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/292893448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:55:18.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:17 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/292893448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:55:18.577 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:55:19.057 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:18 smithi006 ceph-mon[29835]: pgmap v510: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:55:19.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:18 smithi186 ceph-mon[34668]: pgmap v510: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:55:19.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:18 smithi169 ceph-mon[34075]: pgmap v510: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:55:20.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.891+0000 7fc7d6400700 1 -- 172.21.15.6:0/1561579520 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7d0109680 msgr2=0x7fc7d0110fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:20.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.891+0000 7fc7d6400700 1 --2- 172.21.15.6:0/1561579520 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7d0109680 0x7fc7d0110fe0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc7c80099c0 tx=0x7fc7c8009cd0 comp rx=0 tx=0).stop 2024-05-09T04:55:20.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.891+0000 7fc7d6400700 1 -- 172.21.15.6:0/1561579520 shutdown_connections 2024-05-09T04:55:20.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.891+0000 7fc7d6400700 1 --2- 172.21.15.6:0/1561579520 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7d0109680 0x7fc7d0110fe0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:20.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.891+0000 7fc7d6400700 1 --2- 172.21.15.6:0/1561579520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7d0104400 0x7fc7d0104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:20.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.891+0000 7fc7d6400700 1 --2- 172.21.15.6:0/1561579520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7d0108d30 0x7fc7d0109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:20.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.891+0000 7fc7d6400700 1 -- 172.21.15.6:0/1561579520 >> 172.21.15.6:0/1561579520 conn(0x7fc7d0079d10 msgr2=0x7fc7d007a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:20.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.892+0000 7fc7d6400700 1 -- 172.21.15.6:0/1561579520 shutdown_connections 2024-05-09T04:55:20.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.892+0000 7fc7d6400700 1 -- 172.21.15.6:0/1561579520 wait complete. 2024-05-09T04:55:20.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.893+0000 7fc7d6400700 1 Processor -- start 2024-05-09T04:55:20.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.893+0000 7fc7d6400700 1 -- start start 2024-05-09T04:55:20.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.893+0000 7fc7d6400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7d0104400 0x7fc7d01a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:20.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.893+0000 7fc7d6400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7d0108d30 0x7fc7d01a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:20.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.893+0000 7fc7d6400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7d0109680 0x7fc7d019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:20.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.893+0000 7fc7d6400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d01142b0 con 0x7fc7d0109680 2024-05-09T04:55:20.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.893+0000 7fc7d6400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d0114130 con 0x7fc7d0104400 2024-05-09T04:55:20.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.893+0000 7fc7d6400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d0114430 con 0x7fc7d0108d30 2024-05-09T04:55:20.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.894+0000 7fc7d5a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7d0109680 0x7fc7d019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:20.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.894+0000 7fc7d5a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7d0109680 0x7fc7d019a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39380/0 (socket says 172.21.15.6:39380) 2024-05-09T04:55:20.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.894+0000 7fc7d5a00700 1 -- 172.21.15.6:0/1689451405 learned_addr learned my addr 172.21.15.6:0/1689451405 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:55:20.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.894+0000 7fc7c7e00700 1 --2- 172.21.15.6:0/1689451405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7d0108d30 0x7fc7d01a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:20.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.894+0000 7fc7d5000700 1 --2- 172.21.15.6:0/1689451405 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7d0104400 0x7fc7d01a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:20.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.894+0000 7fc7d5a00700 1 -- 172.21.15.6:0/1689451405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7d0108d30 msgr2=0x7fc7d01a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:20.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.894+0000 7fc7d5a00700 1 --2- 172.21.15.6:0/1689451405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7d0108d30 0x7fc7d01a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:20.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.894+0000 7fc7d5a00700 1 -- 172.21.15.6:0/1689451405 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7d0104400 msgr2=0x7fc7d01a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:20.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.894+0000 7fc7d5a00700 1 --2- 172.21.15.6:0/1689451405 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7d0104400 0x7fc7d01a07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:20.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.894+0000 7fc7d5a00700 1 -- 172.21.15.6:0/1689451405 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7d019b0a0 con 0x7fc7d0109680 2024-05-09T04:55:20.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.894+0000 7fc7d5000700 1 --2- 172.21.15.6:0/1689451405 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7d0104400 0x7fc7d01a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:55:20.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.895+0000 7fc7d5a00700 1 --2- 172.21.15.6:0/1689451405 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7d0109680 0x7fc7d019a9f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8005910 tx=0x7fc7c80055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:20.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.895+0000 7fc7ce000700 1 -- 172.21.15.6:0/1689451405 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8026070 con 0x7fc7d0109680 2024-05-09T04:55:20.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.895+0000 7fc7d6400700 1 -- 172.21.15.6:0/1689451405 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7d019b330 con 0x7fc7d0109680 2024-05-09T04:55:20.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.895+0000 7fc7ce000700 1 -- 172.21.15.6:0/1689451405 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7c800f460 con 0x7fc7d0109680 2024-05-09T04:55:20.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.895+0000 7fc7d6400700 1 -- 172.21.15.6:0/1689451405 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7d01a7660 con 0x7fc7d0109680 2024-05-09T04:55:20.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.896+0000 7fc7ce000700 1 -- 172.21.15.6:0/1689451405 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8030060 con 0x7fc7d0109680 2024-05-09T04:55:20.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.896+0000 7fc7d6400700 1 -- 172.21.15.6:0/1689451405 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7d00512c0 con 0x7fc7d0109680 2024-05-09T04:55:20.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.899+0000 7fc7ce000700 1 -- 172.21.15.6:0/1689451405 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) ==== 149702+0+0 (secure 0 0 0) 0x7fc7c8029070 con 0x7fc7d0109680 2024-05-09T04:55:20.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.901+0000 7fc7ce000700 1 --2- 172.21.15.6:0/1689451405 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fc7b40b2910 0x7fc7b40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:20.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.901+0000 7fc7ce000700 1 -- 172.21.15.6:0/1689451405 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fc7c80e3350 con 0x7fc7d0109680 2024-05-09T04:55:20.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.901+0000 7fc7d5000700 1 --2- 172.21.15.6:0/1689451405 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fc7b40b2910 0x7fc7b40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:20.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.902+0000 7fc7d5000700 1 --2- 172.21.15.6:0/1689451405 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fc7b40b2910 0x7fc7b40b4e20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc7d019bb30 tx=0x7fc7c0009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:20.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:20.906+0000 7fc7ce000700 1 -- 172.21.15.6:0/1689451405 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc7c8013070 con 0x7fc7d0109680 2024-05-09T04:55:21.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:21.145+0000 7fc7d6400700 1 -- 172.21.15.6:0/1689451405 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc7d01a7830 con 0x7fc7d0109680 2024-05-09T04:55:21.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:21.150+0000 7fc7ce000700 1 -- 172.21.15.6:0/1689451405 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24659 (secure 0 0 0) 0x7fc7c80aabc0 con 0x7fc7d0109680 2024-05-09T04:55:21.150 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:55:21.150 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:20 smithi006 ceph-mon[29835]: pgmap v511: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:55:21.151 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":89,"fsid":"f74d813c-0dbd-11ef-bc97-c7b262605968","created":"2024-05-09T04:39:35.287470+0000","modified":"2024-05-09T04:50:57.686719+0000","last_up_change":"2024-05-09T04:48:16.919842+0000","last_in_change":"2024-05-09T04:47:56.452645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:43:47.538306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:49:28.805447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T04:49:32.970287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T04:49:45.637000+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1b732fd3-e648-4b0d-aece-083bdaaa5d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6803","nonce":1738750914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6805","nonce":1738750914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6809","nonce":1738750914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6807","nonce":1738750914}]},"public_addr":"172.21.15.6:6803/1738750914","cluster_addr":"172.21.15.6:6805/1738750914","heartbeat_back_addr":"172.21.15.6:6809/1738750914","heartbeat_front_addr":"172.21.15.6:6807/1738750914","state":["exists","up"]},{"osd":1,"uuid":"9770c07d-4243-43bf-a0cd-9d2ac558556c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6801","nonce":2552207737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6803","nonce":2552207737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6807","nonce":2552207737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6805","nonce":2552207737}]},"public_addr":"172.21.15.169:6801/2552207737","cluster_addr":"172.21.15.169:6803/2552207737","heartbeat_back_addr":"172.21.15.169:6807/2552207737","heartbeat_front_addr":"172.21.15.169:6805/2552207737","state":["exists","up"]},{"osd":2,"uuid":"3f50ed89-b2a8-415e-b78d-8510d98d9368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6801","nonce":231435105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6803","nonce":231435105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6807","nonce":231435105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6805","nonce":231435105}]},"public_addr":"172.21.15.186:6801/231435105","cluster_addr":"172.21.15.186:6803/231435105","heartbeat_back_addr":"172.21.15.186:6807/231435105","heartbeat_front_addr":"172.21.15.186:6805/231435105","state":["exists","up"]},{"osd":3,"uuid":"afcbc36b-5669-4044-8dc5-2ebfe7970774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6811","nonce":2846613703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6813","nonce":2846613703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6817","nonce":2846613703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6815","nonce":2846613703}]},"public_addr":"172.21.15.6:6811/2846613703","cluster_addr":"172.21.15.6:6813/2846613703","heartbeat_back_addr":"172.21.15.6:6817/2846613703","heartbeat_front_addr":"172.21.15.6:6815/2846613703","state":["exists","up"]},{"osd":4,"uuid":"bb84cc5e-0297-4e23-be50-7ee2f7e59665","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6809","nonce":1928722453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6811","nonce":1928722453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6815","nonce":1928722453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6813","nonce":1928722453}]},"public_addr":"172.21.15.169:6809/1928722453","cluster_addr":"172.21.15.169:6811/1928722453","heartbeat_back_addr":"172.21.15.169:6815/1928722453","heartbeat_front_addr":"172.21.15.169:6813/1928722453","state":["exists","up"]},{"osd":5,"uuid":"830dc7f3-352e-4b01-8d4c-a9b68dcb40d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6809","nonce":3644445439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6811","nonce":3644445439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6815","nonce":3644445439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6813","nonce":3644445439}]},"public_addr":"172.21.15.186:6809/3644445439","cluster_addr":"172.21.15.186:6811/3644445439","heartbeat_back_addr":"172.21.15.186:6815/3644445439","heartbeat_front_addr":"172.21.15.186:6813/3644445439","state":["exists","up"]},{"osd":6,"uuid":"2e8478cb-c8c9-47ce-a3a5-48ffc959ab24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6819","nonce":3068252891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6821","nonce":3068252891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6825","nonce":3068252891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6823","nonce":3068252891}]},"public_addr":"172.21.15.6:6819/3068252891","cluster_addr":"172.21.15.6:6821/3068252891","heartbeat_back_addr":"172.21.15.6:6825/3068252891","heartbeat_front_addr":"172.21.15.6:6823/3068252891","state":["exists","up"]},{"osd":7,"uuid":"0f432ee2-42a4-4efa-9233-8e5a2fd65d10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6817","nonce":1303584711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6819","nonce":1303584711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6823","nonce":1303584711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6821","nonce":1303584711}]},"public_addr":"172.21.15.169:6817/1303584711","cluster_addr":"172.21.15.169:6819/1303584711","heartbeat_back_addr":"172.21.15.169:6823/1303584711","heartbeat_front_addr":"172.21.15.169:6821/1303584711","state":["exists","up"]},{"osd":8,"uuid":"cf5dca81-131b-48e7-9417-d01d4be47316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6817","nonce":3475163771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6819","nonce":3475163771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6823","nonce":3475163771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6821","nonce":3475163771}]},"public_addr":"172.21.15.186:6817/3475163771","cluster_addr":"172.21.15.186:6819/3475163771","heartbeat_back_addr":"172.21.15.186:6823/3475163771","heartbeat_front_addr":"172.21.15.186:6821/3475163771","state":["exists","up"]},{"osd":9,"uuid":"bca7bedd-1a57-4465-9033-064ca468add2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6827","nonce":3590990973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6829","nonce":3590990973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6833","nonce":3590990973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6831","nonce":3590990973}]},"public_addr":"172.21.15.6:6827/3590990973","cluster_addr":"172.21.15.6:6829/3590990973","heartbeat_back_addr":"172.21.15.6:6833/3590990973","heartbeat_front_addr":"172.21.15.6:6831/3590990973","state":["exists","up"]},{"osd":10,"uuid":"97b0e058-64f8-42ee-9b99-3ef8e39c8a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6825","nonce":61007341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6827","nonce":61007341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6831","nonce":61007341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6829","nonce":61007341}]},"public_addr":"172.21.15.169:6825/61007341","cluster_addr":"172.21.15.169:6827/61007341","heartbeat_back_addr":"172.21.15.169:6831/61007341","heartbeat_front_addr":"172.21.15.169:6829/61007341","state":["exists","up"]},{"osd":11,"uuid":"7f2220b0-24cf-48b2-95ae-1d66e00e57fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6825","nonce":1138940441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6827","nonce":1138940441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6831","nonce":1138940441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6829","nonce":1138940441}]},"public_addr":"172.21.15.186:6825/1138940441","cluster_addr":"172.21.15.186:6827/1138940441","heartbeat_back_addr":"172.21.15.186:6831/1138940441","heartbeat_front_addr":"172.21.15.186:6829/1138940441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:53.766295+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:20.195243+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:44.153812+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:20.472086+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:48.415371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:13.795415+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:50.236398+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:18.066623+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:43.781009+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:47:20.084719+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:47:48.219901+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:48:14.965536+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/3742411767":"2024-05-10T04:40:55.357057+0000","172.21.15.6:0/2410771702":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6801/937713898":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6800/937713898":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6801/3021503778":"2024-05-10T04:40:19.639151+0000","172.21.15.6:6800/3021503778":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/2559510850":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/3810474743":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/4133970555":"2024-05-10T04:40:55.357057+0000","172.21.15.6:0/664817743":"2024-05-10T04:40:19.639151+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:55:21.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:21.153+0000 7fc7d6400700 1 -- 172.21.15.6:0/1689451405 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fc7b40b2910 msgr2=0x7fc7b40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:21.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:21.153+0000 7fc7d6400700 1 --2- 172.21.15.6:0/1689451405 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fc7b40b2910 0x7fc7b40b4e20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc7d019bb30 tx=0x7fc7c0009040 comp rx=0 tx=0).stop 2024-05-09T04:55:21.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:21.153+0000 7fc7d6400700 1 -- 172.21.15.6:0/1689451405 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7d0109680 msgr2=0x7fc7d019a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:21.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:21.153+0000 7fc7d6400700 1 --2- 172.21.15.6:0/1689451405 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7d0109680 0x7fc7d019a9f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8005910 tx=0x7fc7c80055f0 comp rx=0 tx=0).stop 2024-05-09T04:55:21.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:21.153+0000 7fc7d6400700 1 -- 172.21.15.6:0/1689451405 shutdown_connections 2024-05-09T04:55:21.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:21.153+0000 7fc7d6400700 1 --2- 172.21.15.6:0/1689451405 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7d0109680 0x7fc7d019a9f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:21.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:21.153+0000 7fc7d6400700 1 --2- 172.21.15.6:0/1689451405 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fc7b40b2910 0x7fc7b40b4e20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:21.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:21.153+0000 7fc7d6400700 1 --2- 172.21.15.6:0/1689451405 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7d0104400 0x7fc7d01a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:21.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:21.153+0000 7fc7d6400700 1 --2- 172.21.15.6:0/1689451405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7d0108d30 0x7fc7d01a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:21.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:21.153+0000 7fc7d6400700 1 -- 172.21.15.6:0/1689451405 >> 172.21.15.6:0/1689451405 conn(0x7fc7d0079d10 msgr2=0x7fc7d010b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:21.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:21.154+0000 7fc7d6400700 1 -- 172.21.15.6:0/1689451405 shutdown_connections 2024-05-09T04:55:21.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:21.154+0000 7fc7d6400700 1 -- 172.21.15.6:0/1689451405 wait complete. 2024-05-09T04:55:21.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:20 smithi186 ceph-mon[34668]: pgmap v511: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:55:21.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:20 smithi169 ceph-mon[34075]: pgmap v511: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:55:22.112 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-05-09T04:55:22.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:21 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/1689451405' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:55:22.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:21 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/1689451405' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:55:22.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:21 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1689451405' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:55:22.581 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:55:23.048 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:22 smithi006 ceph-mon[29835]: pgmap v512: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:55:23.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:22 smithi186 ceph-mon[34668]: pgmap v512: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:55:23.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:22 smithi169 ceph-mon[34075]: pgmap v512: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:55:25.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:24 smithi186 ceph-mon[34668]: pgmap v513: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:55:25.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:24 smithi169 ceph-mon[34075]: pgmap v513: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:55:25.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:24 smithi006 ceph-mon[29835]: pgmap v513: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:55:25.837 INFO:teuthology.orchestra.run.smithi006.stdout:{"run_dir":"/var/run/ceph"} 2024-05-09T04:55:25.838 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-05-09T04:55:25.841 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 1722 2024-05-09T04:55:25.841 DEBUG:tasks.quiescer:Yielding 2024-05-09T04:55:25.841 INFO:teuthology.run_tasks:Running task workunit... 2024-05-09T04:55:25.854 INFO:tasks.workunit:Pulling workunits from ref c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T04:55:25.855 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-09T04:55:25.855 INFO:tasks.workunit:timeout=3h 2024-05-09T04:55:25.855 INFO:tasks.workunit:cleanup=True 2024-05-09T04:55:25.855 DEBUG:teuthology.orchestra.run.smithi006:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-09T04:55:25.855 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:55:25.912 INFO:teuthology.orchestra.run.smithi006.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-09T04:55:25.913 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-09T04:55:25.913 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 72h/114d Inode: 1099511628283 Links: 2 2024-05-09T04:55:25.913 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T04:55:25.913 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-05-09T04:55:25.913 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2024-05-09 04:51:23.868504855 +0000 2024-05-09T04:55:25.913 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2024-05-09 04:51:23.868504855 +0000 2024-05-09T04:55:25.913 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2024-05-09 04:54:55.020364151 +0000 2024-05-09T04:55:25.913 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2024-05-09 04:51:23.868504855 +0000 2024-05-09T04:55:25.913 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-09T04:55:25.914 DEBUG:teuthology.orchestra.run.smithi006:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-09T04:55:26.049 DEBUG:teuthology.orchestra.run.smithi006:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T04:55:26.103 INFO:tasks.workunit.client.0.smithi006.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-09T04:55:26.360 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:55:27.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:26 smithi006 ceph-mon[29835]: pgmap v514: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:55:27.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:26 smithi186 ceph-mon[34668]: pgmap v514: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:55:27.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:26 smithi169 ceph-mon[34075]: pgmap v514: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:55:28.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.616+0000 7ff875200700 1 -- 172.21.15.6:0/2857591704 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8701045f0 msgr2=0x7ff870104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:28.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.616+0000 7ff875200700 1 --2- 172.21.15.6:0/2857591704 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8701045f0 0x7ff870104a00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff86000b0d0 tx=0x7ff86000b3e0 comp rx=0 tx=0).stop 2024-05-09T04:55:28.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.616+0000 7ff875200700 1 -- 172.21.15.6:0/2857591704 shutdown_connections 2024-05-09T04:55:28.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.616+0000 7ff875200700 1 --2- 172.21.15.6:0/2857591704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff870109870 0x7ff8701111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:28.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.616+0000 7ff875200700 1 --2- 172.21.15.6:0/2857591704 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8701045f0 0x7ff870104a00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:28.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.616+0000 7ff875200700 1 --2- 172.21.15.6:0/2857591704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff870108f20 0x7ff870109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:28.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.616+0000 7ff875200700 1 -- 172.21.15.6:0/2857591704 >> 172.21.15.6:0/2857591704 conn(0x7ff870100040 msgr2=0x7ff870102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:28.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.616+0000 7ff875200700 1 -- 172.21.15.6:0/2857591704 shutdown_connections 2024-05-09T04:55:28.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.616+0000 7ff875200700 1 -- 172.21.15.6:0/2857591704 wait complete. 2024-05-09T04:55:28.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.618+0000 7ff875200700 1 Processor -- start 2024-05-09T04:55:28.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.618+0000 7ff875200700 1 -- start start 2024-05-09T04:55:28.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.618+0000 7ff875200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8701045f0 0x7ff87019d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:28.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.618+0000 7ff875200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff870108f20 0x7ff87019d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:28.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.618+0000 7ff875200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff870109870 0x7ff8701a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:28.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.618+0000 7ff875200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff8701144a0 con 0x7ff870108f20 2024-05-09T04:55:28.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.618+0000 7ff875200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff870114320 con 0x7ff870109870 2024-05-09T04:55:28.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.618+0000 7ff875200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff870114620 con 0x7ff8701045f0 2024-05-09T04:55:28.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.619+0000 7ff86de00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8701045f0 0x7ff87019d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:28.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.619+0000 7ff86de00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8701045f0 0x7ff87019d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:33532/0 (socket says 172.21.15.6:33532) 2024-05-09T04:55:28.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.619+0000 7ff86de00700 1 -- 172.21.15.6:0/3632604702 learned_addr learned my addr 172.21.15.6:0/3632604702 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:55:28.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.619+0000 7ff86e800700 1 --2- 172.21.15.6:0/3632604702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff870109870 0x7ff8701a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:28.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.619+0000 7ff865400700 1 --2- 172.21.15.6:0/3632604702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff870108f20 0x7ff87019d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:28.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.619+0000 7ff86de00700 1 -- 172.21.15.6:0/3632604702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff870109870 msgr2=0x7ff8701a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:28.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.619+0000 7ff86de00700 1 --2- 172.21.15.6:0/3632604702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff870109870 0x7ff8701a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:28.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.619+0000 7ff86de00700 1 -- 172.21.15.6:0/3632604702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff870108f20 msgr2=0x7ff87019d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:28.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.619+0000 7ff86de00700 1 --2- 172.21.15.6:0/3632604702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff870108f20 0x7ff87019d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:28.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.619+0000 7ff86de00700 1 -- 172.21.15.6:0/3632604702 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8701a1fc0 con 0x7ff8701045f0 2024-05-09T04:55:28.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.620+0000 7ff86de00700 1 --2- 172.21.15.6:0/3632604702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8701045f0 0x7ff87019d0f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff860007a30 tx=0x7ff860007b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:28.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.620+0000 7ff867400700 1 -- 172.21.15.6:0/3632604702 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86001f070 con 0x7ff8701045f0 2024-05-09T04:55:28.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.620+0000 7ff867400700 1 -- 172.21.15.6:0/3632604702 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff860005520 con 0x7ff8701045f0 2024-05-09T04:55:28.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.620+0000 7ff875200700 1 -- 172.21.15.6:0/3632604702 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8701a2250 con 0x7ff8701045f0 2024-05-09T04:55:28.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.621+0000 7ff867400700 1 -- 172.21.15.6:0/3632604702 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86001b690 con 0x7ff8701045f0 2024-05-09T04:55:28.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.621+0000 7ff875200700 1 -- 172.21.15.6:0/3632604702 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8701a9bd0 con 0x7ff8701045f0 2024-05-09T04:55:28.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.622+0000 7ff875200700 1 -- 172.21.15.6:0/3632604702 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff870105aa0 con 0x7ff8701045f0 2024-05-09T04:55:28.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.624+0000 7ff867400700 1 -- 172.21.15.6:0/3632604702 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 19) ==== 149702+0+0 (secure 0 0 0) 0x7ff860022070 con 0x7ff8701045f0 2024-05-09T04:55:28.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.626+0000 7ff867400700 1 --2- 172.21.15.6:0/3632604702 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff8540b2990 0x7ff8540b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:28.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.626+0000 7ff867400700 1 -- 172.21.15.6:0/3632604702 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7ff86000f020 con 0x7ff8701045f0 2024-05-09T04:55:28.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.626+0000 7ff865400700 1 --2- 172.21.15.6:0/3632604702 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff8540b2990 0x7ff8540b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:28.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.627+0000 7ff865400700 1 --2- 172.21.15.6:0/3632604702 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff8540b2990 0x7ff8540b4ea0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff87010a810 tx=0x7ff858009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:28.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.631+0000 7ff867400700 1 -- 172.21.15.6:0/3632604702 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff860029020 con 0x7ff8701045f0 2024-05-09T04:55:28.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.922+0000 7ff875200700 1 -- 172.21.15.6:0/3632604702 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff87019e450 con 0x7ff8701045f0 2024-05-09T04:55:28.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.923+0000 7ff867400700 1 -- 172.21.15.6:0/3632604702 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11927 (secure 0 0 0) 0x7ff8600ab040 con 0x7ff8701045f0 2024-05-09T04:55:28.925 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:55:28.925 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/162094599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":162094599},{"type":"v1","addr":"172.21.15.6:6835","nonce":162094599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/589294363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":589294363},{"type":"v1","addr":"172.21.15.6:6841","nonce":589294363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:49:36.920376+0000","modified":"2024-05-09T04:54:56.247431+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24554,"mds_1":24530,"mds_2":24512,"mds_3":24511,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24442":{"gid":24442,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6833/2977145206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2977145206},{"type":"v1","addr":"172.21.15.186:6833","nonce":2977145206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6833/133345948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":133345948},{"type":"v1","addr":"172.21.15.169:6833","nonce":133345948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.6:6837/1281521088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":1281521088},{"type":"v1","addr":"172.21.15.6:6837","nonce":1281521088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2268322007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2268322007},{"type":"v1","addr":"172.21.15.186:6835","nonce":2268322007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/3811005052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3811005052},{"type":"v1","addr":"172.21.15.169:6835","nonce":3811005052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.186:6837/1647607118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1647607118},{"type":"v1","addr":"172.21.15.186:6837","nonce":1647607118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":32,"addr":"172.21.15.186:6839/3954883105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":3954883105},{"type":"v1","addr":"172.21.15.186:6839","nonce":3954883105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.169:6837/1845715294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1845715294},{"type":"v1","addr":"172.21.15.169:6837","nonce":1845715294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.6:6839/2780306492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":2780306492},{"type":"v1","addr":"172.21.15.6:6839","nonce":2780306492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.169:6839/614388771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":614388771},{"type":"v1","addr":"172.21.15.169:6839","nonce":614388771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24512,24554,24530,24493,24511]},"id":1}]} 2024-05-09T04:55:28.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.926+0000 7ff875200700 1 -- 172.21.15.6:0/3632604702 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff8540b2990 msgr2=0x7ff8540b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:28.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.926+0000 7ff875200700 1 --2- 172.21.15.6:0/3632604702 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff8540b2990 0x7ff8540b4ea0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff87010a810 tx=0x7ff858009260 comp rx=0 tx=0).stop 2024-05-09T04:55:28.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.926+0000 7ff875200700 1 -- 172.21.15.6:0/3632604702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8701045f0 msgr2=0x7ff87019d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:28.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.926+0000 7ff875200700 1 --2- 172.21.15.6:0/3632604702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8701045f0 0x7ff87019d0f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff860007a30 tx=0x7ff860007b10 comp rx=0 tx=0).stop 2024-05-09T04:55:28.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.926+0000 7ff875200700 1 -- 172.21.15.6:0/3632604702 shutdown_connections 2024-05-09T04:55:28.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.926+0000 7ff875200700 1 --2- 172.21.15.6:0/3632604702 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff8540b2990 0x7ff8540b4ea0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:28.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.926+0000 7ff875200700 1 --2- 172.21.15.6:0/3632604702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff870109870 0x7ff8701a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:28.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.926+0000 7ff875200700 1 --2- 172.21.15.6:0/3632604702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8701045f0 0x7ff87019d0f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:28.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.926+0000 7ff875200700 1 --2- 172.21.15.6:0/3632604702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff870108f20 0x7ff87019d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:28.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.926+0000 7ff875200700 1 -- 172.21.15.6:0/3632604702 >> 172.21.15.6:0/3632604702 conn(0x7ff870100040 msgr2=0x7ff8701086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:28.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.927+0000 7ff875200700 1 -- 172.21.15.6:0/3632604702 shutdown_connections 2024-05-09T04:55:28.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:28.927+0000 7ff875200700 1 -- 172.21.15.6:0/3632604702 wait complete. 2024-05-09T04:55:28.929 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 30 2024-05-09T04:55:29.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:28 smithi006 ceph-mon[29835]: pgmap v515: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-09T04:55:29.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:28 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/3632604702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:55:29.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:28 smithi186 ceph-mon[34668]: pgmap v515: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-09T04:55:29.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:28 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/3632604702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:55:29.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:28 smithi169 ceph-mon[34075]: pgmap v515: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-09T04:55:29.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:28 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/3632604702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:55:29.877 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-09T04:49:36.920376+0000', 'modified': '2024-05-09T04:54:56.247431+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24554, 'mds_1': 24530, 'mds_2': 24512, 'mds_3': 24511, 'mds_4': 24493}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24442': {'gid': 24442, 'name': 'l', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6833/2977145206', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6832', 'nonce': 2977145206}, {'type': 'v1', 'addr': '172.21.15.186:6833', 'nonce': 2977145206}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24461': {'gid': 24461, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.169:6833/133345948', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6832', 'nonce': 133345948}, {'type': 'v1', 'addr': '172.21.15.169:6833', 'nonce': 133345948}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'd', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.6:6837/1281521088', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.6:6836', 'nonce': 1281521088}, {'type': 'v1', 'addr': '172.21.15.6:6837', 'nonce': 1281521088}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6835/2268322007', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6834', 'nonce': 2268322007}, {'type': 'v1', 'addr': '172.21.15.186:6835', 'nonce': 2268322007}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24491': {'gid': 24491, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.169:6835/3811005052', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6834', 'nonce': 3811005052}, {'type': 'v1', 'addr': '172.21.15.169:6835', 'nonce': 3811005052}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'c', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 33, 'addr': '172.21.15.186:6837/1647607118', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6836', 'nonce': 1647607118}, {'type': 'v1', 'addr': '172.21.15.186:6837', 'nonce': 1647607118}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'i', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 32, 'addr': '172.21.15.186:6839/3954883105', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6838', 'nonce': 3954883105}, {'type': 'v1', 'addr': '172.21.15.186:6839', 'nonce': 3954883105}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 32, 'addr': '172.21.15.169:6837/1845715294', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6836', 'nonce': 1845715294}, {'type': 'v1', 'addr': '172.21.15.169:6837', 'nonce': 1845715294}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24530': {'gid': 24530, 'name': 'g', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 30, 'addr': '172.21.15.6:6839/2780306492', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.6:6838', 'nonce': 2780306492}, {'type': 'v1', 'addr': '172.21.15.6:6839', 'nonce': 2780306492}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24554': {'gid': 24554, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.169:6839/614388771', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6838', 'nonce': 614388771}, {'type': 'v1', 'addr': '172.21.15.169:6839', 'nonce': 614388771}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24554, 'qdb_cluster': [24512, 24554, 24530, 24493, 24511]} 2024-05-09T04:55:29.877 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-09T04:55:29.877 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.c status 2024-05-09T04:55:30.363 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:55:31.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:30 smithi006 ceph-mon[29835]: pgmap v516: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T04:55:31.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:30 smithi186 ceph-mon[34668]: pgmap v516: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T04:55:31.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:30 smithi169 ceph-mon[34075]: pgmap v516: 129 pgs: 129 active+clean; 657 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T04:55:32.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.778+0000 7ff915a00700 1 -- 172.21.15.6:0/321892924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff910110cb0 msgr2=0x7ff9101130d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:32.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.778+0000 7ff915a00700 1 --2- 172.21.15.6:0/321892924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff910110cb0 0x7ff9101130d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff90800b0a0 tx=0x7ff90800b3b0 comp rx=0 tx=0).stop 2024-05-09T04:55:32.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.778+0000 7ff915a00700 1 -- 172.21.15.6:0/321892924 shutdown_connections 2024-05-09T04:55:32.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.778+0000 7ff915a00700 1 --2- 172.21.15.6:0/321892924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff910110cb0 0x7ff9101130d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:32.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.778+0000 7ff915a00700 1 --2- 172.21.15.6:0/321892924 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff910103100 0x7ff910103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:32.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.778+0000 7ff915a00700 1 --2- 172.21.15.6:0/321892924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff910103a50 0x7ff910110770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:32.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.778+0000 7ff915a00700 1 -- 172.21.15.6:0/321892924 >> 172.21.15.6:0/321892924 conn(0x7ff9100fedc0 msgr2=0x7ff9101011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:32.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.778+0000 7ff915a00700 1 -- 172.21.15.6:0/321892924 shutdown_connections 2024-05-09T04:55:32.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.778+0000 7ff915a00700 1 -- 172.21.15.6:0/321892924 wait complete. 2024-05-09T04:55:32.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.779+0000 7ff915a00700 1 Processor -- start 2024-05-09T04:55:32.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.780+0000 7ff915a00700 1 -- start start 2024-05-09T04:55:32.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.780+0000 7ff915a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff910103100 0x7ff91019cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:32.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.780+0000 7ff915a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff910103a50 0x7ff91019d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:32.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.780+0000 7ff915a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff910110cb0 0x7ff9101a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:32.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.780+0000 7ff915a00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff9101166c0 con 0x7ff910110cb0 2024-05-09T04:55:32.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.780+0000 7ff915a00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff910116540 con 0x7ff910103100 2024-05-09T04:55:32.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.780+0000 7ff915a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff910116840 con 0x7ff910103a50 2024-05-09T04:55:32.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.781+0000 7ff90ea00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff910103100 0x7ff91019cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:32.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.781+0000 7ff907e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff910103a50 0x7ff91019d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:32.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.781+0000 7ff90ea00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff910103100 0x7ff91019cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:41056/0 (socket says 172.21.15.6:41056) 2024-05-09T04:55:32.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.781+0000 7ff90ea00700 1 -- 172.21.15.6:0/37987729 learned_addr learned my addr 172.21.15.6:0/37987729 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:55:32.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.781+0000 7ff90f400700 1 --2- 172.21.15.6:0/37987729 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff910110cb0 0x7ff9101a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:32.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.781+0000 7ff90ea00700 1 -- 172.21.15.6:0/37987729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff910103a50 msgr2=0x7ff91019d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:32.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.781+0000 7ff90ea00700 1 --2- 172.21.15.6:0/37987729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff910103a50 0x7ff91019d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:32.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.781+0000 7ff90ea00700 1 -- 172.21.15.6:0/37987729 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff910110cb0 msgr2=0x7ff9101a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:32.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.781+0000 7ff90ea00700 1 --2- 172.21.15.6:0/37987729 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff910110cb0 0x7ff9101a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:32.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.781+0000 7ff90ea00700 1 -- 172.21.15.6:0/37987729 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9101a1e80 con 0x7ff910103100 2024-05-09T04:55:32.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.782+0000 7ff90ea00700 1 --2- 172.21.15.6:0/37987729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff910103100 0x7ff91019cfb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff90000eb30 tx=0x7ff90000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:32.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.782+0000 7ff907400700 1 -- 172.21.15.6:0/37987729 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90000cbd0 con 0x7ff910103100 2024-05-09T04:55:32.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.782+0000 7ff907400700 1 -- 172.21.15.6:0/37987729 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9000044d0 con 0x7ff910103100 2024-05-09T04:55:32.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.782+0000 7ff915a00700 1 -- 172.21.15.6:0/37987729 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9101a2170 con 0x7ff910103100 2024-05-09T04:55:32.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.782+0000 7ff907400700 1 -- 172.21.15.6:0/37987729 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff900003e30 con 0x7ff910103100 2024-05-09T04:55:32.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.782+0000 7ff915a00700 1 -- 172.21.15.6:0/37987729 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9101a9a90 con 0x7ff910103100 2024-05-09T04:55:32.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.786+0000 7ff907400700 1 -- 172.21.15.6:0/37987729 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 19) ==== 149702+0+0 (secure 0 0 0) 0x7ff900004000 con 0x7ff910103100 2024-05-09T04:55:32.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.787+0000 7ff924632b80 1 Processor -- start 2024-05-09T04:55:32.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.788+0000 7ff924632b80 1 -- start start 2024-05-09T04:55:32.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.788+0000 7ff907400700 1 --2- 172.21.15.6:0/37987729 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff8f40b2910 0x7ff8f40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:32.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.788+0000 7ff924632b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55581c7cd450 0x55581c94fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:32.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.788+0000 7ff924632b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55581c950410 0x55581c94a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:32.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.788+0000 7ff924632b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55581c94aa00 0x55581c94ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:32.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.788+0000 7ff924632b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55581c86d260 con 0x55581c94aa00 2024-05-09T04:55:32.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.788+0000 7ff924632b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55581c80b090 con 0x55581c950410 2024-05-09T04:55:32.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.788+0000 7ff924632b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55581c869220 con 0x55581c7cd450 2024-05-09T04:55:32.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.788+0000 7ff907400700 1 -- 172.21.15.6:0/37987729 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7ff9000e2b10 con 0x7ff910103100 2024-05-09T04:55:32.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.788+0000 7ff90f400700 1 --2- 172.21.15.6:0/37987729 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff8f40b2910 0x7ff8f40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:32.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.789+0000 7ff90ea00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55581c7cd450 0x55581c94fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:32.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.789+0000 7ff90ea00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55581c7cd450 0x55581c94fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:33572/0 (socket says 172.21.15.6:33572) 2024-05-09T04:55:32.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.789+0000 7ff90ea00700 1 -- 172.21.15.6:0/4121483212 learned_addr learned my addr 172.21.15.6:0/4121483212 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:55:32.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.789+0000 7ff90f400700 1 --2- 172.21.15.6:0/4121483212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55581c94aa00 0x55581c94ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:32.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.789+0000 7ff907e00700 1 --2- 172.21.15.6:0/4121483212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55581c950410 0x55581c94a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:32.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.789+0000 7ff90ea00700 1 -- 172.21.15.6:0/4121483212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55581c950410 msgr2=0x55581c94a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:32.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.789+0000 7ff90ea00700 1 --2- 172.21.15.6:0/4121483212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55581c950410 0x55581c94a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:32.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.789+0000 7ff90ea00700 1 -- 172.21.15.6:0/4121483212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55581c94aa00 msgr2=0x55581c94ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:32.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.789+0000 7ff90ea00700 1 --2- 172.21.15.6:0/4121483212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55581c94aa00 0x55581c94ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:32.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.789+0000 7ff90ea00700 1 -- 172.21.15.6:0/4121483212 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55581c6e8440 con 0x55581c7cd450 2024-05-09T04:55:32.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.789+0000 7ff907e00700 1 --2- 172.21.15.6:0/4121483212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55581c950410 0x55581c94a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:55:32.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.790+0000 7ff90ea00700 1 --2- 172.21.15.6:0/4121483212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55581c7cd450 0x55581c94fed0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff9000a9b80 tx=0x7ff9000a9af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ef838c898d0bd87 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:32.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.790+0000 7ff90f400700 1 --2- 172.21.15.6:0/37987729 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff8f40b2910 0x7ff8f40b4e20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff90800b780 tx=0x7ff908009320 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:32.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.790+0000 7ff8ffe00700 1 -- 172.21.15.6:0/4121483212 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9000a9360 con 0x55581c7cd450 2024-05-09T04:55:32.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.790+0000 7ff8ffe00700 1 -- 172.21.15.6:0/4121483212 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9000b2390 con 0x55581c7cd450 2024-05-09T04:55:32.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.790+0000 7ff8ffe00700 1 -- 172.21.15.6:0/4121483212 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9000aa0b0 con 0x55581c7cd450 2024-05-09T04:55:32.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.791+0000 7ff924632b80 1 -- 172.21.15.6:0/4121483212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55581c7cd450 msgr2=0x55581c94fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:32.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.791+0000 7ff924632b80 1 --2- 172.21.15.6:0/4121483212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55581c7cd450 0x55581c94fed0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff9000a9b80 tx=0x7ff9000a9af0 comp rx=0 tx=0).stop 2024-05-09T04:55:32.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.791+0000 7ff924632b80 1 -- 172.21.15.6:0/4121483212 shutdown_connections 2024-05-09T04:55:32.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.791+0000 7ff924632b80 1 --2- 172.21.15.6:0/4121483212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55581c94aa00 0x55581c94ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:32.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.791+0000 7ff924632b80 1 --2- 172.21.15.6:0/4121483212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55581c7cd450 0x55581c94fed0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:32.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.791+0000 7ff924632b80 1 --2- 172.21.15.6:0/4121483212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55581c950410 0x55581c94a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:32.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.791+0000 7ff924632b80 1 -- 172.21.15.6:0/4121483212 >> 172.21.15.6:0/4121483212 conn(0x55581c811e20 msgr2=0x55581c944e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:32.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.791+0000 7ff924632b80 1 -- 172.21.15.6:0/4121483212 shutdown_connections 2024-05-09T04:55:32.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.791+0000 7ff924632b80 1 -- 172.21.15.6:0/4121483212 wait complete. 2024-05-09T04:55:32.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.792+0000 7ff924632b80 1 -- start start 2024-05-09T04:55:32.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.792+0000 7ff924632b80 1 Processor -- start 2024-05-09T04:55:32.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.793+0000 7ff924632b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55581c7cd450 0x55581c953bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:32.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.793+0000 7ff924632b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55581c94aa00 0x55581c954110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:32.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.793+0000 7ff924632b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55581c950410 0x55581c954650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:32.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.793+0000 7ff924632b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55581c956ea0 con 0x55581c950410 2024-05-09T04:55:32.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.793+0000 7ff924632b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55581c7d2670 con 0x55581c7cd450 2024-05-09T04:55:32.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.793+0000 7ff924632b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55581c957020 con 0x55581c94aa00 2024-05-09T04:55:32.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.793+0000 7ff924632b80 10 client.0 _ll_register_callbacks cb 0x55581c804540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:55:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.793+0000 7ff90f400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55581c950410 0x55581c954650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.793+0000 7ff907e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55581c94aa00 0x55581c954110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.793+0000 7ff90f400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55581c950410 0x55581c954650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:40648/0 (socket says 172.21.15.6:40648) 2024-05-09T04:55:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.793+0000 7ff90ea00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55581c7cd450 0x55581c953bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.793+0000 7ff90f400700 1 -- 172.21.15.6:0/2652132185 learned_addr learned my addr 172.21.15.6:0/2652132185 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:55:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.794+0000 7ff90f400700 1 -- 172.21.15.6:0/2652132185 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55581c94aa00 msgr2=0x55581c954110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.794+0000 7ff90f400700 1 --2- 172.21.15.6:0/2652132185 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55581c94aa00 0x55581c954110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.794+0000 7ff90f400700 1 -- 172.21.15.6:0/2652132185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55581c7cd450 msgr2=0x55581c953bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.794+0000 7ff90f400700 1 --2- 172.21.15.6:0/2652132185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55581c7cd450 0x55581c953bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.794+0000 7ff90f400700 1 -- 172.21.15.6:0/2652132185 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55581c960350 con 0x55581c950410 2024-05-09T04:55:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.794+0000 7ff90ea00700 1 --2- 172.21.15.6:0/2652132185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55581c7cd450 0x55581c953bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:55:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.794+0000 7ff90f400700 1 --2- 172.21.15.6:0/2652132185 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55581c950410 0x55581c954650 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff90800b0a0 tx=0x7ff908006210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce9ca5f7e25ca9a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.794+0000 7ff8fcc00700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-05-09T04:55:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.794+0000 7ff8fcc00700 1 -- 172.21.15.6:0/2652132185 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff908031060 con 0x55581c950410 2024-05-09T04:55:32.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.794+0000 7ff924632b80 1 -- 172.21.15.6:0/2652132185 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55581c916820 con 0x55581c950410 2024-05-09T04:55:32.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.794+0000 7ff8fcc00700 1 -- 172.21.15.6:0/2652132185 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7ff908021460 con 0x55581c950410 2024-05-09T04:55:32.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.795+0000 7ff8fcc00700 1 -- 172.21.15.6:0/2652132185 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff908038050 con 0x55581c950410 2024-05-09T04:55:32.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.795+0000 7ff924632b80 10 client.15150 fetch_fsmap learned FSMap version 30 2024-05-09T04:55:32.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.795+0000 7ff924632b80 1 -- 172.21.15.6:0/2652132185 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55581c961ca0 con 0x55581c950410 2024-05-09T04:55:32.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.795+0000 7ff8fcc00700 1 -- 172.21.15.6:0/2652132185 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55581c916820 con 0x55581c950410 2024-05-09T04:55:32.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.795+0000 7ff8fcc00700 1 -- 172.21.15.6:0/2652132185 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff908011070 con 0x55581c950410 2024-05-09T04:55:32.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.796+0000 7ff924632b80 10 client.15150 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T04:55:32.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.796+0000 7ff924632b80 10 client.15150 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T04:55:32.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.796+0000 7ff924632b80 20 client.15150 populate_metadata read hostname 'smithi006' 2024-05-09T04:55:32.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.796+0000 7ff924632b80 1 --2- 172.21.15.6:0/2652132185 >> [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] conn(0x55581c9642b0 0x55581c965720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:32.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.796+0000 7ff924632b80 4 client.15150 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:55:32.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.796+0000 7ff924632b80 1 -- 172.21.15.6:0/2652132185 --> [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55581c87ac40 con 0x55581c9642b0 2024-05-09T04:55:32.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.796+0000 7ff90ea00700 1 --2- 172.21.15.6:0/2652132185 >> [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] conn(0x55581c9642b0 0x55581c965720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:32.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.797+0000 7ff90ea00700 1 --2- 172.21.15.6:0/2652132185 >> [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] conn(0x55581c9642b0 0x55581c965720 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=73577a501ae209ae server_cookie=c1b4ab5b71ec707c in_seq=0 out_seq=0 2024-05-09T04:55:32.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.797+0000 7ff8fcc00700 10 client.15150 ms_handle_connect on v2:172.21.15.186:6836/1647607118 2024-05-09T04:55:32.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.798+0000 7ff8fcc00700 1 -- 172.21.15.6:0/2652132185 <== mds.4 v2:172.21.15.186:6836/1647607118 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55581c87ac40 con 0x55581c9642b0 2024-05-09T04:55:32.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.798+0000 7ff8fcc00700 10 client.15150 handle_command_reply: tid=0 2024-05-09T04:55:32.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.798+0000 7ff924632b80 1 client.15150 shutdown 2024-05-09T04:55:32.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.798+0000 7ff924632b80 1 -- 172.21.15.6:0/2652132185 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55581c950410 msgr2=0x55581c954650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:32.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.798+0000 7ff924632b80 1 --2- 172.21.15.6:0/2652132185 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55581c950410 0x55581c954650 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff90800b0a0 tx=0x7ff908006210 comp rx=0 tx=0).stop 2024-05-09T04:55:32.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.798+0000 7ff924632b80 1 -- 172.21.15.6:0/2652132185 shutdown_connections 2024-05-09T04:55:32.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.798+0000 7ff924632b80 1 --2- 172.21.15.6:0/2652132185 >> [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] conn(0x55581c9642b0 0x55581c965720 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:32.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.798+0000 7ff924632b80 1 --2- 172.21.15.6:0/2652132185 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55581c950410 0x55581c954650 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:32.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.798+0000 7ff924632b80 1 --2- 172.21.15.6:0/2652132185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55581c7cd450 0x55581c953bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:32.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.798+0000 7ff924632b80 1 --2- 172.21.15.6:0/2652132185 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55581c94aa00 0x55581c954110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:32.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.798+0000 7ff8fcc00700 0 client.15150 ms_handle_reset on v2:172.21.15.186:6836/1647607118 2024-05-09T04:55:32.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.798+0000 7ff924632b80 1 -- 172.21.15.6:0/2652132185 >> 172.21.15.6:0/2652132185 conn(0x55581c811e20 msgr2=0x55581c944e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:32.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.798+0000 7ff924632b80 1 -- 172.21.15.6:0/2652132185 shutdown_connections 2024-05-09T04:55:32.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.798+0000 7ff924632b80 1 -- 172.21.15.6:0/2652132185 wait complete. 2024-05-09T04:55:32.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.799+0000 7ff924632b80 20 client.15150 trim_cache size 0 max 16384 2024-05-09T04:55:32.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.822+0000 7ff924632b80 1 Processor -- start 2024-05-09T04:55:32.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.822+0000 7ff924632b80 1 -- start start 2024-05-09T04:55:32.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.823+0000 7ff924632b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55581c7cd450 0x55581c94a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:32.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.823+0000 7ff924632b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55581c94aa00 0x55581c951830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:32.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.823+0000 7ff924632b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55581c950410 0x55581c952ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:32.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.823+0000 7ff924632b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55581c963a60 con 0x55581c7cd450 2024-05-09T04:55:32.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.823+0000 7ff924632b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55581c964130 con 0x55581c950410 2024-05-09T04:55:32.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.823+0000 7ff924632b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55581c964800 con 0x55581c94aa00 2024-05-09T04:55:32.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.823+0000 7ff90ea00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55581c7cd450 0x55581c94a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:32.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.823+0000 7ff90ea00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55581c7cd450 0x55581c94a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:40662/0 (socket says 172.21.15.6:40662) 2024-05-09T04:55:32.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.823+0000 7ff90ea00700 1 -- 172.21.15.6:0/2278603245 learned_addr learned my addr 172.21.15.6:0/2278603245 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:55:32.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.823+0000 7ff907e00700 1 --2- 172.21.15.6:0/2278603245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55581c94aa00 0x55581c951830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:32.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.823+0000 7ff90f400700 1 --2- 172.21.15.6:0/2278603245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55581c950410 0x55581c952ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:32.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.824+0000 7ff90ea00700 1 -- 172.21.15.6:0/2278603245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55581c94aa00 msgr2=0x55581c951830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:32.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.824+0000 7ff90ea00700 1 --2- 172.21.15.6:0/2278603245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55581c94aa00 0x55581c951830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:32.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.824+0000 7ff90ea00700 1 -- 172.21.15.6:0/2278603245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55581c950410 msgr2=0x55581c952ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:32.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.824+0000 7ff90ea00700 1 --2- 172.21.15.6:0/2278603245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55581c950410 0x55581c952ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:32.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.824+0000 7ff90ea00700 1 -- 172.21.15.6:0/2278603245 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55581c946060 con 0x55581c7cd450 2024-05-09T04:55:32.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.824+0000 7ff90ea00700 1 --2- 172.21.15.6:0/2278603245 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55581c7cd450 0x55581c94a4c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff900003fd0 tx=0x7ff9000c0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b2216e809ca32e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:32.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.824+0000 7ff8ff400700 1 -- 172.21.15.6:0/2278603245 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90006e070 con 0x55581c7cd450 2024-05-09T04:55:32.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.824+0000 7ff8ff400700 1 -- 172.21.15.6:0/2278603245 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff90001e4f0 con 0x55581c7cd450 2024-05-09T04:55:32.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.824+0000 7ff8ff400700 1 -- 172.21.15.6:0/2278603245 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90008e030 con 0x55581c7cd450 2024-05-09T04:55:32.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.825+0000 7ff924632b80 1 -- 172.21.15.6:0/2278603245 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55581c7cd450 msgr2=0x55581c94a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:32.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.825+0000 7ff924632b80 1 --2- 172.21.15.6:0/2278603245 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55581c7cd450 0x55581c94a4c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff900003fd0 tx=0x7ff9000c0040 comp rx=0 tx=0).stop 2024-05-09T04:55:32.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.825+0000 7ff924632b80 1 -- 172.21.15.6:0/2278603245 shutdown_connections 2024-05-09T04:55:32.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.825+0000 7ff924632b80 1 --2- 172.21.15.6:0/2278603245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55581c950410 0x55581c952ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:32.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.825+0000 7ff924632b80 1 --2- 172.21.15.6:0/2278603245 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55581c7cd450 0x55581c94a4c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:32.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.825+0000 7ff924632b80 1 --2- 172.21.15.6:0/2278603245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55581c94aa00 0x55581c951830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:32.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.825+0000 7ff924632b80 1 -- 172.21.15.6:0/2278603245 >> 172.21.15.6:0/2278603245 conn(0x55581c811e20 msgr2=0x55581c94ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:32.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.825+0000 7ff924632b80 1 -- 172.21.15.6:0/2278603245 shutdown_connections 2024-05-09T04:55:32.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.826+0000 7ff924632b80 1 -- 172.21.15.6:0/2278603245 wait complete. 2024-05-09T04:55:32.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.826+0000 7ff924632b80 1 -- start start 2024-05-09T04:55:32.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.826+0000 7ff924632b80 1 Processor -- start 2024-05-09T04:55:32.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.827+0000 7ff924632b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55581c7cd450 0x55581c96efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:32.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.827+0000 7ff924632b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55581c94aa00 0x55581c9494b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:32.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.827+0000 7ff924632b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55581c950410 0x55581c9499f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:32.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.827+0000 7ff924632b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55581c9655a0 con 0x55581c94aa00 2024-05-09T04:55:32.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.827+0000 7ff924632b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55581c964ed0 con 0x55581c7cd450 2024-05-09T04:55:32.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.827+0000 7ff924632b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55581c965c70 con 0x55581c950410 2024-05-09T04:55:32.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.827+0000 7ff924632b80 10 client.0 _ll_register_callbacks cb 0x55581c804540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:55:32.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.827+0000 7ff907e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55581c94aa00 0x55581c9494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:32.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.827+0000 7ff907e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55581c94aa00 0x55581c9494b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:40664/0 (socket says 172.21.15.6:40664) 2024-05-09T04:55:32.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.827+0000 7ff907e00700 1 -- 172.21.15.6:0/1629496435 learned_addr learned my addr 172.21.15.6:0/1629496435 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:55:32.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.827+0000 7ff90ea00700 1 --2- 172.21.15.6:0/1629496435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55581c7cd450 0x55581c96efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:32.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.827+0000 7ff90f400700 1 --2- 172.21.15.6:0/1629496435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55581c950410 0x55581c9499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:32.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.828+0000 7ff907e00700 1 -- 172.21.15.6:0/1629496435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55581c950410 msgr2=0x55581c9499f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:32.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.828+0000 7ff907e00700 1 --2- 172.21.15.6:0/1629496435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55581c950410 0x55581c9499f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:32.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.828+0000 7ff907e00700 1 -- 172.21.15.6:0/1629496435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55581c7cd450 msgr2=0x55581c96efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:32.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.828+0000 7ff907e00700 1 --2- 172.21.15.6:0/1629496435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55581c7cd450 0x55581c96efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:32.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.828+0000 7ff907e00700 1 -- 172.21.15.6:0/1629496435 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55581c94a030 con 0x55581c94aa00 2024-05-09T04:55:32.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.828+0000 7ff90ea00700 1 --2- 172.21.15.6:0/1629496435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55581c7cd450 0x55581c96efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:55:32.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.828+0000 7ff907e00700 1 --2- 172.21.15.6:0/1629496435 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55581c94aa00 0x55581c9494b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff9101048a0 tx=0x7ff8f8003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af46518ba610f78 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:32.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.828+0000 7ff8fcc00700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-05-09T04:55:32.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.828+0000 7ff8fcc00700 1 -- 172.21.15.6:0/1629496435 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f8022060 con 0x55581c94aa00 2024-05-09T04:55:32.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.828+0000 7ff8fcc00700 1 -- 172.21.15.6:0/1629496435 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7ff8f8010840 con 0x55581c94aa00 2024-05-09T04:55:32.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.828+0000 7ff924632b80 1 -- 172.21.15.6:0/1629496435 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55581c874ce0 con 0x55581c94aa00 2024-05-09T04:55:32.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.829+0000 7ff8fcc00700 1 -- 172.21.15.6:0/1629496435 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff8f8029050 con 0x55581c94aa00 2024-05-09T04:55:32.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.829+0000 7ff924632b80 10 client.15156 fetch_fsmap learned FSMap version 30 2024-05-09T04:55:32.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.829+0000 7ff924632b80 1 -- 172.21.15.6:0/1629496435 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55581c96e9a0 con 0x55581c94aa00 2024-05-09T04:55:32.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.829+0000 7ff8fcc00700 1 -- 172.21.15.6:0/1629496435 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55581c874ce0 con 0x55581c94aa00 2024-05-09T04:55:32.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.829+0000 7ff8fcc00700 1 -- 172.21.15.6:0/1629496435 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff8f801c910 con 0x55581c94aa00 2024-05-09T04:55:32.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.829+0000 7ff924632b80 10 client.15156 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T04:55:32.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.829+0000 7ff924632b80 10 client.15156 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T04:55:32.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.830+0000 7ff924632b80 20 client.15156 populate_metadata read hostname 'smithi006' 2024-05-09T04:55:32.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.830+0000 7ff924632b80 1 --2- 172.21.15.6:0/1629496435 >> [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] conn(0x55581c9d19f0 0x55581c9d2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:32.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.830+0000 7ff924632b80 4 client.15156 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T04:55:32.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.830+0000 7ff924632b80 1 -- 172.21.15.6:0/1629496435 --> [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] -- command(tid 0: {"prefix": "status"}) -- 0x55581c6e8920 con 0x55581c9d19f0 2024-05-09T04:55:32.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.830+0000 7ff90ea00700 1 --2- 172.21.15.6:0/1629496435 >> [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] conn(0x55581c9d19f0 0x55581c9d2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:32.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.831+0000 7ff90ea00700 1 --2- 172.21.15.6:0/1629496435 >> [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] conn(0x55581c9d19f0 0x55581c9d2e60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1133dc5202bbf900 server_cookie=a9473ac3e1ac037c in_seq=0 out_seq=0 2024-05-09T04:55:32.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.831+0000 7ff8fcc00700 10 client.15156 ms_handle_connect on v2:172.21.15.186:6836/1647607118 2024-05-09T04:55:32.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.831+0000 7ff8fcc00700 1 -- 172.21.15.6:0/1629496435 <== mds.4 v2:172.21.15.186:6836/1647607118 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55581c6e8920 con 0x55581c9d19f0 2024-05-09T04:55:32.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.831+0000 7ff8fcc00700 10 client.15156 handle_command_reply: tid=0 2024-05-09T04:55:32.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.831+0000 7ff924632b80 1 client.15156 shutdown 2024-05-09T04:55:32.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.831+0000 7ff924632b80 1 -- 172.21.15.6:0/1629496435 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55581c94aa00 msgr2=0x55581c9494b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:32.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.831+0000 7ff924632b80 1 --2- 172.21.15.6:0/1629496435 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55581c94aa00 0x55581c9494b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff9101048a0 tx=0x7ff8f8003960 comp rx=0 tx=0).stop 2024-05-09T04:55:32.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.831+0000 7ff924632b80 1 -- 172.21.15.6:0/1629496435 shutdown_connections 2024-05-09T04:55:32.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.831+0000 7ff924632b80 1 --2- 172.21.15.6:0/1629496435 >> [v2:172.21.15.186:6836/1647607118,v1:172.21.15.186:6837/1647607118] conn(0x55581c9d19f0 0x55581c9d2e60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:32.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.831+0000 7ff924632b80 1 --2- 172.21.15.6:0/1629496435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55581c950410 0x55581c9499f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:32.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.831+0000 7ff924632b80 1 --2- 172.21.15.6:0/1629496435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55581c7cd450 0x55581c96efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:32.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.831+0000 7ff924632b80 1 --2- 172.21.15.6:0/1629496435 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55581c94aa00 0x55581c9494b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:32.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.831+0000 7ff924632b80 1 -- 172.21.15.6:0/1629496435 >> 172.21.15.6:0/1629496435 conn(0x55581c811e20 msgr2=0x55581c94ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:32.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.831+0000 7ff8fcc00700 0 client.15156 ms_handle_reset on v2:172.21.15.186:6836/1647607118 2024-05-09T04:55:32.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.832+0000 7ff924632b80 1 -- 172.21.15.6:0/1629496435 shutdown_connections 2024-05-09T04:55:32.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.832+0000 7ff924632b80 1 -- 172.21.15.6:0/1629496435 wait complete. 2024-05-09T04:55:32.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.832+0000 7ff924632b80 20 client.15156 trim_cache size 0 max 16384 2024-05-09T04:55:32.836 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-09T04:55:32.836 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "f74d813c-0dbd-11ef-bc97-c7b262605968", 2024-05-09T04:55:32.836 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 4, 2024-05-09T04:55:32.836 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24493, 2024-05-09T04:55:32.836 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-05-09T04:55:32.836 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-05-09T04:55:32.836 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-05-09T04:55:32.836 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 270.07458303700002, 2024-05-09T04:55:32.836 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 30, 2024-05-09T04:55:32.836 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 89, 2024-05-09T04:55:32.836 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 89, 2024-05-09T04:55:32.836 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 397.35852587300002 2024-05-09T04:55:32.837 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-09T04:55:32.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.832+0000 7ff915a00700 1 -- 172.21.15.6:0/37987729 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff8f40b2910 msgr2=0x7ff8f40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:32.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.832+0000 7ff915a00700 1 --2- 172.21.15.6:0/37987729 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff8f40b2910 0x7ff8f40b4e20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff90800b780 tx=0x7ff908009320 comp rx=0 tx=0).stop 2024-05-09T04:55:32.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.832+0000 7ff915a00700 1 -- 172.21.15.6:0/37987729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff910103100 msgr2=0x7ff91019cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:32.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.832+0000 7ff915a00700 1 --2- 172.21.15.6:0/37987729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff910103100 0x7ff91019cfb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff90000eb30 tx=0x7ff90000ee40 comp rx=0 tx=0).stop 2024-05-09T04:55:32.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.833+0000 7ff915a00700 1 -- 172.21.15.6:0/37987729 shutdown_connections 2024-05-09T04:55:32.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.833+0000 7ff915a00700 1 --2- 172.21.15.6:0/37987729 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff910110cb0 0x7ff9101a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:32.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.833+0000 7ff915a00700 1 --2- 172.21.15.6:0/37987729 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff8f40b2910 0x7ff8f40b4e20 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:32.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.833+0000 7ff915a00700 1 --2- 172.21.15.6:0/37987729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff910103100 0x7ff91019cfb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:32.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.833+0000 7ff915a00700 1 --2- 172.21.15.6:0/37987729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff910103a50 0x7ff91019d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:32.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.833+0000 7ff915a00700 1 -- 172.21.15.6:0/37987729 >> 172.21.15.6:0/37987729 conn(0x7ff9100fedc0 msgr2=0x7ff910112420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:32.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.833+0000 7ff915a00700 1 -- 172.21.15.6:0/37987729 shutdown_connections 2024-05-09T04:55:32.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:32.833+0000 7ff915a00700 1 -- 172.21.15.6:0/37987729 wait complete. 2024-05-09T04:55:33.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:33 smithi006 ceph-mon[29835]: pgmap v517: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-05-09T04:55:33.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:33 smithi186 ceph-mon[34668]: pgmap v517: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-05-09T04:55:33.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:33 smithi169 ceph-mon[34075]: pgmap v517: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-05-09T04:55:33.796 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-09T04:55:34.289 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:55:34.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:34 smithi186 ceph-mon[34668]: pgmap v518: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-09T04:55:34.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:34 smithi169 ceph-mon[34075]: pgmap v518: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-09T04:55:34.445 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:34 smithi006 ceph-mon[29835]: pgmap v518: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-09T04:55:36.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.493+0000 7fec3ca00700 1 -- 172.21.15.6:0/2348492343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec3810e740 msgr2=0x7fec38110c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:36.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.493+0000 7fec3ca00700 1 --2- 172.21.15.6:0/2348492343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec3810e740 0x7fec38110c20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fec30009990 tx=0x7fec30009ca0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.493+0000 7fec3ca00700 1 -- 172.21.15.6:0/2348492343 shutdown_connections 2024-05-09T04:55:36.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.493+0000 7fec3ca00700 1 --2- 172.21.15.6:0/2348492343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec3810e740 0x7fec38110c20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.493+0000 7fec3ca00700 1 --2- 172.21.15.6:0/2348492343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec38105310 0x7fec38105720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.493+0000 7fec3ca00700 1 --2- 172.21.15.6:0/2348492343 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec38105c60 0x7fec3810e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.493+0000 7fec3ca00700 1 -- 172.21.15.6:0/2348492343 >> 172.21.15.6:0/2348492343 conn(0x7fec38078c40 msgr2=0x7fec38079080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:36.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.493+0000 7fec3ca00700 1 -- 172.21.15.6:0/2348492343 shutdown_connections 2024-05-09T04:55:36.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.493+0000 7fec3ca00700 1 -- 172.21.15.6:0/2348492343 wait complete. 2024-05-09T04:55:36.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.494+0000 7fec3ca00700 1 Processor -- start 2024-05-09T04:55:36.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.494+0000 7fec3ca00700 1 -- start start 2024-05-09T04:55:36.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.495+0000 7fec3ca00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec38105310 0x7fec38103000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:36.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.495+0000 7fec3ca00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec38105c60 0x7fec38103540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:36.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.495+0000 7fec3ca00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec3810e740 0x7fec38103a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:36.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.495+0000 7fec3ca00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fec381142a0 con 0x7fec3810e740 2024-05-09T04:55:36.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.495+0000 7fec3ca00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fec38114120 con 0x7fec38105310 2024-05-09T04:55:36.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.495+0000 7fec3ca00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fec38114420 con 0x7fec38105c60 2024-05-09T04:55:36.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.495+0000 7fec37400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec38105310 0x7fec38103000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:36.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.495+0000 7fec37400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec38105310 0x7fec38103000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:41104/0 (socket says 172.21.15.6:41104) 2024-05-09T04:55:36.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.495+0000 7fec36a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec38105c60 0x7fec38103540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:36.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.495+0000 7fec37400700 1 -- 172.21.15.6:0/4137361282 learned_addr learned my addr 172.21.15.6:0/4137361282 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:55:36.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.496+0000 7fec37e00700 1 --2- 172.21.15.6:0/4137361282 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec3810e740 0x7fec38103a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:36.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.496+0000 7fec37400700 1 -- 172.21.15.6:0/4137361282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec38105c60 msgr2=0x7fec38103540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:36.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.496+0000 7fec37400700 1 --2- 172.21.15.6:0/4137361282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec38105c60 0x7fec38103540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.496+0000 7fec37400700 1 -- 172.21.15.6:0/4137361282 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec3810e740 msgr2=0x7fec38103a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:36.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.496+0000 7fec37400700 1 --2- 172.21.15.6:0/4137361282 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec3810e740 0x7fec38103a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.496+0000 7fec37400700 1 -- 172.21.15.6:0/4137361282 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec381a8f50 con 0x7fec38105310 2024-05-09T04:55:36.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.496+0000 7fec37400700 1 --2- 172.21.15.6:0/4137361282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec38105310 0x7fec38103000 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fec2800ef10 tx=0x7fec2800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:36.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.497+0000 7fec2fe00700 1 -- 172.21.15.6:0/4137361282 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2800c9c0 con 0x7fec38105310 2024-05-09T04:55:36.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.497+0000 7fec2fe00700 1 -- 172.21.15.6:0/4137361282 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec28019070 con 0x7fec38105310 2024-05-09T04:55:36.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.497+0000 7fec2fe00700 1 -- 172.21.15.6:0/4137361282 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec28014480 con 0x7fec38105310 2024-05-09T04:55:36.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.497+0000 7fec3ca00700 1 -- 172.21.15.6:0/4137361282 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec381a91e0 con 0x7fec38105310 2024-05-09T04:55:36.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.497+0000 7fec3ca00700 1 -- 172.21.15.6:0/4137361282 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec381a9720 con 0x7fec38105310 2024-05-09T04:55:36.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.502+0000 7fec4cc10b80 1 Processor -- start 2024-05-09T04:55:36.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.502+0000 7fec2fe00700 1 -- 172.21.15.6:0/4137361282 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 19) ==== 149702+0+0 (secure 0 0 0) 0x7fec28012050 con 0x7fec38105310 2024-05-09T04:55:36.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.502+0000 7fec4cc10b80 1 -- start start 2024-05-09T04:55:36.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.503+0000 7fec4cc10b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b2f0ae4450 0x55b2f0c67100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:36.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.503+0000 7fec4cc10b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b2f0c67640 0x55b2f0c61620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:36.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.503+0000 7fec4cc10b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2f0c61b60 0x55b2f0c61f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:36.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.503+0000 7fec4cc10b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55b2f0b84260 con 0x55b2f0c67640 2024-05-09T04:55:36.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.503+0000 7fec4cc10b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b2f0b22090 con 0x55b2f0ae4450 2024-05-09T04:55:36.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.503+0000 7fec4cc10b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55b2f0b80220 con 0x55b2f0c61b60 2024-05-09T04:55:36.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.503+0000 7fec37400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b2f0c67640 0x55b2f0c61620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:36.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.503+0000 7fec37400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b2f0c67640 0x55b2f0c61620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:40696/0 (socket says 172.21.15.6:40696) 2024-05-09T04:55:36.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.503+0000 7fec37e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b2f0ae4450 0x55b2f0c67100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:36.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.503+0000 7fec37400700 1 -- 172.21.15.6:0/3378481452 learned_addr learned my addr 172.21.15.6:0/3378481452 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:55:36.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.503+0000 7fec36a00700 1 --2- 172.21.15.6:0/3378481452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2f0c61b60 0x55b2f0c61f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:36.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.504+0000 7fec37e00700 1 -- 172.21.15.6:0/3378481452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2f0c61b60 msgr2=0x55b2f0c61f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:36.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.504+0000 7fec37e00700 1 --2- 172.21.15.6:0/3378481452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2f0c61b60 0x55b2f0c61f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.504+0000 7fec37e00700 1 -- 172.21.15.6:0/3378481452 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b2f0c67640 msgr2=0x55b2f0c61620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:36.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.504+0000 7fec37e00700 1 --2- 172.21.15.6:0/3378481452 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b2f0c67640 0x55b2f0c61620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.504+0000 7fec37e00700 1 -- 172.21.15.6:0/3378481452 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2f09ff440 con 0x55b2f0ae4450 2024-05-09T04:55:36.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.504+0000 7fec36a00700 1 --2- 172.21.15.6:0/3378481452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2f0c61b60 0x55b2f0c61f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:55:36.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.504+0000 7fec2fe00700 1 --2- 172.21.15.6:0/4137361282 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fec1c0b2910 0x7fec1c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:36.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.504+0000 7fec37e00700 1 --2- 172.21.15.6:0/3378481452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b2f0ae4450 0x55b2f0c67100 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fec3000b2e0 tx=0x7fec300096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a24bb51da7b855c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:36.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.504+0000 7fec2fe00700 1 -- 172.21.15.6:0/4137361282 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fec280e2650 con 0x7fec38105310 2024-05-09T04:55:36.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.504+0000 7fec37400700 1 --2- 172.21.15.6:0/4137361282 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fec1c0b2910 0x7fec1c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:36.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.504+0000 7fec2cc00700 1 -- 172.21.15.6:0/3378481452 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec30027070 con 0x55b2f0ae4450 2024-05-09T04:55:36.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.505+0000 7fec2cc00700 1 -- 172.21.15.6:0/3378481452 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec300056b0 con 0x55b2f0ae4450 2024-05-09T04:55:36.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.505+0000 7fec2cc00700 1 -- 172.21.15.6:0/3378481452 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec30003bb0 con 0x55b2f0ae4450 2024-05-09T04:55:36.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.506+0000 7fec4cc10b80 1 -- 172.21.15.6:0/3378481452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b2f0ae4450 msgr2=0x55b2f0c67100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:36.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.506+0000 7fec4cc10b80 1 --2- 172.21.15.6:0/3378481452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b2f0ae4450 0x55b2f0c67100 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fec3000b2e0 tx=0x7fec300096a0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.506+0000 7fec37400700 1 --2- 172.21.15.6:0/4137361282 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fec1c0b2910 0x7fec1c0b4e20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fec28000f80 tx=0x7fec280e8530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:36.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.506+0000 7fec4cc10b80 1 -- 172.21.15.6:0/3378481452 shutdown_connections 2024-05-09T04:55:36.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.506+0000 7fec4cc10b80 1 --2- 172.21.15.6:0/3378481452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2f0c61b60 0x55b2f0c61f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.506+0000 7fec4cc10b80 1 --2- 172.21.15.6:0/3378481452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b2f0ae4450 0x55b2f0c67100 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.506+0000 7fec4cc10b80 1 --2- 172.21.15.6:0/3378481452 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b2f0c67640 0x55b2f0c61620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.506+0000 7fec4cc10b80 1 -- 172.21.15.6:0/3378481452 >> 172.21.15.6:0/3378481452 conn(0x55b2f0b28e20 msgr2=0x55b2f0c5c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:36.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.506+0000 7fec4cc10b80 1 -- 172.21.15.6:0/3378481452 shutdown_connections 2024-05-09T04:55:36.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.506+0000 7fec4cc10b80 1 -- 172.21.15.6:0/3378481452 wait complete. 2024-05-09T04:55:36.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.507+0000 7fec4cc10b80 1 -- start start 2024-05-09T04:55:36.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.507+0000 7fec4cc10b80 1 Processor -- start 2024-05-09T04:55:36.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.508+0000 7fec4cc10b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b2f0ae4450 0x55b2f0c6ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:36.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.508+0000 7fec4cc10b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2f0c61b60 0x55b2f0c6b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:36.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.508+0000 7fec4cc10b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b2f0c67640 0x55b2f0c6b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:36.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.508+0000 7fec4cc10b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55b2f0c6e070 con 0x55b2f0ae4450 2024-05-09T04:55:36.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.508+0000 7fec4cc10b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b2f0ae9670 con 0x55b2f0c67640 2024-05-09T04:55:36.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.508+0000 7fec4cc10b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55b2f0c6e1f0 con 0x55b2f0c61b60 2024-05-09T04:55:36.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.508+0000 7fec4cc10b80 10 client.0 _ll_register_callbacks cb 0x55b2f0b1b540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:55:36.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.508+0000 7fec37e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b2f0ae4450 0x55b2f0c6ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:36.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.508+0000 7fec37e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b2f0ae4450 0x55b2f0c6ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:40706/0 (socket says 172.21.15.6:40706) 2024-05-09T04:55:36.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.508+0000 7fec37e00700 1 -- 172.21.15.6:0/3868880174 learned_addr learned my addr 172.21.15.6:0/3868880174 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:55:36.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.508+0000 7fec36a00700 1 --2- 172.21.15.6:0/3868880174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2f0c61b60 0x55b2f0c6b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:36.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.508+0000 7fec37e00700 1 --2- 172.21.15.6:0/3868880174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b2f0c67640 0x55b2f0c6b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:36.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.509+0000 7fec37e00700 1 -- 172.21.15.6:0/3868880174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2f0c61b60 msgr2=0x55b2f0c6b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:36.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.509+0000 7fec37e00700 1 --2- 172.21.15.6:0/3868880174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2f0c61b60 0x55b2f0c6b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.509+0000 7fec37e00700 1 -- 172.21.15.6:0/3868880174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b2f0c67640 msgr2=0x55b2f0c6b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:36.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.509+0000 7fec37e00700 1 --2- 172.21.15.6:0/3868880174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b2f0c67640 0x55b2f0c6b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.509+0000 7fec37e00700 1 -- 172.21.15.6:0/3868880174 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2f0c774e0 con 0x55b2f0ae4450 2024-05-09T04:55:36.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.509+0000 7fec37e00700 1 --2- 172.21.15.6:0/3868880174 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b2f0ae4450 0x55b2f0c6ae30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fec30003ba0 tx=0x7fec30003790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b027dfed53200509 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:36.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.509+0000 7fec21600700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-05-09T04:55:36.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.509+0000 7fec21600700 1 -- 172.21.15.6:0/3868880174 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec30027040 con 0x55b2f0ae4450 2024-05-09T04:55:36.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.509+0000 7fec21600700 1 -- 172.21.15.6:0/3868880174 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fec3000b720 con 0x55b2f0ae4450 2024-05-09T04:55:36.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.509+0000 7fec4cc10b80 1 -- 172.21.15.6:0/3868880174 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec300056b0 con 0x55b2f0ae4450 2024-05-09T04:55:36.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.510+0000 7fec21600700 1 -- 172.21.15.6:0/3868880174 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fec3000b920 con 0x55b2f0ae4450 2024-05-09T04:55:36.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.510+0000 7fec4cc10b80 10 client.15168 fetch_fsmap learned FSMap version 30 2024-05-09T04:55:36.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.510+0000 7fec4cc10b80 1 -- 172.21.15.6:0/3868880174 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b2f0c78e50 con 0x55b2f0ae4450 2024-05-09T04:55:36.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.510+0000 7fec21600700 1 -- 172.21.15.6:0/3868880174 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec300056b0 con 0x55b2f0ae4450 2024-05-09T04:55:36.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.510+0000 7fec21600700 1 -- 172.21.15.6:0/3868880174 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fec30022ca0 con 0x55b2f0ae4450 2024-05-09T04:55:36.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.510+0000 7fec4cc10b80 10 client.15168 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T04:55:36.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.511+0000 7fec4cc10b80 10 client.15168 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T04:55:36.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.511+0000 7fec4cc10b80 20 client.15168 populate_metadata read hostname 'smithi006' 2024-05-09T04:55:36.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.511+0000 7fec4cc10b80 1 --2- 172.21.15.6:0/3868880174 >> [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] conn(0x55b2f0c7a1a0 0x55b2f0c7c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:36.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.511+0000 7fec4cc10b80 4 client.15168 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:55:36.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.511+0000 7fec4cc10b80 1 -- 172.21.15.6:0/3868880174 --> [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2f0c2d820 con 0x55b2f0c7a1a0 2024-05-09T04:55:36.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.511+0000 7fec36a00700 1 --2- 172.21.15.6:0/3868880174 >> [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] conn(0x55b2f0c7a1a0 0x55b2f0c7c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:36.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.512+0000 7fec36a00700 1 --2- 172.21.15.6:0/3868880174 >> [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] conn(0x55b2f0c7a1a0 0x55b2f0c7c340 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5c58f5cab77f5301 server_cookie=c0f610b131aebc95 in_seq=0 out_seq=0 2024-05-09T04:55:36.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.512+0000 7fec21600700 10 client.15168 ms_handle_connect on v2:172.21.15.186:6838/3954883105 2024-05-09T04:55:36.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.512+0000 7fec21600700 1 -- 172.21.15.6:0/3868880174 <== mds.3 v2:172.21.15.186:6838/3954883105 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b2f0c2d820 con 0x55b2f0c7a1a0 2024-05-09T04:55:36.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.512+0000 7fec21600700 10 client.15168 handle_command_reply: tid=0 2024-05-09T04:55:36.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.513+0000 7fec4cc10b80 1 client.15168 shutdown 2024-05-09T04:55:36.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.513+0000 7fec4cc10b80 1 -- 172.21.15.6:0/3868880174 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b2f0ae4450 msgr2=0x55b2f0c6ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:36.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.513+0000 7fec4cc10b80 1 --2- 172.21.15.6:0/3868880174 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b2f0ae4450 0x55b2f0c6ae30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fec30003ba0 tx=0x7fec30003790 comp rx=0 tx=0).stop 2024-05-09T04:55:36.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.513+0000 7fec4cc10b80 1 -- 172.21.15.6:0/3868880174 shutdown_connections 2024-05-09T04:55:36.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.513+0000 7fec4cc10b80 1 --2- 172.21.15.6:0/3868880174 >> [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] conn(0x55b2f0c7a1a0 0x55b2f0c7c340 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.513+0000 7fec4cc10b80 1 --2- 172.21.15.6:0/3868880174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b2f0c67640 0x55b2f0c6b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.513+0000 7fec4cc10b80 1 --2- 172.21.15.6:0/3868880174 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b2f0ae4450 0x55b2f0c6ae30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.513+0000 7fec4cc10b80 1 --2- 172.21.15.6:0/3868880174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2f0c61b60 0x55b2f0c6b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.513+0000 7fec21600700 0 client.15168 ms_handle_reset on v2:172.21.15.186:6838/3954883105 2024-05-09T04:55:36.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.513+0000 7fec4cc10b80 1 -- 172.21.15.6:0/3868880174 >> 172.21.15.6:0/3868880174 conn(0x55b2f0b28e20 msgr2=0x55b2f0c5c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:36.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.513+0000 7fec4cc10b80 1 -- 172.21.15.6:0/3868880174 shutdown_connections 2024-05-09T04:55:36.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.513+0000 7fec4cc10b80 1 -- 172.21.15.6:0/3868880174 wait complete. 2024-05-09T04:55:36.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.513+0000 7fec4cc10b80 20 client.15168 trim_cache size 0 max 16384 2024-05-09T04:55:36.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.537+0000 7fec4cc10b80 1 Processor -- start 2024-05-09T04:55:36.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.537+0000 7fec4cc10b80 1 -- start start 2024-05-09T04:55:36.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.537+0000 7fec4cc10b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b2f0ae4450 0x55b2f0c65490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:36.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.537+0000 7fec4cc10b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b2f0c61b60 0x55b2f0c61f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:36.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.537+0000 7fec4cc10b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2f0c67640 0x55b2f0c65eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:36.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.537+0000 7fec4cc10b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55b2f0c7a680 con 0x55b2f0c61b60 2024-05-09T04:55:36.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.537+0000 7fec4cc10b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b2f0c7ad50 con 0x55b2f0ae4450 2024-05-09T04:55:36.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.537+0000 7fec4cc10b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55b2f0c7b420 con 0x55b2f0c67640 2024-05-09T04:55:36.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.538+0000 7fec36a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b2f0c61b60 0x55b2f0c61f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:36.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.538+0000 7fec37e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b2f0ae4450 0x55b2f0c65490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:36.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.538+0000 7fec37e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b2f0ae4450 0x55b2f0c65490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:41134/0 (socket says 172.21.15.6:41134) 2024-05-09T04:55:36.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.538+0000 7fec37e00700 1 -- 172.21.15.6:0/209567934 learned_addr learned my addr 172.21.15.6:0/209567934 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:55:36.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.538+0000 7fec37e00700 1 --2- 172.21.15.6:0/209567934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2f0c67640 0x55b2f0c65eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:36.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.538+0000 7fec37e00700 1 -- 172.21.15.6:0/209567934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2f0c67640 msgr2=0x55b2f0c65eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:36.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.538+0000 7fec37e00700 1 --2- 172.21.15.6:0/209567934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2f0c67640 0x55b2f0c65eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.538+0000 7fec37e00700 1 -- 172.21.15.6:0/209567934 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b2f0c61b60 msgr2=0x55b2f0c61f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:36.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.538+0000 7fec37e00700 1 --2- 172.21.15.6:0/209567934 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b2f0c61b60 0x55b2f0c61f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.538+0000 7fec37e00700 1 -- 172.21.15.6:0/209567934 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2f0c66ad0 con 0x55b2f0ae4450 2024-05-09T04:55:36.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.538+0000 7fec37e00700 1 --2- 172.21.15.6:0/209567934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b2f0ae4450 0x55b2f0c65490 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fec30001eb0 tx=0x7fec3001a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9772e9e514ea3216 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:36.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.539+0000 7fec23e00700 1 -- 172.21.15.6:0/209567934 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3000bac0 con 0x55b2f0ae4450 2024-05-09T04:55:36.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.539+0000 7fec23e00700 1 -- 172.21.15.6:0/209567934 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec3000bc60 con 0x55b2f0ae4450 2024-05-09T04:55:36.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.539+0000 7fec23e00700 1 -- 172.21.15.6:0/209567934 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec30022a60 con 0x55b2f0ae4450 2024-05-09T04:55:36.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.540+0000 7fec4cc10b80 1 -- 172.21.15.6:0/209567934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b2f0ae4450 msgr2=0x55b2f0c65490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:36.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.540+0000 7fec4cc10b80 1 --2- 172.21.15.6:0/209567934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b2f0ae4450 0x55b2f0c65490 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fec30001eb0 tx=0x7fec3001a040 comp rx=0 tx=0).stop 2024-05-09T04:55:36.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.540+0000 7fec4cc10b80 1 -- 172.21.15.6:0/209567934 shutdown_connections 2024-05-09T04:55:36.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.540+0000 7fec4cc10b80 1 --2- 172.21.15.6:0/209567934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2f0c67640 0x55b2f0c65eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.540+0000 7fec4cc10b80 1 --2- 172.21.15.6:0/209567934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b2f0ae4450 0x55b2f0c65490 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.540+0000 7fec4cc10b80 1 --2- 172.21.15.6:0/209567934 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b2f0c61b60 0x55b2f0c61f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.540+0000 7fec4cc10b80 1 -- 172.21.15.6:0/209567934 >> 172.21.15.6:0/209567934 conn(0x55b2f0b28e20 msgr2=0x55b2f0c5bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:36.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.540+0000 7fec4cc10b80 1 -- 172.21.15.6:0/209567934 shutdown_connections 2024-05-09T04:55:36.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.540+0000 7fec4cc10b80 1 -- 172.21.15.6:0/209567934 wait complete. 2024-05-09T04:55:36.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.541+0000 7fec4cc10b80 1 -- start start 2024-05-09T04:55:36.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.541+0000 7fec4cc10b80 1 Processor -- start 2024-05-09T04:55:36.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.541+0000 7fec4cc10b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2f0ae4450 0x55b2f0c6ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:36.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.542+0000 7fec4cc10b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b2f0c61b60 0x55b2f0c6b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:36.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.542+0000 7fec4cc10b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b2f0c67640 0x55b2f0c6b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:36.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.542+0000 7fec4cc10b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55b2f0c7c1c0 con 0x55b2f0c61b60 2024-05-09T04:55:36.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.542+0000 7fec4cc10b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b2f0c7baf0 con 0x55b2f0c67640 2024-05-09T04:55:36.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.542+0000 7fec4cc10b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55b2f0c7c890 con 0x55b2f0ae4450 2024-05-09T04:55:36.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.542+0000 7fec4cc10b80 10 client.0 _ll_register_callbacks cb 0x55b2f0b1b540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:55:36.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.542+0000 7fec36a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b2f0c61b60 0x55b2f0c6b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:36.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.542+0000 7fec36a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b2f0c61b60 0x55b2f0c6b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:40720/0 (socket says 172.21.15.6:40720) 2024-05-09T04:55:36.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.542+0000 7fec37e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2f0ae4450 0x55b2f0c6ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:36.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.542+0000 7fec36a00700 1 -- 172.21.15.6:0/3202691747 learned_addr learned my addr 172.21.15.6:0/3202691747 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:55:36.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.542+0000 7fec37e00700 1 --2- 172.21.15.6:0/3202691747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b2f0c67640 0x55b2f0c6b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:36.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.542+0000 7fec36a00700 1 -- 172.21.15.6:0/3202691747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2f0ae4450 msgr2=0x55b2f0c6ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:36.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.542+0000 7fec36a00700 1 --2- 172.21.15.6:0/3202691747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2f0ae4450 0x55b2f0c6ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.542+0000 7fec36a00700 1 -- 172.21.15.6:0/3202691747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b2f0c67640 msgr2=0x55b2f0c6b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:36.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.542+0000 7fec36a00700 1 --2- 172.21.15.6:0/3202691747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b2f0c67640 0x55b2f0c6b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.542+0000 7fec36a00700 1 -- 172.21.15.6:0/3202691747 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2f0c83e50 con 0x55b2f0c61b60 2024-05-09T04:55:36.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.543+0000 7fec36a00700 1 --2- 172.21.15.6:0/3202691747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b2f0c61b60 0x55b2f0c6b1c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x55b2f0c7ca40 tx=0x7fec240044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26cc05311da23e42 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:36.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.543+0000 7fec21600700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-05-09T04:55:36.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.543+0000 7fec21600700 1 -- 172.21.15.6:0/3202691747 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec24004830 con 0x55b2f0c61b60 2024-05-09T04:55:36.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.543+0000 7fec4cc10b80 1 -- 172.21.15.6:0/3202691747 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec3000bc60 con 0x55b2f0c61b60 2024-05-09T04:55:36.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.543+0000 7fec21600700 1 -- 172.21.15.6:0/3202691747 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fec240049d0 con 0x55b2f0c61b60 2024-05-09T04:55:36.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.543+0000 7fec21600700 1 -- 172.21.15.6:0/3202691747 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fec2402c050 con 0x55b2f0c61b60 2024-05-09T04:55:36.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.543+0000 7fec4cc10b80 10 client.15174 fetch_fsmap learned FSMap version 30 2024-05-09T04:55:36.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.543+0000 7fec4cc10b80 1 -- 172.21.15.6:0/3202691747 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b2f0ce67c0 con 0x55b2f0c61b60 2024-05-09T04:55:36.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.544+0000 7fec21600700 1 -- 172.21.15.6:0/3202691747 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec3000bc60 con 0x55b2f0c61b60 2024-05-09T04:55:36.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.544+0000 7fec21600700 1 -- 172.21.15.6:0/3202691747 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fec2401d4b0 con 0x55b2f0c61b60 2024-05-09T04:55:36.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.544+0000 7fec4cc10b80 10 client.15174 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T04:55:36.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.544+0000 7fec4cc10b80 10 client.15174 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T04:55:36.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.544+0000 7fec4cc10b80 20 client.15174 populate_metadata read hostname 'smithi006' 2024-05-09T04:55:36.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.544+0000 7fec4cc10b80 1 --2- 172.21.15.6:0/3202691747 >> [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] conn(0x55b2f0ce8c70 0x55b2f0cea0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:36.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.544+0000 7fec4cc10b80 4 client.15174 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T04:55:36.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.544+0000 7fec4cc10b80 1 -- 172.21.15.6:0/3202691747 --> [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] -- command(tid 0: {"prefix": "status"}) -- 0x55b2f0b91c40 con 0x55b2f0ce8c70 2024-05-09T04:55:36.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.545+0000 7fec37e00700 1 --2- 172.21.15.6:0/3202691747 >> [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] conn(0x55b2f0ce8c70 0x55b2f0cea0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:36.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.545+0000 7fec37e00700 1 --2- 172.21.15.6:0/3202691747 >> [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] conn(0x55b2f0ce8c70 0x55b2f0cea0e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=459758939014631e server_cookie=67b5e32a0dd3dac4 in_seq=0 out_seq=0 2024-05-09T04:55:36.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.545+0000 7fec21600700 10 client.15174 ms_handle_connect on v2:172.21.15.186:6838/3954883105 2024-05-09T04:55:36.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.546+0000 7fec21600700 1 -- 172.21.15.6:0/3202691747 <== mds.3 v2:172.21.15.186:6838/3954883105 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b2f0b91c40 con 0x55b2f0ce8c70 2024-05-09T04:55:36.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.546+0000 7fec21600700 10 client.15174 handle_command_reply: tid=0 2024-05-09T04:55:36.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.546+0000 7fec4cc10b80 1 client.15174 shutdown 2024-05-09T04:55:36.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.546+0000 7fec4cc10b80 1 -- 172.21.15.6:0/3202691747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b2f0c61b60 msgr2=0x55b2f0c6b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:36.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.546+0000 7fec4cc10b80 1 --2- 172.21.15.6:0/3202691747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b2f0c61b60 0x55b2f0c6b1c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x55b2f0c7ca40 tx=0x7fec240044c0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.546+0000 7fec4cc10b80 1 -- 172.21.15.6:0/3202691747 shutdown_connections 2024-05-09T04:55:36.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.546+0000 7fec4cc10b80 1 --2- 172.21.15.6:0/3202691747 >> [v2:172.21.15.186:6838/3954883105,v1:172.21.15.186:6839/3954883105] conn(0x55b2f0ce8c70 0x55b2f0cea0e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.546+0000 7fec4cc10b80 1 --2- 172.21.15.6:0/3202691747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b2f0c67640 0x55b2f0c6b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.546+0000 7fec4cc10b80 1 --2- 172.21.15.6:0/3202691747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b2f0ae4450 0x55b2f0c6ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.546+0000 7fec4cc10b80 1 --2- 172.21.15.6:0/3202691747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b2f0c61b60 0x55b2f0c6b1c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.546+0000 7fec21600700 0 client.15174 ms_handle_reset on v2:172.21.15.186:6838/3954883105 2024-05-09T04:55:36.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.546+0000 7fec4cc10b80 1 -- 172.21.15.6:0/3202691747 >> 172.21.15.6:0/3202691747 conn(0x55b2f0b28e20 msgr2=0x55b2f0c76440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:36.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.546+0000 7fec4cc10b80 1 -- 172.21.15.6:0/3202691747 shutdown_connections 2024-05-09T04:55:36.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.546+0000 7fec4cc10b80 1 -- 172.21.15.6:0/3202691747 wait complete. 2024-05-09T04:55:36.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.546+0000 7fec4cc10b80 20 client.15174 trim_cache size 0 max 16384 2024-05-09T04:55:36.549 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-09T04:55:36.549 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "f74d813c-0dbd-11ef-bc97-c7b262605968", 2024-05-09T04:55:36.549 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 3, 2024-05-09T04:55:36.549 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24511, 2024-05-09T04:55:36.549 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-05-09T04:55:36.549 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-05-09T04:55:36.549 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-05-09T04:55:36.549 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 274.79863552099999, 2024-05-09T04:55:36.550 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 30, 2024-05-09T04:55:36.550 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 89, 2024-05-09T04:55:36.550 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 89, 2024-05-09T04:55:36.550 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 395.26804967599998 2024-05-09T04:55:36.550 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-09T04:55:36.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.547+0000 7fec3ca00700 1 -- 172.21.15.6:0/4137361282 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fec1c0b2910 msgr2=0x7fec1c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:36.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.547+0000 7fec3ca00700 1 --2- 172.21.15.6:0/4137361282 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fec1c0b2910 0x7fec1c0b4e20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fec28000f80 tx=0x7fec280e8530 comp rx=0 tx=0).stop 2024-05-09T04:55:36.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.547+0000 7fec3ca00700 1 -- 172.21.15.6:0/4137361282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec38105310 msgr2=0x7fec38103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:36.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.547+0000 7fec3ca00700 1 --2- 172.21.15.6:0/4137361282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec38105310 0x7fec38103000 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fec2800ef10 tx=0x7fec2800c3a0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.547+0000 7fec3ca00700 1 -- 172.21.15.6:0/4137361282 shutdown_connections 2024-05-09T04:55:36.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.547+0000 7fec3ca00700 1 --2- 172.21.15.6:0/4137361282 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec3810e740 0x7fec38103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.547+0000 7fec3ca00700 1 --2- 172.21.15.6:0/4137361282 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fec1c0b2910 0x7fec1c0b4e20 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.547+0000 7fec3ca00700 1 --2- 172.21.15.6:0/4137361282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec38105310 0x7fec38103000 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.547+0000 7fec3ca00700 1 --2- 172.21.15.6:0/4137361282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec38105c60 0x7fec38103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:36.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.547+0000 7fec3ca00700 1 -- 172.21.15.6:0/4137361282 >> 172.21.15.6:0/4137361282 conn(0x7fec38078c40 msgr2=0x7fec380ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:36.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.547+0000 7fec3ca00700 1 -- 172.21.15.6:0/4137361282 shutdown_connections 2024-05-09T04:55:36.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:36.548+0000 7fec3ca00700 1 -- 172.21.15.6:0/4137361282 wait complete. 2024-05-09T04:55:37.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:36 smithi186 ceph-mon[34668]: pgmap v519: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-05-09T04:55:37.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:36 smithi169 ceph-mon[34075]: pgmap v519: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-05-09T04:55:37.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:36 smithi006 ceph-mon[29835]: pgmap v519: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-05-09T04:55:37.469 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.h status 2024-05-09T04:55:37.959 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:55:38.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:38 smithi006 ceph-mon[29835]: pgmap v520: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-09T04:55:38.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:38 smithi186 ceph-mon[34668]: pgmap v520: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-09T04:55:38.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:38 smithi169 ceph-mon[34075]: pgmap v520: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-09T04:55:40.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.378+0000 7f6f0e200700 1 -- 172.21.15.6:0/2351472437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f08105480 msgr2=0x7f6f08105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:40.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.378+0000 7f6f0e200700 1 --2- 172.21.15.6:0/2351472437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f08105480 0x7f6f08105890 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8009990 tx=0x7f6ef8009ca0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.379+0000 7f6f0e200700 1 -- 172.21.15.6:0/2351472437 shutdown_connections 2024-05-09T04:55:40.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.379+0000 7f6f0e200700 1 --2- 172.21.15.6:0/2351472437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f0810e8f0 0x7f6f08110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.379+0000 7f6f0e200700 1 --2- 172.21.15.6:0/2351472437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f08105480 0x7f6f08105890 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.379+0000 7f6f0e200700 1 --2- 172.21.15.6:0/2351472437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f08105dd0 0x7f6f0810e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.379+0000 7f6f0e200700 1 -- 172.21.15.6:0/2351472437 >> 172.21.15.6:0/2351472437 conn(0x7f6f080fef30 msgr2=0x7f6f08101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:40.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.379+0000 7f6f0e200700 1 -- 172.21.15.6:0/2351472437 shutdown_connections 2024-05-09T04:55:40.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.379+0000 7f6f0e200700 1 -- 172.21.15.6:0/2351472437 wait complete. 2024-05-09T04:55:40.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.380+0000 7f6f0e200700 1 Processor -- start 2024-05-09T04:55:40.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.380+0000 7f6f0e200700 1 -- start start 2024-05-09T04:55:40.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.381+0000 7f6f0e200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f08105480 0x7f6f08103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:40.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.381+0000 7f6f0e200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f08105dd0 0x7f6f081036b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:40.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.381+0000 7f6f0e200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f0810e8f0 0x7f6f08103bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:40.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.381+0000 7f6f0e200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6f08114490 con 0x7f6f08105dd0 2024-05-09T04:55:40.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.381+0000 7f6f0e200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6f08114310 con 0x7f6f0810e8f0 2024-05-09T04:55:40.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.381+0000 7f6f0e200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6f08114610 con 0x7f6f08105480 2024-05-09T04:55:40.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.381+0000 7f6f06a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f08105dd0 0x7f6f081036b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:40.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.381+0000 7f6f06a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f08105dd0 0x7f6f081036b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58110/0 (socket says 172.21.15.6:58110) 2024-05-09T04:55:40.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.381+0000 7f6f07e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f0810e8f0 0x7f6f08103bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:40.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.381+0000 7f6f06a00700 1 -- 172.21.15.6:0/2416582694 learned_addr learned my addr 172.21.15.6:0/2416582694 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:55:40.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.381+0000 7f6f07400700 1 --2- 172.21.15.6:0/2416582694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f08105480 0x7f6f08103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:40.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.382+0000 7f6f07400700 1 -- 172.21.15.6:0/2416582694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f0810e8f0 msgr2=0x7f6f08103bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:40.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.382+0000 7f6f07400700 1 --2- 172.21.15.6:0/2416582694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f0810e8f0 0x7f6f08103bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.382+0000 7f6f07400700 1 -- 172.21.15.6:0/2416582694 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f08105dd0 msgr2=0x7f6f081036b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:40.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.382+0000 7f6f07400700 1 --2- 172.21.15.6:0/2416582694 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f08105dd0 0x7f6f081036b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.382+0000 7f6f07400700 1 -- 172.21.15.6:0/2416582694 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f081a9160 con 0x7f6f08105480 2024-05-09T04:55:40.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.382+0000 7f6f06a00700 1 --2- 172.21.15.6:0/2416582694 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f08105dd0 0x7f6f081036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:55:40.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.382+0000 7f6f07400700 1 --2- 172.21.15.6:0/2416582694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f08105480 0x7f6f08103170 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6ef80098e0 tx=0x7f6ef80057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:40.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.382+0000 7f6effe00700 1 -- 172.21.15.6:0/2416582694 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef8027070 con 0x7f6f08105480 2024-05-09T04:55:40.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.382+0000 7f6effe00700 1 -- 172.21.15.6:0/2416582694 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ef8022470 con 0x7f6f08105480 2024-05-09T04:55:40.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.382+0000 7f6f0e200700 1 -- 172.21.15.6:0/2416582694 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f081a93f0 con 0x7f6f08105480 2024-05-09T04:55:40.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.382+0000 7f6effe00700 1 -- 172.21.15.6:0/2416582694 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef8017650 con 0x7f6f08105480 2024-05-09T04:55:40.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.383+0000 7f6f0e200700 1 -- 172.21.15.6:0/2416582694 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f081a98a0 con 0x7f6f08105480 2024-05-09T04:55:40.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.387+0000 7f6effe00700 1 -- 172.21.15.6:0/2416582694 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 19) ==== 149702+0+0 (secure 0 0 0) 0x7f6ef802a070 con 0x7f6f08105480 2024-05-09T04:55:40.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.388+0000 7f6f1cd9bb80 1 Processor -- start 2024-05-09T04:55:40.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.388+0000 7f6f1cd9bb80 1 -- start start 2024-05-09T04:55:40.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.389+0000 7f6f1cd9bb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563841dd8450 0x563841f5aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:40.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.389+0000 7f6f1cd9bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563841f5b410 0x563841f554c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:40.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.389+0000 7f6f1cd9bb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563841f55a00 0x563841f55e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:40.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.389+0000 7f6f1cd9bb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x563841e78260 con 0x563841f55a00 2024-05-09T04:55:40.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.389+0000 7f6f1cd9bb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563841e16090 con 0x563841dd8450 2024-05-09T04:55:40.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.389+0000 7f6f1cd9bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x563841e74220 con 0x563841f5b410 2024-05-09T04:55:40.391 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.389+0000 7f6f07e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563841dd8450 0x563841f5aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:40.391 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.389+0000 7f6f07e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563841dd8450 0x563841f5aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:36504/0 (socket says 172.21.15.6:36504) 2024-05-09T04:55:40.391 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.389+0000 7f6f07e00700 1 -- 172.21.15.6:0/3681597090 learned_addr learned my addr 172.21.15.6:0/3681597090 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:55:40.391 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.389+0000 7f6f07400700 1 --2- 172.21.15.6:0/3681597090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563841f5b410 0x563841f554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:40.391 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.389+0000 7f6f06a00700 1 --2- 172.21.15.6:0/3681597090 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563841f55a00 0x563841f55e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:40.391 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.389+0000 7f6effe00700 1 --2- 172.21.15.6:0/2416582694 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f6eec0b2990 0x7f6eec0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:40.391 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.390+0000 7f6effe00700 1 -- 172.21.15.6:0/2416582694 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f6ef80e33c0 con 0x7f6f08105480 2024-05-09T04:55:40.391 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.390+0000 7f6f07e00700 1 -- 172.21.15.6:0/3681597090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563841f5b410 msgr2=0x563841f554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:40.392 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.390+0000 7f6f07e00700 1 --2- 172.21.15.6:0/3681597090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563841f5b410 0x563841f554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.392 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.390+0000 7f6f07e00700 1 -- 172.21.15.6:0/3681597090 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563841f55a00 msgr2=0x563841f55e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:40.392 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.390+0000 7f6f07e00700 1 --2- 172.21.15.6:0/3681597090 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563841f55a00 0x563841f55e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.392 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.390+0000 7f6f07e00700 1 -- 172.21.15.6:0/3681597090 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563841cf3440 con 0x563841dd8450 2024-05-09T04:55:40.392 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.390+0000 7f6f07e00700 1 --2- 172.21.15.6:0/2416582694 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f6eec0b2990 0x7f6eec0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:40.392 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.390+0000 7f6f07400700 1 --2- 172.21.15.6:0/3681597090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563841f5b410 0x563841f554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:55:40.392 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.390+0000 7f6f07e00700 1 --2- 172.21.15.6:0/3681597090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563841dd8450 0x563841f5aed0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6f00011790 tx=0x7f6f00011aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71de5ef934e29c05 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:40.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.391+0000 7f6efcc00700 1 -- 172.21.15.6:0/3681597090 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0001c040 con 0x563841dd8450 2024-05-09T04:55:40.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.391+0000 7f6efcc00700 1 -- 172.21.15.6:0/3681597090 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f00018b30 con 0x563841dd8450 2024-05-09T04:55:40.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.391+0000 7f6efcc00700 1 -- 172.21.15.6:0/3681597090 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f00023910 con 0x563841dd8450 2024-05-09T04:55:40.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.391+0000 7f6f07e00700 1 --2- 172.21.15.6:0/2416582694 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f6eec0b2990 0x7f6eec0b4ea0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6f00022000 tx=0x7f6f0001f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:40.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.392+0000 7f6f1cd9bb80 1 -- 172.21.15.6:0/3681597090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563841dd8450 msgr2=0x563841f5aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:40.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.392+0000 7f6f1cd9bb80 1 --2- 172.21.15.6:0/3681597090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563841dd8450 0x563841f5aed0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6f00011790 tx=0x7f6f00011aa0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.392+0000 7f6f1cd9bb80 1 -- 172.21.15.6:0/3681597090 shutdown_connections 2024-05-09T04:55:40.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.392+0000 7f6f1cd9bb80 1 --2- 172.21.15.6:0/3681597090 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563841f55a00 0x563841f55e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.392+0000 7f6f1cd9bb80 1 --2- 172.21.15.6:0/3681597090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563841dd8450 0x563841f5aed0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.392+0000 7f6f1cd9bb80 1 --2- 172.21.15.6:0/3681597090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563841f5b410 0x563841f554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.392+0000 7f6f1cd9bb80 1 -- 172.21.15.6:0/3681597090 >> 172.21.15.6:0/3681597090 conn(0x563841e1ce20 msgr2=0x563841f4fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:40.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.392+0000 7f6f1cd9bb80 1 -- 172.21.15.6:0/3681597090 shutdown_connections 2024-05-09T04:55:40.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.392+0000 7f6f1cd9bb80 1 -- 172.21.15.6:0/3681597090 wait complete. 2024-05-09T04:55:40.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.393+0000 7f6f1cd9bb80 1 -- start start 2024-05-09T04:55:40.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.393+0000 7f6f1cd9bb80 1 Processor -- start 2024-05-09T04:55:40.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.393+0000 7f6f1cd9bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563841dd8450 0x563841f5eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:40.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.394+0000 7f6f1cd9bb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563841f55a00 0x563841f5f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:40.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.394+0000 7f6f1cd9bb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563841f5b410 0x563841f5f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:40.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.394+0000 7f6f1cd9bb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x563841f61f00 con 0x563841f5b410 2024-05-09T04:55:40.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.394+0000 7f6f1cd9bb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563841ddd670 con 0x563841f55a00 2024-05-09T04:55:40.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.394+0000 7f6f1cd9bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x563841f62080 con 0x563841dd8450 2024-05-09T04:55:40.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.394+0000 7f6f1cd9bb80 10 client.0 _ll_register_callbacks cb 0x563841e0f540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:55:40.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.394+0000 7f6f07400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563841dd8450 0x563841f5eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:40.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.394+0000 7f6f07e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563841f5b410 0x563841f5f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:40.397 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.394+0000 7f6f07400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563841dd8450 0x563841f5eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:48914/0 (socket says 172.21.15.6:48914) 2024-05-09T04:55:40.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.394+0000 7f6f07400700 1 -- 172.21.15.6:0/3385097720 learned_addr learned my addr 172.21.15.6:0/3385097720 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:55:40.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.394+0000 7f6f06a00700 1 --2- 172.21.15.6:0/3385097720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563841f55a00 0x563841f5f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:40.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.394+0000 7f6f06a00700 1 -- 172.21.15.6:0/3385097720 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563841dd8450 msgr2=0x563841f5eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:40.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.394+0000 7f6f06a00700 1 --2- 172.21.15.6:0/3385097720 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563841dd8450 0x563841f5eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.394+0000 7f6f06a00700 1 -- 172.21.15.6:0/3385097720 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563841f5b410 msgr2=0x563841f5f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:40.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.394+0000 7f6f06a00700 1 --2- 172.21.15.6:0/3385097720 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563841f5b410 0x563841f5f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.394+0000 7f6f06a00700 1 -- 172.21.15.6:0/3385097720 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563841f6b3b0 con 0x563841f55a00 2024-05-09T04:55:40.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.395+0000 7f6f06a00700 1 --2- 172.21.15.6:0/3385097720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563841f55a00 0x563841f5f080 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4011ce0 tx=0x7f6ef4009d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=477f24c9ba950eae server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:40.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.395+0000 7f6ef1600700 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-05-09T04:55:40.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.395+0000 7f6ef1600700 1 -- 172.21.15.6:0/3385097720 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4004260 con 0x563841f55a00 2024-05-09T04:55:40.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.395+0000 7f6f1cd9bb80 1 -- 172.21.15.6:0/3385097720 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f00018b30 con 0x563841f55a00 2024-05-09T04:55:40.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.395+0000 7f6ef1600700 1 -- 172.21.15.6:0/3385097720 <== mon.2 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f6ef4004400 con 0x563841f55a00 2024-05-09T04:55:40.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.395+0000 7f6ef1600700 1 -- 172.21.15.6:0/3385097720 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6ef4025050 con 0x563841f55a00 2024-05-09T04:55:40.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.395+0000 7f6f1cd9bb80 10 client.25334 fetch_fsmap learned FSMap version 30 2024-05-09T04:55:40.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.396+0000 7f6f1cd9bb80 1 -- 172.21.15.6:0/3385097720 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563841f6ccc0 con 0x563841f55a00 2024-05-09T04:55:40.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.396+0000 7f6ef1600700 1 -- 172.21.15.6:0/3385097720 <== mon.2 v2:172.21.15.169:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f00018b30 con 0x563841f55a00 2024-05-09T04:55:40.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.396+0000 7f6ef1600700 1 -- 172.21.15.6:0/3385097720 <== mon.2 v2:172.21.15.169:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6ef4003b80 con 0x563841f55a00 2024-05-09T04:55:40.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.396+0000 7f6f1cd9bb80 10 client.25334 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T04:55:40.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.396+0000 7f6f1cd9bb80 10 client.25334 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-09T04:55:40.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.396+0000 7f6f1cd9bb80 20 client.25334 populate_metadata read hostname 'smithi006' 2024-05-09T04:55:40.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.396+0000 7f6f1cd9bb80 1 --2- 172.21.15.6:0/3385097720 >> [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] conn(0x563841f6f2c0 0x563841f70730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:40.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.396+0000 7f6f1cd9bb80 4 client.25334 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:55:40.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.396+0000 7f6f1cd9bb80 1 -- 172.21.15.6:0/3385097720 --> [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563841f21820 con 0x563841f6f2c0 2024-05-09T04:55:40.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.397+0000 7f6f07400700 1 --2- 172.21.15.6:0/3385097720 >> [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] conn(0x563841f6f2c0 0x563841f70730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:40.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.397+0000 7f6f07400700 1 --2- 172.21.15.6:0/3385097720 >> [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] conn(0x563841f6f2c0 0x563841f70730 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b225247b806a821f server_cookie=993443c287ca866a in_seq=0 out_seq=0 2024-05-09T04:55:40.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.397+0000 7f6ef1600700 10 client.25334 ms_handle_connect on v2:172.21.15.169:6836/1845715294 2024-05-09T04:55:40.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.398+0000 7f6ef1600700 1 -- 172.21.15.6:0/3385097720 <== mds.2 v2:172.21.15.169:6836/1845715294 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563841f21820 con 0x563841f6f2c0 2024-05-09T04:55:40.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.398+0000 7f6ef1600700 10 client.25334 handle_command_reply: tid=0 2024-05-09T04:55:40.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.398+0000 7f6f1cd9bb80 1 client.25334 shutdown 2024-05-09T04:55:40.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.399+0000 7f6f1cd9bb80 1 -- 172.21.15.6:0/3385097720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563841f55a00 msgr2=0x563841f5f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:40.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.399+0000 7f6f1cd9bb80 1 --2- 172.21.15.6:0/3385097720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563841f55a00 0x563841f5f080 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4011ce0 tx=0x7f6ef4009d70 comp rx=0 tx=0).stop 2024-05-09T04:55:40.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.399+0000 7f6f1cd9bb80 1 -- 172.21.15.6:0/3385097720 shutdown_connections 2024-05-09T04:55:40.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.399+0000 7f6f1cd9bb80 1 --2- 172.21.15.6:0/3385097720 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563841f5b410 0x563841f5f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.399+0000 7f6f1cd9bb80 1 --2- 172.21.15.6:0/3385097720 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563841dd8450 0x563841f5eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.399+0000 7f6f1cd9bb80 1 --2- 172.21.15.6:0/3385097720 >> [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] conn(0x563841f6f2c0 0x563841f70730 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.399+0000 7f6f1cd9bb80 1 --2- 172.21.15.6:0/3385097720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563841f55a00 0x563841f5f080 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.399+0000 7f6f1cd9bb80 1 -- 172.21.15.6:0/3385097720 >> 172.21.15.6:0/3385097720 conn(0x563841e1ce20 msgr2=0x563841f4fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:40.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.399+0000 7f6ef1600700 0 client.25334 ms_handle_reset on v2:172.21.15.169:6836/1845715294 2024-05-09T04:55:40.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.399+0000 7f6f1cd9bb80 1 -- 172.21.15.6:0/3385097720 shutdown_connections 2024-05-09T04:55:40.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.399+0000 7f6f1cd9bb80 1 -- 172.21.15.6:0/3385097720 wait complete. 2024-05-09T04:55:40.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.399+0000 7f6f1cd9bb80 20 client.25334 trim_cache size 0 max 16384 2024-05-09T04:55:40.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.422+0000 7f6f1cd9bb80 1 Processor -- start 2024-05-09T04:55:40.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.423+0000 7f6f1cd9bb80 1 -- start start 2024-05-09T04:55:40.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.423+0000 7f6f1cd9bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563841dd8450 0x563841f554c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:40.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.423+0000 7f6f1cd9bb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563841f55a00 0x563841f59330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:40.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.423+0000 7f6f1cd9bb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563841f5b410 0x563841f5f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:40.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.423+0000 7f6f1cd9bb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x563841f6ea70 con 0x563841f55a00 2024-05-09T04:55:40.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.423+0000 7f6f1cd9bb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563841f6f140 con 0x563841f5b410 2024-05-09T04:55:40.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.423+0000 7f6f1cd9bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x563841f6f810 con 0x563841dd8450 2024-05-09T04:55:40.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.423+0000 7f6f06a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563841f55a00 0x563841f59330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:40.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.423+0000 7f6f07400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563841dd8450 0x563841f554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:40.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.423+0000 7f6f07e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563841f5b410 0x563841f5f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:40.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.423+0000 7f6f07400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563841dd8450 0x563841f554c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:48926/0 (socket says 172.21.15.6:48926) 2024-05-09T04:55:40.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.424+0000 7f6f06a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563841f55a00 0x563841f59330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58142/0 (socket says 172.21.15.6:58142) 2024-05-09T04:55:40.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.424+0000 7f6f06a00700 1 -- 172.21.15.6:0/724624980 learned_addr learned my addr 172.21.15.6:0/724624980 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:55:40.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.424+0000 7f6f07e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563841f5b410 0x563841f5f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:36522/0 (socket says 172.21.15.6:36522) 2024-05-09T04:55:40.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.424+0000 7f6f07e00700 1 -- 172.21.15.6:0/724624980 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563841dd8450 msgr2=0x563841f554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:40.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.424+0000 7f6f07e00700 1 --2- 172.21.15.6:0/724624980 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563841dd8450 0x563841f554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.424+0000 7f6f07e00700 1 -- 172.21.15.6:0/724624980 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563841f55a00 msgr2=0x563841f59330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:40.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.424+0000 7f6f07e00700 1 --2- 172.21.15.6:0/724624980 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563841f55a00 0x563841f59330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.424+0000 7f6f07e00700 1 -- 172.21.15.6:0/724624980 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563841f62030 con 0x563841f5b410 2024-05-09T04:55:40.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.424+0000 7f6f07400700 1 --2- 172.21.15.6:0/724624980 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563841dd8450 0x563841f554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:55:40.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.424+0000 7f6f07e00700 1 --2- 172.21.15.6:0/724624980 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563841f5b410 0x563841f5f400 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6f000234c0 tx=0x7f6f000094b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53b74654fbf9300a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:40.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.425+0000 7f6ef3e00700 1 -- 172.21.15.6:0/724624980 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0001c050 con 0x563841f5b410 2024-05-09T04:55:40.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.425+0000 7f6ef3e00700 1 -- 172.21.15.6:0/724624980 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f00023760 con 0x563841f5b410 2024-05-09T04:55:40.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.425+0000 7f6ef3e00700 1 -- 172.21.15.6:0/724624980 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f00018c20 con 0x563841f5b410 2024-05-09T04:55:40.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.426+0000 7f6f1cd9bb80 1 -- 172.21.15.6:0/724624980 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563841f5b410 msgr2=0x563841f5f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:40.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.426+0000 7f6f1cd9bb80 1 --2- 172.21.15.6:0/724624980 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563841f5b410 0x563841f5f400 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6f000234c0 tx=0x7f6f000094b0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.426+0000 7f6f1cd9bb80 1 -- 172.21.15.6:0/724624980 shutdown_connections 2024-05-09T04:55:40.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.426+0000 7f6f1cd9bb80 1 --2- 172.21.15.6:0/724624980 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563841f5b410 0x563841f5f400 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.426+0000 7f6f1cd9bb80 1 --2- 172.21.15.6:0/724624980 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563841dd8450 0x563841f554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.426+0000 7f6f1cd9bb80 1 --2- 172.21.15.6:0/724624980 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563841f55a00 0x563841f59330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.426+0000 7f6f1cd9bb80 1 -- 172.21.15.6:0/724624980 >> 172.21.15.6:0/724624980 conn(0x563841e1ce20 msgr2=0x563841f5d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:40.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.426+0000 7f6f1cd9bb80 1 -- 172.21.15.6:0/724624980 shutdown_connections 2024-05-09T04:55:40.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.426+0000 7f6f1cd9bb80 1 -- 172.21.15.6:0/724624980 wait complete. 2024-05-09T04:55:40.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.427+0000 7f6f1cd9bb80 1 -- start start 2024-05-09T04:55:40.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.427+0000 7f6f1cd9bb80 1 Processor -- start 2024-05-09T04:55:40.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.428+0000 7f6f1cd9bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563841dd8450 0x563841f4f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:40.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.428+0000 7f6f1cd9bb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563841f55a00 0x563841f4f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:40.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.428+0000 7f6f1cd9bb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563841f5b410 0x563841f78230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:40.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.428+0000 7f6f1cd9bb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x563841f705b0 con 0x563841f55a00 2024-05-09T04:55:40.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.428+0000 7f6f1cd9bb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563841f6fee0 con 0x563841f5b410 2024-05-09T04:55:40.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.428+0000 7f6f1cd9bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x563841f70c80 con 0x563841dd8450 2024-05-09T04:55:40.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.428+0000 7f6f1cd9bb80 10 client.0 _ll_register_callbacks cb 0x563841e0f540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:55:40.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.428+0000 7f6f06a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563841f55a00 0x563841f4f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:40.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.428+0000 7f6f06a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563841f55a00 0x563841f4f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58150/0 (socket says 172.21.15.6:58150) 2024-05-09T04:55:40.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.428+0000 7f6f06a00700 1 -- 172.21.15.6:0/1418628389 learned_addr learned my addr 172.21.15.6:0/1418628389 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:55:40.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.428+0000 7f6f07e00700 1 --2- 172.21.15.6:0/1418628389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563841f5b410 0x563841f78230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:40.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.428+0000 7f6f07400700 1 --2- 172.21.15.6:0/1418628389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563841dd8450 0x563841f4f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:40.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.428+0000 7f6f06a00700 1 -- 172.21.15.6:0/1418628389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563841dd8450 msgr2=0x563841f4f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:40.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.428+0000 7f6f06a00700 1 --2- 172.21.15.6:0/1418628389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563841dd8450 0x563841f4f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.429+0000 7f6f06a00700 1 -- 172.21.15.6:0/1418628389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563841f5b410 msgr2=0x563841f78230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:40.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.429+0000 7f6f06a00700 1 --2- 172.21.15.6:0/1418628389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563841f5b410 0x563841f78230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.429+0000 7f6f06a00700 1 -- 172.21.15.6:0/1418628389 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563841f502b0 con 0x563841f55a00 2024-05-09T04:55:40.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.429+0000 7f6f07e00700 1 --2- 172.21.15.6:0/1418628389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563841f5b410 0x563841f78230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:55:40.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.429+0000 7f6f06a00700 1 --2- 172.21.15.6:0/1418628389 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563841f55a00 0x563841f4f9d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4003ef0 tx=0x7f6ef4005580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c3c03d33f72076 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:40.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.429+0000 7f6ef1600700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-05-09T04:55:40.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.429+0000 7f6ef1600700 1 -- 172.21.15.6:0/1418628389 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef402d030 con 0x563841f55a00 2024-05-09T04:55:40.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.429+0000 7f6ef1600700 1 -- 172.21.15.6:0/1418628389 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f6ef4017040 con 0x563841f55a00 2024-05-09T04:55:40.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.429+0000 7f6f1cd9bb80 1 -- 172.21.15.6:0/1418628389 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f00023760 con 0x563841f55a00 2024-05-09T04:55:40.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.430+0000 7f6ef1600700 1 -- 172.21.15.6:0/1418628389 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6ef403a050 con 0x563841f55a00 2024-05-09T04:55:40.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.430+0000 7f6f1cd9bb80 10 client.15192 fetch_fsmap learned FSMap version 30 2024-05-09T04:55:40.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.430+0000 7f6f1cd9bb80 1 -- 172.21.15.6:0/1418628389 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563841fdacd0 con 0x563841f55a00 2024-05-09T04:55:40.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.430+0000 7f6ef1600700 1 -- 172.21.15.6:0/1418628389 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f00023760 con 0x563841f55a00 2024-05-09T04:55:40.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.430+0000 7f6ef1600700 1 -- 172.21.15.6:0/1418628389 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6ef401a580 con 0x563841f55a00 2024-05-09T04:55:40.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.430+0000 7f6f1cd9bb80 10 client.15192 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T04:55:40.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.430+0000 7f6f1cd9bb80 10 client.15192 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-09T04:55:40.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.430+0000 7f6f1cd9bb80 20 client.15192 populate_metadata read hostname 'smithi006' 2024-05-09T04:55:40.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.430+0000 7f6f1cd9bb80 1 --2- 172.21.15.6:0/1418628389 >> [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] conn(0x563841fdd0b0 0x563841fde520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:40.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.430+0000 7f6f1cd9bb80 4 client.15192 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T04:55:40.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.430+0000 7f6f1cd9bb80 1 -- 172.21.15.6:0/1418628389 --> [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] -- command(tid 0: {"prefix": "status"}) -- 0x563841e85c40 con 0x563841fdd0b0 2024-05-09T04:55:40.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.431+0000 7f6f07400700 1 --2- 172.21.15.6:0/1418628389 >> [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] conn(0x563841fdd0b0 0x563841fde520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:40.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.431+0000 7f6f07400700 1 --2- 172.21.15.6:0/1418628389 >> [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] conn(0x563841fdd0b0 0x563841fde520 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2a3ab7c041010c1d server_cookie=e123853f41777a48 in_seq=0 out_seq=0 2024-05-09T04:55:40.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.431+0000 7f6ef1600700 10 client.15192 ms_handle_connect on v2:172.21.15.169:6836/1845715294 2024-05-09T04:55:40.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.432+0000 7f6ef1600700 1 -- 172.21.15.6:0/1418628389 <== mds.2 v2:172.21.15.169:6836/1845715294 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x563841e85c40 con 0x563841fdd0b0 2024-05-09T04:55:40.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.432+0000 7f6ef1600700 10 client.15192 handle_command_reply: tid=0 2024-05-09T04:55:40.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.432+0000 7f6f1cd9bb80 1 client.15192 shutdown 2024-05-09T04:55:40.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.432+0000 7f6f1cd9bb80 1 -- 172.21.15.6:0/1418628389 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563841f55a00 msgr2=0x563841f4f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:40.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.432+0000 7f6f1cd9bb80 1 --2- 172.21.15.6:0/1418628389 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563841f55a00 0x563841f4f9d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4003ef0 tx=0x7f6ef4005580 comp rx=0 tx=0).stop 2024-05-09T04:55:40.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.432+0000 7f6f1cd9bb80 1 -- 172.21.15.6:0/1418628389 shutdown_connections 2024-05-09T04:55:40.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.432+0000 7f6f1cd9bb80 1 --2- 172.21.15.6:0/1418628389 >> [v2:172.21.15.169:6836/1845715294,v1:172.21.15.169:6837/1845715294] conn(0x563841fdd0b0 0x563841fde520 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.432+0000 7f6f1cd9bb80 1 --2- 172.21.15.6:0/1418628389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563841f5b410 0x563841f78230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.432+0000 7f6f1cd9bb80 1 --2- 172.21.15.6:0/1418628389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563841dd8450 0x563841f4f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.432+0000 7f6f1cd9bb80 1 --2- 172.21.15.6:0/1418628389 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563841f55a00 0x563841f4f9d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.432+0000 7f6f1cd9bb80 1 -- 172.21.15.6:0/1418628389 >> 172.21.15.6:0/1418628389 conn(0x563841e1ce20 msgr2=0x563841f6a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:40.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.432+0000 7f6ef1600700 0 client.15192 ms_handle_reset on v2:172.21.15.169:6836/1845715294 2024-05-09T04:55:40.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.432+0000 7f6f1cd9bb80 1 -- 172.21.15.6:0/1418628389 shutdown_connections 2024-05-09T04:55:40.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.432+0000 7f6f1cd9bb80 1 -- 172.21.15.6:0/1418628389 wait complete. 2024-05-09T04:55:40.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.432+0000 7f6f1cd9bb80 20 client.15192 trim_cache size 0 max 16384 2024-05-09T04:55:40.437 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-09T04:55:40.437 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "f74d813c-0dbd-11ef-bc97-c7b262605968", 2024-05-09T04:55:40.437 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 2, 2024-05-09T04:55:40.437 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24512, 2024-05-09T04:55:40.437 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-05-09T04:55:40.437 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-05-09T04:55:40.437 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-05-09T04:55:40.437 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 279.69943398800001, 2024-05-09T04:55:40.437 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 30, 2024-05-09T04:55:40.437 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 89, 2024-05-09T04:55:40.437 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 89, 2024-05-09T04:55:40.437 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 402.08883759600002 2024-05-09T04:55:40.437 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-09T04:55:40.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.433+0000 7f6f0e200700 1 -- 172.21.15.6:0/2416582694 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f6eec0b2990 msgr2=0x7f6eec0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:40.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.433+0000 7f6f0e200700 1 --2- 172.21.15.6:0/2416582694 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f6eec0b2990 0x7f6eec0b4ea0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6f00022000 tx=0x7f6f0001f000 comp rx=0 tx=0).stop 2024-05-09T04:55:40.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.433+0000 7f6f0e200700 1 -- 172.21.15.6:0/2416582694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f08105480 msgr2=0x7f6f08103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:40.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.433+0000 7f6f0e200700 1 --2- 172.21.15.6:0/2416582694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f08105480 0x7f6f08103170 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6ef80098e0 tx=0x7f6ef80057c0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.433+0000 7f6f0e200700 1 -- 172.21.15.6:0/2416582694 shutdown_connections 2024-05-09T04:55:40.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.433+0000 7f6f0e200700 1 --2- 172.21.15.6:0/2416582694 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f6eec0b2990 0x7f6eec0b4ea0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.433+0000 7f6f0e200700 1 --2- 172.21.15.6:0/2416582694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f0810e8f0 0x7f6f08103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.433+0000 7f6f0e200700 1 --2- 172.21.15.6:0/2416582694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f08105480 0x7f6f08103170 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.433+0000 7f6f0e200700 1 --2- 172.21.15.6:0/2416582694 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f08105dd0 0x7f6f081036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:40.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.433+0000 7f6f0e200700 1 -- 172.21.15.6:0/2416582694 >> 172.21.15.6:0/2416582694 conn(0x7f6f080fef30 msgr2=0x7f6f080ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:40.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.433+0000 7f6f0e200700 1 -- 172.21.15.6:0/2416582694 shutdown_connections 2024-05-09T04:55:40.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:40.434+0000 7f6f0e200700 1 -- 172.21.15.6:0/2416582694 wait complete. 2024-05-09T04:55:41.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:40 smithi186 ceph-mon[34668]: pgmap v521: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 852 B/s wr, 10 op/s 2024-05-09T04:55:41.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:40 smithi169 ceph-mon[34075]: pgmap v521: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 852 B/s wr, 10 op/s 2024-05-09T04:55:41.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:40 smithi006 ceph-mon[29835]: pgmap v521: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 852 B/s wr, 10 op/s 2024-05-09T04:55:41.337 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.g status 2024-05-09T04:55:41.810 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:55:43.031 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:42 smithi006 ceph-mon[29835]: pgmap v522: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-09T04:55:43.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:42 smithi186 ceph-mon[34668]: pgmap v522: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-09T04:55:43.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:42 smithi169 ceph-mon[34075]: pgmap v522: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-09T04:55:44.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.093+0000 7f6597000700 1 -- 172.21.15.6:0/795838501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6590115180 msgr2=0x7f65901175a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:44.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.093+0000 7f6597000700 1 --2- 172.21.15.6:0/795838501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6590115180 0x7f65901175a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f658800b0a0 tx=0x7f658800b3b0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.093+0000 7f6597000700 1 -- 172.21.15.6:0/795838501 shutdown_connections 2024-05-09T04:55:44.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.093+0000 7f6597000700 1 --2- 172.21.15.6:0/795838501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6590115180 0x7f65901175a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.093+0000 7f6597000700 1 --2- 172.21.15.6:0/795838501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6590075b60 0x7f6590075f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.093+0000 7f6597000700 1 --2- 172.21.15.6:0/795838501 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f65900764b0 0x7f6590114c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.093+0000 7f6597000700 1 -- 172.21.15.6:0/795838501 >> 172.21.15.6:0/795838501 conn(0x7f6590079d10 msgr2=0x7f659007a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:44.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.093+0000 7f6597000700 1 -- 172.21.15.6:0/795838501 shutdown_connections 2024-05-09T04:55:44.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.093+0000 7f6597000700 1 -- 172.21.15.6:0/795838501 wait complete. 2024-05-09T04:55:44.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.094+0000 7f6597000700 1 Processor -- start 2024-05-09T04:55:44.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.095+0000 7f6597000700 1 -- start start 2024-05-09T04:55:44.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.095+0000 7f6597000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6590075b60 0x7f65901a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:44.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.095+0000 7f6597000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65900764b0 0x7f65901a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:44.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.095+0000 7f6597000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6590115180 0x7f65901a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:44.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.095+0000 7f6597000700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f659011a880 con 0x7f6590075b60 2024-05-09T04:55:44.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.095+0000 7f6597000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f659011a700 con 0x7f65900764b0 2024-05-09T04:55:44.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.095+0000 7f6597000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f659011aa00 con 0x7f6590115180 2024-05-09T04:55:44.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.096+0000 7f6595c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6590075b60 0x7f65901a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:44.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.096+0000 7f6595200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65900764b0 0x7f65901a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:44.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.096+0000 7f6595c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6590075b60 0x7f65901a11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58178/0 (socket says 172.21.15.6:58178) 2024-05-09T04:55:44.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.096+0000 7f6595c00700 1 -- 172.21.15.6:0/3070225899 learned_addr learned my addr 172.21.15.6:0/3070225899 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:55:44.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.096+0000 7f6596600700 1 --2- 172.21.15.6:0/3070225899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6590115180 0x7f65901a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:44.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.096+0000 7f6595c00700 1 -- 172.21.15.6:0/3070225899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6590115180 msgr2=0x7f65901a5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:44.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.096+0000 7f6595c00700 1 --2- 172.21.15.6:0/3070225899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6590115180 0x7f65901a5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.096+0000 7f6595c00700 1 -- 172.21.15.6:0/3070225899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65900764b0 msgr2=0x7f65901a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:44.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.096+0000 7f6595c00700 1 --2- 172.21.15.6:0/3070225899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65900764b0 0x7f65901a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.096+0000 7f6595c00700 1 -- 172.21.15.6:0/3070225899 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65901a60c0 con 0x7f6590075b60 2024-05-09T04:55:44.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.097+0000 7f6595c00700 1 --2- 172.21.15.6:0/3070225899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6590075b60 0x7f65901a11f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f658400e9a0 tx=0x7f658400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:44.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.097+0000 7f658e000700 1 -- 172.21.15.6:0/3070225899 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658400ca10 con 0x7f6590075b60 2024-05-09T04:55:44.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.097+0000 7f6597000700 1 -- 172.21.15.6:0/3070225899 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65901a63b0 con 0x7f6590075b60 2024-05-09T04:55:44.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.097+0000 7f658e000700 1 -- 172.21.15.6:0/3070225899 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f658400cbb0 con 0x7f6590075b60 2024-05-09T04:55:44.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.097+0000 7f658e000700 1 -- 172.21.15.6:0/3070225899 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6584018680 con 0x7f6590075b60 2024-05-09T04:55:44.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.097+0000 7f6597000700 1 -- 172.21.15.6:0/3070225899 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65901adcd0 con 0x7f6590075b60 2024-05-09T04:55:44.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.101+0000 7f658e000700 1 -- 172.21.15.6:0/3070225899 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) ==== 149702+0+0 (secure 0 0 0) 0x7f6584018820 con 0x7f6590075b60 2024-05-09T04:55:44.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.102+0000 7f65a71cab80 1 Processor -- start 2024-05-09T04:55:44.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.102+0000 7f65a71cab80 1 -- start start 2024-05-09T04:55:44.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.103+0000 7f658e000700 1 --2- 172.21.15.6:0/3070225899 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f65780b2910 0x7f65780b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:44.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.103+0000 7f65a71cab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a58a97e450 0x55a58ab01100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:44.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.103+0000 7f65a71cab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a58ab01640 0x55a58aafb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:44.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.103+0000 7f65a71cab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a58aafbb60 0x55a58aafbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:44.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.103+0000 7f65a71cab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a58aa1e260 con 0x55a58a97e450 2024-05-09T04:55:44.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.103+0000 7f65a71cab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a58a9bc090 con 0x55a58aafbb60 2024-05-09T04:55:44.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.103+0000 7f65a71cab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a58aa1a220 con 0x55a58ab01640 2024-05-09T04:55:44.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.103+0000 7f658e000700 1 -- 172.21.15.6:0/3070225899 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f65840e29b0 con 0x7f6590075b60 2024-05-09T04:55:44.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.103+0000 7f6596600700 1 --2- 172.21.15.6:0/3070225899 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f65780b2910 0x7f65780b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:44.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.103+0000 7f6595c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a58a97e450 0x55a58ab01100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:44.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.103+0000 7f6595200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a58ab01640 0x55a58aafb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:44.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.103+0000 7f6595200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a58ab01640 0x55a58aafb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:48972/0 (socket says 172.21.15.6:48972) 2024-05-09T04:55:44.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.104+0000 7f6595200700 1 -- 172.21.15.6:0/403274303 learned_addr learned my addr 172.21.15.6:0/403274303 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:55:44.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.104+0000 7f6596600700 1 --2- 172.21.15.6:0/403274303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a58aafbb60 0x55a58aafbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:44.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.104+0000 7f6595200700 1 -- 172.21.15.6:0/403274303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a58aafbb60 msgr2=0x55a58aafbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:44.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.104+0000 7f6595200700 1 --2- 172.21.15.6:0/403274303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a58aafbb60 0x55a58aafbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.104+0000 7f6595200700 1 -- 172.21.15.6:0/403274303 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a58a97e450 msgr2=0x55a58ab01100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:44.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.104+0000 7f6595200700 1 --2- 172.21.15.6:0/403274303 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a58a97e450 0x55a58ab01100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.104+0000 7f6595200700 1 -- 172.21.15.6:0/403274303 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a58a899440 con 0x55a58ab01640 2024-05-09T04:55:44.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.105+0000 7f6595200700 1 --2- 172.21.15.6:0/403274303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a58ab01640 0x55a58aafb620 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f657c009ca0 tx=0x7f657c011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2380c535a82409ae server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:44.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.105+0000 7f6596600700 1 --2- 172.21.15.6:0/3070225899 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f65780b2910 0x7f65780b4e20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6588009fd0 tx=0x7f6588009860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:44.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.105+0000 7f6582a00700 1 -- 172.21.15.6:0/403274303 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c011d80 con 0x55a58ab01640 2024-05-09T04:55:44.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.105+0000 7f6582a00700 1 -- 172.21.15.6:0/403274303 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f657c004ce0 con 0x55a58ab01640 2024-05-09T04:55:44.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.105+0000 7f6582a00700 1 -- 172.21.15.6:0/403274303 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c019690 con 0x55a58ab01640 2024-05-09T04:55:44.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.106+0000 7f65a71cab80 1 -- 172.21.15.6:0/403274303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a58ab01640 msgr2=0x55a58aafb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:44.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.106+0000 7f65a71cab80 1 --2- 172.21.15.6:0/403274303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a58ab01640 0x55a58aafb620 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f657c009ca0 tx=0x7f657c011440 comp rx=0 tx=0).stop 2024-05-09T04:55:44.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.106+0000 7f65a71cab80 1 -- 172.21.15.6:0/403274303 shutdown_connections 2024-05-09T04:55:44.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.106+0000 7f65a71cab80 1 --2- 172.21.15.6:0/403274303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a58aafbb60 0x55a58aafbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.106+0000 7f65a71cab80 1 --2- 172.21.15.6:0/403274303 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a58a97e450 0x55a58ab01100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.106+0000 7f65a71cab80 1 --2- 172.21.15.6:0/403274303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a58ab01640 0x55a58aafb620 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.106+0000 7f65a71cab80 1 -- 172.21.15.6:0/403274303 >> 172.21.15.6:0/403274303 conn(0x55a58a9c2e20 msgr2=0x55a58aaf6050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:44.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.106+0000 7f65a71cab80 1 -- 172.21.15.6:0/403274303 shutdown_connections 2024-05-09T04:55:44.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.106+0000 7f65a71cab80 1 -- 172.21.15.6:0/403274303 wait complete. 2024-05-09T04:55:44.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.107+0000 7f65a71cab80 1 -- start start 2024-05-09T04:55:44.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.107+0000 7f65a71cab80 1 Processor -- start 2024-05-09T04:55:44.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.108+0000 7f65a71cab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a58a97e450 0x55a58ab04d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:44.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.108+0000 7f65a71cab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a58aafbb60 0x55a58ab05270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:44.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.108+0000 7f65a71cab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a58ab01640 0x55a58ab057b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:44.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.108+0000 7f65a71cab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a58ab080d0 con 0x55a58a97e450 2024-05-09T04:55:44.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.108+0000 7f65a71cab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a58a983670 con 0x55a58ab01640 2024-05-09T04:55:44.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.108+0000 7f65a71cab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a58ab08250 con 0x55a58aafbb60 2024-05-09T04:55:44.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.108+0000 7f65a71cab80 10 client.0 _ll_register_callbacks cb 0x55a58a9b5540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:55:44.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.108+0000 7f6595c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a58a97e450 0x55a58ab04d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:44.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.108+0000 7f6595200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a58aafbb60 0x55a58ab05270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:44.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.108+0000 7f6595200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a58aafbb60 0x55a58ab05270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:48986/0 (socket says 172.21.15.6:48986) 2024-05-09T04:55:44.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.108+0000 7f6596600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a58ab01640 0x55a58ab057b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:44.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.108+0000 7f6595c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a58a97e450 0x55a58ab04d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58202/0 (socket says 172.21.15.6:58202) 2024-05-09T04:55:44.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.108+0000 7f6595c00700 1 -- 172.21.15.6:0/3856581774 learned_addr learned my addr 172.21.15.6:0/3856581774 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:55:44.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.109+0000 7f6595c00700 1 -- 172.21.15.6:0/3856581774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a58aafbb60 msgr2=0x55a58ab05270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:44.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.109+0000 7f6595c00700 1 --2- 172.21.15.6:0/3856581774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a58aafbb60 0x55a58ab05270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.109+0000 7f6595c00700 1 -- 172.21.15.6:0/3856581774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a58ab01640 msgr2=0x55a58ab057b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:44.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.109+0000 7f6595c00700 1 --2- 172.21.15.6:0/3856581774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a58ab01640 0x55a58ab057b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.109+0000 7f6595c00700 1 -- 172.21.15.6:0/3856581774 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a58ab11540 con 0x55a58a97e450 2024-05-09T04:55:44.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.109+0000 7f6595c00700 1 --2- 172.21.15.6:0/3856581774 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a58a97e450 0x55a58ab04d30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f658400e970 tx=0x7f65840b22b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9827b97e4f94d6a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:44.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.109+0000 7f6577400700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-05-09T04:55:44.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.109+0000 7f6577400700 1 -- 172.21.15.6:0/3856581774 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65840a9a90 con 0x55a58a97e450 2024-05-09T04:55:44.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.109+0000 7f6577400700 1 -- 172.21.15.6:0/3856581774 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f658409d4c0 con 0x55a58a97e450 2024-05-09T04:55:44.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.109+0000 7f65a71cab80 1 -- 172.21.15.6:0/3856581774 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f657c004ce0 con 0x55a58a97e450 2024-05-09T04:55:44.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.110+0000 7f6577400700 1 -- 172.21.15.6:0/3856581774 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f658409d6c0 con 0x55a58a97e450 2024-05-09T04:55:44.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.110+0000 7f65a71cab80 10 client.15204 fetch_fsmap learned FSMap version 30 2024-05-09T04:55:44.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.110+0000 7f65a71cab80 1 -- 172.21.15.6:0/3856581774 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a58ab12e30 con 0x55a58a97e450 2024-05-09T04:55:44.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.110+0000 7f6577400700 1 -- 172.21.15.6:0/3856581774 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f657c004ce0 con 0x55a58a97e450 2024-05-09T04:55:44.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.110+0000 7f6577400700 1 -- 172.21.15.6:0/3856581774 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f65840b2460 con 0x55a58a97e450 2024-05-09T04:55:44.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.110+0000 7f65a71cab80 10 client.15204 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T04:55:44.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.110+0000 7f65a71cab80 10 client.15204 resolve_mds: resolved name 'g' to daemon mds.g 2024-05-09T04:55:44.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.110+0000 7f65a71cab80 20 client.15204 populate_metadata read hostname 'smithi006' 2024-05-09T04:55:44.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.110+0000 7f65a71cab80 1 --2- 172.21.15.6:0/3856581774 >> [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] conn(0x55a58ab153f0 0x55a58ab16860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:44.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.111+0000 7f65a71cab80 4 client.15204 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:55:44.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.111+0000 7f65a71cab80 1 -- 172.21.15.6:0/3856581774 --> [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a58aac7820 con 0x55a58ab153f0 2024-05-09T04:55:44.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.111+0000 7f6595200700 1 --2- 172.21.15.6:0/3856581774 >> [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] conn(0x55a58ab153f0 0x55a58ab16860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:44.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.112+0000 7f6595200700 1 --2- 172.21.15.6:0/3856581774 >> [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] conn(0x55a58ab153f0 0x55a58ab16860 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=81978d6b832f340c server_cookie=fe885f18dc191f73 in_seq=0 out_seq=0 2024-05-09T04:55:44.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.112+0000 7f6577400700 10 client.15204 ms_handle_connect on v2:172.21.15.6:6838/2780306492 2024-05-09T04:55:44.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.112+0000 7f6577400700 1 -- 172.21.15.6:0/3856581774 <== mds.1 v2:172.21.15.6:6838/2780306492 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a58aac7820 con 0x55a58ab153f0 2024-05-09T04:55:44.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.112+0000 7f6577400700 10 client.15204 handle_command_reply: tid=0 2024-05-09T04:55:44.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.113+0000 7f65a71cab80 1 client.15204 shutdown 2024-05-09T04:55:44.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.113+0000 7f65a71cab80 1 -- 172.21.15.6:0/3856581774 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a58a97e450 msgr2=0x55a58ab04d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:44.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.113+0000 7f65a71cab80 1 --2- 172.21.15.6:0/3856581774 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a58a97e450 0x55a58ab04d30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f658400e970 tx=0x7f65840b22b0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.113+0000 7f65a71cab80 1 -- 172.21.15.6:0/3856581774 shutdown_connections 2024-05-09T04:55:44.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.113+0000 7f65a71cab80 1 --2- 172.21.15.6:0/3856581774 >> [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] conn(0x55a58ab153f0 0x55a58ab16860 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.113+0000 7f65a71cab80 1 --2- 172.21.15.6:0/3856581774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a58ab01640 0x55a58ab057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.113+0000 7f65a71cab80 1 --2- 172.21.15.6:0/3856581774 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a58a97e450 0x55a58ab04d30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.113+0000 7f65a71cab80 1 --2- 172.21.15.6:0/3856581774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a58aafbb60 0x55a58ab05270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.113+0000 7f6577400700 0 client.15204 ms_handle_reset on v2:172.21.15.6:6838/2780306492 2024-05-09T04:55:44.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.113+0000 7f65a71cab80 1 -- 172.21.15.6:0/3856581774 >> 172.21.15.6:0/3856581774 conn(0x55a58a9c2e20 msgr2=0x55a58aaf6010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:44.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.113+0000 7f65a71cab80 1 -- 172.21.15.6:0/3856581774 shutdown_connections 2024-05-09T04:55:44.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.113+0000 7f65a71cab80 1 -- 172.21.15.6:0/3856581774 wait complete. 2024-05-09T04:55:44.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.113+0000 7f65a71cab80 20 client.15204 trim_cache size 0 max 16384 2024-05-09T04:55:44.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.137+0000 7f65a71cab80 1 Processor -- start 2024-05-09T04:55:44.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.137+0000 7f65a71cab80 1 -- start start 2024-05-09T04:55:44.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.138+0000 7f65a71cab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a58a97e450 0x55a58aaf5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:44.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.138+0000 7f65a71cab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a58aafbb60 0x55a58aaf6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:44.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.138+0000 7f65a71cab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a58ab01640 0x55a58ab1fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:44.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.138+0000 7f65a71cab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a58ab14ba0 con 0x55a58ab01640 2024-05-09T04:55:44.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.138+0000 7f65a71cab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a58ab15270 con 0x55a58a97e450 2024-05-09T04:55:44.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.138+0000 7f65a71cab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a58ab15940 con 0x55a58aafbb60 2024-05-09T04:55:44.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.138+0000 7f6596600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a58ab01640 0x55a58ab1fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:44.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.138+0000 7f6596600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a58ab01640 0x55a58ab1fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58206/0 (socket says 172.21.15.6:58206) 2024-05-09T04:55:44.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.138+0000 7f6596600700 1 -- 172.21.15.6:0/1819990149 learned_addr learned my addr 172.21.15.6:0/1819990149 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:55:44.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.138+0000 7f6595200700 1 --2- 172.21.15.6:0/1819990149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a58aafbb60 0x55a58aaf6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:44.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.138+0000 7f6595c00700 1 --2- 172.21.15.6:0/1819990149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a58a97e450 0x55a58aaf5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:44.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.139+0000 7f6595200700 1 -- 172.21.15.6:0/1819990149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a58a97e450 msgr2=0x55a58aaf5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:44.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.139+0000 7f6595200700 1 --2- 172.21.15.6:0/1819990149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a58a97e450 0x55a58aaf5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.139+0000 7f6595200700 1 -- 172.21.15.6:0/1819990149 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a58ab01640 msgr2=0x55a58ab1fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:44.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.139+0000 7f6595200700 1 --2- 172.21.15.6:0/1819990149 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a58ab01640 0x55a58ab1fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.139+0000 7f6595200700 1 -- 172.21.15.6:0/1819990149 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a58ab08200 con 0x55a58aafbb60 2024-05-09T04:55:44.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.139+0000 7f6595200700 1 --2- 172.21.15.6:0/1819990149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a58aafbb60 0x55a58aaf6210 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f657c01b040 tx=0x7f657c011eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9bf169859d15a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:44.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.140+0000 7f6582000700 1 -- 172.21.15.6:0/1819990149 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c0050d0 con 0x55a58aafbb60 2024-05-09T04:55:44.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.140+0000 7f6582000700 1 -- 172.21.15.6:0/1819990149 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f657c005270 con 0x55a58aafbb60 2024-05-09T04:55:44.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.140+0000 7f6582000700 1 -- 172.21.15.6:0/1819990149 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c02a250 con 0x55a58aafbb60 2024-05-09T04:55:44.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.141+0000 7f65a71cab80 1 -- 172.21.15.6:0/1819990149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a58aafbb60 msgr2=0x55a58aaf6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:44.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.141+0000 7f65a71cab80 1 --2- 172.21.15.6:0/1819990149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a58aafbb60 0x55a58aaf6210 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f657c01b040 tx=0x7f657c011eb0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.141+0000 7f65a71cab80 1 -- 172.21.15.6:0/1819990149 shutdown_connections 2024-05-09T04:55:44.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.141+0000 7f65a71cab80 1 --2- 172.21.15.6:0/1819990149 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a58ab01640 0x55a58ab1fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.141+0000 7f65a71cab80 1 --2- 172.21.15.6:0/1819990149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a58a97e450 0x55a58aaf5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.141+0000 7f65a71cab80 1 --2- 172.21.15.6:0/1819990149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a58aafbb60 0x55a58aaf6210 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.141+0000 7f65a71cab80 1 -- 172.21.15.6:0/1819990149 >> 172.21.15.6:0/1819990149 conn(0x55a58a9c2e20 msgr2=0x55a58ab142a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:44.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.141+0000 7f65a71cab80 1 -- 172.21.15.6:0/1819990149 shutdown_connections 2024-05-09T04:55:44.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.141+0000 7f65a71cab80 1 -- 172.21.15.6:0/1819990149 wait complete. 2024-05-09T04:55:44.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.142+0000 7f65a71cab80 1 -- start start 2024-05-09T04:55:44.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.142+0000 7f65a71cab80 1 Processor -- start 2024-05-09T04:55:44.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.142+0000 7f65a71cab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a58a97e450 0x55a58aafa710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:44.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.142+0000 7f65a71cab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a58aafbb60 0x55a58aafac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:44.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.142+0000 7f65a71cab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a58ab01640 0x55a58aafb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:44.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.142+0000 7f65a71cab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55a58ab166e0 con 0x55a58a97e450 2024-05-09T04:55:44.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.142+0000 7f65a71cab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a58ab16010 con 0x55a58aafbb60 2024-05-09T04:55:44.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.142+0000 7f65a71cab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a58ab16db0 con 0x55a58ab01640 2024-05-09T04:55:44.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.143+0000 7f65a71cab80 10 client.0 _ll_register_callbacks cb 0x55a58a9b5540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:55:44.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.143+0000 7f6595c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a58a97e450 0x55a58aafa710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:44.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.143+0000 7f6595c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a58a97e450 0x55a58aafa710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58220/0 (socket says 172.21.15.6:58220) 2024-05-09T04:55:44.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.143+0000 7f6595c00700 1 -- 172.21.15.6:0/3468772326 learned_addr learned my addr 172.21.15.6:0/3468772326 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:55:44.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.143+0000 7f6596600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a58ab01640 0x55a58aafb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:44.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.143+0000 7f6595200700 1 --2- 172.21.15.6:0/3468772326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a58aafbb60 0x55a58aafac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:44.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.143+0000 7f6595c00700 1 -- 172.21.15.6:0/3468772326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a58ab01640 msgr2=0x55a58aafb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:44.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.143+0000 7f6595c00700 1 --2- 172.21.15.6:0/3468772326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a58ab01640 0x55a58aafb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.143+0000 7f6595c00700 1 -- 172.21.15.6:0/3468772326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a58aafbb60 msgr2=0x55a58aafac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:44.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.143+0000 7f6595c00700 1 --2- 172.21.15.6:0/3468772326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a58aafbb60 0x55a58aafac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.143+0000 7f6595c00700 1 -- 172.21.15.6:0/3468772326 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a58ab1e450 con 0x55a58a97e450 2024-05-09T04:55:44.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.144+0000 7f6595c00700 1 --2- 172.21.15.6:0/3468772326 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a58a97e450 0x55a58aafa710 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f65840a5b10 tx=0x7f65840dfec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9df83579ad4cb4a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:44.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.144+0000 7f6577400700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-05-09T04:55:44.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.144+0000 7f6577400700 1 -- 172.21.15.6:0/3468772326 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65840a9a90 con 0x55a58a97e450 2024-05-09T04:55:44.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.144+0000 7f65a71cab80 1 -- 172.21.15.6:0/3468772326 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f657c005270 con 0x55a58a97e450 2024-05-09T04:55:44.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.144+0000 7f6577400700 1 -- 172.21.15.6:0/3468772326 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f65840cc390 con 0x55a58a97e450 2024-05-09T04:55:44.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.144+0000 7f6577400700 1 -- 172.21.15.6:0/3468772326 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f65840a93e0 con 0x55a58a97e450 2024-05-09T04:55:44.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.144+0000 7f65a71cab80 10 client.15210 fetch_fsmap learned FSMap version 30 2024-05-09T04:55:44.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.144+0000 7f65a71cab80 1 -- 172.21.15.6:0/3468772326 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a58ab80cc0 con 0x55a58a97e450 2024-05-09T04:55:44.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.144+0000 7f6577400700 1 -- 172.21.15.6:0/3468772326 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f657c005270 con 0x55a58a97e450 2024-05-09T04:55:44.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.145+0000 7f6577400700 1 -- 172.21.15.6:0/3468772326 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f65840a5400 con 0x55a58a97e450 2024-05-09T04:55:44.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.145+0000 7f65a71cab80 10 client.15210 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T04:55:44.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.145+0000 7f65a71cab80 10 client.15210 resolve_mds: resolved name 'g' to daemon mds.g 2024-05-09T04:55:44.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.145+0000 7f65a71cab80 20 client.15210 populate_metadata read hostname 'smithi006' 2024-05-09T04:55:44.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.145+0000 7f65a71cab80 1 --2- 172.21.15.6:0/3468772326 >> [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] conn(0x55a58ab831a0 0x55a58ab84610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:44.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.145+0000 7f65a71cab80 4 client.15210 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T04:55:44.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.145+0000 7f65a71cab80 1 -- 172.21.15.6:0/3468772326 --> [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] -- command(tid 0: {"prefix": "status"}) -- 0x55a58aa2bc40 con 0x55a58ab831a0 2024-05-09T04:55:44.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.145+0000 7f6595200700 1 --2- 172.21.15.6:0/3468772326 >> [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] conn(0x55a58ab831a0 0x55a58ab84610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:44.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.146+0000 7f6595200700 1 --2- 172.21.15.6:0/3468772326 >> [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] conn(0x55a58ab831a0 0x55a58ab84610 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3917c3e52a1dceb4 server_cookie=50a224dec7be1ec5 in_seq=0 out_seq=0 2024-05-09T04:55:44.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.146+0000 7f6577400700 10 client.15210 ms_handle_connect on v2:172.21.15.6:6838/2780306492 2024-05-09T04:55:44.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.146+0000 7f6577400700 1 -- 172.21.15.6:0/3468772326 <== mds.1 v2:172.21.15.6:6838/2780306492 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a58aa2bc40 con 0x55a58ab831a0 2024-05-09T04:55:44.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.146+0000 7f6577400700 10 client.15210 handle_command_reply: tid=0 2024-05-09T04:55:44.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.146+0000 7f65a71cab80 1 client.15210 shutdown 2024-05-09T04:55:44.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.146+0000 7f65a71cab80 1 -- 172.21.15.6:0/3468772326 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a58a97e450 msgr2=0x55a58aafa710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:44.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.146+0000 7f65a71cab80 1 --2- 172.21.15.6:0/3468772326 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a58a97e450 0x55a58aafa710 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f65840a5b10 tx=0x7f65840dfec0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.146+0000 7f65a71cab80 1 -- 172.21.15.6:0/3468772326 shutdown_connections 2024-05-09T04:55:44.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.146+0000 7f65a71cab80 1 --2- 172.21.15.6:0/3468772326 >> [v2:172.21.15.6:6838/2780306492,v1:172.21.15.6:6839/2780306492] conn(0x55a58ab831a0 0x55a58ab84610 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.147+0000 7f65a71cab80 1 --2- 172.21.15.6:0/3468772326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a58ab01640 0x55a58aafb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.147+0000 7f65a71cab80 1 --2- 172.21.15.6:0/3468772326 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a58a97e450 0x55a58aafa710 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.147+0000 7f65a71cab80 1 --2- 172.21.15.6:0/3468772326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a58aafbb60 0x55a58aafac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.147+0000 7f65a71cab80 1 -- 172.21.15.6:0/3468772326 >> 172.21.15.6:0/3468772326 conn(0x55a58a9c2e20 msgr2=0x55a58aafea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:44.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.147+0000 7f6577400700 0 client.15210 ms_handle_reset on v2:172.21.15.6:6838/2780306492 2024-05-09T04:55:44.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.147+0000 7f65a71cab80 1 -- 172.21.15.6:0/3468772326 shutdown_connections 2024-05-09T04:55:44.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.147+0000 7f65a71cab80 1 -- 172.21.15.6:0/3468772326 wait complete. 2024-05-09T04:55:44.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.147+0000 7f65a71cab80 20 client.15210 trim_cache size 0 max 16384 2024-05-09T04:55:44.151 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-09T04:55:44.151 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "f74d813c-0dbd-11ef-bc97-c7b262605968", 2024-05-09T04:55:44.151 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 1, 2024-05-09T04:55:44.151 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24530, 2024-05-09T04:55:44.151 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-05-09T04:55:44.151 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-05-09T04:55:44.151 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-05-09T04:55:44.151 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 284.42026892199999, 2024-05-09T04:55:44.151 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 30, 2024-05-09T04:55:44.151 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 89, 2024-05-09T04:55:44.151 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 89, 2024-05-09T04:55:44.151 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 397.46163973300003 2024-05-09T04:55:44.151 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-09T04:55:44.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.148+0000 7f6597000700 1 -- 172.21.15.6:0/3070225899 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f65780b2910 msgr2=0x7f65780b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:44.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.148+0000 7f6597000700 1 --2- 172.21.15.6:0/3070225899 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f65780b2910 0x7f65780b4e20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6588009fd0 tx=0x7f6588009860 comp rx=0 tx=0).stop 2024-05-09T04:55:44.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.148+0000 7f6597000700 1 -- 172.21.15.6:0/3070225899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6590075b60 msgr2=0x7f65901a11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:44.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.148+0000 7f6597000700 1 --2- 172.21.15.6:0/3070225899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6590075b60 0x7f65901a11f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f658400e9a0 tx=0x7f658400ecb0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.148+0000 7f6597000700 1 -- 172.21.15.6:0/3070225899 shutdown_connections 2024-05-09T04:55:44.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.148+0000 7f6597000700 1 --2- 172.21.15.6:0/3070225899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6590115180 0x7f65901a5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.148+0000 7f6597000700 1 --2- 172.21.15.6:0/3070225899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6590075b60 0x7f65901a11f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.148+0000 7f6597000700 1 --2- 172.21.15.6:0/3070225899 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f65780b2910 0x7f65780b4e20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.148+0000 7f6597000700 1 --2- 172.21.15.6:0/3070225899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65900764b0 0x7f65901a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:44.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.148+0000 7f6597000700 1 -- 172.21.15.6:0/3070225899 >> 172.21.15.6:0/3070225899 conn(0x7f6590079d10 msgr2=0x7f6590100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:44.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.148+0000 7f6597000700 1 -- 172.21.15.6:0/3070225899 shutdown_connections 2024-05-09T04:55:44.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:44.148+0000 7f6597000700 1 -- 172.21.15.6:0/3070225899 wait complete. 2024-05-09T04:55:44.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:44 smithi006 ceph-mon[29835]: pgmap v523: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:55:45.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:44 smithi186 ceph-mon[34668]: pgmap v523: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:55:45.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:44 smithi169 ceph-mon[34075]: pgmap v523: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:55:45.279 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-09T04:55:45.755 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:55:46.089 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:45 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:55:46.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:45 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:55:46.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:45 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:55:47.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:46 smithi186 ceph-mon[34668]: pgmap v524: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:55:47.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:46 smithi169 ceph-mon[34075]: pgmap v524: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:55:47.188 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:46 smithi006 ceph-mon[29835]: pgmap v524: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:55:48.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:47 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:55:48.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:47 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:55:48.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:47 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:55:48.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:47 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:55:48.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:47 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:55:48.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:47 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:55:48.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.173+0000 7fce4d800700 1 -- 172.21.15.6:0/1235168480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce4810b8a0 msgr2=0x7fce48113200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:48.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.173+0000 7fce4d800700 1 --2- 172.21.15.6:0/1235168480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce4810b8a0 0x7fce48113200 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fce40009fc0 tx=0x7fce4000b330 comp rx=0 tx=0).stop 2024-05-09T04:55:48.175 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:47 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:55:48.175 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:47 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:55:48.175 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:47 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:55:48.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.173+0000 7fce4d800700 1 -- 172.21.15.6:0/1235168480 shutdown_connections 2024-05-09T04:55:48.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.173+0000 7fce4d800700 1 --2- 172.21.15.6:0/1235168480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce4810b8a0 0x7fce48113200 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:48.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.173+0000 7fce4d800700 1 --2- 172.21.15.6:0/1235168480 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce48106620 0x7fce48106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:48.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.173+0000 7fce4d800700 1 --2- 172.21.15.6:0/1235168480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce4810af50 0x7fce4810b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:48.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.173+0000 7fce4d800700 1 -- 172.21.15.6:0/1235168480 >> 172.21.15.6:0/1235168480 conn(0x7fce48102030 msgr2=0x7fce48104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:48.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.173+0000 7fce4d800700 1 -- 172.21.15.6:0/1235168480 shutdown_connections 2024-05-09T04:55:48.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.173+0000 7fce4d800700 1 -- 172.21.15.6:0/1235168480 wait complete. 2024-05-09T04:55:48.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.175+0000 7fce4d800700 1 Processor -- start 2024-05-09T04:55:48.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.175+0000 7fce4d800700 1 -- start start 2024-05-09T04:55:48.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.176+0000 7fce4d800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce48106620 0x7fce4819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:48.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.176+0000 7fce4d800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce4810af50 0x7fce4819d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:48.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.176+0000 7fce4d800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce4810b8a0 0x7fce481a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:48.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.176+0000 7fce4d800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fce48116520 con 0x7fce4810b8a0 2024-05-09T04:55:48.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.176+0000 7fce4d800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fce481163a0 con 0x7fce4810af50 2024-05-09T04:55:48.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.176+0000 7fce4d800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fce481166a0 con 0x7fce48106620 2024-05-09T04:55:48.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.176+0000 7fce4ce00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce4810b8a0 0x7fce481a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:48.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.176+0000 7fce47e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce48106620 0x7fce4819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:48.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.176+0000 7fce4ce00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce4810b8a0 0x7fce481a15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33930/0 (socket says 172.21.15.6:33930) 2024-05-09T04:55:48.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.176+0000 7fce4ce00700 1 -- 172.21.15.6:0/3831838428 learned_addr learned my addr 172.21.15.6:0/3831838428 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:55:48.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.176+0000 7fce47400700 1 --2- 172.21.15.6:0/3831838428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce4810af50 0x7fce4819d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:48.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.177+0000 7fce4ce00700 1 -- 172.21.15.6:0/3831838428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce48106620 msgr2=0x7fce4819cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:48.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.177+0000 7fce4ce00700 1 --2- 172.21.15.6:0/3831838428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce48106620 0x7fce4819cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:48.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.177+0000 7fce4ce00700 1 -- 172.21.15.6:0/3831838428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce4810af50 msgr2=0x7fce4819d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:48.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.177+0000 7fce4ce00700 1 --2- 172.21.15.6:0/3831838428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce4810af50 0x7fce4819d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:48.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.177+0000 7fce4ce00700 1 -- 172.21.15.6:0/3831838428 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce481a1c80 con 0x7fce4810b8a0 2024-05-09T04:55:48.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.177+0000 7fce47400700 1 --2- 172.21.15.6:0/3831838428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce4810af50 0x7fce4819d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:55:48.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.177+0000 7fce47e00700 1 --2- 172.21.15.6:0/3831838428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce48106620 0x7fce4819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:55:48.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.177+0000 7fce4ce00700 1 --2- 172.21.15.6:0/3831838428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce4810b8a0 0x7fce481a15e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fce40009790 tx=0x7fce4000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:48.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.177+0000 7fce44c00700 1 -- 172.21.15.6:0/3831838428 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce40021070 con 0x7fce4810b8a0 2024-05-09T04:55:48.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.177+0000 7fce44c00700 1 -- 172.21.15.6:0/3831838428 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce4000ce60 con 0x7fce4810b8a0 2024-05-09T04:55:48.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.177+0000 7fce4d800700 1 -- 172.21.15.6:0/3831838428 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce481a1f10 con 0x7fce4810b8a0 2024-05-09T04:55:48.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.178+0000 7fce44c00700 1 -- 172.21.15.6:0/3831838428 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce40005040 con 0x7fce4810b8a0 2024-05-09T04:55:48.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.178+0000 7fce4d800700 1 -- 172.21.15.6:0/3831838428 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce481a9890 con 0x7fce4810b8a0 2024-05-09T04:55:48.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.182+0000 7fce44c00700 1 -- 172.21.15.6:0/3831838428 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 19) ==== 149702+0+0 (secure 0 0 0) 0x7fce40024070 con 0x7fce4810b8a0 2024-05-09T04:55:48.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.183+0000 7fce5d907b80 1 Processor -- start 2024-05-09T04:55:48.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.183+0000 7fce44c00700 1 --2- 172.21.15.6:0/3831838428 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fce2c0b2910 0x7fce2c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:48.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.183+0000 7fce5d907b80 1 -- start start 2024-05-09T04:55:48.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.183+0000 7fce44c00700 1 -- 172.21.15.6:0/3831838428 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fce4000f020 con 0x7fce4810b8a0 2024-05-09T04:55:48.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.184+0000 7fce47400700 1 --2- 172.21.15.6:0/3831838428 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fce2c0b2910 0x7fce2c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:48.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.184+0000 7fce5d907b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e89065b450 0x55e8907de100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:48.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.184+0000 7fce5d907b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e8907de640 0x55e8907d8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:48.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.184+0000 7fce5d907b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8907d8b60 0x55e8907d8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:48.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.184+0000 7fce5d907b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55e8906fb260 con 0x55e8907d8b60 2024-05-09T04:55:48.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.184+0000 7fce5d907b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e890699090 con 0x55e8907de640 2024-05-09T04:55:48.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.184+0000 7fce5d907b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55e8906f7220 con 0x55e89065b450 2024-05-09T04:55:48.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.184+0000 7fce47400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e8907de640 0x55e8907d8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:48.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.184+0000 7fce4ce00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8907d8b60 0x55e8907d8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:48.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.184+0000 7fce4ce00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8907d8b60 0x55e8907d8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33944/0 (socket says 172.21.15.6:33944) 2024-05-09T04:55:48.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.184+0000 7fce4ce00700 1 -- 172.21.15.6:0/343018330 learned_addr learned my addr 172.21.15.6:0/343018330 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:55:48.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.184+0000 7fce47e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e89065b450 0x55e8907de100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:48.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.185+0000 7fce47400700 1 --2- 172.21.15.6:0/3831838428 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fce2c0b2910 0x7fce2c0b4e20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fce48109c30 tx=0x7fce34009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:48.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.185+0000 7fce47400700 1 -- 172.21.15.6:0/343018330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e89065b450 msgr2=0x55e8907de100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:48.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.185+0000 7fce47400700 1 --2- 172.21.15.6:0/343018330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e89065b450 0x55e8907de100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:48.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.185+0000 7fce47400700 1 -- 172.21.15.6:0/343018330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8907d8b60 msgr2=0x55e8907d8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:48.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.185+0000 7fce47400700 1 --2- 172.21.15.6:0/343018330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8907d8b60 0x55e8907d8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:48.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.185+0000 7fce47400700 1 -- 172.21.15.6:0/343018330 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e890576440 con 0x55e8907de640 2024-05-09T04:55:48.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.185+0000 7fce47400700 1 --2- 172.21.15.6:0/343018330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e8907de640 0x55e8907d8620 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fce4806ceb0 tx=0x7fce34019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8610f580ee8a4f5a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:48.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.186+0000 7fce3d600700 1 -- 172.21.15.6:0/343018330 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce340038c0 con 0x55e8907de640 2024-05-09T04:55:48.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.186+0000 7fce3d600700 1 -- 172.21.15.6:0/343018330 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce3401c440 con 0x55e8907de640 2024-05-09T04:55:48.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.186+0000 7fce3d600700 1 -- 172.21.15.6:0/343018330 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3401b6c0 con 0x55e8907de640 2024-05-09T04:55:48.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.187+0000 7fce5d907b80 1 -- 172.21.15.6:0/343018330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e8907de640 msgr2=0x55e8907d8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:48.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.187+0000 7fce5d907b80 1 --2- 172.21.15.6:0/343018330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e8907de640 0x55e8907d8620 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fce4806ceb0 tx=0x7fce34019000 comp rx=0 tx=0).stop 2024-05-09T04:55:48.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.187+0000 7fce5d907b80 1 -- 172.21.15.6:0/343018330 shutdown_connections 2024-05-09T04:55:48.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.187+0000 7fce5d907b80 1 --2- 172.21.15.6:0/343018330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8907d8b60 0x55e8907d8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:48.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.187+0000 7fce5d907b80 1 --2- 172.21.15.6:0/343018330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e89065b450 0x55e8907de100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:48.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.187+0000 7fce5d907b80 1 --2- 172.21.15.6:0/343018330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e8907de640 0x55e8907d8620 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:48.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.187+0000 7fce5d907b80 1 -- 172.21.15.6:0/343018330 >> 172.21.15.6:0/343018330 conn(0x55e89069fe20 msgr2=0x55e8907d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:48.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.187+0000 7fce5d907b80 1 -- 172.21.15.6:0/343018330 shutdown_connections 2024-05-09T04:55:48.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.187+0000 7fce5d907b80 1 -- 172.21.15.6:0/343018330 wait complete. 2024-05-09T04:55:48.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.188+0000 7fce5d907b80 1 -- start start 2024-05-09T04:55:48.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.188+0000 7fce5d907b80 1 Processor -- start 2024-05-09T04:55:48.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.189+0000 7fce5d907b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e89065b450 0x55e8907e1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:48.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.189+0000 7fce5d907b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e8907d8b60 0x55e8907e22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:48.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.189+0000 7fce5d907b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8907de640 0x55e8907e27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:48.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.189+0000 7fce5d907b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55e8907e50d0 con 0x55e8907de640 2024-05-09T04:55:48.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.189+0000 7fce5d907b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e890660670 con 0x55e8907d8b60 2024-05-09T04:55:48.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.189+0000 7fce5d907b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55e8907e5250 con 0x55e89065b450 2024-05-09T04:55:48.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.189+0000 7fce5d907b80 10 client.0 _ll_register_callbacks cb 0x55e890692540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:55:48.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.189+0000 7fce4ce00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8907de640 0x55e8907e27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:48.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.189+0000 7fce4ce00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8907de640 0x55e8907e27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33950/0 (socket says 172.21.15.6:33950) 2024-05-09T04:55:48.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.189+0000 7fce4ce00700 1 -- 172.21.15.6:0/841812781 learned_addr learned my addr 172.21.15.6:0/841812781 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:55:48.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.189+0000 7fce47400700 1 --2- 172.21.15.6:0/841812781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e8907d8b60 0x55e8907e22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:48.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.189+0000 7fce47e00700 1 --2- 172.21.15.6:0/841812781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e89065b450 0x55e8907e1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:48.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.190+0000 7fce4ce00700 1 -- 172.21.15.6:0/841812781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e89065b450 msgr2=0x55e8907e1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:48.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.190+0000 7fce4ce00700 1 --2- 172.21.15.6:0/841812781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e89065b450 0x55e8907e1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:48.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.190+0000 7fce4ce00700 1 -- 172.21.15.6:0/841812781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e8907d8b60 msgr2=0x55e8907e22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:48.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.190+0000 7fce4ce00700 1 --2- 172.21.15.6:0/841812781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e8907d8b60 0x55e8907e22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:48.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.190+0000 7fce4ce00700 1 -- 172.21.15.6:0/841812781 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8907ee5a0 con 0x55e8907de640 2024-05-09T04:55:48.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.190+0000 7fce4ce00700 1 --2- 172.21.15.6:0/841812781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8907de640 0x55e8907e27e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fce400aac90 tx=0x7fce40007d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f927e60970432cf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:48.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.190+0000 7fce32000700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-05-09T04:55:48.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.190+0000 7fce32000700 1 -- 172.21.15.6:0/841812781 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce400ab450 con 0x55e8907de640 2024-05-09T04:55:48.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.190+0000 7fce32000700 1 -- 172.21.15.6:0/841812781 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fce400c5350 con 0x55e8907de640 2024-05-09T04:55:48.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.190+0000 7fce5d907b80 1 -- 172.21.15.6:0/841812781 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce3401c440 con 0x55e8907de640 2024-05-09T04:55:48.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.191+0000 7fce32000700 1 -- 172.21.15.6:0/841812781 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fce400aad50 con 0x55e8907de640 2024-05-09T04:55:48.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.191+0000 7fce5d907b80 10 client.15222 fetch_fsmap learned FSMap version 30 2024-05-09T04:55:48.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.191+0000 7fce5d907b80 1 -- 172.21.15.6:0/841812781 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e8907efe50 con 0x55e8907de640 2024-05-09T04:55:48.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.191+0000 7fce32000700 1 -- 172.21.15.6:0/841812781 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce3401c440 con 0x55e8907de640 2024-05-09T04:55:48.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.191+0000 7fce32000700 1 -- 172.21.15.6:0/841812781 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fce40011070 con 0x55e8907de640 2024-05-09T04:55:48.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.191+0000 7fce5d907b80 10 client.15222 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T04:55:48.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.191+0000 7fce5d907b80 10 client.15222 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T04:55:48.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.191+0000 7fce5d907b80 20 client.15222 populate_metadata read hostname 'smithi006' 2024-05-09T04:55:48.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.191+0000 7fce5d907b80 1 --2- 172.21.15.6:0/841812781 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x55e8907f2540 0x55e8907f39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:48.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.191+0000 7fce5d907b80 4 client.15222 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:55:48.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.192+0000 7fce5d907b80 1 -- 172.21.15.6:0/841812781 --> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8907a4820 con 0x55e8907f2540 2024-05-09T04:55:48.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.192+0000 7fce47e00700 1 --2- 172.21.15.6:0/841812781 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x55e8907f2540 0x55e8907f39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:48.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.192+0000 7fce47e00700 1 --2- 172.21.15.6:0/841812781 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x55e8907f2540 0x55e8907f39b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20302d55f5207f87 server_cookie=72694497f089598a in_seq=0 out_seq=0 2024-05-09T04:55:48.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.192+0000 7fce32000700 10 client.15222 ms_handle_connect on v2:172.21.15.169:6838/614388771 2024-05-09T04:55:48.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.193+0000 7fce32000700 1 -- 172.21.15.6:0/841812781 <== mds.0 v2:172.21.15.169:6838/614388771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e8907a4820 con 0x55e8907f2540 2024-05-09T04:55:48.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.193+0000 7fce32000700 10 client.15222 handle_command_reply: tid=0 2024-05-09T04:55:48.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.193+0000 7fce5d907b80 1 client.15222 shutdown 2024-05-09T04:55:48.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.194+0000 7fce5d907b80 1 -- 172.21.15.6:0/841812781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8907de640 msgr2=0x55e8907e27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:48.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.194+0000 7fce5d907b80 1 --2- 172.21.15.6:0/841812781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8907de640 0x55e8907e27e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fce400aac90 tx=0x7fce40007d50 comp rx=0 tx=0).stop 2024-05-09T04:55:48.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.194+0000 7fce5d907b80 1 -- 172.21.15.6:0/841812781 shutdown_connections 2024-05-09T04:55:48.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.194+0000 7fce5d907b80 1 --2- 172.21.15.6:0/841812781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8907de640 0x55e8907e27e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:48.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.194+0000 7fce5d907b80 1 --2- 172.21.15.6:0/841812781 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x55e8907f2540 0x55e8907f39b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:48.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.194+0000 7fce5d907b80 1 --2- 172.21.15.6:0/841812781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e89065b450 0x55e8907e1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:48.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.194+0000 7fce5d907b80 1 --2- 172.21.15.6:0/841812781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e8907d8b60 0x55e8907e22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:48.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.194+0000 7fce5d907b80 1 -- 172.21.15.6:0/841812781 >> 172.21.15.6:0/841812781 conn(0x55e89069fe20 msgr2=0x55e8907d3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:48.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.194+0000 7fce32000700 0 client.15222 ms_handle_reset on v2:172.21.15.169:6838/614388771 2024-05-09T04:55:48.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.194+0000 7fce5d907b80 1 -- 172.21.15.6:0/841812781 shutdown_connections 2024-05-09T04:55:48.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.194+0000 7fce5d907b80 1 -- 172.21.15.6:0/841812781 wait complete. 2024-05-09T04:55:48.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.194+0000 7fce5d907b80 20 client.15222 trim_cache size 0 max 16384 2024-05-09T04:55:48.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.218+0000 7fce5d907b80 1 Processor -- start 2024-05-09T04:55:48.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.218+0000 7fce5d907b80 1 -- start start 2024-05-09T04:55:48.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.218+0000 7fce5d907b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e89065b450 0x55e8907e2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:48.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.218+0000 7fce5d907b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8907d8b60 0x55e8907dce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:48.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.218+0000 7fce5d907b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8907de640 0x55e8907dd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:48.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.218+0000 7fce5d907b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55e8907f1cf0 con 0x55e8907de640 2024-05-09T04:55:48.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.218+0000 7fce5d907b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e8907f23c0 con 0x55e89065b450 2024-05-09T04:55:48.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.218+0000 7fce5d907b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55e8907f2a90 con 0x55e8907d8b60 2024-05-09T04:55:48.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.218+0000 7fce4ce00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8907de640 0x55e8907dd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:48.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.219+0000 7fce4ce00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8907de640 0x55e8907dd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33956/0 (socket says 172.21.15.6:33956) 2024-05-09T04:55:48.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.219+0000 7fce4ce00700 1 -- 172.21.15.6:0/2323083414 learned_addr learned my addr 172.21.15.6:0/2323083414 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:55:48.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.219+0000 7fce47e00700 1 --2- 172.21.15.6:0/2323083414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e89065b450 0x55e8907e2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:48.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.219+0000 7fce47400700 1 --2- 172.21.15.6:0/2323083414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8907d8b60 0x55e8907dce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:48.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.219+0000 7fce47e00700 1 -- 172.21.15.6:0/2323083414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8907d8b60 msgr2=0x55e8907dce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:48.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.219+0000 7fce47e00700 1 --2- 172.21.15.6:0/2323083414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8907d8b60 0x55e8907dce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:48.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.219+0000 7fce47e00700 1 -- 172.21.15.6:0/2323083414 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8907de640 msgr2=0x55e8907dd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:48.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.219+0000 7fce47e00700 1 --2- 172.21.15.6:0/2323083414 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8907de640 0x55e8907dd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:48.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.219+0000 7fce47e00700 1 -- 172.21.15.6:0/2323083414 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8907ce2f0 con 0x55e89065b450 2024-05-09T04:55:48.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.219+0000 7fce47e00700 1 --2- 172.21.15.6:0/2323083414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e89065b450 0x55e8907e2620 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x55e8907f4080 tx=0x7fce38009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=395257c3b4b768c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:48.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.220+0000 7fce3cc00700 1 -- 172.21.15.6:0/2323083414 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce38004830 con 0x55e89065b450 2024-05-09T04:55:48.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.220+0000 7fce3cc00700 1 -- 172.21.15.6:0/2323083414 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce38005020 con 0x55e89065b450 2024-05-09T04:55:48.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.220+0000 7fce3cc00700 1 -- 172.21.15.6:0/2323083414 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3800f9e0 con 0x55e89065b450 2024-05-09T04:55:48.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.220+0000 7fce5d907b80 1 -- 172.21.15.6:0/2323083414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e89065b450 msgr2=0x55e8907e2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:48.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.220+0000 7fce5d907b80 1 --2- 172.21.15.6:0/2323083414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e89065b450 0x55e8907e2620 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x55e8907f4080 tx=0x7fce38009310 comp rx=0 tx=0).stop 2024-05-09T04:55:48.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.221+0000 7fce5d907b80 1 -- 172.21.15.6:0/2323083414 shutdown_connections 2024-05-09T04:55:48.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.221+0000 7fce5d907b80 1 --2- 172.21.15.6:0/2323083414 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8907de640 0x55e8907dd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:48.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.221+0000 7fce5d907b80 1 --2- 172.21.15.6:0/2323083414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e89065b450 0x55e8907e2620 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:48.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.221+0000 7fce5d907b80 1 --2- 172.21.15.6:0/2323083414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8907d8b60 0x55e8907dce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:48.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.221+0000 7fce5d907b80 1 -- 172.21.15.6:0/2323083414 >> 172.21.15.6:0/2323083414 conn(0x55e89069fe20 msgr2=0x55e8907d2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:48.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.221+0000 7fce5d907b80 1 -- 172.21.15.6:0/2323083414 shutdown_connections 2024-05-09T04:55:48.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.221+0000 7fce5d907b80 1 -- 172.21.15.6:0/2323083414 wait complete. 2024-05-09T04:55:48.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.221+0000 7fce5d907b80 1 -- start start 2024-05-09T04:55:48.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.221+0000 7fce5d907b80 1 Processor -- start 2024-05-09T04:55:48.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.222+0000 7fce5d907b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e89065b450 0x55e8907ded10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:48.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.222+0000 7fce5d907b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e8907d8b60 0x55e8907df250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:48.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.222+0000 7fce5d907b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8907de640 0x55e8907df790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:48.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.222+0000 7fce5d907b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55e8907f3830 con 0x55e8907de640 2024-05-09T04:55:48.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.222+0000 7fce5d907b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e8907f3160 con 0x55e8907d8b60 2024-05-09T04:55:48.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.222+0000 7fce5d907b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55e8907f3f00 con 0x55e89065b450 2024-05-09T04:55:48.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.222+0000 7fce5d907b80 10 client.0 _ll_register_callbacks cb 0x55e890692540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:55:48.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.222+0000 7fce47e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e89065b450 0x55e8907ded10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:48.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.222+0000 7fce47e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e89065b450 0x55e8907ded10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:54192/0 (socket says 172.21.15.6:54192) 2024-05-09T04:55:48.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.222+0000 7fce47e00700 1 -- 172.21.15.6:0/546731735 learned_addr learned my addr 172.21.15.6:0/546731735 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:55:48.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.222+0000 7fce47400700 1 --2- 172.21.15.6:0/546731735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e8907d8b60 0x55e8907df250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:48.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.223+0000 7fce4ce00700 1 --2- 172.21.15.6:0/546731735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8907de640 0x55e8907df790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:48.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.223+0000 7fce47400700 1 -- 172.21.15.6:0/546731735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e89065b450 msgr2=0x55e8907ded10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:48.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.223+0000 7fce47400700 1 --2- 172.21.15.6:0/546731735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e89065b450 0x55e8907ded10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:48.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.223+0000 7fce47400700 1 -- 172.21.15.6:0/546731735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8907de640 msgr2=0x55e8907df790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:48.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.223+0000 7fce47400700 1 --2- 172.21.15.6:0/546731735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8907de640 0x55e8907df790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:48.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.223+0000 7fce47400700 1 -- 172.21.15.6:0/546731735 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8907fb3c0 con 0x55e8907d8b60 2024-05-09T04:55:48.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.223+0000 7fce47400700 1 --2- 172.21.15.6:0/546731735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e8907d8b60 0x55e8907df250 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fce3401ccf0 tx=0x7fce34003d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57fd71332031743e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:48.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.223+0000 7fce32000700 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-05-09T04:55:48.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.224+0000 7fce32000700 1 -- 172.21.15.6:0/546731735 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce34035020 con 0x55e8907d8b60 2024-05-09T04:55:48.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.224+0000 7fce5d907b80 1 -- 172.21.15.6:0/546731735 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce38005020 con 0x55e8907d8b60 2024-05-09T04:55:48.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.224+0000 7fce32000700 1 -- 172.21.15.6:0/546731735 <== mon.2 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fce340054a0 con 0x55e8907d8b60 2024-05-09T04:55:48.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.224+0000 7fce32000700 1 -- 172.21.15.6:0/546731735 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fce34040050 con 0x55e8907d8b60 2024-05-09T04:55:48.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.224+0000 7fce5d907b80 10 client.25376 fetch_fsmap learned FSMap version 30 2024-05-09T04:55:48.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.224+0000 7fce5d907b80 1 -- 172.21.15.6:0/546731735 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e89085dd30 con 0x55e8907d8b60 2024-05-09T04:55:48.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.224+0000 7fce32000700 1 -- 172.21.15.6:0/546731735 <== mon.2 v2:172.21.15.169:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce38005020 con 0x55e8907d8b60 2024-05-09T04:55:48.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.225+0000 7fce32000700 1 -- 172.21.15.6:0/546731735 <== mon.2 v2:172.21.15.169:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fce340047a0 con 0x55e8907d8b60 2024-05-09T04:55:48.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.225+0000 7fce5d907b80 10 client.25376 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T04:55:48.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.225+0000 7fce5d907b80 10 client.25376 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T04:55:48.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.225+0000 7fce5d907b80 20 client.25376 populate_metadata read hostname 'smithi006' 2024-05-09T04:55:48.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.225+0000 7fce5d907b80 1 --2- 172.21.15.6:0/546731735 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x55e890860010 0x55e890861480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:48.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.225+0000 7fce5d907b80 4 client.25376 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T04:55:48.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.225+0000 7fce5d907b80 1 -- 172.21.15.6:0/546731735 --> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] -- command(tid 0: {"prefix": "status"}) -- 0x55e890708c40 con 0x55e890860010 2024-05-09T04:55:48.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.225+0000 7fce47e00700 1 --2- 172.21.15.6:0/546731735 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x55e890860010 0x55e890861480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:48.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.226+0000 7fce47e00700 1 --2- 172.21.15.6:0/546731735 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x55e890860010 0x55e890861480 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cebe73413831f2f server_cookie=863795663d86b6b3 in_seq=0 out_seq=0 2024-05-09T04:55:48.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.226+0000 7fce32000700 10 client.25376 ms_handle_connect on v2:172.21.15.169:6838/614388771 2024-05-09T04:55:48.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.226+0000 7fce32000700 1 -- 172.21.15.6:0/546731735 <== mds.0 v2:172.21.15.169:6838/614388771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e890708c40 con 0x55e890860010 2024-05-09T04:55:48.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.226+0000 7fce32000700 10 client.25376 handle_command_reply: tid=0 2024-05-09T04:55:48.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.226+0000 7fce5d907b80 1 client.25376 shutdown 2024-05-09T04:55:48.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.226+0000 7fce5d907b80 1 -- 172.21.15.6:0/546731735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e8907d8b60 msgr2=0x55e8907df250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:48.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.226+0000 7fce5d907b80 1 --2- 172.21.15.6:0/546731735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e8907d8b60 0x55e8907df250 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fce3401ccf0 tx=0x7fce34003d20 comp rx=0 tx=0).stop 2024-05-09T04:55:48.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.227+0000 7fce5d907b80 1 -- 172.21.15.6:0/546731735 shutdown_connections 2024-05-09T04:55:48.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.227+0000 7fce5d907b80 1 --2- 172.21.15.6:0/546731735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8907de640 0x55e8907df790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:48.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.227+0000 7fce5d907b80 1 --2- 172.21.15.6:0/546731735 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x55e890860010 0x55e890861480 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:48.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.227+0000 7fce5d907b80 1 --2- 172.21.15.6:0/546731735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e89065b450 0x55e8907ded10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:48.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.227+0000 7fce5d907b80 1 --2- 172.21.15.6:0/546731735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e8907d8b60 0x55e8907df250 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:48.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.227+0000 7fce32000700 0 client.25376 ms_handle_reset on v2:172.21.15.169:6838/614388771 2024-05-09T04:55:48.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.227+0000 7fce5d907b80 1 -- 172.21.15.6:0/546731735 >> 172.21.15.6:0/546731735 conn(0x55e89069fe20 msgr2=0x55e8907dba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:48.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.227+0000 7fce5d907b80 1 -- 172.21.15.6:0/546731735 shutdown_connections 2024-05-09T04:55:48.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.227+0000 7fce5d907b80 1 -- 172.21.15.6:0/546731735 wait complete. 2024-05-09T04:55:48.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.227+0000 7fce5d907b80 20 client.25376 trim_cache size 0 max 16384 2024-05-09T04:55:48.232 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-09T04:55:48.232 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "f74d813c-0dbd-11ef-bc97-c7b262605968", 2024-05-09T04:55:48.232 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 0, 2024-05-09T04:55:48.232 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24554, 2024-05-09T04:55:48.232 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2024-05-09T04:55:48.232 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2024-05-09T04:55:48.232 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2024-05-09T04:55:48.232 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 371.07139686400001, 2024-05-09T04:55:48.232 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 30, 2024-05-09T04:55:48.232 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 89, 2024-05-09T04:55:48.232 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 89, 2024-05-09T04:55:48.232 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 392.37480007400001 2024-05-09T04:55:48.232 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-09T04:55:48.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.228+0000 7fce4d800700 1 -- 172.21.15.6:0/3831838428 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fce2c0b2910 msgr2=0x7fce2c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:48.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.228+0000 7fce4d800700 1 --2- 172.21.15.6:0/3831838428 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fce2c0b2910 0x7fce2c0b4e20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fce48109c30 tx=0x7fce34009260 comp rx=0 tx=0).stop 2024-05-09T04:55:48.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.228+0000 7fce4d800700 1 -- 172.21.15.6:0/3831838428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce4810b8a0 msgr2=0x7fce481a15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:48.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.228+0000 7fce4d800700 1 --2- 172.21.15.6:0/3831838428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce4810b8a0 0x7fce481a15e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fce40009790 tx=0x7fce4000cc30 comp rx=0 tx=0).stop 2024-05-09T04:55:48.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.228+0000 7fce4d800700 1 -- 172.21.15.6:0/3831838428 shutdown_connections 2024-05-09T04:55:48.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.228+0000 7fce4d800700 1 --2- 172.21.15.6:0/3831838428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce4810b8a0 0x7fce481a15e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:48.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.228+0000 7fce4d800700 1 --2- 172.21.15.6:0/3831838428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce48106620 0x7fce4819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:48.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.228+0000 7fce4d800700 1 --2- 172.21.15.6:0/3831838428 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fce2c0b2910 0x7fce2c0b4e20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:48.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.228+0000 7fce4d800700 1 --2- 172.21.15.6:0/3831838428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce4810af50 0x7fce4819d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:48.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.228+0000 7fce4d800700 1 -- 172.21.15.6:0/3831838428 >> 172.21.15.6:0/3831838428 conn(0x7fce48102030 msgr2=0x7fce4810a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:48.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.228+0000 7fce4d800700 1 -- 172.21.15.6:0/3831838428 shutdown_connections 2024-05-09T04:55:48.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:55:48.228+0000 7fce4d800700 1 -- 172.21.15.6:0/3831838428 wait complete. 2024-05-09T04:55:48.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:48 smithi006 ceph-mon[29835]: pgmap v525: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:55:49.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:48 smithi186 ceph-mon[34668]: pgmap v525: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:55:49.111 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-05-09T04:55:49.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:48 smithi169 ceph-mon[34075]: pgmap v525: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:55:51.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:50 smithi186 ceph-mon[34668]: pgmap v526: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:55:51.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:50 smithi169 ceph-mon[34075]: pgmap v526: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:55:51.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:50 smithi006 ceph-mon[29835]: pgmap v526: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:55:53.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:52 smithi186 ceph-mon[34668]: pgmap v527: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:55:53.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:52 smithi169 ceph-mon[34075]: pgmap v527: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:55:53.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:52 smithi006 ceph-mon[29835]: pgmap v527: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:55:55.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:54 smithi186 ceph-mon[34668]: pgmap v528: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:55:55.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:54 smithi169 ceph-mon[34075]: pgmap v528: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:55:55.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:54 smithi006 ceph-mon[29835]: pgmap v528: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:55:57.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:56 smithi169 ceph-mon[34075]: pgmap v529: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:55:57.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:56 smithi006 ceph-mon[29835]: pgmap v529: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:55:57.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:56 smithi186 ceph-mon[34668]: pgmap v529: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:55:59.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:55:58 smithi169 ceph-mon[34075]: pgmap v530: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:55:59.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:55:58 smithi006 ceph-mon[29835]: pgmap v530: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:55:59.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:55:58 smithi186 ceph-mon[34668]: pgmap v530: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:56:01.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:00 smithi169 ceph-mon[34075]: pgmap v531: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:56:01.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:00 smithi006 ceph-mon[29835]: pgmap v531: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:56:01.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:00 smithi186 ceph-mon[34668]: pgmap v531: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:56:03.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:02 smithi169 ceph-mon[34075]: pgmap v532: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:56:03.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:02 smithi006 ceph-mon[29835]: pgmap v532: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:56:03.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:02 smithi186 ceph-mon[34668]: pgmap v532: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:56:04.113 INFO:tasks.quiescer.fs.[cephfs]:Total duration (49.9) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-09T04:56:04.113 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.95 2024-05-09T04:56:04.113 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:56:04.594 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:56:05.084 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:04 smithi006 ceph-mon[29835]: pgmap v533: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-09T04:56:05.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:04 smithi186 ceph-mon[34668]: pgmap v533: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-09T04:56:05.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:04 smithi169 ceph-mon[34075]: pgmap v533: 129 pgs: 129 active+clean; 662 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-09T04:56:06.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.890+0000 7f2bc4800700 1 -- 172.21.15.6:0/1766117366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2bbc110ba0 msgr2=0x7f2bbc113000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:06.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.890+0000 7f2bc4800700 1 --2- 172.21.15.6:0/1766117366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2bbc110ba0 0x7f2bbc113000 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800b0a0 tx=0x7f2bb800b3b0 comp rx=0 tx=0).stop 2024-05-09T04:56:06.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.890+0000 7f2bc4800700 1 -- 172.21.15.6:0/1766117366 shutdown_connections 2024-05-09T04:56:06.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.890+0000 7f2bc4800700 1 --2- 172.21.15.6:0/1766117366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2bbc110ba0 0x7f2bbc113000 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:06.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.890+0000 7f2bc4800700 1 --2- 172.21.15.6:0/1766117366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bbc075d30 0x7f2bbc076140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:06.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.890+0000 7f2bc4800700 1 --2- 172.21.15.6:0/1766117366 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bbc076680 0x7f2bbc110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:06.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.890+0000 7f2bc4800700 1 -- 172.21.15.6:0/1766117366 >> 172.21.15.6:0/1766117366 conn(0x7f2bbc100040 msgr2=0x7f2bbc102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:06.893 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:06 smithi006 ceph-mon[29835]: pgmap v534: 129 pgs: 129 active+clean; 667 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-09T04:56:06.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.890+0000 7f2bc4800700 1 -- 172.21.15.6:0/1766117366 shutdown_connections 2024-05-09T04:56:06.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.890+0000 7f2bc4800700 1 -- 172.21.15.6:0/1766117366 wait complete. 2024-05-09T04:56:06.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.892+0000 7f2bc4800700 1 Processor -- start 2024-05-09T04:56:06.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.892+0000 7f2bc4800700 1 -- start start 2024-05-09T04:56:06.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.892+0000 7f2bc4800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bbc075d30 0x7f2bbc19d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:06.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.892+0000 7f2bc4800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2bbc076680 0x7f2bbc19d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:06.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.892+0000 7f2bc4800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bbc110ba0 0x7f2bbc1a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:06.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.892+0000 7f2bc4800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc1166d0 con 0x7f2bbc076680 2024-05-09T04:56:06.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.892+0000 7f2bc4800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc116550 con 0x7f2bbc075d30 2024-05-09T04:56:06.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.892+0000 7f2bc4800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc116850 con 0x7f2bbc110ba0 2024-05-09T04:56:06.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.893+0000 7f2bc1e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bbc075d30 0x7f2bbc19d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:06.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.893+0000 7f2bc1400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2bbc076680 0x7f2bbc19d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:06.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.893+0000 7f2bc1e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bbc075d30 0x7f2bbc19d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:47272/0 (socket says 172.21.15.6:47272) 2024-05-09T04:56:06.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.893+0000 7f2bc1e00700 1 -- 172.21.15.6:0/2274217138 learned_addr learned my addr 172.21.15.6:0/2274217138 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:56:06.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.893+0000 7f2bc2800700 1 --2- 172.21.15.6:0/2274217138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bbc110ba0 0x7f2bbc1a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:06.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.893+0000 7f2bc1e00700 1 -- 172.21.15.6:0/2274217138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bbc110ba0 msgr2=0x7f2bbc1a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:06.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.893+0000 7f2bc1e00700 1 --2- 172.21.15.6:0/2274217138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bbc110ba0 0x7f2bbc1a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:06.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.893+0000 7f2bc1e00700 1 -- 172.21.15.6:0/2274217138 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2bbc076680 msgr2=0x7f2bbc19d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:06.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.893+0000 7f2bc1e00700 1 --2- 172.21.15.6:0/2274217138 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2bbc076680 0x7f2bbc19d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:06.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.893+0000 7f2bc1e00700 1 -- 172.21.15.6:0/2274217138 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bbc1a1ef0 con 0x7f2bbc075d30 2024-05-09T04:56:06.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.894+0000 7f2bc1e00700 1 --2- 172.21.15.6:0/2274217138 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bbc075d30 0x7f2bbc19d020 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000e9a0 tx=0x7f2bb000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:06.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.894+0000 7f2bb6a00700 1 -- 172.21.15.6:0/2274217138 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb000ca10 con 0x7f2bbc075d30 2024-05-09T04:56:06.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.894+0000 7f2bc4800700 1 -- 172.21.15.6:0/2274217138 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bbc1a21e0 con 0x7f2bbc075d30 2024-05-09T04:56:06.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.894+0000 7f2bb6a00700 1 -- 172.21.15.6:0/2274217138 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bb000cbb0 con 0x7f2bbc075d30 2024-05-09T04:56:06.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.894+0000 7f2bb6a00700 1 -- 172.21.15.6:0/2274217138 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb0018680 con 0x7f2bbc075d30 2024-05-09T04:56:06.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.894+0000 7f2bc4800700 1 -- 172.21.15.6:0/2274217138 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bbc1a9b00 con 0x7f2bbc075d30 2024-05-09T04:56:06.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.896+0000 7f2bc4800700 1 -- 172.21.15.6:0/2274217138 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bbc0512c0 con 0x7f2bbc075d30 2024-05-09T04:56:06.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.898+0000 7f2bb6a00700 1 -- 172.21.15.6:0/2274217138 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 19) ==== 149702+0+0 (secure 0 0 0) 0x7f2bb0010430 con 0x7f2bbc075d30 2024-05-09T04:56:06.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.899+0000 7f2bb6a00700 1 --2- 172.21.15.6:0/2274217138 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f2ba40b2910 0x7f2ba40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:06.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.900+0000 7f2bb6a00700 1 -- 172.21.15.6:0/2274217138 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f2bb00e2970 con 0x7f2bbc075d30 2024-05-09T04:56:06.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.900+0000 7f2bc1400700 1 --2- 172.21.15.6:0/2274217138 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f2ba40b2910 0x7f2ba40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:06.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.900+0000 7f2bc1400700 1 --2- 172.21.15.6:0/2274217138 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f2ba40b2910 0x7f2ba40b4e20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc103480 tx=0x7f2bac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:06.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:06.904+0000 7f2bb6a00700 1 -- 172.21.15.6:0/2274217138 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2bb00aa150 con 0x7f2bbc075d30 2024-05-09T04:56:07.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:07.191+0000 7f2bc4800700 1 -- 172.21.15.6:0/2274217138 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2bbc19e380 con 0x7f2bbc075d30 2024-05-09T04:56:07.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:07.192+0000 7f2bb6a00700 1 -- 172.21.15.6:0/2274217138 <== mon.2 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11927 (secure 0 0 0) 0x7f2bb00b21d0 con 0x7f2bbc075d30 2024-05-09T04:56:07.194 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:56:07.194 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/162094599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":162094599},{"type":"v1","addr":"172.21.15.6:6835","nonce":162094599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/589294363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":589294363},{"type":"v1","addr":"172.21.15.6:6841","nonce":589294363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:49:36.920376+0000","modified":"2024-05-09T04:54:56.247431+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24554,"mds_1":24530,"mds_2":24512,"mds_3":24511,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24442":{"gid":24442,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6833/2977145206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2977145206},{"type":"v1","addr":"172.21.15.186:6833","nonce":2977145206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6833/133345948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":133345948},{"type":"v1","addr":"172.21.15.169:6833","nonce":133345948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.6:6837/1281521088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":1281521088},{"type":"v1","addr":"172.21.15.6:6837","nonce":1281521088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2268322007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2268322007},{"type":"v1","addr":"172.21.15.186:6835","nonce":2268322007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/3811005052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3811005052},{"type":"v1","addr":"172.21.15.169:6835","nonce":3811005052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.186:6837/1647607118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1647607118},{"type":"v1","addr":"172.21.15.186:6837","nonce":1647607118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":32,"addr":"172.21.15.186:6839/3954883105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":3954883105},{"type":"v1","addr":"172.21.15.186:6839","nonce":3954883105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.169:6837/1845715294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1845715294},{"type":"v1","addr":"172.21.15.169:6837","nonce":1845715294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.6:6839/2780306492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":2780306492},{"type":"v1","addr":"172.21.15.6:6839","nonce":2780306492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.169:6839/614388771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":614388771},{"type":"v1","addr":"172.21.15.169:6839","nonce":614388771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24512,24554,24530,24493,24511]},"id":1}]} 2024-05-09T04:56:07.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:07.196+0000 7f2bc4800700 1 -- 172.21.15.6:0/2274217138 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f2ba40b2910 msgr2=0x7f2ba40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:07.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:07.196+0000 7f2bc4800700 1 --2- 172.21.15.6:0/2274217138 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f2ba40b2910 0x7f2ba40b4e20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc103480 tx=0x7f2bac009260 comp rx=0 tx=0).stop 2024-05-09T04:56:07.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:07.196+0000 7f2bc4800700 1 -- 172.21.15.6:0/2274217138 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bbc075d30 msgr2=0x7f2bbc19d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:07.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:07.196+0000 7f2bc4800700 1 --2- 172.21.15.6:0/2274217138 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bbc075d30 0x7f2bbc19d020 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000e9a0 tx=0x7f2bb000ecb0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:07.196+0000 7f2bc4800700 1 -- 172.21.15.6:0/2274217138 shutdown_connections 2024-05-09T04:56:07.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:07.196+0000 7f2bc4800700 1 --2- 172.21.15.6:0/2274217138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bbc110ba0 0x7f2bbc1a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:07.196+0000 7f2bc4800700 1 --2- 172.21.15.6:0/2274217138 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f2ba40b2910 0x7f2ba40b4e20 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:07.196+0000 7f2bc4800700 1 --2- 172.21.15.6:0/2274217138 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bbc075d30 0x7f2bbc19d020 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:07.196+0000 7f2bc4800700 1 --2- 172.21.15.6:0/2274217138 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2bbc076680 0x7f2bbc19d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:07.196+0000 7f2bc4800700 1 -- 172.21.15.6:0/2274217138 >> 172.21.15.6:0/2274217138 conn(0x7f2bbc100040 msgr2=0x7f2bbc100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:07.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:07.196+0000 7f2bc4800700 1 -- 172.21.15.6:0/2274217138 shutdown_connections 2024-05-09T04:56:07.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:07.196+0000 7f2bc4800700 1 -- 172.21.15.6:0/2274217138 wait complete. 2024-05-09T04:56:07.199 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 30 2024-05-09T04:56:07.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:06 smithi186 ceph-mon[34668]: pgmap v534: 129 pgs: 129 active+clean; 667 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-09T04:56:07.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:06 smithi169 ceph-mon[34075]: pgmap v534: 129 pgs: 129 active+clean; 667 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-09T04:56:08.115 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24554 quiesce db / --timeout 90 --expiration 144.95 --await-for 45.0' 2024-05-09T04:56:08.115 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24554 quiesce db / --timeout 90 --expiration 144.95 --await-for 45.0 2024-05-09T04:56:08.185 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:07 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/2274217138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:56:08.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:07 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/2274217138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:56:08.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:07 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/2274217138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:56:08.583 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:56:09.070 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:08 smithi006 ceph-mon[29835]: pgmap v535: 129 pgs: 129 active+clean; 667 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-09T04:56:09.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:08 smithi186 ceph-mon[34668]: pgmap v535: 129 pgs: 129 active+clean; 667 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-09T04:56:09.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:08 smithi169 ceph-mon[34075]: pgmap v535: 129 pgs: 129 active+clean; 667 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-09T04:56:11.073 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:10 smithi006 ceph-mon[29835]: pgmap v536: 129 pgs: 129 active+clean; 667 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-09T04:56:11.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.071+0000 7fd7d8c00700 1 -- 172.21.15.6:0/233928806 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7d410af50 msgr2=0x7fd7d410b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:11.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.071+0000 7fd7d8c00700 1 --2- 172.21.15.6:0/233928806 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7d410af50 0x7fd7d410b360 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0099c0 tx=0x7fd7bc009cd0 comp rx=0 tx=0).stop 2024-05-09T04:56:11.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.071+0000 7fd7d8c00700 1 -- 172.21.15.6:0/233928806 shutdown_connections 2024-05-09T04:56:11.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.071+0000 7fd7d8c00700 1 --2- 172.21.15.6:0/233928806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7d410b8a0 0x7fd7d4113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:11.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.071+0000 7fd7d8c00700 1 --2- 172.21.15.6:0/233928806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7d4106620 0x7fd7d4106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:11.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.071+0000 7fd7d8c00700 1 --2- 172.21.15.6:0/233928806 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7d410af50 0x7fd7d410b360 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:11.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.071+0000 7fd7d8c00700 1 -- 172.21.15.6:0/233928806 >> 172.21.15.6:0/233928806 conn(0x7fd7d4102030 msgr2=0x7fd7d4104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:11.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.071+0000 7fd7d8c00700 1 -- 172.21.15.6:0/233928806 shutdown_connections 2024-05-09T04:56:11.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.071+0000 7fd7d8c00700 1 -- 172.21.15.6:0/233928806 wait complete. 2024-05-09T04:56:11.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.073+0000 7fd7d8c00700 1 Processor -- start 2024-05-09T04:56:11.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.073+0000 7fd7d8c00700 1 -- start start 2024-05-09T04:56:11.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.073+0000 7fd7d8c00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7d4106620 0x7fd7d419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:11.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.073+0000 7fd7d8c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7d410af50 0x7fd7d419d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:11.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.073+0000 7fd7d8c00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7d410b8a0 0x7fd7d41a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:11.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.073+0000 7fd7d8c00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d41164b0 con 0x7fd7d410b8a0 2024-05-09T04:56:11.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.073+0000 7fd7d8c00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d4116330 con 0x7fd7d4106620 2024-05-09T04:56:11.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.073+0000 7fd7d8c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d4116630 con 0x7fd7d410af50 2024-05-09T04:56:11.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.074+0000 7fd7d3400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7d4106620 0x7fd7d419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:11.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.074+0000 7fd7d3e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7d410b8a0 0x7fd7d41a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:11.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.074+0000 7fd7d3400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7d4106620 0x7fd7d419ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:47286/0 (socket says 172.21.15.6:47286) 2024-05-09T04:56:11.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.074+0000 7fd7d3400700 1 -- 172.21.15.6:0/536559346 learned_addr learned my addr 172.21.15.6:0/536559346 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:56:11.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.074+0000 7fd7cbe00700 1 --2- 172.21.15.6:0/536559346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7d410af50 0x7fd7d419d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:11.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.074+0000 7fd7d3400700 1 -- 172.21.15.6:0/536559346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7d410af50 msgr2=0x7fd7d419d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:11.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.074+0000 7fd7d3400700 1 --2- 172.21.15.6:0/536559346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7d410af50 0x7fd7d419d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:11.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.074+0000 7fd7d3400700 1 -- 172.21.15.6:0/536559346 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7d410b8a0 msgr2=0x7fd7d41a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:11.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.074+0000 7fd7d3400700 1 --2- 172.21.15.6:0/536559346 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7d410b8a0 0x7fd7d41a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:11.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.074+0000 7fd7d3400700 1 -- 172.21.15.6:0/536559346 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d41a1c60 con 0x7fd7d4106620 2024-05-09T04:56:11.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.075+0000 7fd7d3400700 1 --2- 172.21.15.6:0/536559346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7d4106620 0x7fd7d419ce40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd7c400e9a0 tx=0x7fd7c400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:11.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.075+0000 7fd7d0c00700 1 -- 172.21.15.6:0/536559346 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c400ca10 con 0x7fd7d4106620 2024-05-09T04:56:11.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.075+0000 7fd7d0c00700 1 -- 172.21.15.6:0/536559346 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7c400cbb0 con 0x7fd7d4106620 2024-05-09T04:56:11.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.075+0000 7fd7d0c00700 1 -- 172.21.15.6:0/536559346 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4018680 con 0x7fd7d4106620 2024-05-09T04:56:11.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.075+0000 7fd7d8c00700 1 -- 172.21.15.6:0/536559346 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7d41a1f50 con 0x7fd7d4106620 2024-05-09T04:56:11.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.075+0000 7fd7d8c00700 1 -- 172.21.15.6:0/536559346 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7d41a24b0 con 0x7fd7d4106620 2024-05-09T04:56:11.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.081+0000 7fd7d0c00700 1 -- 172.21.15.6:0/536559346 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 19) ==== 149702+0+0 (secure 0 0 0) 0x7fd7c4010430 con 0x7fd7d4106620 2024-05-09T04:56:11.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.082+0000 7fd7e8de8b80 1 Processor -- start 2024-05-09T04:56:11.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.082+0000 7fd7e8de8b80 1 -- start start 2024-05-09T04:56:11.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.082+0000 7fd7e8de8b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e25b74640 0x557e25cf74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:11.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.082+0000 7fd7e8de8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e25cf7a30 0x557e25cf15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:11.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.082+0000 7fd7e8de8b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557e25cf1af0 0x557e25cf1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:11.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.082+0000 7fd7e8de8b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x557e25c143f0 con 0x557e25cf1af0 2024-05-09T04:56:11.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.082+0000 7fd7e8de8b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557e25bb21c0 con 0x557e25b74640 2024-05-09T04:56:11.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.082+0000 7fd7e8de8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x557e25c103b0 con 0x557e25cf7a30 2024-05-09T04:56:11.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.083+0000 7fd7d3e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e25b74640 0x557e25cf74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:11.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.083+0000 7fd7cbe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557e25cf1af0 0x557e25cf1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:11.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.083+0000 7fd7d3e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e25b74640 0x557e25cf74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:47288/0 (socket says 172.21.15.6:47288) 2024-05-09T04:56:11.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.083+0000 7fd7d3e00700 1 -- 172.21.15.6:0/2457889263 learned_addr learned my addr 172.21.15.6:0/2457889263 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:56:11.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.083+0000 7fd7d0c00700 1 --2- 172.21.15.6:0/536559346 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd7b80b2990 0x7fd7b80b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:11.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.083+0000 7fd7d3400700 1 --2- 172.21.15.6:0/2457889263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e25cf7a30 0x557e25cf15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:11.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.083+0000 7fd7cbe00700 1 -- 172.21.15.6:0/2457889263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e25cf7a30 msgr2=0x557e25cf15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:11.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.083+0000 7fd7cbe00700 1 --2- 172.21.15.6:0/2457889263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e25cf7a30 0x557e25cf15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:11.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.083+0000 7fd7cbe00700 1 -- 172.21.15.6:0/2457889263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e25b74640 msgr2=0x557e25cf74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:11.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.083+0000 7fd7cbe00700 1 --2- 172.21.15.6:0/2457889263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e25b74640 0x557e25cf74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:11.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.084+0000 7fd7cbe00700 1 -- 172.21.15.6:0/2457889263 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e25a8f630 con 0x557e25cf1af0 2024-05-09T04:56:11.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.084+0000 7fd7d3e00700 1 --2- 172.21.15.6:0/536559346 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd7b80b2990 0x7fd7b80b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:11.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.084+0000 7fd7d0c00700 1 -- 172.21.15.6:0/536559346 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fd7c40e2810 con 0x7fd7d4106620 2024-05-09T04:56:11.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.084+0000 7fd7cbe00700 1 --2- 172.21.15.6:0/2457889263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557e25cf1af0 0x557e25cf1f00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00b580 tx=0x7fd7bc005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d1918b280e53d11 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:11.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.084+0000 7fd7c8c00700 1 -- 172.21.15.6:0/2457889263 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc027070 con 0x557e25cf1af0 2024-05-09T04:56:11.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.084+0000 7fd7c8c00700 1 -- 172.21.15.6:0/2457889263 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7bc0037c0 con 0x557e25cf1af0 2024-05-09T04:56:11.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.084+0000 7fd7c8c00700 1 -- 172.21.15.6:0/2457889263 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc00b9f0 con 0x557e25cf1af0 2024-05-09T04:56:11.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.085+0000 7fd7e8de8b80 1 -- 172.21.15.6:0/2457889263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557e25cf1af0 msgr2=0x557e25cf1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:11.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.085+0000 7fd7e8de8b80 1 --2- 172.21.15.6:0/2457889263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557e25cf1af0 0x557e25cf1f00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00b580 tx=0x7fd7bc005040 comp rx=0 tx=0).stop 2024-05-09T04:56:11.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.086+0000 7fd7d3e00700 1 --2- 172.21.15.6:0/536559346 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd7b80b2990 0x7fd7b80b4ea0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x557e25cf6fb0 tx=0x7fd7cc004b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:11.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.086+0000 7fd7e8de8b80 1 -- 172.21.15.6:0/2457889263 shutdown_connections 2024-05-09T04:56:11.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.086+0000 7fd7e8de8b80 1 --2- 172.21.15.6:0/2457889263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557e25cf1af0 0x557e25cf1f00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:11.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.086+0000 7fd7e8de8b80 1 --2- 172.21.15.6:0/2457889263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e25b74640 0x557e25cf74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:11.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.086+0000 7fd7e8de8b80 1 --2- 172.21.15.6:0/2457889263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e25cf7a30 0x557e25cf15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:11.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.086+0000 7fd7e8de8b80 1 -- 172.21.15.6:0/2457889263 >> 172.21.15.6:0/2457889263 conn(0x557e25b75930 msgr2=0x557e25cec430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:11.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.086+0000 7fd7e8de8b80 1 -- 172.21.15.6:0/2457889263 shutdown_connections 2024-05-09T04:56:11.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.086+0000 7fd7e8de8b80 1 -- 172.21.15.6:0/2457889263 wait complete. 2024-05-09T04:56:11.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.087+0000 7fd7e8de8b80 1 -- start start 2024-05-09T04:56:11.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.087+0000 7fd7e8de8b80 1 Processor -- start 2024-05-09T04:56:11.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.088+0000 7fd7e8de8b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e25b74640 0x557e25cfb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:11.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.088+0000 7fd7e8de8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e25cf1af0 0x557e25cfb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:11.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.088+0000 7fd7e8de8b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557e25cf7a30 0x557e25cfbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:11.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.088+0000 7fd7e8de8b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x557e25cfe4f0 con 0x557e25cf7a30 2024-05-09T04:56:11.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.088+0000 7fd7e8de8b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557e25bae470 con 0x557e25b74640 2024-05-09T04:56:11.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.088+0000 7fd7e8de8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x557e25cfe670 con 0x557e25cf1af0 2024-05-09T04:56:11.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.088+0000 7fd7e8de8b80 10 client.0 _ll_register_callbacks cb 0x557e25bab730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:56:11.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.088+0000 7fd7d3400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e25b74640 0x557e25cfb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:11.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.089+0000 7fd7d3e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557e25cf7a30 0x557e25cfbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:11.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.089+0000 7fd7d3400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e25b74640 0x557e25cfb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:47290/0 (socket says 172.21.15.6:47290) 2024-05-09T04:56:11.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.089+0000 7fd7cbe00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e25cf1af0 0x557e25cfb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:11.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.089+0000 7fd7d3400700 1 -- 172.21.15.6:0/2576101002 learned_addr learned my addr 172.21.15.6:0/2576101002 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:56:11.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.089+0000 7fd7d3400700 1 -- 172.21.15.6:0/2576101002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e25cf1af0 msgr2=0x557e25cfb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:11.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.089+0000 7fd7d3400700 1 --2- 172.21.15.6:0/2576101002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e25cf1af0 0x557e25cfb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:11.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.089+0000 7fd7d3400700 1 -- 172.21.15.6:0/2576101002 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557e25cf7a30 msgr2=0x557e25cfbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:11.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.089+0000 7fd7d3400700 1 --2- 172.21.15.6:0/2576101002 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557e25cf7a30 0x557e25cfbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:11.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.089+0000 7fd7d3400700 1 -- 172.21.15.6:0/2576101002 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e25bd99f0 con 0x557e25b74640 2024-05-09T04:56:11.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.089+0000 7fd7d3400700 1 --2- 172.21.15.6:0/2576101002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e25b74640 0x557e25cfb040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd7c40a98e0 tx=0x7fd7c40a9e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75d18cfe49babe2a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:11.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.089+0000 7fd7c1600700 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-05-09T04:56:11.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.090+0000 7fd7c1600700 1 -- 172.21.15.6:0/2576101002 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c40a9720 con 0x557e25b74640 2024-05-09T04:56:11.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.090+0000 7fd7c1600700 1 -- 172.21.15.6:0/2576101002 <== mon.2 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fd7c40a8bb0 con 0x557e25b74640 2024-05-09T04:56:11.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.090+0000 7fd7e8de8b80 1 -- 172.21.15.6:0/2576101002 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7bc0037c0 con 0x557e25b74640 2024-05-09T04:56:11.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.090+0000 7fd7c1600700 1 -- 172.21.15.6:0/2576101002 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd7c40a90c0 con 0x557e25b74640 2024-05-09T04:56:11.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.090+0000 7fd7e8de8b80 10 client.25394 fetch_fsmap learned FSMap version 30 2024-05-09T04:56:11.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.090+0000 7fd7e8de8b80 1 -- 172.21.15.6:0/2576101002 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557e25d09090 con 0x557e25b74640 2024-05-09T04:56:11.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.090+0000 7fd7c1600700 1 -- 172.21.15.6:0/2576101002 <== mon.2 v2:172.21.15.169:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7bc0037c0 con 0x557e25b74640 2024-05-09T04:56:11.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.091+0000 7fd7c1600700 1 -- 172.21.15.6:0/2576101002 <== mon.2 v2:172.21.15.169:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd7c40a8620 con 0x557e25b74640 2024-05-09T04:56:11.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.091+0000 7fd7e8de8b80 10 client.25394 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T04:56:11.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.091+0000 7fd7e8de8b80 10 client.25394 resolve_mds: validated gid 24554 aka daemon mds.b 2024-05-09T04:56:11.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.091+0000 7fd7e8de8b80 20 client.25394 populate_metadata read hostname 'smithi006' 2024-05-09T04:56:11.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.091+0000 7fd7e8de8b80 1 --2- 172.21.15.6:0/2576101002 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x557e25d0b620 0x557e25d0da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:11.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.091+0000 7fd7e8de8b80 4 client.25394 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:56:11.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.091+0000 7fd7e8de8b80 1 -- 172.21.15.6:0/2576101002 --> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e25c21dd0 con 0x557e25d0b620 2024-05-09T04:56:11.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.092+0000 7fd7cbe00700 1 --2- 172.21.15.6:0/2576101002 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x557e25d0b620 0x557e25d0da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:11.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.093+0000 7fd7cbe00700 1 --2- 172.21.15.6:0/2576101002 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x557e25d0b620 0x557e25d0da80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e39ea3ddf9bd094 server_cookie=215551324b36b3f1 in_seq=0 out_seq=0 2024-05-09T04:56:11.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.093+0000 7fd7c1600700 10 client.25394 ms_handle_connect on v2:172.21.15.169:6838/614388771 2024-05-09T04:56:11.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.094+0000 7fd7c1600700 1 -- 172.21.15.6:0/2576101002 <== mds.0 v2:172.21.15.169:6838/614388771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557e25c21dd0 con 0x557e25d0b620 2024-05-09T04:56:11.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.094+0000 7fd7c1600700 10 client.25394 handle_command_reply: tid=0 2024-05-09T04:56:11.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.094+0000 7fd7e8de8b80 1 client.25394 shutdown 2024-05-09T04:56:11.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.094+0000 7fd7e8de8b80 1 -- 172.21.15.6:0/2576101002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e25b74640 msgr2=0x557e25cfb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:11.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.094+0000 7fd7e8de8b80 1 --2- 172.21.15.6:0/2576101002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e25b74640 0x557e25cfb040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd7c40a98e0 tx=0x7fd7c40a9e90 comp rx=0 tx=0).stop 2024-05-09T04:56:11.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.095+0000 7fd7e8de8b80 1 -- 172.21.15.6:0/2576101002 shutdown_connections 2024-05-09T04:56:11.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.095+0000 7fd7e8de8b80 1 --2- 172.21.15.6:0/2576101002 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557e25cf7a30 0x557e25cfbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:11.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.095+0000 7fd7e8de8b80 1 --2- 172.21.15.6:0/2576101002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e25b74640 0x557e25cfb040 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:11.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.095+0000 7fd7e8de8b80 1 --2- 172.21.15.6:0/2576101002 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x557e25d0b620 0x557e25d0da80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:11.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.095+0000 7fd7e8de8b80 1 --2- 172.21.15.6:0/2576101002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e25cf1af0 0x557e25cfb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:11.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.095+0000 7fd7e8de8b80 1 -- 172.21.15.6:0/2576101002 >> 172.21.15.6:0/2576101002 conn(0x557e25b75930 msgr2=0x557e25cec170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:11.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.095+0000 7fd7c1600700 0 client.25394 ms_handle_reset on v2:172.21.15.169:6838/614388771 2024-05-09T04:56:11.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.095+0000 7fd7e8de8b80 1 -- 172.21.15.6:0/2576101002 shutdown_connections 2024-05-09T04:56:11.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.095+0000 7fd7e8de8b80 1 -- 172.21.15.6:0/2576101002 wait complete. 2024-05-09T04:56:11.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.095+0000 7fd7e8de8b80 20 client.25394 trim_cache size 0 max 16384 2024-05-09T04:56:11.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.139+0000 7fd7e8de8b80 1 Processor -- start 2024-05-09T04:56:11.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.139+0000 7fd7e8de8b80 1 -- start start 2024-05-09T04:56:11.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.140+0000 7fd7e8de8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e25b74640 0x557e25cf15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:11.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.140+0000 7fd7e8de8b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557e25cf1af0 0x557e25d075b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:11.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.140+0000 7fd7e8de8b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e25cf7a30 0x557e25d13240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:11.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.140+0000 7fd7e8de8b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d41b20e0 con 0x557e25cf1af0 2024-05-09T04:56:11.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.140+0000 7fd7e8de8b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d41b3c20 con 0x557e25cf7a30 2024-05-09T04:56:11.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.140+0000 7fd7e8de8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d41b27b0 con 0x557e25b74640 2024-05-09T04:56:11.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.140+0000 7fd7cbe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557e25cf1af0 0x557e25d075b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:11.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.140+0000 7fd7d3400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e25b74640 0x557e25cf15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:11.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.140+0000 7fd7d3400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e25b74640 0x557e25cf15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:48344/0 (socket says 172.21.15.6:48344) 2024-05-09T04:56:11.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.140+0000 7fd7cbe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557e25cf1af0 0x557e25d075b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54788/0 (socket says 172.21.15.6:54788) 2024-05-09T04:56:11.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.140+0000 7fd7d3400700 1 -- 172.21.15.6:0/3389406436 learned_addr learned my addr 172.21.15.6:0/3389406436 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:56:11.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.140+0000 7fd7d3e00700 1 --2- 172.21.15.6:0/3389406436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e25cf7a30 0x557e25d13240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:11.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.141+0000 7fd7d3400700 1 -- 172.21.15.6:0/3389406436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e25cf7a30 msgr2=0x557e25d13240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:11.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.141+0000 7fd7d3400700 1 --2- 172.21.15.6:0/3389406436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e25cf7a30 0x557e25d13240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:11.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.141+0000 7fd7d3400700 1 -- 172.21.15.6:0/3389406436 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557e25cf1af0 msgr2=0x557e25d075b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:11.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.141+0000 7fd7d3400700 1 --2- 172.21.15.6:0/3389406436 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557e25cf1af0 0x557e25d075b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:11.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.141+0000 7fd7d3400700 1 -- 172.21.15.6:0/3389406436 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e25ce5e70 con 0x557e25b74640 2024-05-09T04:56:11.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.141+0000 7fd7d3400700 1 --2- 172.21.15.6:0/3389406436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e25b74640 0x557e25cf15b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd7c400c2d0 tx=0x7fd7c4009790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=819b28cfdcbe4af7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:11.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.141+0000 7fd7c3e00700 1 -- 172.21.15.6:0/3389406436 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c40a92a0 con 0x557e25b74640 2024-05-09T04:56:11.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.142+0000 7fd7c3e00700 1 -- 172.21.15.6:0/3389406436 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7c40b20f0 con 0x557e25b74640 2024-05-09T04:56:11.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.142+0000 7fd7c3e00700 1 -- 172.21.15.6:0/3389406436 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c40b27b0 con 0x557e25b74640 2024-05-09T04:56:11.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.143+0000 7fd7e8de8b80 1 -- 172.21.15.6:0/3389406436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e25b74640 msgr2=0x557e25cf15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:11.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.143+0000 7fd7e8de8b80 1 --2- 172.21.15.6:0/3389406436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e25b74640 0x557e25cf15b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd7c400c2d0 tx=0x7fd7c4009790 comp rx=0 tx=0).stop 2024-05-09T04:56:11.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.143+0000 7fd7e8de8b80 1 -- 172.21.15.6:0/3389406436 shutdown_connections 2024-05-09T04:56:11.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.143+0000 7fd7e8de8b80 1 --2- 172.21.15.6:0/3389406436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e25cf7a30 0x557e25d13240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:11.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.143+0000 7fd7e8de8b80 1 --2- 172.21.15.6:0/3389406436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e25b74640 0x557e25cf15b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:11.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.143+0000 7fd7e8de8b80 1 --2- 172.21.15.6:0/3389406436 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557e25cf1af0 0x557e25d075b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:11.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.143+0000 7fd7e8de8b80 1 -- 172.21.15.6:0/3389406436 >> 172.21.15.6:0/3389406436 conn(0x557e25b75930 msgr2=0x557e25cf5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:11.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.143+0000 7fd7e8de8b80 1 -- 172.21.15.6:0/3389406436 shutdown_connections 2024-05-09T04:56:11.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.143+0000 7fd7e8de8b80 1 -- 172.21.15.6:0/3389406436 wait complete. 2024-05-09T04:56:11.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.143+0000 7fd7e8de8b80 1 -- start start 2024-05-09T04:56:11.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.144+0000 7fd7e8de8b80 1 Processor -- start 2024-05-09T04:56:11.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.144+0000 7fd7e8de8b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e25b74640 0x557e25d03ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:11.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.144+0000 7fd7e8de8b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557e25cf1af0 0x557e25d00900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:11.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.144+0000 7fd7e8de8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e25cf7a30 0x557e25d00e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:11.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.144+0000 7fd7e8de8b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d41b3550 con 0x557e25cf1af0 2024-05-09T04:56:11.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.144+0000 7fd7e8de8b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d41b2e80 con 0x557e25b74640 2024-05-09T04:56:11.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.144+0000 7fd7e8de8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d41b0080 con 0x557e25cf7a30 2024-05-09T04:56:11.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.144+0000 7fd7e8de8b80 10 client.0 _ll_register_callbacks cb 0x557e25bab730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:56:11.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.145+0000 7fd7cbe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557e25cf1af0 0x557e25d00900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:11.146 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.145+0000 7fd7cbe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557e25cf1af0 0x557e25d00900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54792/0 (socket says 172.21.15.6:54792) 2024-05-09T04:56:11.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.145+0000 7fd7cbe00700 1 -- 172.21.15.6:0/1170861255 learned_addr learned my addr 172.21.15.6:0/1170861255 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:56:11.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.145+0000 7fd7d3400700 1 --2- 172.21.15.6:0/1170861255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e25b74640 0x557e25d03ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:11.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.145+0000 7fd7d3e00700 1 --2- 172.21.15.6:0/1170861255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e25cf7a30 0x557e25d00e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:11.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.145+0000 7fd7cbe00700 1 -- 172.21.15.6:0/1170861255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e25cf7a30 msgr2=0x557e25d00e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:11.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.145+0000 7fd7cbe00700 1 --2- 172.21.15.6:0/1170861255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e25cf7a30 0x557e25d00e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:11.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.145+0000 7fd7cbe00700 1 -- 172.21.15.6:0/1170861255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e25b74640 msgr2=0x557e25d03ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:11.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.145+0000 7fd7cbe00700 1 --2- 172.21.15.6:0/1170861255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e25b74640 0x557e25d03ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:11.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.145+0000 7fd7cbe00700 1 -- 172.21.15.6:0/1170861255 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e25d01380 con 0x557e25cf1af0 2024-05-09T04:56:11.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.145+0000 7fd7cbe00700 1 --2- 172.21.15.6:0/1170861255 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557e25cf1af0 0x557e25d00900 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc009d10 tx=0x7fd7bc005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b95ba2f778e6dc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:11.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.146+0000 7fd7c1600700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-05-09T04:56:11.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.146+0000 7fd7c1600700 1 -- 172.21.15.6:0/1170861255 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc038020 con 0x557e25cf1af0 2024-05-09T04:56:11.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.146+0000 7fd7c1600700 1 -- 172.21.15.6:0/1170861255 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fd7bc005580 con 0x557e25cf1af0 2024-05-09T04:56:11.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.146+0000 7fd7e8de8b80 1 -- 172.21.15.6:0/1170861255 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557e25c1be70 con 0x557e25cf1af0 2024-05-09T04:56:11.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.146+0000 7fd7c1600700 1 -- 172.21.15.6:0/1170861255 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd7bc043050 con 0x557e25cf1af0 2024-05-09T04:56:11.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.146+0000 7fd7e8de8b80 10 client.15252 fetch_fsmap learned FSMap version 30 2024-05-09T04:56:11.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.146+0000 7fd7e8de8b80 1 -- 172.21.15.6:0/1170861255 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557e25d769e0 con 0x557e25cf1af0 2024-05-09T04:56:11.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.147+0000 7fd7c1600700 1 -- 172.21.15.6:0/1170861255 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557e25c1be70 con 0x557e25cf1af0 2024-05-09T04:56:11.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.147+0000 7fd7c1600700 1 -- 172.21.15.6:0/1170861255 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd7bc02e040 con 0x557e25cf1af0 2024-05-09T04:56:11.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.147+0000 7fd7e8de8b80 10 client.15252 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T04:56:11.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.147+0000 7fd7e8de8b80 10 client.15252 resolve_mds: validated gid 24554 aka daemon mds.b 2024-05-09T04:56:11.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.147+0000 7fd7e8de8b80 20 client.15252 populate_metadata read hostname 'smithi006' 2024-05-09T04:56:11.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.147+0000 7fd7e8de8b80 1 --2- 172.21.15.6:0/1170861255 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x557e25d76ce0 0x557e25d79100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:11.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.147+0000 7fd7e8de8b80 4 client.15252 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.95, "await_for": 45.0}] 2024-05-09T04:56:11.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.147+0000 7fd7e8de8b80 1 -- 172.21.15.6:0/1170861255 --> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.95, "await_for": 45.0}) -- 0x557e25a8fb10 con 0x557e25d76ce0 2024-05-09T04:56:11.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.148+0000 7fd7d3400700 1 --2- 172.21.15.6:0/1170861255 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x557e25d76ce0 0x557e25d79100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:11.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.148+0000 7fd7c1600700 10 client.15252 ms_handle_connect on v2:172.21.15.169:6838/614388771 2024-05-09T04:56:11.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.148+0000 7fd7d3400700 1 --2- 172.21.15.6:0/1170861255 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x557e25d76ce0 0x557e25d79100 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=874e748ec254f306 server_cookie=1884ea1e1a396179 in_seq=0 out_seq=0 2024-05-09T04:56:11.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.160+0000 7fd7c1600700 1 -- 172.21.15.6:0/1170861255 <== mds.0 v2:172.21.15.169:6838/614388771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x557e25a8fb10 con 0x557e25d76ce0 2024-05-09T04:56:11.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.160+0000 7fd7c1600700 10 client.15252 handle_command_reply: tid=0 2024-05-09T04:56:11.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.160+0000 7fd7e8de8b80 1 client.15252 shutdown 2024-05-09T04:56:11.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.160+0000 7fd7e8de8b80 1 -- 172.21.15.6:0/1170861255 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557e25cf1af0 msgr2=0x557e25d00900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:11.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.160+0000 7fd7e8de8b80 1 --2- 172.21.15.6:0/1170861255 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557e25cf1af0 0x557e25d00900 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc009d10 tx=0x7fd7bc005ae0 comp rx=0 tx=0).stop 2024-05-09T04:56:11.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.160+0000 7fd7e8de8b80 1 -- 172.21.15.6:0/1170861255 shutdown_connections 2024-05-09T04:56:11.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.160+0000 7fd7e8de8b80 1 --2- 172.21.15.6:0/1170861255 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x557e25d76ce0 0x557e25d79100 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:11.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.160+0000 7fd7e8de8b80 1 --2- 172.21.15.6:0/1170861255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557e25cf7a30 0x557e25d00e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:11.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.160+0000 7fd7e8de8b80 1 --2- 172.21.15.6:0/1170861255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e25b74640 0x557e25d03ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:11.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.160+0000 7fd7e8de8b80 1 --2- 172.21.15.6:0/1170861255 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557e25cf1af0 0x557e25d00900 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:11.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.160+0000 7fd7c1600700 0 client.15252 ms_handle_reset on v2:172.21.15.169:6838/614388771 2024-05-09T04:56:11.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.160+0000 7fd7e8de8b80 1 -- 172.21.15.6:0/1170861255 >> 172.21.15.6:0/1170861255 conn(0x557e25b75930 msgr2=0x557e25cf6860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:11.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.160+0000 7fd7e8de8b80 1 -- 172.21.15.6:0/1170861255 shutdown_connections 2024-05-09T04:56:11.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.160+0000 7fd7e8de8b80 1 -- 172.21.15.6:0/1170861255 wait complete. 2024-05-09T04:56:11.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.161+0000 7fd7e8de8b80 20 client.15252 trim_cache size 0 max 16384 2024-05-09T04:56:11.163 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-09T04:56:11.163 INFO:teuthology.orchestra.run.smithi006.stdout: "epoch": 30, 2024-05-09T04:56:11.163 INFO:teuthology.orchestra.run.smithi006.stdout: "leader": 24554, 2024-05-09T04:56:11.163 INFO:teuthology.orchestra.run.smithi006.stdout: "set_version": 2, 2024-05-09T04:56:11.163 INFO:teuthology.orchestra.run.smithi006.stdout: "sets": { 2024-05-09T04:56:11.163 INFO:teuthology.orchestra.run.smithi006.stdout: "a3128f52": { 2024-05-09T04:56:11.163 INFO:teuthology.orchestra.run.smithi006.stdout: "version": 2, 2024-05-09T04:56:11.163 INFO:teuthology.orchestra.run.smithi006.stdout: "age_ref": 0.0, 2024-05-09T04:56:11.163 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-05-09T04:56:11.163 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "QUIESCED", 2024-05-09T04:56:11.163 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-05-09T04:56:11.163 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:56:11.163 INFO:teuthology.orchestra.run.smithi006.stdout: "timeout": 90.0, 2024-05-09T04:56:11.164 INFO:teuthology.orchestra.run.smithi006.stdout: "expiration": 144.9, 2024-05-09T04:56:11.164 INFO:teuthology.orchestra.run.smithi006.stdout: "members": { 2024-05-09T04:56:11.164 INFO:teuthology.orchestra.run.smithi006.stdout: "file:/": { 2024-05-09T04:56:11.164 INFO:teuthology.orchestra.run.smithi006.stdout: "excluded": false, 2024-05-09T04:56:11.164 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-05-09T04:56:11.164 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "QUIESCED", 2024-05-09T04:56:11.164 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-05-09T04:56:11.164 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:56:11.164 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:56:11.164 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:56:11.164 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:56:11.164 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:56:11.164 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-09T04:56:11.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.161+0000 7fd7d8c00700 1 -- 172.21.15.6:0/536559346 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd7b80b2990 msgr2=0x7fd7b80b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:11.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.161+0000 7fd7d8c00700 1 --2- 172.21.15.6:0/536559346 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd7b80b2990 0x7fd7b80b4ea0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x557e25cf6fb0 tx=0x7fd7cc004b40 comp rx=0 tx=0).stop 2024-05-09T04:56:11.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.161+0000 7fd7d8c00700 1 -- 172.21.15.6:0/536559346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7d4106620 msgr2=0x7fd7d419ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:11.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.161+0000 7fd7d8c00700 1 --2- 172.21.15.6:0/536559346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7d4106620 0x7fd7d419ce40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd7c400e9a0 tx=0x7fd7c400ecb0 comp rx=0 tx=0).stop 2024-05-09T04:56:11.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.161+0000 7fd7d8c00700 1 -- 172.21.15.6:0/536559346 shutdown_connections 2024-05-09T04:56:11.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.161+0000 7fd7d8c00700 1 --2- 172.21.15.6:0/536559346 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7d410b8a0 0x7fd7d41a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:11.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.161+0000 7fd7d8c00700 1 --2- 172.21.15.6:0/536559346 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd7b80b2990 0x7fd7b80b4ea0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:11.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.161+0000 7fd7d8c00700 1 --2- 172.21.15.6:0/536559346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7d4106620 0x7fd7d419ce40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:11.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.161+0000 7fd7d8c00700 1 --2- 172.21.15.6:0/536559346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7d410af50 0x7fd7d419d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:11.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.162+0000 7fd7d8c00700 1 -- 172.21.15.6:0/536559346 >> 172.21.15.6:0/536559346 conn(0x7fd7d4102030 msgr2=0x7fd7d410a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:11.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.162+0000 7fd7d8c00700 1 -- 172.21.15.6:0/536559346 shutdown_connections 2024-05-09T04:56:11.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:11.162+0000 7fd7d8c00700 1 -- 172.21.15.6:0/536559346 wait complete. 2024-05-09T04:56:11.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:10 smithi186 ceph-mon[34668]: pgmap v536: 129 pgs: 129 active+clean; 667 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-09T04:56:11.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:10 smithi169 ceph-mon[34075]: pgmap v536: 129 pgs: 129 active+clean; 667 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-09T04:56:12.183 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a3128f52', quiesce took 8.1 seconds. Will release after: 16.85 2024-05-09T04:56:13.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:12 smithi006 ceph-mon[29835]: pgmap v537: 129 pgs: 129 active+clean; 667 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-09T04:56:13.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:12 smithi186 ceph-mon[34668]: pgmap v537: 129 pgs: 129 active+clean; 667 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-09T04:56:13.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:12 smithi169 ceph-mon[34075]: pgmap v537: 129 pgs: 129 active+clean; 667 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-09T04:56:15.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:14 smithi006 ceph-mon[29835]: pgmap v538: 129 pgs: 129 active+clean; 667 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-09T04:56:15.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:14 smithi186 ceph-mon[34668]: pgmap v538: 129 pgs: 129 active+clean; 667 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-09T04:56:15.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:14 smithi169 ceph-mon[34075]: pgmap v538: 129 pgs: 129 active+clean; 667 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-09T04:56:17.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:16 smithi006 ceph-mon[29835]: pgmap v539: 129 pgs: 129 active+clean; 667 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 596 B/s wr, 11 op/s 2024-05-09T04:56:17.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:16 smithi186 ceph-mon[34668]: pgmap v539: 129 pgs: 129 active+clean; 667 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 596 B/s wr, 11 op/s 2024-05-09T04:56:17.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:16 smithi169 ceph-mon[34075]: pgmap v539: 129 pgs: 129 active+clean; 667 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 596 B/s wr, 11 op/s 2024-05-09T04:56:19.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:18 smithi006 ceph-mon[29835]: pgmap v540: 129 pgs: 129 active+clean; 667 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-09T04:56:19.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:18 smithi186 ceph-mon[34668]: pgmap v540: 129 pgs: 129 active+clean; 667 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-09T04:56:19.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:18 smithi169 ceph-mon[34075]: pgmap v540: 129 pgs: 129 active+clean; 667 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-09T04:56:21.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:20 smithi006 ceph-mon[29835]: pgmap v541: 129 pgs: 129 active+clean; 667 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-09T04:56:21.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:20 smithi186 ceph-mon[34668]: pgmap v541: 129 pgs: 129 active+clean; 667 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-09T04:56:21.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:20 smithi169 ceph-mon[34075]: pgmap v541: 129 pgs: 129 active+clean; 667 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-09T04:56:23.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:22 smithi006 ceph-mon[29835]: pgmap v542: 129 pgs: 129 active+clean; 667 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-09T04:56:23.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:22 smithi186 ceph-mon[34668]: pgmap v542: 129 pgs: 129 active+clean; 667 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-09T04:56:23.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:22 smithi169 ceph-mon[34075]: pgmap v542: 129 pgs: 129 active+clean; 667 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-09T04:56:25.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:24 smithi006 ceph-mon[29835]: pgmap v543: 129 pgs: 129 active+clean; 667 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-09T04:56:25.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:24 smithi186 ceph-mon[34668]: pgmap v543: 129 pgs: 129 active+clean; 667 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-09T04:56:25.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:24 smithi169 ceph-mon[34075]: pgmap v543: 129 pgs: 129 active+clean; 667 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-09T04:56:27.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:26 smithi006 ceph-mon[29835]: pgmap v544: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 170 B/s wr, 11 op/s 2024-05-09T04:56:27.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:26 smithi186 ceph-mon[34668]: pgmap v544: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 170 B/s wr, 11 op/s 2024-05-09T04:56:27.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:26 smithi169 ceph-mon[34075]: pgmap v544: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 170 B/s wr, 11 op/s 2024-05-09T04:56:28.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:27 smithi186 ceph-mon[34668]: mgrmap e20: x(active, since 15m), standbys: z, y 2024-05-09T04:56:28.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:27 smithi169 ceph-mon[34075]: mgrmap e20: x(active, since 15m), standbys: z, y 2024-05-09T04:56:28.446 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:27 smithi006 ceph-mon[29835]: mgrmap e20: x(active, since 15m), standbys: z, y 2024-05-09T04:56:29.035 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:56:29.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:28 smithi186 ceph-mon[34668]: pgmap v545: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T04:56:29.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:28 smithi169 ceph-mon[34075]: pgmap v545: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T04:56:29.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:28 smithi006 ceph-mon[29835]: pgmap v545: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T04:56:29.575 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:56:31.294 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:30 smithi006 ceph-mon[29835]: pgmap v546: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-09T04:56:31.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:30 smithi186 ceph-mon[34668]: pgmap v546: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-09T04:56:31.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:30 smithi169 ceph-mon[34075]: pgmap v546: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-09T04:56:31.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.959+0000 7fa2e2a00700 1 -- 172.21.15.6:0/432993897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2dc108d30 msgr2=0x7fa2dc109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:31.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.959+0000 7fa2e2a00700 1 --2- 172.21.15.6:0/432993897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2dc108d30 0x7fa2dc109140 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8009990 tx=0x7fa2c8009ca0 comp rx=0 tx=0).stop 2024-05-09T04:56:31.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.959+0000 7fa2e2a00700 1 -- 172.21.15.6:0/432993897 shutdown_connections 2024-05-09T04:56:31.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.959+0000 7fa2e2a00700 1 --2- 172.21.15.6:0/432993897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2dc109680 0x7fa2dc110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:31.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.959+0000 7fa2e2a00700 1 --2- 172.21.15.6:0/432993897 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2dc104400 0x7fa2dc104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:31.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.959+0000 7fa2e2a00700 1 --2- 172.21.15.6:0/432993897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2dc108d30 0x7fa2dc109140 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:31.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.959+0000 7fa2e2a00700 1 -- 172.21.15.6:0/432993897 >> 172.21.15.6:0/432993897 conn(0x7fa2dc079d10 msgr2=0x7fa2dc07a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:31.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.960+0000 7fa2e2a00700 1 -- 172.21.15.6:0/432993897 shutdown_connections 2024-05-09T04:56:31.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.960+0000 7fa2e2a00700 1 -- 172.21.15.6:0/432993897 wait complete. 2024-05-09T04:56:31.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.962+0000 7fa2e2a00700 1 Processor -- start 2024-05-09T04:56:31.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.962+0000 7fa2e2a00700 1 -- start start 2024-05-09T04:56:31.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.962+0000 7fa2e2a00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2dc104400 0x7fa2dc10b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:31.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.962+0000 7fa2e2a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2dc108d30 0x7fa2dc10b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:31.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.962+0000 7fa2e2a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2dc109680 0x7fa2dc10bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:31.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.962+0000 7fa2e2a00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc1142e0 con 0x7fa2dc104400 2024-05-09T04:56:31.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.962+0000 7fa2e2a00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc114160 con 0x7fa2dc109680 2024-05-09T04:56:31.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.962+0000 7fa2e2a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc114460 con 0x7fa2dc108d30 2024-05-09T04:56:31.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.963+0000 7fa2e0c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2dc108d30 0x7fa2dc10b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:31.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.963+0000 7fa2e0c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2dc108d30 0x7fa2dc10b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:49594/0 (socket says 172.21.15.6:49594) 2024-05-09T04:56:31.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.963+0000 7fa2e0c00700 1 -- 172.21.15.6:0/634716098 learned_addr learned my addr 172.21.15.6:0/634716098 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:56:31.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.963+0000 7fa2e2000700 1 --2- 172.21.15.6:0/634716098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2dc109680 0x7fa2dc10bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:31.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.964+0000 7fa2e0c00700 1 -- 172.21.15.6:0/634716098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2dc109680 msgr2=0x7fa2dc10bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:31.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.964+0000 7fa2e0c00700 1 --2- 172.21.15.6:0/634716098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2dc109680 0x7fa2dc10bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:31.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.964+0000 7fa2e0c00700 1 -- 172.21.15.6:0/634716098 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2dc104400 msgr2=0x7fa2dc10b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:31.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.964+0000 7fa2e0c00700 1 --2- 172.21.15.6:0/634716098 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2dc104400 0x7fa2dc10b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:31.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.964+0000 7fa2e0c00700 1 -- 172.21.15.6:0/634716098 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2dc1ad570 con 0x7fa2dc108d30 2024-05-09T04:56:31.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.964+0000 7fa2e2000700 1 --2- 172.21.15.6:0/634716098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2dc109680 0x7fa2dc10bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:56:31.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.964+0000 7fa2e0c00700 1 --2- 172.21.15.6:0/634716098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2dc108d30 0x7fa2dc10b5a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8000c00 tx=0x7fa2c8009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:31.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.966+0000 7fa2da000700 1 -- 172.21.15.6:0/634716098 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c800fac0 con 0x7fa2dc108d30 2024-05-09T04:56:31.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.966+0000 7fa2da000700 1 -- 172.21.15.6:0/634716098 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2c8004ce0 con 0x7fa2dc108d30 2024-05-09T04:56:31.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.966+0000 7fa2da000700 1 -- 172.21.15.6:0/634716098 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c80055c0 con 0x7fa2dc108d30 2024-05-09T04:56:31.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.966+0000 7fa2e2a00700 1 -- 172.21.15.6:0/634716098 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2dc1ad7a0 con 0x7fa2dc108d30 2024-05-09T04:56:31.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.966+0000 7fa2e2a00700 1 -- 172.21.15.6:0/634716098 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2dc1adb40 con 0x7fa2dc108d30 2024-05-09T04:56:31.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.968+0000 7fa2e2a00700 1 -- 172.21.15.6:0/634716098 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2dc1058b0 con 0x7fa2dc108d30 2024-05-09T04:56:31.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.972+0000 7fa2da000700 1 -- 172.21.15.6:0/634716098 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7fa2c8028020 con 0x7fa2dc108d30 2024-05-09T04:56:31.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.975+0000 7fa2da000700 1 --2- 172.21.15.6:0/634716098 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa2c40b2900 0x7fa2c40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:31.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.975+0000 7fa2da000700 1 -- 172.21.15.6:0/634716098 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fa2c80e3650 con 0x7fa2dc108d30 2024-05-09T04:56:31.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.976+0000 7fa2e1600700 1 --2- 172.21.15.6:0/634716098 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa2c40b2900 0x7fa2c40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:31.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.977+0000 7fa2e1600700 1 --2- 172.21.15.6:0/634716098 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa2c40b2900 0x7fa2c40b4e10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0005950 tx=0x7fa2d00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:31.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:31.979+0000 7fa2da000700 1 -- 172.21.15.6:0/634716098 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa2c8013070 con 0x7fa2dc108d30 2024-05-09T04:56:32.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:32.375+0000 7fa2e2a00700 1 -- 172.21.15.6:0/634716098 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2dc1ade60 con 0x7fa2dc108d30 2024-05-09T04:56:32.378 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:56:32.378 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/162094599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":162094599},{"type":"v1","addr":"172.21.15.6:6835","nonce":162094599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/589294363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":589294363},{"type":"v1","addr":"172.21.15.6:6841","nonce":589294363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:49:36.920376+0000","modified":"2024-05-09T04:54:56.247431+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24554,"mds_1":24530,"mds_2":24512,"mds_3":24511,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24442":{"gid":24442,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6833/2977145206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2977145206},{"type":"v1","addr":"172.21.15.186:6833","nonce":2977145206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6833/133345948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":133345948},{"type":"v1","addr":"172.21.15.169:6833","nonce":133345948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.6:6837/1281521088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":1281521088},{"type":"v1","addr":"172.21.15.6:6837","nonce":1281521088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2268322007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2268322007},{"type":"v1","addr":"172.21.15.186:6835","nonce":2268322007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/3811005052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3811005052},{"type":"v1","addr":"172.21.15.169:6835","nonce":3811005052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.186:6837/1647607118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1647607118},{"type":"v1","addr":"172.21.15.186:6837","nonce":1647607118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":32,"addr":"172.21.15.186:6839/3954883105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":3954883105},{"type":"v1","addr":"172.21.15.186:6839","nonce":3954883105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.169:6837/1845715294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1845715294},{"type":"v1","addr":"172.21.15.169:6837","nonce":1845715294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.6:6839/2780306492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":2780306492},{"type":"v1","addr":"172.21.15.6:6839","nonce":2780306492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.169:6839/614388771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":614388771},{"type":"v1","addr":"172.21.15.169:6839","nonce":614388771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24512,24554,24530,24493,24511]},"id":1}]} 2024-05-09T04:56:32.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:32.377+0000 7fa2da000700 1 -- 172.21.15.6:0/634716098 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11927 (secure 0 0 0) 0x7fa2c80ab020 con 0x7fa2dc108d30 2024-05-09T04:56:32.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:32.380+0000 7fa2e2a00700 1 -- 172.21.15.6:0/634716098 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa2c40b2900 msgr2=0x7fa2c40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:32.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:32.380+0000 7fa2e2a00700 1 --2- 172.21.15.6:0/634716098 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa2c40b2900 0x7fa2c40b4e10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0005950 tx=0x7fa2d00058e0 comp rx=0 tx=0).stop 2024-05-09T04:56:32.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:32.380+0000 7fa2e2a00700 1 -- 172.21.15.6:0/634716098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2dc108d30 msgr2=0x7fa2dc10b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:32.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:32.380+0000 7fa2e2a00700 1 --2- 172.21.15.6:0/634716098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2dc108d30 0x7fa2dc10b5a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8000c00 tx=0x7fa2c8009580 comp rx=0 tx=0).stop 2024-05-09T04:56:32.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:32.380+0000 7fa2e2a00700 1 -- 172.21.15.6:0/634716098 shutdown_connections 2024-05-09T04:56:32.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:32.381+0000 7fa2e2a00700 1 --2- 172.21.15.6:0/634716098 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fa2c40b2900 0x7fa2c40b4e10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:32.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:32.381+0000 7fa2e2a00700 1 --2- 172.21.15.6:0/634716098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2dc109680 0x7fa2dc10bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:32.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:32.381+0000 7fa2e2a00700 1 --2- 172.21.15.6:0/634716098 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa2dc104400 0x7fa2dc10b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:32.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:32.381+0000 7fa2e2a00700 1 --2- 172.21.15.6:0/634716098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2dc108d30 0x7fa2dc10b5a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:32.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:32.381+0000 7fa2e2a00700 1 -- 172.21.15.6:0/634716098 >> 172.21.15.6:0/634716098 conn(0x7fa2dc079d10 msgr2=0x7fa2dc10f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:32.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:32.381+0000 7fa2e2a00700 1 -- 172.21.15.6:0/634716098 shutdown_connections 2024-05-09T04:56:32.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:32.381+0000 7fa2e2a00700 1 -- 172.21.15.6:0/634716098 wait complete. 2024-05-09T04:56:32.383 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 30 2024-05-09T04:56:33.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:32 smithi006 ceph-mon[29835]: pgmap v547: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-05-09T04:56:33.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:32 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/634716098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:56:33.340 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24554 quiesce db --set-id a3128f52 --release --await' 2024-05-09T04:56:33.340 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24554 quiesce db --set-id a3128f52 --release --await 2024-05-09T04:56:33.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:32 smithi186 ceph-mon[34668]: pgmap v547: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-05-09T04:56:33.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:32 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/634716098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:56:33.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:32 smithi169 ceph-mon[34075]: pgmap v547: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-05-09T04:56:33.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:32 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/634716098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:56:33.805 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:56:35.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:34 smithi006 ceph-mon[29835]: pgmap v548: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-09T04:56:35.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:34 smithi186 ceph-mon[34668]: pgmap v548: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-09T04:56:35.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:34 smithi169 ceph-mon[34075]: pgmap v548: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-09T04:56:36.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.227+0000 7f20b1600700 1 -- 172.21.15.6:0/3750960638 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ac1030b0 msgr2=0x7f20ac1034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:36.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.227+0000 7f20b1600700 1 --2- 172.21.15.6:0/3750960638 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ac1030b0 0x7f20ac1034c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f209c009990 tx=0x7f209c009ca0 comp rx=0 tx=0).stop 2024-05-09T04:56:36.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.227+0000 7f20b1600700 1 -- 172.21.15.6:0/3750960638 shutdown_connections 2024-05-09T04:56:36.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.227+0000 7f20b1600700 1 --2- 172.21.15.6:0/3750960638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20ac108500 0x7f20ac1089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:36.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.227+0000 7f20b1600700 1 --2- 172.21.15.6:0/3750960638 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ac1030b0 0x7f20ac1034c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:36.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.227+0000 7f20b1600700 1 --2- 172.21.15.6:0/3750960638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20ac103a00 0x7f20ac107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:36.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.227+0000 7f20b1600700 1 -- 172.21.15.6:0/3750960638 >> 172.21.15.6:0/3750960638 conn(0x7f20ac078bf0 msgr2=0x7f20ac079030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:36.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.228+0000 7f20b1600700 1 -- 172.21.15.6:0/3750960638 shutdown_connections 2024-05-09T04:56:36.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.228+0000 7f20b1600700 1 -- 172.21.15.6:0/3750960638 wait complete. 2024-05-09T04:56:36.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.229+0000 7f20b1600700 1 Processor -- start 2024-05-09T04:56:36.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.230+0000 7f20b1600700 1 -- start start 2024-05-09T04:56:36.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.230+0000 7f20b1600700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ac1030b0 0x7f20ac1a1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:36.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.230+0000 7f20b1600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20ac103a00 0x7f20ac1a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:36.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.230+0000 7f20b1600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20ac108500 0x7f20ac1a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:36.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.230+0000 7f20b1600700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac11a8d0 con 0x7f20ac1030b0 2024-05-09T04:56:36.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.230+0000 7f20b1600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac11a750 con 0x7f20ac108500 2024-05-09T04:56:36.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.230+0000 7f20b1600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac11aa50 con 0x7f20ac103a00 2024-05-09T04:56:36.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.231+0000 7f20abe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ac1030b0 0x7f20ac1a1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:36.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.231+0000 7f20abe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ac1030b0 0x7f20ac1a1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53074/0 (socket says 172.21.15.6:53074) 2024-05-09T04:56:36.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.231+0000 7f20abe00700 1 -- 172.21.15.6:0/2469738124 learned_addr learned my addr 172.21.15.6:0/2469738124 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:56:36.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.231+0000 7f20ab400700 1 --2- 172.21.15.6:0/2469738124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20ac103a00 0x7f20ac1a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:36.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.231+0000 7f20b0c00700 1 --2- 172.21.15.6:0/2469738124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20ac108500 0x7f20ac1a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:36.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.231+0000 7f20ab400700 1 -- 172.21.15.6:0/2469738124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20ac108500 msgr2=0x7f20ac1a5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:36.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.231+0000 7f20ab400700 1 --2- 172.21.15.6:0/2469738124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20ac108500 0x7f20ac1a5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:36.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.231+0000 7f20ab400700 1 -- 172.21.15.6:0/2469738124 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ac1030b0 msgr2=0x7f20ac1a1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:36.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.231+0000 7f20ab400700 1 --2- 172.21.15.6:0/2469738124 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ac1030b0 0x7f20ac1a1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:36.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.231+0000 7f20ab400700 1 -- 172.21.15.6:0/2469738124 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20ac1a60d0 con 0x7f20ac103a00 2024-05-09T04:56:36.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.232+0000 7f20abe00700 1 --2- 172.21.15.6:0/2469738124 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ac1030b0 0x7f20ac1a1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:56:36.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.232+0000 7f20ab400700 1 --2- 172.21.15.6:0/2469738124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20ac103a00 0x7f20ac1a17a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f209800e9f0 tx=0x7f209800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:36.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.232+0000 7f20a8c00700 1 -- 172.21.15.6:0/2469738124 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209800c9c0 con 0x7f20ac103a00 2024-05-09T04:56:36.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.232+0000 7f20a8c00700 1 -- 172.21.15.6:0/2469738124 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f209800cb60 con 0x7f20ac103a00 2024-05-09T04:56:36.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.232+0000 7f20a8c00700 1 -- 172.21.15.6:0/2469738124 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2098018680 con 0x7f20ac103a00 2024-05-09T04:56:36.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.232+0000 7f20b1600700 1 -- 172.21.15.6:0/2469738124 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20ac1a63c0 con 0x7f20ac103a00 2024-05-09T04:56:36.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.232+0000 7f20b1600700 1 -- 172.21.15.6:0/2469738124 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20ac1adce0 con 0x7f20ac103a00 2024-05-09T04:56:36.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.238+0000 7f20a8c00700 1 -- 172.21.15.6:0/2469738124 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f2098010430 con 0x7f20ac103a00 2024-05-09T04:56:36.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.239+0000 7f20a8c00700 1 --2- 172.21.15.6:0/2469738124 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f20900b2900 0x7f20900b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:36.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.239+0000 7f20c1831b80 1 Processor -- start 2024-05-09T04:56:36.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.239+0000 7f20a8c00700 1 -- 172.21.15.6:0/2469738124 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f20980e26d0 con 0x7f20ac103a00 2024-05-09T04:56:36.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.240+0000 7f20abe00700 1 --2- 172.21.15.6:0/2469738124 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f20900b2900 0x7f20900b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:36.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.240+0000 7f20c1831b80 1 -- start start 2024-05-09T04:56:36.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.241+0000 7f20c1831b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622784ae5a0 0x562278631450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:36.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.241+0000 7f20c1831b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562278631990 0x56227862b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:36.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.241+0000 7f20c1831b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56227862ba50 0x56227862be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:36.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.241+0000 7f20c1831b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x56227854e350 con 0x5622784ae5a0 2024-05-09T04:56:36.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.241+0000 7f20c1831b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5622784ec120 con 0x56227862ba50 2024-05-09T04:56:36.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.241+0000 7f20c1831b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56227854a310 con 0x562278631990 2024-05-09T04:56:36.242 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.241+0000 7f20abe00700 1 --2- 172.21.15.6:0/2469738124 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f20900b2900 0x7f20900b4e10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f209c006010 tx=0x7f209c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:36.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.241+0000 7f20ab400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562278631990 0x56227862b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:36.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.241+0000 7f20ab400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562278631990 0x56227862b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:49618/0 (socket says 172.21.15.6:49618) 2024-05-09T04:56:36.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.241+0000 7f20ab400700 1 -- 172.21.15.6:0/2963497321 learned_addr learned my addr 172.21.15.6:0/2963497321 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:56:36.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.241+0000 7f20b0c00700 1 --2- 172.21.15.6:0/2963497321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56227862ba50 0x56227862be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:36.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.242+0000 7f20ab400700 1 -- 172.21.15.6:0/2963497321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56227862ba50 msgr2=0x56227862be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:36.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.242+0000 7f20ab400700 1 --2- 172.21.15.6:0/2963497321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56227862ba50 0x56227862be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:36.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.242+0000 7f20ab400700 1 -- 172.21.15.6:0/2963497321 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622784ae5a0 msgr2=0x562278631450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:36.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.242+0000 7f20ab400700 1 --2- 172.21.15.6:0/2963497321 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622784ae5a0 0x562278631450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:36.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.242+0000 7f20ab400700 1 -- 172.21.15.6:0/2963497321 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622783c9590 con 0x562278631990 2024-05-09T04:56:36.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.242+0000 7f20ab400700 1 --2- 172.21.15.6:0/2963497321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562278631990 0x56227862b510 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f20980a6b80 tx=0x7f20980b2180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d67b83db3430dc72 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:36.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.242+0000 7f20a1600700 1 -- 172.21.15.6:0/2963497321 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20980aa010 con 0x562278631990 2024-05-09T04:56:36.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.242+0000 7f20a1600700 1 -- 172.21.15.6:0/2963497321 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20980a9910 con 0x562278631990 2024-05-09T04:56:36.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.242+0000 7f20a1600700 1 -- 172.21.15.6:0/2963497321 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20980e2480 con 0x562278631990 2024-05-09T04:56:36.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.243+0000 7f20c1831b80 1 -- 172.21.15.6:0/2963497321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562278631990 msgr2=0x56227862b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:36.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.243+0000 7f20c1831b80 1 --2- 172.21.15.6:0/2963497321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562278631990 0x56227862b510 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f20980a6b80 tx=0x7f20980b2180 comp rx=0 tx=0).stop 2024-05-09T04:56:36.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.243+0000 7f20c1831b80 1 -- 172.21.15.6:0/2963497321 shutdown_connections 2024-05-09T04:56:36.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.243+0000 7f20c1831b80 1 --2- 172.21.15.6:0/2963497321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56227862ba50 0x56227862be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:36.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.243+0000 7f20c1831b80 1 --2- 172.21.15.6:0/2963497321 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622784ae5a0 0x562278631450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:36.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.243+0000 7f20c1831b80 1 --2- 172.21.15.6:0/2963497321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562278631990 0x56227862b510 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:36.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.243+0000 7f20c1831b80 1 -- 172.21.15.6:0/2963497321 >> 172.21.15.6:0/2963497321 conn(0x5622784af890 msgr2=0x562278626390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:36.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.244+0000 7f20c1831b80 1 -- 172.21.15.6:0/2963497321 shutdown_connections 2024-05-09T04:56:36.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.244+0000 7f20c1831b80 1 -- 172.21.15.6:0/2963497321 wait complete. 2024-05-09T04:56:36.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.244+0000 7f20c1831b80 1 -- start start 2024-05-09T04:56:36.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.244+0000 7f20c1831b80 1 Processor -- start 2024-05-09T04:56:36.247 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.245+0000 7f20c1831b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622784ae5a0 0x562278635140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:36.247 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.245+0000 7f20c1831b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56227862ba50 0x562278635680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:36.247 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.245+0000 7f20c1831b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562278631990 0x562278635bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:36.247 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.245+0000 7f20c1831b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x562278638400 con 0x5622784ae5a0 2024-05-09T04:56:36.248 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.245+0000 7f20c1831b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5622784e83d0 con 0x562278631990 2024-05-09T04:56:36.248 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.245+0000 7f20c1831b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x562278638580 con 0x56227862ba50 2024-05-09T04:56:36.248 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.245+0000 7f20c1831b80 10 client.0 _ll_register_callbacks cb 0x5622784e5690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:56:36.248 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.246+0000 7f20abe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622784ae5a0 0x562278635140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:36.248 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.246+0000 7f20abe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622784ae5a0 0x562278635140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53098/0 (socket says 172.21.15.6:53098) 2024-05-09T04:56:36.248 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.246+0000 7f20abe00700 1 -- 172.21.15.6:0/3750932947 learned_addr learned my addr 172.21.15.6:0/3750932947 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:56:36.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.246+0000 7f20b0c00700 1 --2- 172.21.15.6:0/3750932947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562278631990 0x562278635bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:36.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.246+0000 7f20ab400700 1 --2- 172.21.15.6:0/3750932947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56227862ba50 0x562278635680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:36.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.246+0000 7f20abe00700 1 -- 172.21.15.6:0/3750932947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56227862ba50 msgr2=0x562278635680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:36.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.246+0000 7f20abe00700 1 --2- 172.21.15.6:0/3750932947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56227862ba50 0x562278635680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:36.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.246+0000 7f20abe00700 1 -- 172.21.15.6:0/3750932947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562278631990 msgr2=0x562278635bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:36.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.246+0000 7f20abe00700 1 --2- 172.21.15.6:0/3750932947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562278631990 0x562278635bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:36.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.246+0000 7f20abe00700 1 -- 172.21.15.6:0/3750932947 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562278513950 con 0x5622784ae5a0 2024-05-09T04:56:36.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.247+0000 7f20abe00700 1 --2- 172.21.15.6:0/3750932947 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622784ae5a0 0x562278635140 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f209c00fb90 tx=0x7f209c00fea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c445797a4957f2e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:36.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.247+0000 7f2096000700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-05-09T04:56:36.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.247+0000 7f2096000700 1 -- 172.21.15.6:0/3750932947 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c009530 con 0x5622784ae5a0 2024-05-09T04:56:36.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.247+0000 7f2096000700 1 -- 172.21.15.6:0/3750932947 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f209c0247a0 con 0x5622784ae5a0 2024-05-09T04:56:36.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.247+0000 7f20c1831b80 1 -- 172.21.15.6:0/3750932947 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20980a9910 con 0x5622784ae5a0 2024-05-09T04:56:36.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.248+0000 7f2096000700 1 -- 172.21.15.6:0/3750932947 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f209c0249a0 con 0x5622784ae5a0 2024-05-09T04:56:36.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.248+0000 7f20c1831b80 10 client.15264 fetch_fsmap learned FSMap version 30 2024-05-09T04:56:36.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.248+0000 7f20c1831b80 1 -- 172.21.15.6:0/3750932947 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5622786540e0 con 0x5622784ae5a0 2024-05-09T04:56:36.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.248+0000 7f2096000700 1 -- 172.21.15.6:0/3750932947 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20980a9910 con 0x5622784ae5a0 2024-05-09T04:56:36.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.248+0000 7f2096000700 1 -- 172.21.15.6:0/3750932947 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f209c022070 con 0x5622784ae5a0 2024-05-09T04:56:36.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.248+0000 7f20c1831b80 10 client.15264 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T04:56:36.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.248+0000 7f20c1831b80 10 client.15264 resolve_mds: validated gid 24554 aka daemon mds.b 2024-05-09T04:56:36.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.249+0000 7f20c1831b80 20 client.15264 populate_metadata read hostname 'smithi006' 2024-05-09T04:56:36.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.249+0000 7f20c1831b80 1 --2- 172.21.15.6:0/3750932947 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x562278656ad0 0x562278658f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:36.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.249+0000 7f20c1831b80 4 client.15264 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:56:36.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.249+0000 7f20c1831b80 1 -- 172.21.15.6:0/3750932947 --> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56227855bd30 con 0x562278656ad0 2024-05-09T04:56:36.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.249+0000 7f20ab400700 1 --2- 172.21.15.6:0/3750932947 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x562278656ad0 0x562278658f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:36.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.250+0000 7f20ab400700 1 --2- 172.21.15.6:0/3750932947 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x562278656ad0 0x562278658f60 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1305b5a06b97f70f server_cookie=76897f9fa00f4470 in_seq=0 out_seq=0 2024-05-09T04:56:36.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.250+0000 7f2096000700 10 client.15264 ms_handle_connect on v2:172.21.15.169:6838/614388771 2024-05-09T04:56:36.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.251+0000 7f2096000700 1 -- 172.21.15.6:0/3750932947 <== mds.0 v2:172.21.15.169:6838/614388771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56227855bd30 con 0x562278656ad0 2024-05-09T04:56:36.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.251+0000 7f2096000700 10 client.15264 handle_command_reply: tid=0 2024-05-09T04:56:36.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.251+0000 7f20c1831b80 1 client.15264 shutdown 2024-05-09T04:56:36.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.251+0000 7f20c1831b80 1 -- 172.21.15.6:0/3750932947 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622784ae5a0 msgr2=0x562278635140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:36.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.251+0000 7f20c1831b80 1 --2- 172.21.15.6:0/3750932947 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622784ae5a0 0x562278635140 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f209c00fb90 tx=0x7f209c00fea0 comp rx=0 tx=0).stop 2024-05-09T04:56:36.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.251+0000 7f20c1831b80 1 -- 172.21.15.6:0/3750932947 shutdown_connections 2024-05-09T04:56:36.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.251+0000 7f20c1831b80 1 --2- 172.21.15.6:0/3750932947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562278631990 0x562278635bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:36.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.251+0000 7f20c1831b80 1 --2- 172.21.15.6:0/3750932947 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5622784ae5a0 0x562278635140 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:36.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.251+0000 7f20c1831b80 1 --2- 172.21.15.6:0/3750932947 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x562278656ad0 0x562278658f60 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:36.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.251+0000 7f20c1831b80 1 --2- 172.21.15.6:0/3750932947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56227862ba50 0x562278635680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:36.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.251+0000 7f20c1831b80 1 -- 172.21.15.6:0/3750932947 >> 172.21.15.6:0/3750932947 conn(0x5622784af890 msgr2=0x5622786260d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:36.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.251+0000 7f2096000700 0 client.15264 ms_handle_reset on v2:172.21.15.169:6838/614388771 2024-05-09T04:56:36.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.252+0000 7f20c1831b80 1 -- 172.21.15.6:0/3750932947 shutdown_connections 2024-05-09T04:56:36.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.252+0000 7f20c1831b80 1 -- 172.21.15.6:0/3750932947 wait complete. 2024-05-09T04:56:36.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.252+0000 7f20c1831b80 20 client.15264 trim_cache size 0 max 16384 2024-05-09T04:56:36.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.273+0000 7f20c1831b80 1 Processor -- start 2024-05-09T04:56:36.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.274+0000 7f20c1831b80 1 -- start start 2024-05-09T04:56:36.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.274+0000 7f20c1831b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5622784ae5a0 0x56227862b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:36.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.274+0000 7f20c1831b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56227862ba50 0x56227862f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:36.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.274+0000 7f20c1831b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562278631990 0x562278635a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:36.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.274+0000 7f20c1831b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac1adf80 con 0x562278631990 2024-05-09T04:56:36.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.274+0000 7f20c1831b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac1afaa0 con 0x5622784ae5a0 2024-05-09T04:56:36.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.274+0000 7f20c1831b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac1ae630 con 0x56227862ba50 2024-05-09T04:56:36.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.275+0000 7f20abe00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5622784ae5a0 0x56227862b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:36.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.275+0000 7f20abe00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5622784ae5a0 0x56227862b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:60376/0 (socket says 172.21.15.6:60376) 2024-05-09T04:56:36.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.275+0000 7f20abe00700 1 -- 172.21.15.6:0/3858496799 learned_addr learned my addr 172.21.15.6:0/3858496799 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:56:36.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.275+0000 7f20b0c00700 1 --2- 172.21.15.6:0/3858496799 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562278631990 0x562278635a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:36.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.275+0000 7f20ab400700 1 --2- 172.21.15.6:0/3858496799 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56227862ba50 0x56227862f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:36.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.275+0000 7f20abe00700 1 -- 172.21.15.6:0/3858496799 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56227862ba50 msgr2=0x56227862f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:36.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.275+0000 7f20abe00700 1 --2- 172.21.15.6:0/3858496799 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56227862ba50 0x56227862f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:36.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.275+0000 7f20abe00700 1 -- 172.21.15.6:0/3858496799 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562278631990 msgr2=0x562278635a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:36.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.275+0000 7f20abe00700 1 --2- 172.21.15.6:0/3858496799 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562278631990 0x562278635a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:36.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.275+0000 7f20abe00700 1 -- 172.21.15.6:0/3858496799 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622786568b0 con 0x5622784ae5a0 2024-05-09T04:56:36.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.275+0000 7f20b0c00700 1 --2- 172.21.15.6:0/3858496799 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562278631990 0x562278635a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:56:36.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.276+0000 7f20abe00700 1 --2- 172.21.15.6:0/3858496799 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5622784ae5a0 0x56227862b510 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f209c0098e0 tx=0x7f209c024900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a48591dfe856f77 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:36.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.276+0000 7f20a0c00700 1 -- 172.21.15.6:0/3858496799 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c03d070 con 0x5622784ae5a0 2024-05-09T04:56:36.278 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.276+0000 7f20a0c00700 1 -- 172.21.15.6:0/3858496799 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f209c017690 con 0x5622784ae5a0 2024-05-09T04:56:36.278 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.276+0000 7f20a0c00700 1 -- 172.21.15.6:0/3858496799 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c0249a0 con 0x5622784ae5a0 2024-05-09T04:56:36.278 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.277+0000 7f20c1831b80 1 -- 172.21.15.6:0/3858496799 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5622784ae5a0 msgr2=0x56227862b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:36.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.277+0000 7f20c1831b80 1 --2- 172.21.15.6:0/3858496799 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5622784ae5a0 0x56227862b510 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f209c0098e0 tx=0x7f209c024900 comp rx=0 tx=0).stop 2024-05-09T04:56:36.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.277+0000 7f20c1831b80 1 -- 172.21.15.6:0/3858496799 shutdown_connections 2024-05-09T04:56:36.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.277+0000 7f20c1831b80 1 --2- 172.21.15.6:0/3858496799 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562278631990 0x562278635a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:36.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.277+0000 7f20c1831b80 1 --2- 172.21.15.6:0/3858496799 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5622784ae5a0 0x56227862b510 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:36.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.277+0000 7f20c1831b80 1 --2- 172.21.15.6:0/3858496799 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56227862ba50 0x56227862f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:36.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.277+0000 7f20c1831b80 1 -- 172.21.15.6:0/3858496799 >> 172.21.15.6:0/3858496799 conn(0x5622784af890 msgr2=0x562278633c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:36.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.277+0000 7f20c1831b80 1 -- 172.21.15.6:0/3858496799 shutdown_connections 2024-05-09T04:56:36.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.278+0000 7f20c1831b80 1 -- 172.21.15.6:0/3858496799 wait complete. 2024-05-09T04:56:36.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.278+0000 7f20c1831b80 1 -- start start 2024-05-09T04:56:36.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.278+0000 7f20c1831b80 1 Processor -- start 2024-05-09T04:56:36.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.279+0000 7f20c1831b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5622784ae5a0 0x56227864ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:36.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.279+0000 7f20c1831b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56227862ba50 0x56227864b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:36.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.279+0000 7f20c1831b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562278631990 0x56227864bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:36.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.279+0000 7f20c1831b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac1af3d0 con 0x562278631990 2024-05-09T04:56:36.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.279+0000 7f20c1831b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac1aed00 con 0x5622784ae5a0 2024-05-09T04:56:36.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.279+0000 7f20c1831b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac06a620 con 0x56227862ba50 2024-05-09T04:56:36.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.279+0000 7f20c1831b80 10 client.0 _ll_register_callbacks cb 0x5622784e5690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:56:36.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.280+0000 7f20abe00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5622784ae5a0 0x56227864ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:36.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.280+0000 7f20abe00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5622784ae5a0 0x56227864ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:60388/0 (socket says 172.21.15.6:60388) 2024-05-09T04:56:36.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.280+0000 7f20abe00700 1 -- 172.21.15.6:0/4095711334 learned_addr learned my addr 172.21.15.6:0/4095711334 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:56:36.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.280+0000 7f20ab400700 1 --2- 172.21.15.6:0/4095711334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56227862ba50 0x56227864b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:36.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.280+0000 7f20b0c00700 1 --2- 172.21.15.6:0/4095711334 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562278631990 0x56227864bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:36.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.280+0000 7f20abe00700 1 -- 172.21.15.6:0/4095711334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56227862ba50 msgr2=0x56227864b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:36.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.280+0000 7f20abe00700 1 --2- 172.21.15.6:0/4095711334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56227862ba50 0x56227864b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:36.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.280+0000 7f20abe00700 1 -- 172.21.15.6:0/4095711334 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562278631990 msgr2=0x56227864bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:36.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.280+0000 7f20abe00700 1 --2- 172.21.15.6:0/4095711334 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562278631990 0x56227864bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:36.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.280+0000 7f20abe00700 1 -- 172.21.15.6:0/4095711334 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56227864c320 con 0x5622784ae5a0 2024-05-09T04:56:36.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.280+0000 7f20b0c00700 1 --2- 172.21.15.6:0/4095711334 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562278631990 0x56227864bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:56:36.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.281+0000 7f20abe00700 1 --2- 172.21.15.6:0/4095711334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5622784ae5a0 0x56227864ebf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f209c00b4c0 tx=0x7f209c017af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb6eaed3f1786671 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:36.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.281+0000 7f2096000700 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-05-09T04:56:36.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.281+0000 7f2096000700 1 -- 172.21.15.6:0/4095711334 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c03d030 con 0x5622784ae5a0 2024-05-09T04:56:36.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.281+0000 7f20c1831b80 1 -- 172.21.15.6:0/4095711334 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f209c017690 con 0x5622784ae5a0 2024-05-09T04:56:36.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.281+0000 7f2096000700 1 -- 172.21.15.6:0/4095711334 <== mon.2 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f209c02f620 con 0x5622784ae5a0 2024-05-09T04:56:36.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.282+0000 7f2096000700 1 -- 172.21.15.6:0/4095711334 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f209c04f050 con 0x5622784ae5a0 2024-05-09T04:56:36.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.282+0000 7f20c1831b80 10 client.25424 fetch_fsmap learned FSMap version 30 2024-05-09T04:56:36.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.282+0000 7f20c1831b80 1 -- 172.21.15.6:0/4095711334 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5622786c1fb0 con 0x5622784ae5a0 2024-05-09T04:56:36.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.282+0000 7f2096000700 1 -- 172.21.15.6:0/4095711334 <== mon.2 v2:172.21.15.169:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f209c017690 con 0x5622784ae5a0 2024-05-09T04:56:36.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.283+0000 7f2096000700 1 -- 172.21.15.6:0/4095711334 <== mon.2 v2:172.21.15.169:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f209c022070 con 0x5622784ae5a0 2024-05-09T04:56:36.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.283+0000 7f20c1831b80 10 client.25424 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T04:56:36.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.283+0000 7f20c1831b80 10 client.25424 resolve_mds: validated gid 24554 aka daemon mds.b 2024-05-09T04:56:36.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.283+0000 7f20c1831b80 20 client.25424 populate_metadata read hostname 'smithi006' 2024-05-09T04:56:36.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.283+0000 7f20c1831b80 1 --2- 172.21.15.6:0/4095711334 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x5622786c2570 0x5622786c4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:36.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.283+0000 7f20c1831b80 4 client.25424 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a3128f52", "release": true, "await": true}] 2024-05-09T04:56:36.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.283+0000 7f20c1831b80 1 -- 172.21.15.6:0/4095711334 --> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a3128f52", "release": true, "await": true}) -- 0x562278555dd0 con 0x5622786c2570 2024-05-09T04:56:36.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.283+0000 7f20ab400700 1 --2- 172.21.15.6:0/4095711334 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x5622786c2570 0x5622786c4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:36.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.284+0000 7f20ab400700 1 --2- 172.21.15.6:0/4095711334 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x5622786c2570 0x5622786c4990 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b11444828f03a78 server_cookie=b2ad40acce2ffd7a in_seq=0 out_seq=0 2024-05-09T04:56:36.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.284+0000 7f2096000700 10 client.25424 ms_handle_connect on v2:172.21.15.169:6838/614388771 2024-05-09T04:56:36.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.287+0000 7f2096000700 1 -- 172.21.15.6:0/4095711334 <== mds.0 v2:172.21.15.169:6838/614388771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x562278555dd0 con 0x5622786c2570 2024-05-09T04:56:36.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.288+0000 7f2096000700 10 client.25424 handle_command_reply: tid=0 2024-05-09T04:56:36.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.288+0000 7f20c1831b80 1 client.25424 shutdown 2024-05-09T04:56:36.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.288+0000 7f20c1831b80 1 -- 172.21.15.6:0/4095711334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5622784ae5a0 msgr2=0x56227864ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:36.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.288+0000 7f20c1831b80 1 --2- 172.21.15.6:0/4095711334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5622784ae5a0 0x56227864ebf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f209c00b4c0 tx=0x7f209c017af0 comp rx=0 tx=0).stop 2024-05-09T04:56:36.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.288+0000 7f20c1831b80 1 -- 172.21.15.6:0/4095711334 shutdown_connections 2024-05-09T04:56:36.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.288+0000 7f20c1831b80 1 --2- 172.21.15.6:0/4095711334 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562278631990 0x56227864bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:36.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.288+0000 7f20c1831b80 1 --2- 172.21.15.6:0/4095711334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5622784ae5a0 0x56227864ebf0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:36.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.288+0000 7f20c1831b80 1 --2- 172.21.15.6:0/4095711334 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x5622786c2570 0x5622786c4990 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:36.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.288+0000 7f20c1831b80 1 --2- 172.21.15.6:0/4095711334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56227862ba50 0x56227864b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:36.291 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-09T04:56:36.291 INFO:teuthology.orchestra.run.smithi006.stdout: "epoch": 30, 2024-05-09T04:56:36.291 INFO:teuthology.orchestra.run.smithi006.stdout: "leader": 24554, 2024-05-09T04:56:36.291 INFO:teuthology.orchestra.run.smithi006.stdout: "set_version": 4, 2024-05-09T04:56:36.291 INFO:teuthology.orchestra.run.smithi006.stdout: "sets": { 2024-05-09T04:56:36.291 INFO:teuthology.orchestra.run.smithi006.stdout: "a3128f52": { 2024-05-09T04:56:36.291 INFO:teuthology.orchestra.run.smithi006.stdout: "version": 4, 2024-05-09T04:56:36.291 INFO:teuthology.orchestra.run.smithi006.stdout: "age_ref": 0.0, 2024-05-09T04:56:36.291 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-05-09T04:56:36.292 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RELEASED", 2024-05-09T04:56:36.292 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-05-09T04:56:36.292 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:56:36.292 INFO:teuthology.orchestra.run.smithi006.stdout: "timeout": 90.0, 2024-05-09T04:56:36.292 INFO:teuthology.orchestra.run.smithi006.stdout: "expiration": 144.9, 2024-05-09T04:56:36.292 INFO:teuthology.orchestra.run.smithi006.stdout: "members": { 2024-05-09T04:56:36.292 INFO:teuthology.orchestra.run.smithi006.stdout: "file:/": { 2024-05-09T04:56:36.292 INFO:teuthology.orchestra.run.smithi006.stdout: "excluded": false, 2024-05-09T04:56:36.292 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-05-09T04:56:36.292 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RELEASED", 2024-05-09T04:56:36.293 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-05-09T04:56:36.293 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:56:36.293 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:56:36.293 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:56:36.293 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:56:36.293 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:56:36.293 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-09T04:56:36.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.288+0000 7f20c1831b80 1 -- 172.21.15.6:0/4095711334 >> 172.21.15.6:0/4095711334 conn(0x5622784af890 msgr2=0x5622786366e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:36.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.288+0000 7f2096000700 0 client.25424 ms_handle_reset on v2:172.21.15.169:6838/614388771 2024-05-09T04:56:36.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.288+0000 7f20c1831b80 1 -- 172.21.15.6:0/4095711334 shutdown_connections 2024-05-09T04:56:36.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.288+0000 7f20c1831b80 1 -- 172.21.15.6:0/4095711334 wait complete. 2024-05-09T04:56:36.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.288+0000 7f20c1831b80 20 client.25424 trim_cache size 0 max 16384 2024-05-09T04:56:36.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.291+0000 7f20b1600700 1 -- 172.21.15.6:0/2469738124 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f20900b2900 msgr2=0x7f20900b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:36.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.291+0000 7f20b1600700 1 --2- 172.21.15.6:0/2469738124 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f20900b2900 0x7f20900b4e10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f209c006010 tx=0x7f209c00b260 comp rx=0 tx=0).stop 2024-05-09T04:56:36.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.291+0000 7f20b1600700 1 -- 172.21.15.6:0/2469738124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20ac103a00 msgr2=0x7f20ac1a17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:36.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.291+0000 7f20b1600700 1 --2- 172.21.15.6:0/2469738124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20ac103a00 0x7f20ac1a17a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f209800e9f0 tx=0x7f209800ed00 comp rx=0 tx=0).stop 2024-05-09T04:56:36.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.292+0000 7f20b1600700 1 -- 172.21.15.6:0/2469738124 shutdown_connections 2024-05-09T04:56:36.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.292+0000 7f20b1600700 1 --2- 172.21.15.6:0/2469738124 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f20900b2900 0x7f20900b4e10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:36.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.292+0000 7f20b1600700 1 --2- 172.21.15.6:0/2469738124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20ac108500 0x7f20ac1a5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:36.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.292+0000 7f20b1600700 1 --2- 172.21.15.6:0/2469738124 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ac1030b0 0x7f20ac1a1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:36.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.292+0000 7f20b1600700 1 --2- 172.21.15.6:0/2469738124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20ac103a00 0x7f20ac1a17a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:36.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.292+0000 7f20b1600700 1 -- 172.21.15.6:0/2469738124 >> 172.21.15.6:0/2469738124 conn(0x7f20ac078bf0 msgr2=0x7f20ac105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:36.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.292+0000 7f20b1600700 1 -- 172.21.15.6:0/2469738124 shutdown_connections 2024-05-09T04:56:36.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:36.292+0000 7f20b1600700 1 -- 172.21.15.6:0/2469738124 wait complete. 2024-05-09T04:56:37.230 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:36 smithi006 ceph-mon[29835]: pgmap v549: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-09T04:56:37.277 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a3128f52', total seconds elapsed: 33.2 2024-05-09T04:56:37.277 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.95 2024-05-09T04:56:37.278 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:56:37.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:36 smithi186 ceph-mon[34668]: pgmap v549: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-09T04:56:37.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:36 smithi169 ceph-mon[34075]: pgmap v549: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-09T04:56:37.861 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:56:39.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:38 smithi006 ceph-mon[29835]: pgmap v550: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 511 B/s wr, 10 op/s 2024-05-09T04:56:39.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:38 smithi186 ceph-mon[34668]: pgmap v550: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 511 B/s wr, 10 op/s 2024-05-09T04:56:39.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:38 smithi169 ceph-mon[34075]: pgmap v550: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 511 B/s wr, 10 op/s 2024-05-09T04:56:40.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.303+0000 7f7da8e00700 1 -- 172.21.15.6:0/3226330655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7da4105be0 msgr2=0x7f7da410e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:40.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.303+0000 7f7da8e00700 1 --2- 172.21.15.6:0/3226330655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7da4105be0 0x7f7da410e130 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7d90009990 tx=0x7f7d90009ca0 comp rx=0 tx=0).stop 2024-05-09T04:56:40.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.303+0000 7f7da8e00700 1 -- 172.21.15.6:0/3226330655 shutdown_connections 2024-05-09T04:56:40.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.303+0000 7f7da8e00700 1 --2- 172.21.15.6:0/3226330655 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7da410e700 0x7f7da4110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:40.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.303+0000 7f7da8e00700 1 --2- 172.21.15.6:0/3226330655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7da4105290 0x7f7da41056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:40.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.303+0000 7f7da8e00700 1 --2- 172.21.15.6:0/3226330655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7da4105be0 0x7f7da410e130 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:40.305 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.303+0000 7f7da8e00700 1 -- 172.21.15.6:0/3226330655 >> 172.21.15.6:0/3226330655 conn(0x7f7da4078ac0 msgr2=0x7f7da4078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:40.305 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.304+0000 7f7da8e00700 1 -- 172.21.15.6:0/3226330655 shutdown_connections 2024-05-09T04:56:40.305 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.304+0000 7f7da8e00700 1 -- 172.21.15.6:0/3226330655 wait complete. 2024-05-09T04:56:40.306 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.306+0000 7f7da8e00700 1 Processor -- start 2024-05-09T04:56:40.306 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.306+0000 7f7da8e00700 1 -- start start 2024-05-09T04:56:40.306 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.306+0000 7f7da8e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7da4105290 0x7f7da4106100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:40.306 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.307+0000 7f7da8e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7da4105be0 0x7f7da4106640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:40.306 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.307+0000 7f7da8e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7da410e700 0x7f7da4106b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:40.307 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.307+0000 7f7da8e00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7da41142b0 con 0x7f7da4105290 2024-05-09T04:56:40.307 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.307+0000 7f7da8e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7da4114130 con 0x7f7da4105be0 2024-05-09T04:56:40.307 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.307+0000 7f7da8e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7da4114430 con 0x7f7da410e700 2024-05-09T04:56:40.307 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.307+0000 7f7da3400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7da4105290 0x7f7da4106100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:40.307 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.307+0000 7f7da2a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7da4105be0 0x7f7da4106640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:40.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.307+0000 7f7da3e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7da410e700 0x7f7da4106b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:40.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.307+0000 7f7da3400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7da4105290 0x7f7da4106100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54286/0 (socket says 172.21.15.6:54286) 2024-05-09T04:56:40.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.307+0000 7f7da3400700 1 -- 172.21.15.6:0/1647826592 learned_addr learned my addr 172.21.15.6:0/1647826592 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:56:40.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.308+0000 7f7da3400700 1 -- 172.21.15.6:0/1647826592 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7da410e700 msgr2=0x7f7da4106b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:40.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.308+0000 7f7da3400700 1 --2- 172.21.15.6:0/1647826592 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7da410e700 0x7f7da4106b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:40.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.308+0000 7f7da3400700 1 -- 172.21.15.6:0/1647826592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7da4105be0 msgr2=0x7f7da4106640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:40.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.308+0000 7f7da3400700 1 --2- 172.21.15.6:0/1647826592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7da4105be0 0x7f7da4106640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:40.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.308+0000 7f7da3400700 1 -- 172.21.15.6:0/1647826592 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da41ad4e0 con 0x7f7da4105290 2024-05-09T04:56:40.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.308+0000 7f7da3400700 1 --2- 172.21.15.6:0/1647826592 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7da4105290 0x7f7da4106100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7d9400e9a0 tx=0x7f7d9400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:40.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.309+0000 7f7d9be00700 1 -- 172.21.15.6:0/1647826592 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9400ca10 con 0x7f7da4105290 2024-05-09T04:56:40.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.309+0000 7f7d9be00700 1 -- 172.21.15.6:0/1647826592 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d9400cbb0 con 0x7f7da4105290 2024-05-09T04:56:40.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.309+0000 7f7d9be00700 1 -- 172.21.15.6:0/1647826592 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d94018680 con 0x7f7da4105290 2024-05-09T04:56:40.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.309+0000 7f7da8e00700 1 -- 172.21.15.6:0/1647826592 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da41ad770 con 0x7f7da4105290 2024-05-09T04:56:40.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.309+0000 7f7da8e00700 1 -- 172.21.15.6:0/1647826592 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7da41adc60 con 0x7f7da4105290 2024-05-09T04:56:40.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.311+0000 7f7da8e00700 1 -- 172.21.15.6:0/1647826592 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7da406bee0 con 0x7f7da4105290 2024-05-09T04:56:40.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.313+0000 7f7d9be00700 1 -- 172.21.15.6:0/1647826592 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f7d94010430 con 0x7f7da4105290 2024-05-09T04:56:40.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.314+0000 7f7d9be00700 1 --2- 172.21.15.6:0/1647826592 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f7d880b2900 0x7f7d880b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:40.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.315+0000 7f7d9be00700 1 -- 172.21.15.6:0/1647826592 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f7d940e2860 con 0x7f7da4105290 2024-05-09T04:56:40.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.315+0000 7f7da2a00700 1 --2- 172.21.15.6:0/1647826592 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f7d880b2900 0x7f7d880b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:40.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.316+0000 7f7da2a00700 1 --2- 172.21.15.6:0/1647826592 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f7d880b2900 0x7f7d880b4e10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7da4101f30 tx=0x7f7d90005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:40.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.320+0000 7f7d9be00700 1 -- 172.21.15.6:0/1647826592 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7d94054df0 con 0x7f7da4105290 2024-05-09T04:56:40.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.606+0000 7f7da8e00700 1 -- 172.21.15.6:0/1647826592 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7da410bb60 con 0x7f7da4105290 2024-05-09T04:56:40.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.608+0000 7f7d9be00700 1 -- 172.21.15.6:0/1647826592 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11927 (secure 0 0 0) 0x7f7d940aa1a0 con 0x7f7da4105290 2024-05-09T04:56:40.608 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:56:40.608 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/162094599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":162094599},{"type":"v1","addr":"172.21.15.6:6835","nonce":162094599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/589294363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":589294363},{"type":"v1","addr":"172.21.15.6:6841","nonce":589294363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:49:36.920376+0000","modified":"2024-05-09T04:54:56.247431+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24554,"mds_1":24530,"mds_2":24512,"mds_3":24511,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24442":{"gid":24442,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6833/2977145206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2977145206},{"type":"v1","addr":"172.21.15.186:6833","nonce":2977145206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6833/133345948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":133345948},{"type":"v1","addr":"172.21.15.169:6833","nonce":133345948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.6:6837/1281521088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":1281521088},{"type":"v1","addr":"172.21.15.6:6837","nonce":1281521088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2268322007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2268322007},{"type":"v1","addr":"172.21.15.186:6835","nonce":2268322007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/3811005052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3811005052},{"type":"v1","addr":"172.21.15.169:6835","nonce":3811005052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.186:6837/1647607118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1647607118},{"type":"v1","addr":"172.21.15.186:6837","nonce":1647607118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":32,"addr":"172.21.15.186:6839/3954883105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":3954883105},{"type":"v1","addr":"172.21.15.186:6839","nonce":3954883105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.169:6837/1845715294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1845715294},{"type":"v1","addr":"172.21.15.169:6837","nonce":1845715294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.6:6839/2780306492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":2780306492},{"type":"v1","addr":"172.21.15.6:6839","nonce":2780306492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.169:6839/614388771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":614388771},{"type":"v1","addr":"172.21.15.169:6839","nonce":614388771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24512,24554,24530,24493,24511]},"id":1}]} 2024-05-09T04:56:40.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.611+0000 7f7da8e00700 1 -- 172.21.15.6:0/1647826592 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f7d880b2900 msgr2=0x7f7d880b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:40.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.611+0000 7f7da8e00700 1 --2- 172.21.15.6:0/1647826592 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f7d880b2900 0x7f7d880b4e10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7da4101f30 tx=0x7f7d90005fd0 comp rx=0 tx=0).stop 2024-05-09T04:56:40.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.612+0000 7f7da8e00700 1 -- 172.21.15.6:0/1647826592 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7da4105290 msgr2=0x7f7da4106100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:40.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.612+0000 7f7da8e00700 1 --2- 172.21.15.6:0/1647826592 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7da4105290 0x7f7da4106100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7d9400e9a0 tx=0x7f7d9400ecb0 comp rx=0 tx=0).stop 2024-05-09T04:56:40.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.612+0000 7f7da8e00700 1 -- 172.21.15.6:0/1647826592 shutdown_connections 2024-05-09T04:56:40.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.612+0000 7f7da8e00700 1 --2- 172.21.15.6:0/1647826592 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7da410e700 0x7f7da4106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:40.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.612+0000 7f7da8e00700 1 --2- 172.21.15.6:0/1647826592 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7da4105290 0x7f7da4106100 secure :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7d9400e9a0 tx=0x7f7d9400ecb0 comp rx=0 tx=0).stop 2024-05-09T04:56:40.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.612+0000 7f7da8e00700 1 --2- 172.21.15.6:0/1647826592 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f7d880b2900 0x7f7d880b4e10 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:40.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.612+0000 7f7da8e00700 1 --2- 172.21.15.6:0/1647826592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7da4105be0 0x7f7da4106640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:40.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.612+0000 7f7da8e00700 1 -- 172.21.15.6:0/1647826592 >> 172.21.15.6:0/1647826592 conn(0x7f7da4078ac0 msgr2=0x7f7da4104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:40.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.613+0000 7f7da8e00700 1 -- 172.21.15.6:0/1647826592 shutdown_connections 2024-05-09T04:56:40.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:40.613+0000 7f7da8e00700 1 -- 172.21.15.6:0/1647826592 wait complete. 2024-05-09T04:56:40.614 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 30 2024-05-09T04:56:41.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:40 smithi186 ceph-mon[34668]: pgmap v551: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-09T04:56:41.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:40 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/1647826592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:56:41.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:40 smithi169 ceph-mon[34075]: pgmap v551: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-09T04:56:41.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:40 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/1647826592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:56:41.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:40 smithi006 ceph-mon[29835]: pgmap v551: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-09T04:56:41.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:40 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1647826592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:56:41.526 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24554 quiesce db / --timeout 90 --expiration 144.95 --await-for 45.0' 2024-05-09T04:56:41.527 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24554 quiesce db / --timeout 90 --expiration 144.95 --await-for 45.0 2024-05-09T04:56:42.148 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:56:43.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:42 smithi186 ceph-mon[34668]: pgmap v552: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 341 B/s wr, 11 op/s 2024-05-09T04:56:43.356 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:42 smithi006 ceph-mon[29835]: pgmap v552: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 341 B/s wr, 11 op/s 2024-05-09T04:56:43.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:42 smithi169 ceph-mon[34075]: pgmap v552: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 341 B/s wr, 11 op/s 2024-05-09T04:56:44.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.494+0000 7fe51be00700 1 -- 172.21.15.6:0/1288395711 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe51c10e700 msgr2=0x7fe51c110be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:44.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.494+0000 7fe51be00700 1 --2- 172.21.15.6:0/1288395711 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe51c10e700 0x7fe51c110be0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe51400b0a0 tx=0x7fe51400b3b0 comp rx=0 tx=0).stop 2024-05-09T04:56:44.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.494+0000 7fe51be00700 1 -- 172.21.15.6:0/1288395711 shutdown_connections 2024-05-09T04:56:44.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.495+0000 7fe51be00700 1 --2- 172.21.15.6:0/1288395711 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe51c10e700 0x7fe51c110be0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:44.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.495+0000 7fe51be00700 1 --2- 172.21.15.6:0/1288395711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe51c105290 0x7fe51c1056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:44.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.495+0000 7fe51be00700 1 --2- 172.21.15.6:0/1288395711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe51c105be0 0x7fe51c10e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:44.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.495+0000 7fe51be00700 1 -- 172.21.15.6:0/1288395711 >> 172.21.15.6:0/1288395711 conn(0x7fe51c078ac0 msgr2=0x7fe51c078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:44.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.495+0000 7fe51be00700 1 -- 172.21.15.6:0/1288395711 shutdown_connections 2024-05-09T04:56:44.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.496+0000 7fe51be00700 1 -- 172.21.15.6:0/1288395711 wait complete. 2024-05-09T04:56:44.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.497+0000 7fe51be00700 1 Processor -- start 2024-05-09T04:56:44.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.498+0000 7fe51be00700 1 -- start start 2024-05-09T04:56:44.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.498+0000 7fe51be00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe51c105290 0x7fe51c1a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:44.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.498+0000 7fe51be00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe51c105be0 0x7fe51c1a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:44.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.499+0000 7fe51a000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe51c105be0 0x7fe51c1a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:44.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.499+0000 7fe51a000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe51c105be0 0x7fe51c1a0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54298/0 (socket says 172.21.15.6:54298) 2024-05-09T04:56:44.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.499+0000 7fe51be00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe51c10e700 0x7fe51c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:44.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.499+0000 7fe51be00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c114260 con 0x7fe51c105be0 2024-05-09T04:56:44.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.499+0000 7fe51be00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1140e0 con 0x7fe51c10e700 2024-05-09T04:56:44.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.499+0000 7fe51aa00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe51c105290 0x7fe51c1a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:44.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.499+0000 7fe51a000700 1 -- 172.21.15.6:0/1821608229 learned_addr learned my addr 172.21.15.6:0/1821608229 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:56:44.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.499+0000 7fe51be00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1143e0 con 0x7fe51c105290 2024-05-09T04:56:44.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.500+0000 7fe51b400700 1 --2- 172.21.15.6:0/1821608229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe51c10e700 0x7fe51c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:44.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.500+0000 7fe51a000700 1 -- 172.21.15.6:0/1821608229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe51c105290 msgr2=0x7fe51c1a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:44.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.500+0000 7fe51a000700 1 --2- 172.21.15.6:0/1821608229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe51c105290 0x7fe51c1a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:44.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.500+0000 7fe51a000700 1 -- 172.21.15.6:0/1821608229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe51c10e700 msgr2=0x7fe51c19a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:44.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.500+0000 7fe51a000700 1 --2- 172.21.15.6:0/1821608229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe51c10e700 0x7fe51c19a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:44.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.500+0000 7fe51a000700 1 -- 172.21.15.6:0/1821608229 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe51c19b090 con 0x7fe51c105be0 2024-05-09T04:56:44.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.500+0000 7fe51b400700 1 --2- 172.21.15.6:0/1821608229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe51c10e700 0x7fe51c19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:56:44.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.501+0000 7fe51a000700 1 --2- 172.21.15.6:0/1821608229 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe51c105be0 0x7fe51c1a0d00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe50800e9a0 tx=0x7fe50800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:44.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.501+0000 7fe513400700 1 -- 172.21.15.6:0/1821608229 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50800cbd0 con 0x7fe51c105be0 2024-05-09T04:56:44.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.501+0000 7fe51be00700 1 -- 172.21.15.6:0/1821608229 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe51c19b380 con 0x7fe51c105be0 2024-05-09T04:56:44.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.502+0000 7fe513400700 1 -- 172.21.15.6:0/1821608229 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe50800cd70 con 0x7fe51c105be0 2024-05-09T04:56:44.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.502+0000 7fe513400700 1 -- 172.21.15.6:0/1821608229 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe508018680 con 0x7fe51c105be0 2024-05-09T04:56:44.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.502+0000 7fe51be00700 1 -- 172.21.15.6:0/1821608229 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe51c1a7650 con 0x7fe51c105be0 2024-05-09T04:56:44.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.509+0000 7fe53040fb80 1 Processor -- start 2024-05-09T04:56:44.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.509+0000 7fe53040fb80 1 -- start start 2024-05-09T04:56:44.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.509+0000 7fe53040fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c8e365640 0x563c8e4e87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:44.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.509+0000 7fe53040fb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c8e4e8cf0 0x563c8e4e2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:44.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.509+0000 7fe53040fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c8e4e2fa0 0x563c8e4e33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:44.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.509+0000 7fe53040fb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x563c8e405450 con 0x563c8e4e8cf0 2024-05-09T04:56:44.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.509+0000 7fe53040fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563c8e3a3280 con 0x563c8e365640 2024-05-09T04:56:44.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.509+0000 7fe53040fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x563c8e401410 con 0x563c8e4e2fa0 2024-05-09T04:56:44.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.511+0000 7fe513400700 1 -- 172.21.15.6:0/1821608229 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7fe508018820 con 0x7fe51c105be0 2024-05-09T04:56:44.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.511+0000 7fe51b400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c8e365640 0x563c8e4e87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:44.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.511+0000 7fe51b400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c8e365640 0x563c8e4e87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:47304/0 (socket says 172.21.15.6:47304) 2024-05-09T04:56:44.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.511+0000 7fe51b400700 1 -- 172.21.15.6:0/526808124 learned_addr learned my addr 172.21.15.6:0/526808124 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:56:44.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.511+0000 7fe51aa00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c8e4e8cf0 0x563c8e4e2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:44.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.512+0000 7fe51b400700 1 -- 172.21.15.6:0/526808124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c8e4e2fa0 msgr2=0x563c8e4e33b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T04:56:44.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.512+0000 7fe51a000700 1 --2- 172.21.15.6:0/526808124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c8e4e2fa0 0x563c8e4e33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:44.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.513+0000 7fe51b400700 1 --2- 172.21.15.6:0/526808124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c8e4e2fa0 0x563c8e4e33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:44.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.513+0000 7fe51b400700 1 -- 172.21.15.6:0/526808124 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c8e4e8cf0 msgr2=0x563c8e4e2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:44.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.513+0000 7fe51b400700 1 --2- 172.21.15.6:0/526808124 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c8e4e8cf0 0x563c8e4e2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:44.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.513+0000 7fe51b400700 1 -- 172.21.15.6:0/526808124 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c8e280630 con 0x563c8e365640 2024-05-09T04:56:44.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.513+0000 7fe513400700 1 --2- 172.21.15.6:0/1821608229 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe5000b2900 0x7fe5000b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:44.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.513+0000 7fe513400700 1 -- 172.21.15.6:0/1821608229 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fe5080e2790 con 0x7fe51c105be0 2024-05-09T04:56:44.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.513+0000 7fe51aa00700 1 --2- 172.21.15.6:0/526808124 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c8e4e8cf0 0x563c8e4e2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:56:44.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.514+0000 7fe51b400700 1 --2- 172.21.15.6:0/526808124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c8e365640 0x563c8e4e87b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe514015000 tx=0x7fe514009d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=507a5186db14ff2c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:44.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.514+0000 7fe51aa00700 1 --2- 172.21.15.6:0/1821608229 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe5000b2900 0x7fe5000b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:44.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.514+0000 7fe507e00700 1 -- 172.21.15.6:0/526808124 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514021070 con 0x563c8e365640 2024-05-09T04:56:44.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.514+0000 7fe507e00700 1 -- 172.21.15.6:0/526808124 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe514006020 con 0x563c8e365640 2024-05-09T04:56:44.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.514+0000 7fe507e00700 1 -- 172.21.15.6:0/526808124 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5140050a0 con 0x563c8e365640 2024-05-09T04:56:44.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.515+0000 7fe53040fb80 1 -- 172.21.15.6:0/526808124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c8e365640 msgr2=0x563c8e4e87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:44.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.515+0000 7fe53040fb80 1 --2- 172.21.15.6:0/526808124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c8e365640 0x563c8e4e87b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe514015000 tx=0x7fe514009d90 comp rx=0 tx=0).stop 2024-05-09T04:56:44.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.516+0000 7fe51aa00700 1 --2- 172.21.15.6:0/1821608229 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe5000b2900 0x7fe5000b4e10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x563c8e4df180 tx=0x7fe50c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:44.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.516+0000 7fe53040fb80 1 -- 172.21.15.6:0/526808124 shutdown_connections 2024-05-09T04:56:44.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.516+0000 7fe53040fb80 1 --2- 172.21.15.6:0/526808124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c8e4e2fa0 0x563c8e4e33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:44.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.516+0000 7fe53040fb80 1 --2- 172.21.15.6:0/526808124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c8e365640 0x563c8e4e87b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:44.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.516+0000 7fe53040fb80 1 --2- 172.21.15.6:0/526808124 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c8e4e8cf0 0x563c8e4e2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:44.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.516+0000 7fe53040fb80 1 -- 172.21.15.6:0/526808124 >> 172.21.15.6:0/526808124 conn(0x563c8e3aa010 msgr2=0x563c8e4dd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:44.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.517+0000 7fe53040fb80 1 -- 172.21.15.6:0/526808124 shutdown_connections 2024-05-09T04:56:44.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.517+0000 7fe53040fb80 1 -- 172.21.15.6:0/526808124 wait complete. 2024-05-09T04:56:44.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.517+0000 7fe53040fb80 1 -- start start 2024-05-09T04:56:44.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.518+0000 7fe53040fb80 1 Processor -- start 2024-05-09T04:56:44.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.518+0000 7fe53040fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c8e365640 0x563c8e4ec540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:44.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.518+0000 7fe53040fb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c8e4e2fa0 0x563c8e4eca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:44.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.518+0000 7fe53040fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c8e4e8cf0 0x563c8e4ecfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:44.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.519+0000 7fe53040fb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x563c8e4f3b10 con 0x563c8e4e2fa0 2024-05-09T04:56:44.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.519+0000 7fe53040fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563c8e36a860 con 0x563c8e4e8cf0 2024-05-09T04:56:44.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.519+0000 7fe53040fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x563c8e4f3c90 con 0x563c8e365640 2024-05-09T04:56:44.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.519+0000 7fe53040fb80 10 client.0 _ll_register_callbacks cb 0x563c8e39c730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:56:44.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.519+0000 7fe51aa00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c8e365640 0x563c8e4ec540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:44.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.519+0000 7fe51aa00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c8e365640 0x563c8e4ec540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:33344/0 (socket says 172.21.15.6:33344) 2024-05-09T04:56:44.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.519+0000 7fe51a000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c8e4e2fa0 0x563c8e4eca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:44.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.519+0000 7fe51aa00700 1 -- 172.21.15.6:0/3728781273 learned_addr learned my addr 172.21.15.6:0/3728781273 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:56:44.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.519+0000 7fe51b400700 1 --2- 172.21.15.6:0/3728781273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c8e4e8cf0 0x563c8e4ecfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:44.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.519+0000 7fe51b400700 1 -- 172.21.15.6:0/3728781273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c8e365640 msgr2=0x563c8e4ec540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:44.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.519+0000 7fe51b400700 1 --2- 172.21.15.6:0/3728781273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c8e365640 0x563c8e4ec540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:44.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.519+0000 7fe51b400700 1 -- 172.21.15.6:0/3728781273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c8e4e2fa0 msgr2=0x563c8e4eca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:44.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.519+0000 7fe51b400700 1 --2- 172.21.15.6:0/3728781273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c8e4e2fa0 0x563c8e4eca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:44.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.519+0000 7fe51b400700 1 -- 172.21.15.6:0/3728781273 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c8e4fd070 con 0x563c8e4e8cf0 2024-05-09T04:56:44.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.520+0000 7fe51b400700 1 --2- 172.21.15.6:0/3728781273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c8e4e8cf0 0x563c8e4ecfc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe51400b3f0 tx=0x7fe514009540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5269e6d203bb89d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:44.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.520+0000 7fe504c00700 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-05-09T04:56:44.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.520+0000 7fe504c00700 1 -- 172.21.15.6:0/3728781273 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514021040 con 0x563c8e4e8cf0 2024-05-09T04:56:44.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.520+0000 7fe504c00700 1 -- 172.21.15.6:0/3728781273 <== mon.2 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fe5140300b0 con 0x563c8e4e8cf0 2024-05-09T04:56:44.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.520+0000 7fe53040fb80 1 -- 172.21.15.6:0/3728781273 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe514006020 con 0x563c8e4e8cf0 2024-05-09T04:56:44.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.520+0000 7fe504c00700 1 -- 172.21.15.6:0/3728781273 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe5140302b0 con 0x563c8e4e8cf0 2024-05-09T04:56:44.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.521+0000 7fe53040fb80 10 client.25442 fetch_fsmap learned FSMap version 30 2024-05-09T04:56:44.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.521+0000 7fe53040fb80 1 -- 172.21.15.6:0/3728781273 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563c8e4ef4c0 con 0x563c8e4e8cf0 2024-05-09T04:56:44.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.521+0000 7fe504c00700 1 -- 172.21.15.6:0/3728781273 <== mon.2 v2:172.21.15.169:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe514006020 con 0x563c8e4e8cf0 2024-05-09T04:56:44.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.521+0000 7fe504c00700 1 -- 172.21.15.6:0/3728781273 <== mon.2 v2:172.21.15.169:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe514011070 con 0x563c8e4e8cf0 2024-05-09T04:56:44.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.521+0000 7fe53040fb80 10 client.25442 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T04:56:44.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.521+0000 7fe53040fb80 10 client.25442 resolve_mds: validated gid 24554 aka daemon mds.b 2024-05-09T04:56:44.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.521+0000 7fe53040fb80 20 client.25442 populate_metadata read hostname 'smithi006' 2024-05-09T04:56:44.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.521+0000 7fe53040fb80 1 --2- 172.21.15.6:0/3728781273 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x563c8e4f1990 0x563c8e4f1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:44.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.521+0000 7fe53040fb80 4 client.25442 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:56:44.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.521+0000 7fe53040fb80 1 -- 172.21.15.6:0/3728781273 --> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c8e4aea10 con 0x563c8e4f1990 2024-05-09T04:56:44.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.522+0000 7fe51aa00700 1 --2- 172.21.15.6:0/3728781273 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x563c8e4f1990 0x563c8e4f1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:44.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.522+0000 7fe51aa00700 1 --2- 172.21.15.6:0/3728781273 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x563c8e4f1990 0x563c8e4f1da0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cb368d9ba844c85 server_cookie=c1dc1782a9a9ea90 in_seq=0 out_seq=0 2024-05-09T04:56:44.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.522+0000 7fe504c00700 10 client.25442 ms_handle_connect on v2:172.21.15.169:6838/614388771 2024-05-09T04:56:44.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.524+0000 7fe504c00700 1 -- 172.21.15.6:0/3728781273 <== mds.0 v2:172.21.15.169:6838/614388771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563c8e4aea10 con 0x563c8e4f1990 2024-05-09T04:56:44.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.524+0000 7fe504c00700 10 client.25442 handle_command_reply: tid=0 2024-05-09T04:56:44.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.524+0000 7fe53040fb80 1 client.25442 shutdown 2024-05-09T04:56:44.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.524+0000 7fe53040fb80 1 -- 172.21.15.6:0/3728781273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c8e4e8cf0 msgr2=0x563c8e4ecfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:44.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.524+0000 7fe53040fb80 1 --2- 172.21.15.6:0/3728781273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c8e4e8cf0 0x563c8e4ecfc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe51400b3f0 tx=0x7fe514009540 comp rx=0 tx=0).stop 2024-05-09T04:56:44.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.524+0000 7fe53040fb80 1 -- 172.21.15.6:0/3728781273 shutdown_connections 2024-05-09T04:56:44.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.524+0000 7fe53040fb80 1 --2- 172.21.15.6:0/3728781273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c8e4e8cf0 0x563c8e4ecfc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:44.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.524+0000 7fe53040fb80 1 --2- 172.21.15.6:0/3728781273 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x563c8e4f1990 0x563c8e4f1da0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:44.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.524+0000 7fe53040fb80 1 --2- 172.21.15.6:0/3728781273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c8e365640 0x563c8e4ec540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:44.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.524+0000 7fe53040fb80 1 --2- 172.21.15.6:0/3728781273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c8e4e2fa0 0x563c8e4eca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:44.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.524+0000 7fe53040fb80 1 -- 172.21.15.6:0/3728781273 >> 172.21.15.6:0/3728781273 conn(0x563c8e3aa010 msgr2=0x563c8e4dcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:44.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.525+0000 7fe504c00700 0 client.25442 ms_handle_reset on v2:172.21.15.169:6838/614388771 2024-05-09T04:56:44.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.525+0000 7fe53040fb80 1 -- 172.21.15.6:0/3728781273 shutdown_connections 2024-05-09T04:56:44.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.525+0000 7fe53040fb80 1 -- 172.21.15.6:0/3728781273 wait complete. 2024-05-09T04:56:44.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.525+0000 7fe53040fb80 20 client.25442 trim_cache size 0 max 16384 2024-05-09T04:56:44.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.546+0000 7fe53040fb80 1 Processor -- start 2024-05-09T04:56:44.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.547+0000 7fe53040fb80 1 -- start start 2024-05-09T04:56:44.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.547+0000 7fe53040fb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c8e365640 0x563c8e4e67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:44.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.547+0000 7fe53040fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c8e4e2fa0 0x563c8e4eb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:44.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.547+0000 7fe53040fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c8e4e8cf0 0x563c8e4ece00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:44.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.547+0000 7fe53040fb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1ade60 con 0x563c8e365640 2024-05-09T04:56:44.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.547+0000 7fe53040fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1af9a0 con 0x563c8e4e2fa0 2024-05-09T04:56:44.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.547+0000 7fe53040fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1ae530 con 0x563c8e4e8cf0 2024-05-09T04:56:44.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.547+0000 7fe51aa00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c8e365640 0x563c8e4e67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:44.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.547+0000 7fe51aa00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c8e365640 0x563c8e4e67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54332/0 (socket says 172.21.15.6:54332) 2024-05-09T04:56:44.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.547+0000 7fe51aa00700 1 -- 172.21.15.6:0/1868314986 learned_addr learned my addr 172.21.15.6:0/1868314986 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:56:44.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.548+0000 7fe51a000700 1 --2- 172.21.15.6:0/1868314986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c8e4e2fa0 0x563c8e4eb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:44.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.548+0000 7fe51b400700 1 --2- 172.21.15.6:0/1868314986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c8e4e8cf0 0x563c8e4ece00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:44.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.548+0000 7fe51aa00700 1 -- 172.21.15.6:0/1868314986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c8e4e8cf0 msgr2=0x563c8e4ece00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:44.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.548+0000 7fe51aa00700 1 --2- 172.21.15.6:0/1868314986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c8e4e8cf0 0x563c8e4ece00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:44.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.548+0000 7fe51aa00700 1 -- 172.21.15.6:0/1868314986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c8e4e2fa0 msgr2=0x563c8e4eb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:44.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.548+0000 7fe51aa00700 1 --2- 172.21.15.6:0/1868314986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c8e4e2fa0 0x563c8e4eb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:44.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.548+0000 7fe51aa00700 1 -- 172.21.15.6:0/1868314986 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c8e4ed630 con 0x563c8e365640 2024-05-09T04:56:44.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.548+0000 7fe51aa00700 1 --2- 172.21.15.6:0/1868314986 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c8e365640 0x563c8e4e67c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe50c015010 tx=0x7fe50c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85c9f6d705e8c93 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:44.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.549+0000 7fe507400700 1 -- 172.21.15.6:0/1868314986 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c004220 con 0x563c8e365640 2024-05-09T04:56:44.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.549+0000 7fe507400700 1 -- 172.21.15.6:0/1868314986 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe50c0043c0 con 0x563c8e365640 2024-05-09T04:56:44.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.549+0000 7fe507400700 1 -- 172.21.15.6:0/1868314986 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c003cb0 con 0x563c8e365640 2024-05-09T04:56:44.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.550+0000 7fe53040fb80 1 -- 172.21.15.6:0/1868314986 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c8e365640 msgr2=0x563c8e4e67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:44.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.550+0000 7fe53040fb80 1 --2- 172.21.15.6:0/1868314986 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c8e365640 0x563c8e4e67c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe50c015010 tx=0x7fe50c011000 comp rx=0 tx=0).stop 2024-05-09T04:56:44.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.550+0000 7fe53040fb80 1 -- 172.21.15.6:0/1868314986 shutdown_connections 2024-05-09T04:56:44.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.550+0000 7fe53040fb80 1 --2- 172.21.15.6:0/1868314986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c8e4e8cf0 0x563c8e4ece00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:44.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.550+0000 7fe53040fb80 1 --2- 172.21.15.6:0/1868314986 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c8e365640 0x563c8e4e67c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:44.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.550+0000 7fe53040fb80 1 --2- 172.21.15.6:0/1868314986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c8e4e2fa0 0x563c8e4eb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:44.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.550+0000 7fe53040fb80 1 -- 172.21.15.6:0/1868314986 >> 172.21.15.6:0/1868314986 conn(0x563c8e3aa010 msgr2=0x563c8e4e9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:44.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.551+0000 7fe53040fb80 1 -- 172.21.15.6:0/1868314986 shutdown_connections 2024-05-09T04:56:44.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.551+0000 7fe53040fb80 1 -- 172.21.15.6:0/1868314986 wait complete. 2024-05-09T04:56:44.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.552+0000 7fe53040fb80 1 -- start start 2024-05-09T04:56:44.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.552+0000 7fe53040fb80 1 Processor -- start 2024-05-09T04:56:44.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.553+0000 7fe53040fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c8e365640 0x563c8e500910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:44.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.553+0000 7fe53040fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c8e4e2fa0 0x563c8e500e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:44.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.553+0000 7fe53040fb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c8e4e8cf0 0x563c8e501390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:44.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.553+0000 7fe53040fb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1af2d0 con 0x563c8e4e8cf0 2024-05-09T04:56:44.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.553+0000 7fe53040fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1aec00 con 0x563c8e365640 2024-05-09T04:56:44.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.553+0000 7fe53040fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1abe00 con 0x563c8e4e2fa0 2024-05-09T04:56:44.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.553+0000 7fe53040fb80 10 client.0 _ll_register_callbacks cb 0x563c8e39c730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:56:44.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.553+0000 7fe51aa00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c8e365640 0x563c8e500910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:44.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.554+0000 7fe51b400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c8e4e8cf0 0x563c8e501390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:44.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.554+0000 7fe51aa00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c8e365640 0x563c8e500910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:47340/0 (socket says 172.21.15.6:47340) 2024-05-09T04:56:44.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.554+0000 7fe51aa00700 1 -- 172.21.15.6:0/1485034401 learned_addr learned my addr 172.21.15.6:0/1485034401 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:56:44.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.554+0000 7fe51a000700 1 --2- 172.21.15.6:0/1485034401 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c8e4e2fa0 0x563c8e500e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:44.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.555+0000 7fe51b400700 1 -- 172.21.15.6:0/1485034401 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c8e4e2fa0 msgr2=0x563c8e500e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:44.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.555+0000 7fe51b400700 1 --2- 172.21.15.6:0/1485034401 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c8e4e2fa0 0x563c8e500e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:44.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.555+0000 7fe51b400700 1 -- 172.21.15.6:0/1485034401 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c8e365640 msgr2=0x563c8e500910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:44.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.555+0000 7fe51b400700 1 --2- 172.21.15.6:0/1485034401 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c8e365640 0x563c8e500910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:44.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.555+0000 7fe51b400700 1 -- 172.21.15.6:0/1485034401 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c8e5665b0 con 0x563c8e4e8cf0 2024-05-09T04:56:44.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.555+0000 7fe51b400700 1 --2- 172.21.15.6:0/1485034401 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c8e4e8cf0 0x563c8e501390 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe5140058c0 tx=0x7fe514005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c719537583baca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:44.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.555+0000 7fe504c00700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-05-09T04:56:44.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.555+0000 7fe504c00700 1 -- 172.21.15.6:0/1485034401 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514021070 con 0x563c8e4e8cf0 2024-05-09T04:56:44.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.556+0000 7fe504c00700 1 -- 172.21.15.6:0/1485034401 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fe514005e70 con 0x563c8e4e8cf0 2024-05-09T04:56:44.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.556+0000 7fe53040fb80 1 -- 172.21.15.6:0/1485034401 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe50c0043c0 con 0x563c8e4e8cf0 2024-05-09T04:56:44.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.556+0000 7fe504c00700 1 -- 172.21.15.6:0/1485034401 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe514006070 con 0x563c8e4e8cf0 2024-05-09T04:56:44.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.556+0000 7fe53040fb80 10 client.15291 fetch_fsmap learned FSMap version 30 2024-05-09T04:56:44.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.556+0000 7fe53040fb80 1 -- 172.21.15.6:0/1485034401 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563c8e5677a0 con 0x563c8e4e8cf0 2024-05-09T04:56:44.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.557+0000 7fe504c00700 1 -- 172.21.15.6:0/1485034401 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe50c0043c0 con 0x563c8e4e8cf0 2024-05-09T04:56:44.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.557+0000 7fe504c00700 1 -- 172.21.15.6:0/1485034401 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe514011040 con 0x563c8e4e8cf0 2024-05-09T04:56:44.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.557+0000 7fe53040fb80 10 client.15291 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T04:56:44.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.557+0000 7fe53040fb80 10 client.15291 resolve_mds: validated gid 24554 aka daemon mds.b 2024-05-09T04:56:44.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.557+0000 7fe53040fb80 20 client.15291 populate_metadata read hostname 'smithi006' 2024-05-09T04:56:44.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.557+0000 7fe53040fb80 1 --2- 172.21.15.6:0/1485034401 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x563c8e567cc0 0x563c8e56a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:44.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.557+0000 7fe53040fb80 4 client.15291 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.95, "await_for": 45.0}] 2024-05-09T04:56:44.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.557+0000 7fe53040fb80 1 -- 172.21.15.6:0/1485034401 --> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.95, "await_for": 45.0}) -- 0x563c8e4fce10 con 0x563c8e567cc0 2024-05-09T04:56:44.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.557+0000 7fe51aa00700 1 --2- 172.21.15.6:0/1485034401 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x563c8e567cc0 0x563c8e56a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:44.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.558+0000 7fe51aa00700 1 --2- 172.21.15.6:0/1485034401 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x563c8e567cc0 0x563c8e56a0e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18cf1bd5271ca751 server_cookie=a7b6bbeefd00594f in_seq=0 out_seq=0 2024-05-09T04:56:44.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.558+0000 7fe504c00700 10 client.15291 ms_handle_connect on v2:172.21.15.169:6838/614388771 2024-05-09T04:56:44.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.568+0000 7fe504c00700 1 -- 172.21.15.6:0/1485034401 <== mds.0 v2:172.21.15.169:6838/614388771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x563c8e4fce10 con 0x563c8e567cc0 2024-05-09T04:56:44.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.568+0000 7fe504c00700 10 client.15291 handle_command_reply: tid=0 2024-05-09T04:56:44.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.568+0000 7fe53040fb80 1 client.15291 shutdown 2024-05-09T04:56:44.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.568+0000 7fe53040fb80 1 -- 172.21.15.6:0/1485034401 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c8e4e8cf0 msgr2=0x563c8e501390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:44.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.568+0000 7fe53040fb80 1 --2- 172.21.15.6:0/1485034401 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c8e4e8cf0 0x563c8e501390 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe5140058c0 tx=0x7fe514005b70 comp rx=0 tx=0).stop 2024-05-09T04:56:44.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.569+0000 7fe53040fb80 1 -- 172.21.15.6:0/1485034401 shutdown_connections 2024-05-09T04:56:44.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.569+0000 7fe53040fb80 1 --2- 172.21.15.6:0/1485034401 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563c8e4e8cf0 0x563c8e501390 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:44.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.569+0000 7fe53040fb80 1 --2- 172.21.15.6:0/1485034401 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563c8e365640 0x563c8e500910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:44.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.569+0000 7fe53040fb80 1 --2- 172.21.15.6:0/1485034401 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x563c8e567cc0 0x563c8e56a0e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:44.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.569+0000 7fe504c00700 0 client.15291 ms_handle_reset on v2:172.21.15.169:6838/614388771 2024-05-09T04:56:44.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.569+0000 7fe53040fb80 1 --2- 172.21.15.6:0/1485034401 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563c8e4e2fa0 0x563c8e500e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:44.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.569+0000 7fe53040fb80 1 -- 172.21.15.6:0/1485034401 >> 172.21.15.6:0/1485034401 conn(0x563c8e3aa010 msgr2=0x563c8e4eb490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:44.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.569+0000 7fe53040fb80 1 -- 172.21.15.6:0/1485034401 shutdown_connections 2024-05-09T04:56:44.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.570+0000 7fe53040fb80 1 -- 172.21.15.6:0/1485034401 wait complete. 2024-05-09T04:56:44.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.570+0000 7fe53040fb80 20 client.15291 trim_cache size 0 max 16384 2024-05-09T04:56:44.570 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-09T04:56:44.571 INFO:teuthology.orchestra.run.smithi006.stdout: "epoch": 30, 2024-05-09T04:56:44.571 INFO:teuthology.orchestra.run.smithi006.stdout: "leader": 24554, 2024-05-09T04:56:44.571 INFO:teuthology.orchestra.run.smithi006.stdout: "set_version": 6, 2024-05-09T04:56:44.571 INFO:teuthology.orchestra.run.smithi006.stdout: "sets": { 2024-05-09T04:56:44.571 INFO:teuthology.orchestra.run.smithi006.stdout: "98112720": { 2024-05-09T04:56:44.571 INFO:teuthology.orchestra.run.smithi006.stdout: "version": 6, 2024-05-09T04:56:44.571 INFO:teuthology.orchestra.run.smithi006.stdout: "age_ref": 0.0, 2024-05-09T04:56:44.571 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-05-09T04:56:44.571 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "QUIESCED", 2024-05-09T04:56:44.571 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-05-09T04:56:44.571 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:56:44.571 INFO:teuthology.orchestra.run.smithi006.stdout: "timeout": 90.0, 2024-05-09T04:56:44.571 INFO:teuthology.orchestra.run.smithi006.stdout: "expiration": 144.9, 2024-05-09T04:56:44.572 INFO:teuthology.orchestra.run.smithi006.stdout: "members": { 2024-05-09T04:56:44.572 INFO:teuthology.orchestra.run.smithi006.stdout: "file:/": { 2024-05-09T04:56:44.572 INFO:teuthology.orchestra.run.smithi006.stdout: "excluded": false, 2024-05-09T04:56:44.572 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-05-09T04:56:44.572 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "QUIESCED", 2024-05-09T04:56:44.572 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-05-09T04:56:44.572 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:56:44.572 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:56:44.572 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:56:44.572 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:56:44.572 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:56:44.572 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-09T04:56:44.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.571+0000 7fe51be00700 1 -- 172.21.15.6:0/1821608229 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe5000b2900 msgr2=0x7fe5000b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:44.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.571+0000 7fe51be00700 1 --2- 172.21.15.6:0/1821608229 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe5000b2900 0x7fe5000b4e10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x563c8e4df180 tx=0x7fe50c009260 comp rx=0 tx=0).stop 2024-05-09T04:56:44.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.571+0000 7fe51be00700 1 -- 172.21.15.6:0/1821608229 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe51c105be0 msgr2=0x7fe51c1a0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:44.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.571+0000 7fe51be00700 1 --2- 172.21.15.6:0/1821608229 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe51c105be0 0x7fe51c1a0d00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe50800e9a0 tx=0x7fe50800ecb0 comp rx=0 tx=0).stop 2024-05-09T04:56:44.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.571+0000 7fe51be00700 1 -- 172.21.15.6:0/1821608229 shutdown_connections 2024-05-09T04:56:44.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.571+0000 7fe51be00700 1 --2- 172.21.15.6:0/1821608229 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe5000b2900 0x7fe5000b4e10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:44.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.571+0000 7fe51be00700 1 --2- 172.21.15.6:0/1821608229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe51c10e700 0x7fe51c19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:44.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.571+0000 7fe51be00700 1 --2- 172.21.15.6:0/1821608229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe51c105290 0x7fe51c1a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:44.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.571+0000 7fe51be00700 1 --2- 172.21.15.6:0/1821608229 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe51c105be0 0x7fe51c1a0d00 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:44.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.572+0000 7fe51be00700 1 -- 172.21.15.6:0/1821608229 >> 172.21.15.6:0/1821608229 conn(0x7fe51c078ac0 msgr2=0x7fe51c0fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:44.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.572+0000 7fe51be00700 1 -- 172.21.15.6:0/1821608229 shutdown_connections 2024-05-09T04:56:44.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:56:44.572+0000 7fe51be00700 1 -- 172.21.15.6:0/1821608229 wait complete. 2024-05-09T04:56:45.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:44 smithi186 ceph-mon[34668]: pgmap v553: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:56:45.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:44 smithi169 ceph-mon[34075]: pgmap v553: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:56:45.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:44 smithi006 ceph-mon[29835]: pgmap v553: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:56:45.555 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '98112720', quiesce took 8.3 seconds. Will release after: 16.65 2024-05-09T04:56:47.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:46 smithi186 ceph-mon[34668]: pgmap v554: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:56:47.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:46 smithi169 ceph-mon[34075]: pgmap v554: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:56:47.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:46 smithi006 ceph-mon[29835]: pgmap v554: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:56:48.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:48 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:56:48.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:48 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:56:48.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:48 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:56:49.302 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:49 smithi006 ceph-mon[29835]: pgmap v555: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:56:49.302 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:49 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:56:49.302 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:49 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:56:49.303 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:49 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:56:49.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:49 smithi186 ceph-mon[34668]: pgmap v555: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:56:49.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:49 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:56:49.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:49 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:56:49.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:49 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:56:49.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:49 smithi169 ceph-mon[34075]: pgmap v555: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:56:49.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:49 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:56:49.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:49 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:56:49.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:49 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:56:51.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:51 smithi186 ceph-mon[34668]: pgmap v556: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-09T04:56:51.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:51 smithi169 ceph-mon[34075]: pgmap v556: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-09T04:56:51.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:51 smithi006 ceph-mon[29835]: pgmap v556: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-09T04:56:53.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:53 smithi186 ceph-mon[34668]: pgmap v557: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-09T04:56:53.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:53 smithi169 ceph-mon[34075]: pgmap v557: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-09T04:56:53.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:53 smithi006 ceph-mon[29835]: pgmap v557: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-09T04:56:55.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:55 smithi186 ceph-mon[34668]: pgmap v558: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-09T04:56:55.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:55 smithi169 ceph-mon[34075]: pgmap v558: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-09T04:56:55.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:55 smithi006 ceph-mon[29835]: pgmap v558: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-09T04:56:56.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:56 smithi186 ceph-mon[34668]: pgmap v559: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-09T04:56:56.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:56 smithi169 ceph-mon[34075]: pgmap v559: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-09T04:56:56.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:56 smithi006 ceph-mon[29835]: pgmap v559: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-09T04:56:59.100 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:56:58 smithi186 ceph-mon[34668]: pgmap v560: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-09T04:56:59.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:56:58 smithi169 ceph-mon[34075]: pgmap v560: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-09T04:56:59.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:56:58 smithi006 ceph-mon[29835]: pgmap v560: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-09T04:57:00.209 INFO:tasks.workunit.client.0.smithi006.stderr:Updating files: 58% (7209/12271) Updating files: 59% (7240/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 96% (11789/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-09T04:57:00.920 INFO:tasks.workunit.client.0.smithi006.stderr:Note: switching to 'c9da4049f638c0f84bdd9b4e33c42e86046ac554'. 2024-05-09T04:57:00.920 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-09T04:57:00.920 INFO:tasks.workunit.client.0.smithi006.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-09T04:57:00.920 INFO:tasks.workunit.client.0.smithi006.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-09T04:57:00.920 INFO:tasks.workunit.client.0.smithi006.stderr:state without impacting any branches by switching back to a branch. 2024-05-09T04:57:00.920 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-09T04:57:00.920 INFO:tasks.workunit.client.0.smithi006.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-09T04:57:00.921 INFO:tasks.workunit.client.0.smithi006.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-09T04:57:00.921 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-09T04:57:00.921 INFO:tasks.workunit.client.0.smithi006.stderr: git switch -c 2024-05-09T04:57:00.921 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-09T04:57:00.921 INFO:tasks.workunit.client.0.smithi006.stderr:Or undo this operation with: 2024-05-09T04:57:00.921 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-09T04:57:00.921 INFO:tasks.workunit.client.0.smithi006.stderr: git switch - 2024-05-09T04:57:00.921 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-09T04:57:00.921 INFO:tasks.workunit.client.0.smithi006.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-09T04:57:00.921 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-09T04:57:00.921 INFO:tasks.workunit.client.0.smithi006.stderr:HEAD is now at c9da4049f63 Merge PR #57081 into wip-pdonnell-testing-20240508.183908-debug 2024-05-09T04:57:00.936 DEBUG:teuthology.orchestra.run.smithi006:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-09T04:57:00.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:00 smithi006 ceph-mon[29835]: pgmap v561: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-09T04:57:00.994 INFO:tasks.workunit.client.0.smithi006.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-09T04:57:01.002 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-09T04:57:01.002 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-09T04:57:01.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:00 smithi186 ceph-mon[34668]: pgmap v561: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-09T04:57:01.118 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:00 smithi169 ceph-mon[34075]: pgmap v561: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-09T04:57:01.883 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-09T04:57:01.997 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-09T04:57:02.091 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-09T04:57:02.098 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-09T04:57:02.098 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-09T04:57:02.196 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-09T04:57:02.206 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:57:02.207 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T04:57:02.207 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-09T04:57:02.308 INFO:tasks.workunit:Running workunits matching suites/fsync-tester.sh on client.0... 2024-05-09T04:57:02.310 INFO:tasks.workunit:Running workunit suites/fsync-tester.sh... 2024-05-09T04:57:02.310 DEBUG:teuthology.orchestra.run.smithi006:workunit test suites/fsync-tester.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c9da4049f638c0f84bdd9b4e33c42e86046ac554 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-05-09T04:57:02.707 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:57:03.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:02 smithi186 ceph-mon[34668]: pgmap v562: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:57:03.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:02 smithi169 ceph-mon[34075]: pgmap v562: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:57:03.168 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:02 smithi006 ceph-mon[29835]: pgmap v562: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:57:04.874 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:04 smithi006 ceph-mon[29835]: pgmap v563: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:57:05.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:04 smithi186 ceph-mon[34668]: pgmap v563: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:57:05.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:04 smithi169 ceph-mon[34075]: pgmap v563: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:57:05.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.252+0000 7f14aae00700 1 -- 172.21.15.6:0/3828834788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14a4105ca0 msgr2=0x7f14a410e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:05.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.252+0000 7f14aae00700 1 --2- 172.21.15.6:0/3828834788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14a4105ca0 0x7f14a410e1f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1490009990 tx=0x7f1490009ca0 comp rx=0 tx=0).stop 2024-05-09T04:57:05.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.252+0000 7f14aae00700 1 -- 172.21.15.6:0/3828834788 shutdown_connections 2024-05-09T04:57:05.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.252+0000 7f14aae00700 1 --2- 172.21.15.6:0/3828834788 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14a410e730 0x7f14a4110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:05.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.252+0000 7f14aae00700 1 --2- 172.21.15.6:0/3828834788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14a4105350 0x7f14a4105760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:05.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.252+0000 7f14aae00700 1 --2- 172.21.15.6:0/3828834788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14a4105ca0 0x7f14a410e1f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:05.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.252+0000 7f14aae00700 1 -- 172.21.15.6:0/3828834788 >> 172.21.15.6:0/3828834788 conn(0x7f14a4078ac0 msgr2=0x7f14a4078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:05.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.252+0000 7f14aae00700 1 -- 172.21.15.6:0/3828834788 shutdown_connections 2024-05-09T04:57:05.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.252+0000 7f14aae00700 1 -- 172.21.15.6:0/3828834788 wait complete. 2024-05-09T04:57:05.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.253+0000 7f14aae00700 1 Processor -- start 2024-05-09T04:57:05.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.254+0000 7f14aae00700 1 -- start start 2024-05-09T04:57:05.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.254+0000 7f14aae00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14a4105350 0x7f14a4103040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:05.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.254+0000 7f14aae00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14a4105ca0 0x7f14a4103580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:05.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.254+0000 7f14aae00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14a410e730 0x7f14a4103ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:05.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.254+0000 7f14aae00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f14a41142b0 con 0x7f14a4105ca0 2024-05-09T04:57:05.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.254+0000 7f14aae00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f14a4114130 con 0x7f14a410e730 2024-05-09T04:57:05.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.254+0000 7f14aae00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f14a4114430 con 0x7f14a4105350 2024-05-09T04:57:05.256 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.255+0000 7f14a9000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14a4105ca0 0x7f14a4103580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:05.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.255+0000 7f14a9a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14a4105350 0x7f14a4103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:05.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.255+0000 7f14aa400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14a410e730 0x7f14a4103ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:05.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.255+0000 7f14a9a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14a4105350 0x7f14a4103040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:42128/0 (socket says 172.21.15.6:42128) 2024-05-09T04:57:05.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.255+0000 7f14a9a00700 1 -- 172.21.15.6:0/706639137 learned_addr learned my addr 172.21.15.6:0/706639137 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:57:05.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.255+0000 7f14aa400700 1 -- 172.21.15.6:0/706639137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14a4105350 msgr2=0x7f14a4103040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:05.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.255+0000 7f14aa400700 1 --2- 172.21.15.6:0/706639137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14a4105350 0x7f14a4103040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:05.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.255+0000 7f14aa400700 1 -- 172.21.15.6:0/706639137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14a4105ca0 msgr2=0x7f14a4103580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:05.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.255+0000 7f14aa400700 1 --2- 172.21.15.6:0/706639137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14a4105ca0 0x7f14a4103580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:05.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.255+0000 7f14aa400700 1 -- 172.21.15.6:0/706639137 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a41a8fe0 con 0x7f14a410e730 2024-05-09T04:57:05.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.255+0000 7f14a9a00700 1 --2- 172.21.15.6:0/706639137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14a4105350 0x7f14a4103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:57:05.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.255+0000 7f14a9000700 1 --2- 172.21.15.6:0/706639137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14a4105ca0 0x7f14a4103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:57:05.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.256+0000 7f14aa400700 1 --2- 172.21.15.6:0/706639137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14a410e730 0x7f14a4103ac0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f149c00efc0 tx=0x7f149c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:05.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.256+0000 7f14a2000700 1 -- 172.21.15.6:0/706639137 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c0092e0 con 0x7f14a410e730 2024-05-09T04:57:05.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.256+0000 7f14aae00700 1 -- 172.21.15.6:0/706639137 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14a41a9270 con 0x7f14a410e730 2024-05-09T04:57:05.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.256+0000 7f14a2000700 1 -- 172.21.15.6:0/706639137 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f149c004bb0 con 0x7f14a410e730 2024-05-09T04:57:05.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.256+0000 7f14a2000700 1 -- 172.21.15.6:0/706639137 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c004020 con 0x7f14a410e730 2024-05-09T04:57:05.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.256+0000 7f14aae00700 1 -- 172.21.15.6:0/706639137 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14a41a97b0 con 0x7f14a410e730 2024-05-09T04:57:05.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.257+0000 7f14aae00700 1 -- 172.21.15.6:0/706639137 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14a40512c0 con 0x7f14a410e730 2024-05-09T04:57:05.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.260+0000 7f14a2000700 1 -- 172.21.15.6:0/706639137 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f149c01b030 con 0x7f14a410e730 2024-05-09T04:57:05.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.261+0000 7f14a2000700 1 --2- 172.21.15.6:0/706639137 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f148c0b2880 0x7f148c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:05.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.262+0000 7f14a2000700 1 -- 172.21.15.6:0/706639137 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f149c0e2a10 con 0x7f14a410e730 2024-05-09T04:57:05.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.262+0000 7f14a9a00700 1 --2- 172.21.15.6:0/706639137 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f148c0b2880 0x7f148c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:05.263 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.262+0000 7f14a9a00700 1 --2- 172.21.15.6:0/706639137 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f148c0b2880 0x7f148c0b4d90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1498011550 tx=0x7f1498009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:05.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.264+0000 7f14a2000700 1 -- 172.21.15.6:0/706639137 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f149c022020 con 0x7f14a410e730 2024-05-09T04:57:05.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.551+0000 7f14aae00700 1 -- 172.21.15.6:0/706639137 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14a402e7b0 con 0x7f14a410e730 2024-05-09T04:57:05.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.553+0000 7f14a2000700 1 -- 172.21.15.6:0/706639137 <== mon.2 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11927 (secure 0 0 0) 0x7f149c010040 con 0x7f14a410e730 2024-05-09T04:57:05.554 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:57:05.554 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/162094599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":162094599},{"type":"v1","addr":"172.21.15.6:6835","nonce":162094599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/589294363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":589294363},{"type":"v1","addr":"172.21.15.6:6841","nonce":589294363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:49:36.920376+0000","modified":"2024-05-09T04:54:56.247431+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24554,"mds_1":24530,"mds_2":24512,"mds_3":24511,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24442":{"gid":24442,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6833/2977145206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2977145206},{"type":"v1","addr":"172.21.15.186:6833","nonce":2977145206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6833/133345948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":133345948},{"type":"v1","addr":"172.21.15.169:6833","nonce":133345948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.6:6837/1281521088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":1281521088},{"type":"v1","addr":"172.21.15.6:6837","nonce":1281521088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2268322007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2268322007},{"type":"v1","addr":"172.21.15.186:6835","nonce":2268322007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/3811005052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3811005052},{"type":"v1","addr":"172.21.15.169:6835","nonce":3811005052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.186:6837/1647607118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1647607118},{"type":"v1","addr":"172.21.15.186:6837","nonce":1647607118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":32,"addr":"172.21.15.186:6839/3954883105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":3954883105},{"type":"v1","addr":"172.21.15.186:6839","nonce":3954883105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.169:6837/1845715294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1845715294},{"type":"v1","addr":"172.21.15.169:6837","nonce":1845715294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.6:6839/2780306492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":2780306492},{"type":"v1","addr":"172.21.15.6:6839","nonce":2780306492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.169:6839/614388771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":614388771},{"type":"v1","addr":"172.21.15.169:6839","nonce":614388771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24512,24554,24530,24493,24511]},"id":1}]} 2024-05-09T04:57:05.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.556+0000 7f14aae00700 1 -- 172.21.15.6:0/706639137 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f148c0b2880 msgr2=0x7f148c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:05.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.556+0000 7f14aae00700 1 --2- 172.21.15.6:0/706639137 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f148c0b2880 0x7f148c0b4d90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1498011550 tx=0x7f1498009260 comp rx=0 tx=0).stop 2024-05-09T04:57:05.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.556+0000 7f14aae00700 1 -- 172.21.15.6:0/706639137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14a410e730 msgr2=0x7f14a4103ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:05.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.556+0000 7f14aae00700 1 --2- 172.21.15.6:0/706639137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14a410e730 0x7f14a4103ac0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f149c00efc0 tx=0x7f149c00c3a0 comp rx=0 tx=0).stop 2024-05-09T04:57:05.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.556+0000 7f14aae00700 1 -- 172.21.15.6:0/706639137 shutdown_connections 2024-05-09T04:57:05.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.556+0000 7f14aae00700 1 --2- 172.21.15.6:0/706639137 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f148c0b2880 0x7f148c0b4d90 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:05.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.556+0000 7f14aae00700 1 --2- 172.21.15.6:0/706639137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14a410e730 0x7f14a4103ac0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:05.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.556+0000 7f14aae00700 1 --2- 172.21.15.6:0/706639137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14a4105350 0x7f14a4103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:05.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.556+0000 7f14aae00700 1 --2- 172.21.15.6:0/706639137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14a4105ca0 0x7f14a4103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:05.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.556+0000 7f14aae00700 1 -- 172.21.15.6:0/706639137 >> 172.21.15.6:0/706639137 conn(0x7f14a4078ac0 msgr2=0x7f14a40ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:05.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.557+0000 7f14aae00700 1 -- 172.21.15.6:0/706639137 shutdown_connections 2024-05-09T04:57:05.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:05.557+0000 7f14aae00700 1 -- 172.21.15.6:0/706639137 wait complete. 2024-05-09T04:57:05.559 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 30 2024-05-09T04:57:06.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:05 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/706639137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:57:06.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:05 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/706639137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:57:06.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:05 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/706639137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:57:06.513 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24554 quiesce db --set-id 98112720 --release --await' 2024-05-09T04:57:06.513 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24554 quiesce db --set-id 98112720 --release --await 2024-05-09T04:57:06.989 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:57:07.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:06 smithi169 ceph-mon[34075]: pgmap v564: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-05-09T04:57:07.139 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:06 smithi006 ceph-mon[29835]: pgmap v564: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-05-09T04:57:07.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:06 smithi186 ceph-mon[34668]: pgmap v564: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-05-09T04:57:09.061 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:08 smithi006 ceph-mon[29835]: pgmap v565: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-05-09T04:57:09.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:08 smithi169 ceph-mon[34075]: pgmap v565: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-05-09T04:57:09.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:08 smithi186 ceph-mon[34668]: pgmap v565: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-05-09T04:57:09.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.503+0000 7f392f200700 1 -- 172.21.15.6:0/798332429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3930104410 msgr2=0x7f3930104820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:09.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.503+0000 7f392f200700 1 --2- 172.21.15.6:0/798332429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3930104410 0x7f3930104820 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3920009990 tx=0x7f3920009ca0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.504+0000 7f392f200700 1 -- 172.21.15.6:0/798332429 shutdown_connections 2024-05-09T04:57:09.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.504+0000 7f392f200700 1 --2- 172.21.15.6:0/798332429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3930109690 0x7f3930110ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.504+0000 7f392f200700 1 --2- 172.21.15.6:0/798332429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3930104410 0x7f3930104820 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.504+0000 7f392f200700 1 --2- 172.21.15.6:0/798332429 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3930108d40 0x7f3930109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.504+0000 7f392f200700 1 -- 172.21.15.6:0/798332429 >> 172.21.15.6:0/798332429 conn(0x7f3930079d50 msgr2=0x7f393007a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:09.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.504+0000 7f392f200700 1 -- 172.21.15.6:0/798332429 shutdown_connections 2024-05-09T04:57:09.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.504+0000 7f392f200700 1 -- 172.21.15.6:0/798332429 wait complete. 2024-05-09T04:57:09.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.505+0000 7f392f200700 1 Processor -- start 2024-05-09T04:57:09.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.505+0000 7f392f200700 1 -- start start 2024-05-09T04:57:09.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.506+0000 7f392f200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3930104410 0x7f393019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:09.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.506+0000 7f392f200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3930108d40 0x7f393019d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:09.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.506+0000 7f392f200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3930109690 0x7f39301a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:09.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.506+0000 7f392f200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3930114310 con 0x7f3930109690 2024-05-09T04:57:09.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.506+0000 7f392f200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3930114190 con 0x7f3930108d40 2024-05-09T04:57:09.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.506+0000 7f392f200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3930114490 con 0x7f3930104410 2024-05-09T04:57:09.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.506+0000 7f392e800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3930109690 0x7f39301a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:09.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.506+0000 7f392e800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3930109690 0x7f39301a1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:40380/0 (socket says 172.21.15.6:40380) 2024-05-09T04:57:09.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.506+0000 7f392e800700 1 -- 172.21.15.6:0/3173300446 learned_addr learned my addr 172.21.15.6:0/3173300446 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:57:09.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.506+0000 7f392de00700 1 --2- 172.21.15.6:0/3173300446 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3930104410 0x7f393019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:09.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.506+0000 7f3927e00700 1 --2- 172.21.15.6:0/3173300446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3930108d40 0x7f393019d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:09.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.507+0000 7f392e800700 1 -- 172.21.15.6:0/3173300446 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3930104410 msgr2=0x7f393019ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:09.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.507+0000 7f392e800700 1 --2- 172.21.15.6:0/3173300446 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3930104410 0x7f393019ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.507+0000 7f392e800700 1 -- 172.21.15.6:0/3173300446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3930108d40 msgr2=0x7f393019d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:09.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.507+0000 7f392e800700 1 --2- 172.21.15.6:0/3173300446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3930108d40 0x7f393019d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.507+0000 7f392e800700 1 -- 172.21.15.6:0/3173300446 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39301a1d30 con 0x7f3930109690 2024-05-09T04:57:09.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.507+0000 7f392e800700 1 --2- 172.21.15.6:0/3173300446 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3930109690 0x7f39301a1690 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f392800ef30 tx=0x7f392800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:09.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.507+0000 7f3926a00700 1 -- 172.21.15.6:0/3173300446 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39280092e0 con 0x7f3930109690 2024-05-09T04:57:09.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.508+0000 7f392f200700 1 -- 172.21.15.6:0/3173300446 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39301a1fc0 con 0x7f3930109690 2024-05-09T04:57:09.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.508+0000 7f392f200700 1 -- 172.21.15.6:0/3173300446 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39301a9940 con 0x7f3930109690 2024-05-09T04:57:09.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.508+0000 7f3926a00700 1 -- 172.21.15.6:0/3173300446 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3928004bb0 con 0x7f3930109690 2024-05-09T04:57:09.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.508+0000 7f3926a00700 1 -- 172.21.15.6:0/3173300446 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3928004020 con 0x7f3930109690 2024-05-09T04:57:09.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.512+0000 7f3926a00700 1 -- 172.21.15.6:0/3173300446 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f392801b030 con 0x7f3930109690 2024-05-09T04:57:09.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.513+0000 7f3943e3cb80 1 Processor -- start 2024-05-09T04:57:09.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.513+0000 7f3943e3cb80 1 -- start start 2024-05-09T04:57:09.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.513+0000 7f3926a00700 1 --2- 172.21.15.6:0/3173300446 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f39140b2900 0x7f39140b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:09.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.513+0000 7f3943e3cb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625a0c9d5a0 0x5625a0e20710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:09.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.513+0000 7f3943e3cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625a0e20c50 0x5625a0e1a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:09.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.513+0000 7f3943e3cb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625a0e1af00 0x5625a0e1b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:09.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.513+0000 7f3943e3cb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5625a0d3d3b0 con 0x5625a0c9d5a0 2024-05-09T04:57:09.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.513+0000 7f3943e3cb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5625a0cdb1e0 con 0x5625a0e1af00 2024-05-09T04:57:09.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.513+0000 7f3943e3cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5625a0d39370 con 0x5625a0e20c50 2024-05-09T04:57:09.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.514+0000 7f3926a00700 1 -- 172.21.15.6:0/3173300446 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f39280e1ee0 con 0x7f3930109690 2024-05-09T04:57:09.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.514+0000 7f392de00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625a0c9d5a0 0x5625a0e20710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:09.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.514+0000 7f3927e00700 1 --2- 172.21.15.6:0/3173300446 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f39140b2900 0x7f39140b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:09.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.514+0000 7f392de00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625a0c9d5a0 0x5625a0e20710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:40384/0 (socket says 172.21.15.6:40384) 2024-05-09T04:57:09.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.514+0000 7f392de00700 1 -- 172.21.15.6:0/3044801865 learned_addr learned my addr 172.21.15.6:0/3044801865 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:57:09.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.514+0000 7f392e800700 1 --2- 172.21.15.6:0/3044801865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625a0e1af00 0x5625a0e1b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:09.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.514+0000 7f3927e00700 1 --2- 172.21.15.6:0/3044801865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625a0e20c50 0x5625a0e1a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:09.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.514+0000 7f392de00700 1 -- 172.21.15.6:0/3044801865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625a0e20c50 msgr2=0x5625a0e1a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:09.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.514+0000 7f392de00700 1 --2- 172.21.15.6:0/3044801865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625a0e20c50 0x5625a0e1a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.514+0000 7f392de00700 1 -- 172.21.15.6:0/3044801865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625a0e1af00 msgr2=0x5625a0e1b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:09.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.514+0000 7f392de00700 1 --2- 172.21.15.6:0/3044801865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625a0e1af00 0x5625a0e1b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.514+0000 7f392de00700 1 -- 172.21.15.6:0/3044801865 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625a0bb8590 con 0x5625a0c9d5a0 2024-05-09T04:57:09.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.515+0000 7f392de00700 1 --2- 172.21.15.6:0/3044801865 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625a0c9d5a0 0x5625a0e20710 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3920005f40 tx=0x7f3920009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d66e424a69a7d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:09.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.515+0000 7f3927e00700 1 --2- 172.21.15.6:0/3173300446 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f39140b2900 0x7f39140b4e10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x5625a0e170e0 tx=0x7f39180098d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:09.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.515+0000 7f391f400700 1 -- 172.21.15.6:0/3044801865 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3920027070 con 0x5625a0c9d5a0 2024-05-09T04:57:09.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.515+0000 7f391f400700 1 -- 172.21.15.6:0/3044801865 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39200054e0 con 0x5625a0c9d5a0 2024-05-09T04:57:09.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.515+0000 7f391f400700 1 -- 172.21.15.6:0/3044801865 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3920003b80 con 0x5625a0c9d5a0 2024-05-09T04:57:09.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.516+0000 7f3943e3cb80 1 -- 172.21.15.6:0/3044801865 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625a0c9d5a0 msgr2=0x5625a0e20710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:09.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.516+0000 7f3943e3cb80 1 --2- 172.21.15.6:0/3044801865 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625a0c9d5a0 0x5625a0e20710 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3920005f40 tx=0x7f3920009510 comp rx=0 tx=0).stop 2024-05-09T04:57:09.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.516+0000 7f3943e3cb80 1 -- 172.21.15.6:0/3044801865 shutdown_connections 2024-05-09T04:57:09.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.516+0000 7f3943e3cb80 1 --2- 172.21.15.6:0/3044801865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625a0e1af00 0x5625a0e1b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.516+0000 7f3943e3cb80 1 --2- 172.21.15.6:0/3044801865 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625a0c9d5a0 0x5625a0e20710 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.516+0000 7f3943e3cb80 1 --2- 172.21.15.6:0/3044801865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625a0e20c50 0x5625a0e1a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.516+0000 7f3943e3cb80 1 -- 172.21.15.6:0/3044801865 >> 172.21.15.6:0/3044801865 conn(0x5625a0ce1f70 msgr2=0x5625a0e15520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:09.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.517+0000 7f3943e3cb80 1 -- 172.21.15.6:0/3044801865 shutdown_connections 2024-05-09T04:57:09.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.517+0000 7f3943e3cb80 1 -- 172.21.15.6:0/3044801865 wait complete. 2024-05-09T04:57:09.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.517+0000 7f3943e3cb80 1 -- start start 2024-05-09T04:57:09.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.517+0000 7f3943e3cb80 1 Processor -- start 2024-05-09T04:57:09.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.518+0000 7f3943e3cb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625a0c9d5a0 0x5625a0e24340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:09.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.518+0000 7f3943e3cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625a0e1af00 0x5625a0e24880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:09.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.518+0000 7f3943e3cb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625a0e20c50 0x5625a0e24dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:09.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.518+0000 7f3943e3cb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x5625a0e27660 con 0x5625a0e20c50 2024-05-09T04:57:09.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.518+0000 7f3943e3cb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5625a0ca27c0 con 0x5625a0c9d5a0 2024-05-09T04:57:09.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.518+0000 7f3943e3cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5625a0e277e0 con 0x5625a0e1af00 2024-05-09T04:57:09.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.518+0000 7f3943e3cb80 10 client.0 _ll_register_callbacks cb 0x5625a0cd4690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:57:09.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.518+0000 7f392de00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625a0c9d5a0 0x5625a0e24340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:09.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.518+0000 7f392e800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625a0e20c50 0x5625a0e24dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:09.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.518+0000 7f3927e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625a0e1af00 0x5625a0e24880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:09.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.518+0000 7f392de00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625a0c9d5a0 0x5625a0e24340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:40688/0 (socket says 172.21.15.6:40688) 2024-05-09T04:57:09.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.518+0000 7f392de00700 1 -- 172.21.15.6:0/4057204847 learned_addr learned my addr 172.21.15.6:0/4057204847 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:57:09.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.519+0000 7f392de00700 1 -- 172.21.15.6:0/4057204847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625a0e1af00 msgr2=0x5625a0e24880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:09.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.519+0000 7f392de00700 1 --2- 172.21.15.6:0/4057204847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625a0e1af00 0x5625a0e24880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.519+0000 7f392de00700 1 -- 172.21.15.6:0/4057204847 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625a0e20c50 msgr2=0x5625a0e24dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:09.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.519+0000 7f392de00700 1 --2- 172.21.15.6:0/4057204847 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625a0e20c50 0x5625a0e24dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.519+0000 7f392de00700 1 -- 172.21.15.6:0/4057204847 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625a0e30ba0 con 0x5625a0c9d5a0 2024-05-09T04:57:09.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.519+0000 7f392e800700 1 --2- 172.21.15.6:0/4057204847 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625a0e20c50 0x5625a0e24dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:57:09.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.519+0000 7f392de00700 1 --2- 172.21.15.6:0/4057204847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625a0c9d5a0 0x5625a0e24340 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3920009990 tx=0x7f39200058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64d81f7abe967898 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:09.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.519+0000 7f3913e00700 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-05-09T04:57:09.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.520+0000 7f3913e00700 1 -- 172.21.15.6:0/4057204847 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3920027040 con 0x5625a0c9d5a0 2024-05-09T04:57:09.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.520+0000 7f3913e00700 1 -- 172.21.15.6:0/4057204847 <== mon.2 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f3920003bf0 con 0x5625a0c9d5a0 2024-05-09T04:57:09.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.520+0000 7f3943e3cb80 1 -- 172.21.15.6:0/4057204847 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39200054e0 con 0x5625a0c9d5a0 2024-05-09T04:57:09.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.520+0000 7f3913e00700 1 -- 172.21.15.6:0/4057204847 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3920005690 con 0x5625a0c9d5a0 2024-05-09T04:57:09.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.520+0000 7f3943e3cb80 10 client.25463 fetch_fsmap learned FSMap version 30 2024-05-09T04:57:09.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.520+0000 7f3943e3cb80 1 -- 172.21.15.6:0/4057204847 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5625a0e32430 con 0x5625a0c9d5a0 2024-05-09T04:57:09.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.520+0000 7f3913e00700 1 -- 172.21.15.6:0/4057204847 <== mon.2 v2:172.21.15.169:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39200054e0 con 0x5625a0c9d5a0 2024-05-09T04:57:09.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.521+0000 7f3913e00700 1 -- 172.21.15.6:0/4057204847 <== mon.2 v2:172.21.15.169:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3920022960 con 0x5625a0c9d5a0 2024-05-09T04:57:09.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.521+0000 7f3943e3cb80 10 client.25463 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T04:57:09.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.521+0000 7f3943e3cb80 10 client.25463 resolve_mds: validated gid 24554 aka daemon mds.b 2024-05-09T04:57:09.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.521+0000 7f3943e3cb80 20 client.25463 populate_metadata read hostname 'smithi006' 2024-05-09T04:57:09.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.521+0000 7f3943e3cb80 1 --2- 172.21.15.6:0/4057204847 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x5625a0e34900 0x5625a0e36d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:09.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.521+0000 7f3943e3cb80 4 client.25463 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:57:09.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.521+0000 7f3943e3cb80 1 -- 172.21.15.6:0/4057204847 --> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625a0de6970 con 0x5625a0e34900 2024-05-09T04:57:09.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.521+0000 7f3927e00700 1 --2- 172.21.15.6:0/4057204847 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x5625a0e34900 0x5625a0e36d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:09.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.522+0000 7f3927e00700 1 --2- 172.21.15.6:0/4057204847 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x5625a0e34900 0x5625a0e36d20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c5d9d88faadf988 server_cookie=e99b4ba57e9d5019 in_seq=0 out_seq=0 2024-05-09T04:57:09.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.522+0000 7f3913e00700 10 client.25463 ms_handle_connect on v2:172.21.15.169:6838/614388771 2024-05-09T04:57:09.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.523+0000 7f3913e00700 1 -- 172.21.15.6:0/4057204847 <== mds.0 v2:172.21.15.169:6838/614388771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5625a0de6970 con 0x5625a0e34900 2024-05-09T04:57:09.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.523+0000 7f3913e00700 10 client.25463 handle_command_reply: tid=0 2024-05-09T04:57:09.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.523+0000 7f3943e3cb80 1 client.25463 shutdown 2024-05-09T04:57:09.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.523+0000 7f3943e3cb80 1 -- 172.21.15.6:0/4057204847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625a0c9d5a0 msgr2=0x5625a0e24340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:09.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.523+0000 7f3943e3cb80 1 --2- 172.21.15.6:0/4057204847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625a0c9d5a0 0x5625a0e24340 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3920009990 tx=0x7f39200058e0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.523+0000 7f3943e3cb80 1 -- 172.21.15.6:0/4057204847 shutdown_connections 2024-05-09T04:57:09.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.523+0000 7f3943e3cb80 1 --2- 172.21.15.6:0/4057204847 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625a0e20c50 0x5625a0e24dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.523+0000 7f3943e3cb80 1 --2- 172.21.15.6:0/4057204847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625a0c9d5a0 0x5625a0e24340 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.523+0000 7f3943e3cb80 1 --2- 172.21.15.6:0/4057204847 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x5625a0e34900 0x5625a0e36d20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.523+0000 7f3943e3cb80 1 --2- 172.21.15.6:0/4057204847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625a0e1af00 0x5625a0e24880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.523+0000 7f3943e3cb80 1 -- 172.21.15.6:0/4057204847 >> 172.21.15.6:0/4057204847 conn(0x5625a0ce1f70 msgr2=0x5625a0e14a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:09.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.523+0000 7f3913e00700 0 client.25463 ms_handle_reset on v2:172.21.15.169:6838/614388771 2024-05-09T04:57:09.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.523+0000 7f3943e3cb80 1 -- 172.21.15.6:0/4057204847 shutdown_connections 2024-05-09T04:57:09.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.524+0000 7f3943e3cb80 1 -- 172.21.15.6:0/4057204847 wait complete. 2024-05-09T04:57:09.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.524+0000 7f3943e3cb80 20 client.25463 trim_cache size 0 max 16384 2024-05-09T04:57:09.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.547+0000 7f3943e3cb80 1 Processor -- start 2024-05-09T04:57:09.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.547+0000 7f3943e3cb80 1 -- start start 2024-05-09T04:57:09.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.547+0000 7f3943e3cb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625a0c9d5a0 0x5625a0e1fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:09.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.547+0000 7f3943e3cb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625a0e1af00 0x5625a0e30530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:09.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.548+0000 7f3943e3cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625a0e20c50 0x5625a0e15070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:09.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.548+0000 7f3943e3cb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f390c00ea40 con 0x5625a0e1af00 2024-05-09T04:57:09.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.548+0000 7f3943e3cb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f390c010580 con 0x5625a0c9d5a0 2024-05-09T04:57:09.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.548+0000 7f3943e3cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f390c00f110 con 0x5625a0e20c50 2024-05-09T04:57:09.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.548+0000 7f3927e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625a0e1af00 0x5625a0e30530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:09.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.548+0000 7f392de00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625a0c9d5a0 0x5625a0e1fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:09.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.548+0000 7f392de00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625a0c9d5a0 0x5625a0e1fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:40690/0 (socket says 172.21.15.6:40690) 2024-05-09T04:57:09.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.548+0000 7f392de00700 1 -- 172.21.15.6:0/3909244336 learned_addr learned my addr 172.21.15.6:0/3909244336 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:57:09.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.548+0000 7f392e800700 1 --2- 172.21.15.6:0/3909244336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625a0e20c50 0x5625a0e15070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:09.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.548+0000 7f392de00700 1 -- 172.21.15.6:0/3909244336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625a0e20c50 msgr2=0x5625a0e15070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:09.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.548+0000 7f392de00700 1 --2- 172.21.15.6:0/3909244336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625a0e20c50 0x5625a0e15070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.548+0000 7f392de00700 1 -- 172.21.15.6:0/3909244336 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625a0e1af00 msgr2=0x5625a0e30530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:09.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.548+0000 7f392de00700 1 --2- 172.21.15.6:0/3909244336 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625a0e1af00 0x5625a0e30530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.548+0000 7f392de00700 1 -- 172.21.15.6:0/3909244336 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625a0e25350 con 0x5625a0c9d5a0 2024-05-09T04:57:09.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.549+0000 7f392de00700 1 --2- 172.21.15.6:0/3909244336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625a0c9d5a0 0x5625a0e1fb70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f39200098e0 tx=0x7f39200092f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92abb905c553efba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:09.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.549+0000 7f391ea00700 1 -- 172.21.15.6:0/3909244336 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392000f660 con 0x5625a0c9d5a0 2024-05-09T04:57:09.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.549+0000 7f391ea00700 1 -- 172.21.15.6:0/3909244336 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3920031070 con 0x5625a0c9d5a0 2024-05-09T04:57:09.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.549+0000 7f391ea00700 1 -- 172.21.15.6:0/3909244336 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39200226e0 con 0x5625a0c9d5a0 2024-05-09T04:57:09.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.550+0000 7f3943e3cb80 1 -- 172.21.15.6:0/3909244336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625a0c9d5a0 msgr2=0x5625a0e1fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:09.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.550+0000 7f3943e3cb80 1 --2- 172.21.15.6:0/3909244336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625a0c9d5a0 0x5625a0e1fb70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f39200098e0 tx=0x7f39200092f0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.550+0000 7f3943e3cb80 1 -- 172.21.15.6:0/3909244336 shutdown_connections 2024-05-09T04:57:09.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.550+0000 7f3943e3cb80 1 --2- 172.21.15.6:0/3909244336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625a0e20c50 0x5625a0e15070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.550+0000 7f3943e3cb80 1 --2- 172.21.15.6:0/3909244336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625a0c9d5a0 0x5625a0e1fb70 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.550+0000 7f3943e3cb80 1 --2- 172.21.15.6:0/3909244336 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625a0e1af00 0x5625a0e30530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.550+0000 7f3943e3cb80 1 -- 172.21.15.6:0/3909244336 >> 172.21.15.6:0/3909244336 conn(0x5625a0ce1f70 msgr2=0x5625a0e1e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:09.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.550+0000 7f3943e3cb80 1 -- 172.21.15.6:0/3909244336 shutdown_connections 2024-05-09T04:57:09.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.550+0000 7f3943e3cb80 1 -- 172.21.15.6:0/3909244336 wait complete. 2024-05-09T04:57:09.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.551+0000 7f3943e3cb80 1 -- start start 2024-05-09T04:57:09.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.551+0000 7f3943e3cb80 1 Processor -- start 2024-05-09T04:57:09.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.552+0000 7f3943e3cb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625a0c9d5a0 0x5625a0e93000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:09.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.552+0000 7f3943e3cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625a0e1af00 0x5625a0e93540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:09.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.552+0000 7f3943e3cb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625a0e20c50 0x5625a0e96fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:09.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.552+0000 7f3943e3cb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f390c00feb0 con 0x5625a0e20c50 2024-05-09T04:57:09.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.552+0000 7f3943e3cb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f390c00f7e0 con 0x5625a0c9d5a0 2024-05-09T04:57:09.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.552+0000 7f3943e3cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f390c00c9e0 con 0x5625a0e1af00 2024-05-09T04:57:09.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.552+0000 7f3943e3cb80 10 client.0 _ll_register_callbacks cb 0x5625a0cd4690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:57:09.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.552+0000 7f392e800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625a0e20c50 0x5625a0e96fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:09.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.552+0000 7f392de00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625a0c9d5a0 0x5625a0e93000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:09.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.552+0000 7f392de00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625a0c9d5a0 0x5625a0e93000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:40700/0 (socket says 172.21.15.6:40700) 2024-05-09T04:57:09.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.552+0000 7f392e800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625a0e20c50 0x5625a0e96fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:40408/0 (socket says 172.21.15.6:40408) 2024-05-09T04:57:09.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.552+0000 7f392de00700 1 -- 172.21.15.6:0/42879757 learned_addr learned my addr 172.21.15.6:0/42879757 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:57:09.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.552+0000 7f3927e00700 1 --2- 172.21.15.6:0/42879757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625a0e1af00 0x5625a0e93540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:09.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.553+0000 7f392de00700 1 -- 172.21.15.6:0/42879757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625a0e1af00 msgr2=0x5625a0e93540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:09.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.553+0000 7f392de00700 1 --2- 172.21.15.6:0/42879757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625a0e1af00 0x5625a0e93540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.553+0000 7f392de00700 1 -- 172.21.15.6:0/42879757 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625a0e20c50 msgr2=0x5625a0e96fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:09.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.553+0000 7f392de00700 1 --2- 172.21.15.6:0/42879757 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625a0e20c50 0x5625a0e96fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.553+0000 7f392de00700 1 -- 172.21.15.6:0/42879757 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625a0e975a0 con 0x5625a0c9d5a0 2024-05-09T04:57:09.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.553+0000 7f3927e00700 1 --2- 172.21.15.6:0/42879757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625a0e1af00 0x5625a0e93540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:57:09.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.553+0000 7f392e800700 1 --2- 172.21.15.6:0/42879757 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625a0e20c50 0x5625a0e96fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:57:09.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.553+0000 7f392de00700 1 --2- 172.21.15.6:0/42879757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625a0c9d5a0 0x5625a0e93000 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3920005b00 tx=0x7f39200059d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9f43fe71d50694b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:09.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.553+0000 7f3913e00700 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-05-09T04:57:09.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.553+0000 7f3913e00700 1 -- 172.21.15.6:0/42879757 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392004a020 con 0x5625a0c9d5a0 2024-05-09T04:57:09.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.553+0000 7f3943e3cb80 1 -- 172.21.15.6:0/42879757 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3920031070 con 0x5625a0c9d5a0 2024-05-09T04:57:09.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.553+0000 7f3913e00700 1 -- 172.21.15.6:0/42879757 <== mon.2 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f3920022a50 con 0x5625a0c9d5a0 2024-05-09T04:57:09.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.554+0000 7f3913e00700 1 -- 172.21.15.6:0/42879757 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3920027030 con 0x5625a0c9d5a0 2024-05-09T04:57:09.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.554+0000 7f3943e3cb80 10 client.25469 fetch_fsmap learned FSMap version 30 2024-05-09T04:57:09.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.554+0000 7f3943e3cb80 1 -- 172.21.15.6:0/42879757 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5625a0e975a0 con 0x5625a0c9d5a0 2024-05-09T04:57:09.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.554+0000 7f3913e00700 1 -- 172.21.15.6:0/42879757 <== mon.2 v2:172.21.15.169:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3920031070 con 0x5625a0c9d5a0 2024-05-09T04:57:09.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.555+0000 7f3913e00700 1 -- 172.21.15.6:0/42879757 <== mon.2 v2:172.21.15.169:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f392000f5f0 con 0x5625a0c9d5a0 2024-05-09T04:57:09.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.555+0000 7f3943e3cb80 10 client.25469 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T04:57:09.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.555+0000 7f3943e3cb80 10 client.25469 resolve_mds: validated gid 24554 aka daemon mds.b 2024-05-09T04:57:09.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.555+0000 7f3943e3cb80 20 client.25469 populate_metadata read hostname 'smithi006' 2024-05-09T04:57:09.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.555+0000 7f3943e3cb80 1 --2- 172.21.15.6:0/42879757 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x5625a0e9fd30 0x5625a0ea2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:09.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.555+0000 7f3943e3cb80 4 client.25469 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "98112720", "release": true, "await": true}] 2024-05-09T04:57:09.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.555+0000 7f3943e3cb80 1 -- 172.21.15.6:0/42879757 --> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] -- command(tid 0: {"prefix": "quiesce db", "set_id": "98112720", "release": true, "await": true}) -- 0x5625a0d4ad90 con 0x5625a0e9fd30 2024-05-09T04:57:09.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.555+0000 7f3927e00700 1 --2- 172.21.15.6:0/42879757 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x5625a0e9fd30 0x5625a0ea2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:09.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.556+0000 7f3927e00700 1 --2- 172.21.15.6:0/42879757 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x5625a0e9fd30 0x5625a0ea2150 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d3dbc210b1ff305 server_cookie=c33ab828fb398f99 in_seq=0 out_seq=0 2024-05-09T04:57:09.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.556+0000 7f3913e00700 10 client.25469 ms_handle_connect on v2:172.21.15.169:6838/614388771 2024-05-09T04:57:09.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.560+0000 7f3913e00700 1 -- 172.21.15.6:0/42879757 <== mds.0 v2:172.21.15.169:6838/614388771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5625a0d4ad90 con 0x5625a0e9fd30 2024-05-09T04:57:09.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.560+0000 7f3913e00700 10 client.25469 handle_command_reply: tid=0 2024-05-09T04:57:09.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.560+0000 7f3943e3cb80 1 client.25469 shutdown 2024-05-09T04:57:09.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.560+0000 7f3943e3cb80 1 -- 172.21.15.6:0/42879757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625a0c9d5a0 msgr2=0x5625a0e93000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:09.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.560+0000 7f3943e3cb80 1 --2- 172.21.15.6:0/42879757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625a0c9d5a0 0x5625a0e93000 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3920005b00 tx=0x7f39200059d0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.560+0000 7f3943e3cb80 1 -- 172.21.15.6:0/42879757 shutdown_connections 2024-05-09T04:57:09.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.560+0000 7f3943e3cb80 1 --2- 172.21.15.6:0/42879757 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625a0e20c50 0x5625a0e96fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.560+0000 7f3943e3cb80 1 --2- 172.21.15.6:0/42879757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625a0c9d5a0 0x5625a0e93000 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.560+0000 7f3943e3cb80 1 --2- 172.21.15.6:0/42879757 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x5625a0e9fd30 0x5625a0ea2150 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.560+0000 7f3943e3cb80 1 --2- 172.21.15.6:0/42879757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625a0e1af00 0x5625a0e93540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.560+0000 7f3943e3cb80 1 -- 172.21.15.6:0/42879757 >> 172.21.15.6:0/42879757 conn(0x5625a0ce1f70 msgr2=0x5625a0e1e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:09.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.560+0000 7f3913e00700 0 client.25469 ms_handle_reset on v2:172.21.15.169:6838/614388771 2024-05-09T04:57:09.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.560+0000 7f3943e3cb80 1 -- 172.21.15.6:0/42879757 shutdown_connections 2024-05-09T04:57:09.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.560+0000 7f3943e3cb80 1 -- 172.21.15.6:0/42879757 wait complete. 2024-05-09T04:57:09.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.561+0000 7f3943e3cb80 20 client.25469 trim_cache size 0 max 16384 2024-05-09T04:57:09.563 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-09T04:57:09.563 INFO:teuthology.orchestra.run.smithi006.stdout: "epoch": 30, 2024-05-09T04:57:09.563 INFO:teuthology.orchestra.run.smithi006.stdout: "leader": 24554, 2024-05-09T04:57:09.563 INFO:teuthology.orchestra.run.smithi006.stdout: "set_version": 8, 2024-05-09T04:57:09.563 INFO:teuthology.orchestra.run.smithi006.stdout: "sets": { 2024-05-09T04:57:09.563 INFO:teuthology.orchestra.run.smithi006.stdout: "98112720": { 2024-05-09T04:57:09.563 INFO:teuthology.orchestra.run.smithi006.stdout: "version": 8, 2024-05-09T04:57:09.563 INFO:teuthology.orchestra.run.smithi006.stdout: "age_ref": 0.0, 2024-05-09T04:57:09.563 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-05-09T04:57:09.563 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RELEASED", 2024-05-09T04:57:09.563 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-05-09T04:57:09.563 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:57:09.564 INFO:teuthology.orchestra.run.smithi006.stdout: "timeout": 90.0, 2024-05-09T04:57:09.564 INFO:teuthology.orchestra.run.smithi006.stdout: "expiration": 144.9, 2024-05-09T04:57:09.564 INFO:teuthology.orchestra.run.smithi006.stdout: "members": { 2024-05-09T04:57:09.564 INFO:teuthology.orchestra.run.smithi006.stdout: "file:/": { 2024-05-09T04:57:09.564 INFO:teuthology.orchestra.run.smithi006.stdout: "excluded": false, 2024-05-09T04:57:09.564 INFO:teuthology.orchestra.run.smithi006.stdout: "state": { 2024-05-09T04:57:09.564 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RELEASED", 2024-05-09T04:57:09.564 INFO:teuthology.orchestra.run.smithi006.stdout: "age": 0.0 2024-05-09T04:57:09.564 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:57:09.564 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:57:09.564 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:57:09.564 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:57:09.564 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:57:09.564 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-09T04:57:09.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.561+0000 7f392f200700 1 -- 172.21.15.6:0/3173300446 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f39140b2900 msgr2=0x7f39140b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:09.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.561+0000 7f392f200700 1 --2- 172.21.15.6:0/3173300446 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f39140b2900 0x7f39140b4e10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x5625a0e170e0 tx=0x7f39180098d0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.561+0000 7f392f200700 1 -- 172.21.15.6:0/3173300446 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3930109690 msgr2=0x7f39301a1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:09.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.561+0000 7f392f200700 1 --2- 172.21.15.6:0/3173300446 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3930109690 0x7f39301a1690 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f392800ef30 tx=0x7f392800c3a0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.562+0000 7f392f200700 1 -- 172.21.15.6:0/3173300446 shutdown_connections 2024-05-09T04:57:09.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.562+0000 7f392f200700 1 --2- 172.21.15.6:0/3173300446 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3930109690 0x7f39301a1690 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.562+0000 7f392f200700 1 --2- 172.21.15.6:0/3173300446 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3930104410 0x7f393019ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.562+0000 7f392f200700 1 --2- 172.21.15.6:0/3173300446 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f39140b2900 0x7f39140b4e10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.562+0000 7f392f200700 1 --2- 172.21.15.6:0/3173300446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3930108d40 0x7f393019d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:09.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.562+0000 7f392f200700 1 -- 172.21.15.6:0/3173300446 >> 172.21.15.6:0/3173300446 conn(0x7f3930079d50 msgr2=0x7f39301025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:09.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.562+0000 7f392f200700 1 -- 172.21.15.6:0/3173300446 shutdown_connections 2024-05-09T04:57:09.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:57:09.562+0000 7f392f200700 1 -- 172.21.15.6:0/3173300446 wait complete. 2024-05-09T04:57:09.573 INFO:tasks.workunit.client.0.smithi006.stderr:+ wget http://download.ceph.com/qa/fsync-tester.c 2024-05-09T04:57:09.604 INFO:tasks.workunit.client.0.smithi006.stderr:--2024-05-09 04:57:09-- http://download.ceph.com/qa/fsync-tester.c 2024-05-09T04:57:09.616 INFO:tasks.workunit.client.0.smithi006.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-05-09T04:57:09.638 INFO:tasks.workunit.client.0.smithi006.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-05-09T04:57:09.658 INFO:tasks.workunit.client.0.smithi006.stderr:HTTP request sent, awaiting response... 200 OK 2024-05-09T04:57:09.658 INFO:tasks.workunit.client.0.smithi006.stderr:Length: 3024 (3.0K) [application/octet-stream] 2024-05-09T04:57:09.661 INFO:tasks.workunit.client.0.smithi006.stderr:Saving to: ‘fsync-tester.c’ 2024-05-09T04:57:09.668 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-09T04:57:09.668 INFO:tasks.workunit.client.0.smithi006.stderr: 0K .. 100% 94.1M=0s 2024-05-09T04:57:09.669 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-09T04:57:09.669 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-09 04:57:09 (94.1 MB/s) - ‘fsync-tester.c’ saved [3024/3024] 2024-05-09T04:57:09.669 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-09T04:57:09.669 INFO:tasks.workunit.client.0.smithi006.stderr:+ gcc -D_GNU_SOURCE fsync-tester.c -o fsync-tester 2024-05-09T04:57:10.045 INFO:tasks.workunit.client.0.smithi006.stderr:+ ./fsync-tester 2024-05-09T04:57:10.612 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '98112720', total seconds elapsed: 33.3 2024-05-09T04:57:11.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:10 smithi006 ceph-mon[29835]: pgmap v566: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-09T04:57:11.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:10 smithi186 ceph-mon[34668]: pgmap v566: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-09T04:57:11.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:10 smithi169 ceph-mon[34075]: pgmap v566: 129 pgs: 129 active+clean; 668 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-09T04:57:13.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:12 smithi006 ceph-mon[29835]: pgmap v567: 129 pgs: 129 active+clean; 697 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-05-09T04:57:13.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:12 smithi186 ceph-mon[34668]: pgmap v567: 129 pgs: 129 active+clean; 697 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-05-09T04:57:13.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:12 smithi169 ceph-mon[34075]: pgmap v567: 129 pgs: 129 active+clean; 697 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-05-09T04:57:15.151 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:14 smithi186 ceph-mon[34668]: pgmap v568: 129 pgs: 129 active+clean; 1.7 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 174 KiB/s wr, 10 op/s 2024-05-09T04:57:15.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:14 smithi006 ceph-mon[29835]: pgmap v568: 129 pgs: 129 active+clean; 1.7 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 174 KiB/s wr, 10 op/s 2024-05-09T04:57:15.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:14 smithi169 ceph-mon[34075]: pgmap v568: 129 pgs: 129 active+clean; 1.7 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 174 KiB/s wr, 10 op/s 2024-05-09T04:57:17.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:16 smithi006 ceph-mon[29835]: pgmap v569: 129 pgs: 129 active+clean; 1.7 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 177 KiB/s wr, 12 op/s 2024-05-09T04:57:17.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:16 smithi186 ceph-mon[34668]: pgmap v569: 129 pgs: 129 active+clean; 1.7 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 177 KiB/s wr, 12 op/s 2024-05-09T04:57:17.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:16 smithi169 ceph-mon[34075]: pgmap v569: 129 pgs: 129 active+clean; 1.7 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 177 KiB/s wr, 12 op/s 2024-05-09T04:57:19.156 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:18 smithi006 ceph-mon[29835]: pgmap v570: 129 pgs: 129 active+clean; 1.7 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 519 KiB/s wr, 11 op/s 2024-05-09T04:57:19.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:18 smithi186 ceph-mon[34668]: pgmap v570: 129 pgs: 129 active+clean; 1.7 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 519 KiB/s wr, 11 op/s 2024-05-09T04:57:19.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:18 smithi169 ceph-mon[34075]: pgmap v570: 129 pgs: 129 active+clean; 1.7 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 519 KiB/s wr, 11 op/s 2024-05-09T04:57:21.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:20 smithi006 ceph-mon[29835]: pgmap v571: 129 pgs: 129 active+clean; 1.7 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 519 KiB/s wr, 12 op/s 2024-05-09T04:57:21.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:20 smithi186 ceph-mon[34668]: pgmap v571: 129 pgs: 129 active+clean; 1.7 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 519 KiB/s wr, 12 op/s 2024-05-09T04:57:21.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:20 smithi169 ceph-mon[34075]: pgmap v571: 129 pgs: 129 active+clean; 1.7 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 519 KiB/s wr, 12 op/s 2024-05-09T04:57:23.169 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:22 smithi169 ceph-mon[34075]: pgmap v572: 129 pgs: 129 active+clean; 1.8 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 521 KiB/s wr, 14 op/s 2024-05-09T04:57:23.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:22 smithi006 ceph-mon[29835]: pgmap v572: 129 pgs: 129 active+clean; 1.8 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 521 KiB/s wr, 14 op/s 2024-05-09T04:57:23.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:22 smithi186 ceph-mon[34668]: pgmap v572: 129 pgs: 129 active+clean; 1.8 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 521 KiB/s wr, 14 op/s 2024-05-09T04:57:25.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:24 smithi006 ceph-mon[29835]: pgmap v573: 129 pgs: 129 active+clean; 1.8 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 945 KiB/s wr, 12 op/s 2024-05-09T04:57:25.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:24 smithi186 ceph-mon[34668]: pgmap v573: 129 pgs: 129 active+clean; 1.8 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 945 KiB/s wr, 12 op/s 2024-05-09T04:57:25.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:24 smithi169 ceph-mon[34075]: pgmap v573: 129 pgs: 129 active+clean; 1.8 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 945 KiB/s wr, 12 op/s 2024-05-09T04:57:27.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:26 smithi006 ceph-mon[29835]: pgmap v574: 129 pgs: 129 active+clean; 1.8 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 777 KiB/s wr, 14 op/s 2024-05-09T04:57:27.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:26 smithi186 ceph-mon[34668]: pgmap v574: 129 pgs: 129 active+clean; 1.8 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 777 KiB/s wr, 14 op/s 2024-05-09T04:57:27.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:26 smithi169 ceph-mon[34075]: pgmap v574: 129 pgs: 129 active+clean; 1.8 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 777 KiB/s wr, 14 op/s 2024-05-09T04:57:29.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:28 smithi006 ceph-mon[29835]: pgmap v575: 129 pgs: 129 active+clean; 1.8 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 1.2 MiB/s wr, 12 op/s 2024-05-09T04:57:29.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:28 smithi186 ceph-mon[34668]: pgmap v575: 129 pgs: 129 active+clean; 1.8 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 1.2 MiB/s wr, 12 op/s 2024-05-09T04:57:29.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:28 smithi169 ceph-mon[34075]: pgmap v575: 129 pgs: 129 active+clean; 1.8 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 1.2 MiB/s wr, 12 op/s 2024-05-09T04:57:31.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:30 smithi006 ceph-mon[29835]: pgmap v576: 129 pgs: 129 active+clean; 1.8 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 859 KiB/s wr, 12 op/s 2024-05-09T04:57:31.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:30 smithi186 ceph-mon[34668]: pgmap v576: 129 pgs: 129 active+clean; 1.8 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 859 KiB/s wr, 12 op/s 2024-05-09T04:57:31.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:30 smithi169 ceph-mon[34075]: pgmap v576: 129 pgs: 129 active+clean; 1.8 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 859 KiB/s wr, 12 op/s 2024-05-09T04:57:33.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:32 smithi006 ceph-mon[29835]: pgmap v577: 129 pgs: 129 active+clean; 34 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.5 MiB/s wr, 15 op/s 2024-05-09T04:57:33.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:32 smithi186 ceph-mon[34668]: pgmap v577: 129 pgs: 129 active+clean; 34 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.5 MiB/s wr, 15 op/s 2024-05-09T04:57:33.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:32 smithi169 ceph-mon[34075]: pgmap v577: 129 pgs: 129 active+clean; 34 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.5 MiB/s wr, 15 op/s 2024-05-09T04:57:35.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:34 smithi006 ceph-mon[29835]: pgmap v578: 129 pgs: 129 active+clean; 74 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 MiB/s wr, 14 op/s 2024-05-09T04:57:35.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:34 smithi186 ceph-mon[34668]: pgmap v578: 129 pgs: 129 active+clean; 74 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 MiB/s wr, 14 op/s 2024-05-09T04:57:35.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:34 smithi169 ceph-mon[34075]: pgmap v578: 129 pgs: 129 active+clean; 74 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 MiB/s wr, 14 op/s 2024-05-09T04:57:37.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:36 smithi006 ceph-mon[29835]: pgmap v579: 129 pgs: 129 active+clean; 74 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.8 MiB/s wr, 16 op/s 2024-05-09T04:57:37.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:36 smithi186 ceph-mon[34668]: pgmap v579: 129 pgs: 129 active+clean; 74 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.8 MiB/s wr, 16 op/s 2024-05-09T04:57:37.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:36 smithi169 ceph-mon[34075]: pgmap v579: 129 pgs: 129 active+clean; 74 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.8 MiB/s wr, 16 op/s 2024-05-09T04:57:39.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:38 smithi006 ceph-mon[29835]: pgmap v580: 129 pgs: 129 active+clean; 166 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 17 MiB/s wr, 16 op/s 2024-05-09T04:57:39.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:38 smithi186 ceph-mon[34668]: pgmap v580: 129 pgs: 129 active+clean; 166 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 17 MiB/s wr, 16 op/s 2024-05-09T04:57:39.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:38 smithi169 ceph-mon[34075]: pgmap v580: 129 pgs: 129 active+clean; 166 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 17 MiB/s wr, 16 op/s 2024-05-09T04:57:41.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:40 smithi186 ceph-mon[34668]: pgmap v581: 129 pgs: 129 active+clean; 242 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 20 op/s 2024-05-09T04:57:41.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:40 smithi169 ceph-mon[34075]: pgmap v581: 129 pgs: 129 active+clean; 242 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 20 op/s 2024-05-09T04:57:41.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:40 smithi006 ceph-mon[29835]: pgmap v581: 129 pgs: 129 active+clean; 242 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 20 op/s 2024-05-09T04:57:43.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:42 smithi186 ceph-mon[34668]: pgmap v582: 129 pgs: 129 active+clean; 258 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 MiB/s wr, 26 op/s 2024-05-09T04:57:43.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:42 smithi169 ceph-mon[34075]: pgmap v582: 129 pgs: 129 active+clean; 258 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 MiB/s wr, 26 op/s 2024-05-09T04:57:43.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:42 smithi006 ceph-mon[29835]: pgmap v582: 129 pgs: 129 active+clean; 258 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 MiB/s wr, 26 op/s 2024-05-09T04:57:45.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:44 smithi186 ceph-mon[34668]: pgmap v583: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 26 MiB/s wr, 24 op/s 2024-05-09T04:57:45.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:44 smithi169 ceph-mon[34075]: pgmap v583: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 26 MiB/s wr, 24 op/s 2024-05-09T04:57:45.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:44 smithi006 ceph-mon[29835]: pgmap v583: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 26 MiB/s wr, 24 op/s 2024-05-09T04:57:47.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:46 smithi186 ceph-mon[34668]: pgmap v584: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 24 op/s 2024-05-09T04:57:47.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:46 smithi169 ceph-mon[34075]: pgmap v584: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 24 op/s 2024-05-09T04:57:47.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:46 smithi006 ceph-mon[29835]: pgmap v584: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 24 op/s 2024-05-09T04:57:49.250 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:48 smithi006 ceph-mon[29835]: pgmap v585: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 MiB/s wr, 22 op/s 2024-05-09T04:57:49.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:48 smithi186 ceph-mon[34668]: pgmap v585: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 MiB/s wr, 22 op/s 2024-05-09T04:57:49.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:48 smithi169 ceph-mon[34075]: pgmap v585: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 MiB/s wr, 22 op/s 2024-05-09T04:57:50.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:49 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:57:50.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:49 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:57:50.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:49 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:57:51.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:50 smithi186 ceph-mon[34668]: pgmap v586: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 19 op/s 2024-05-09T04:57:51.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:50 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:57:51.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:50 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:57:51.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:50 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:57:51.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:50 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:57:51.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:50 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:57:51.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:50 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:57:51.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:50 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:57:51.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:50 smithi169 ceph-mon[34075]: pgmap v586: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 19 op/s 2024-05-09T04:57:51.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:50 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:57:51.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:50 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:57:51.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:50 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:57:51.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:50 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:57:51.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:50 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:57:51.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:50 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:57:51.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:50 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:57:51.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:50 smithi006 ceph-mon[29835]: pgmap v586: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 19 op/s 2024-05-09T04:57:51.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:50 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:57:51.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:50 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:57:51.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:50 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:57:51.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:50 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:57:51.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:50 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:57:51.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:50 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:57:51.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:50 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:57:52.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:51 smithi186 ceph-mon[34668]: Adjusting osd_memory_target on smithi006 to 177.5M 2024-05-09T04:57:52.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:51 smithi186 ceph-mon[34668]: Unable to set osd_memory_target on smithi006 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-09T04:57:52.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:51 smithi169 ceph-mon[34075]: Adjusting osd_memory_target on smithi006 to 177.5M 2024-05-09T04:57:52.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:51 smithi169 ceph-mon[34075]: Unable to set osd_memory_target on smithi006 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-09T04:57:52.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:51 smithi006 ceph-mon[29835]: Adjusting osd_memory_target on smithi006 to 177.5M 2024-05-09T04:57:52.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:51 smithi006 ceph-mon[29835]: Unable to set osd_memory_target on smithi006 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-09T04:57:53.268 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:52 smithi169 ceph-mon[34075]: pgmap v587: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.8 MiB/s wr, 18 op/s 2024-05-09T04:57:53.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:52 smithi186 ceph-mon[34668]: pgmap v587: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.8 MiB/s wr, 18 op/s 2024-05-09T04:57:53.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:52 smithi006 ceph-mon[29835]: pgmap v587: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.8 MiB/s wr, 18 op/s 2024-05-09T04:57:55.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:55 smithi186 ceph-mon[34668]: pgmap v588: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.2 MiB/s wr, 12 op/s 2024-05-09T04:57:55.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:55 smithi169 ceph-mon[34075]: pgmap v588: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.2 MiB/s wr, 12 op/s 2024-05-09T04:57:55.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:55 smithi006 ceph-mon[29835]: pgmap v588: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.2 MiB/s wr, 12 op/s 2024-05-09T04:57:57.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:57 smithi186 ceph-mon[34668]: pgmap v589: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 777 KiB/s wr, 14 op/s 2024-05-09T04:57:57.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:57 smithi169 ceph-mon[34075]: pgmap v589: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 777 KiB/s wr, 14 op/s 2024-05-09T04:57:57.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:57 smithi006 ceph-mon[29835]: pgmap v589: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 777 KiB/s wr, 14 op/s 2024-05-09T04:57:59.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:57:59 smithi186 ceph-mon[34668]: pgmap v590: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.2 MiB/s wr, 12 op/s 2024-05-09T04:57:59.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:57:59 smithi169 ceph-mon[34075]: pgmap v590: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.2 MiB/s wr, 12 op/s 2024-05-09T04:57:59.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:57:59 smithi006 ceph-mon[29835]: pgmap v590: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.2 MiB/s wr, 12 op/s 2024-05-09T04:58:01.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:01 smithi186 ceph-mon[34668]: pgmap v591: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 774 KiB/s wr, 12 op/s 2024-05-09T04:58:01.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:01 smithi169 ceph-mon[34075]: pgmap v591: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 774 KiB/s wr, 12 op/s 2024-05-09T04:58:01.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:01 smithi006 ceph-mon[29835]: pgmap v591: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 774 KiB/s wr, 12 op/s 2024-05-09T04:58:03.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:03 smithi186 ceph-mon[34668]: pgmap v592: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 777 KiB/s wr, 14 op/s 2024-05-09T04:58:03.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:03 smithi169 ceph-mon[34075]: pgmap v592: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 777 KiB/s wr, 14 op/s 2024-05-09T04:58:03.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:03 smithi006 ceph-mon[29835]: pgmap v592: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 777 KiB/s wr, 14 op/s 2024-05-09T04:58:05.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:05 smithi186 ceph-mon[34668]: pgmap v593: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.2 MiB/s wr, 12 op/s 2024-05-09T04:58:05.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:05 smithi169 ceph-mon[34075]: pgmap v593: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.2 MiB/s wr, 12 op/s 2024-05-09T04:58:05.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:05 smithi006 ceph-mon[29835]: pgmap v593: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.2 MiB/s wr, 12 op/s 2024-05-09T04:58:07.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:07 smithi186 ceph-mon[34668]: pgmap v594: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 862 KiB/s wr, 14 op/s 2024-05-09T04:58:07.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:07 smithi169 ceph-mon[34075]: pgmap v594: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 862 KiB/s wr, 14 op/s 2024-05-09T04:58:07.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:07 smithi006 ceph-mon[29835]: pgmap v594: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 862 KiB/s wr, 14 op/s 2024-05-09T04:58:08.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:08 smithi186 ceph-mon[34668]: pgmap v595: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.3 MiB/s wr, 12 op/s 2024-05-09T04:58:08.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:08 smithi169 ceph-mon[34075]: pgmap v595: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.3 MiB/s wr, 12 op/s 2024-05-09T04:58:08.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:08 smithi006 ceph-mon[29835]: pgmap v595: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.3 MiB/s wr, 12 op/s 2024-05-09T04:58:11.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:10 smithi006 ceph-mon[29835]: pgmap v596: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.8 MiB/s wr, 13 op/s 2024-05-09T04:58:11.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:10 smithi186 ceph-mon[34668]: pgmap v596: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.8 MiB/s wr, 13 op/s 2024-05-09T04:58:11.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:10 smithi169 ceph-mon[34075]: pgmap v596: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.8 MiB/s wr, 13 op/s 2024-05-09T04:58:11.387 INFO:tasks.workunit.client.0.smithi006.stdout:setting up random write file 2024-05-09T04:58:11.387 INFO:tasks.workunit.client.0.smithi006.stdout:done setting up random write file 2024-05-09T04:58:11.387 INFO:tasks.workunit.client.0.smithi006.stdout:starting fsync run 2024-05-09T04:58:11.387 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0010s fsync time: 0.0438s 2024-05-09T04:58:11.387 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0007s fsync time: 0.0405s 2024-05-09T04:58:11.387 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0012s fsync time: 0.0335s 2024-05-09T04:58:11.387 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0445s 2024-05-09T04:58:11.387 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0015s fsync time: 0.0200s 2024-05-09T04:58:11.387 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0014s fsync time: 0.0244s 2024-05-09T04:58:11.387 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0292s 2024-05-09T04:58:11.387 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0014s fsync time: 0.0206s 2024-05-09T04:58:11.388 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0014s fsync time: 0.0380s 2024-05-09T04:58:11.388 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0384s 2024-05-09T04:58:11.388 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0009s fsync time: 0.0199s 2024-05-09T04:58:11.388 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0009s fsync time: 0.0244s 2024-05-09T04:58:11.388 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0014s fsync time: 0.0242s 2024-05-09T04:58:11.388 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0197s 2024-05-09T04:58:11.388 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0234s 2024-05-09T04:58:11.388 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0024s fsync time: 0.0237s 2024-05-09T04:58:11.388 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0223s 2024-05-09T04:58:11.388 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0245s 2024-05-09T04:58:11.388 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0267s 2024-05-09T04:58:11.388 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0009s fsync time: 0.0662s 2024-05-09T04:58:11.388 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0155s 2024-05-09T04:58:11.388 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0234s 2024-05-09T04:58:11.388 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0354s 2024-05-09T04:58:11.389 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0160s 2024-05-09T04:58:11.389 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0233s 2024-05-09T04:58:11.389 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0019s fsync time: 0.0632s 2024-05-09T04:58:11.389 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0281s 2024-05-09T04:58:11.389 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0159s 2024-05-09T04:58:11.389 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0369s 2024-05-09T04:58:11.389 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0014s fsync time: 0.0170s 2024-05-09T04:58:11.389 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0337s 2024-05-09T04:58:11.389 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0184s 2024-05-09T04:58:11.389 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0260s 2024-05-09T04:58:11.389 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0151s 2024-05-09T04:58:11.389 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0258s 2024-05-09T04:58:11.389 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0183s 2024-05-09T04:58:11.389 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0014s fsync time: 0.0231s 2024-05-09T04:58:11.389 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0387s 2024-05-09T04:58:11.390 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0014s fsync time: 0.0153s 2024-05-09T04:58:11.390 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0235s 2024-05-09T04:58:11.390 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0182s 2024-05-09T04:58:11.390 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0009s fsync time: 0.0158s 2024-05-09T04:58:11.390 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0237s 2024-05-09T04:58:11.390 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0246s 2024-05-09T04:58:11.390 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0014s fsync time: 0.0177s 2024-05-09T04:58:11.390 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0015s fsync time: 0.0256s 2024-05-09T04:58:11.390 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0233s 2024-05-09T04:58:11.390 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0160s 2024-05-09T04:58:11.390 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0236s 2024-05-09T04:58:11.390 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0014s fsync time: 0.0175s 2024-05-09T04:58:11.390 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0238s 2024-05-09T04:58:11.391 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0009s fsync time: 0.0160s 2024-05-09T04:58:11.391 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0366s 2024-05-09T04:58:11.391 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0014s fsync time: 0.0353s 2024-05-09T04:58:11.391 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0243s 2024-05-09T04:58:11.391 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0233s 2024-05-09T04:58:11.391 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0010s fsync time: 0.0188s 2024-05-09T04:58:11.391 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0232s 2024-05-09T04:58:11.391 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0263s 2024-05-09T04:58:11.391 INFO:tasks.workunit.client.0.smithi006.stdout:write time: 0.0013s fsync time: 0.0260s 2024-05-09T04:58:11.391 INFO:tasks.workunit.client.0.smithi006.stdout:run done 60 fsyncs total, killing random writer 2024-05-09T04:58:11.391 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-05-09T04:58:11.392 INFO:tasks.workunit.client.0.smithi006.stdout:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-05-09T04:58:11.392 INFO:tasks.workunit.client.0.smithi006.stderr:+ whereis lsof 2024-05-09T04:58:11.459 INFO:tasks.workunit.client.0.smithi006.stdout:lsof: /usr/bin/lsof /usr/share/man/man1/lsof.1.gz 2024-05-09T04:58:11.459 INFO:tasks.workunit.client.0.smithi006.stderr:+ lsof 2024-05-09T04:58:11.479 INFO:tasks.workunit.client.0.smithi006.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/6348e3a836580359ded965ccca9c5ed8f431b9fb889efea942c3c25368e48089/merged 2024-05-09T04:58:11.480 INFO:tasks.workunit.client.0.smithi006.stderr: Output information may be incomplete. 2024-05-09T04:58:11.480 INFO:tasks.workunit.client.0.smithi006.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/7682fa9a25a21a009a11dcd956dff23541b57dc43a16dd45cdf2c3fc32a924a9/merged 2024-05-09T04:58:11.480 INFO:tasks.workunit.client.0.smithi006.stderr: Output information may be incomplete. 2024-05-09T04:58:11.487 INFO:tasks.workunit.client.0.smithi006.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/d8cc6174a1bff49d41794a2957f9da81d345fd1193f201e5098f10c6b1944d6d/merged 2024-05-09T04:58:11.487 INFO:tasks.workunit.client.0.smithi006.stderr: Output information may be incomplete. 2024-05-09T04:58:11.487 INFO:tasks.workunit.client.0.smithi006.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/71ef540917fd7849848b8d5a3106baecf1537bcfcf8f91aeece86cdbabfa6006/merged 2024-05-09T04:58:11.488 INFO:tasks.workunit.client.0.smithi006.stderr: Output information may be incomplete. 2024-05-09T04:58:11.488 INFO:tasks.workunit.client.0.smithi006.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/d5bd3a81624b5d2029f395d4ea08cf58a48ac78335629ba8b15d427be31df430/merged 2024-05-09T04:58:11.488 INFO:tasks.workunit.client.0.smithi006.stderr: Output information may be incomplete. 2024-05-09T04:58:11.488 INFO:tasks.workunit.client.0.smithi006.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/c9e5db8715bbbbd5752c6903193b81fc7fe47982d435bd33bf8747c71b78eb00/merged 2024-05-09T04:58:11.488 INFO:tasks.workunit.client.0.smithi006.stderr: Output information may be incomplete. 2024-05-09T04:58:11.488 INFO:tasks.workunit.client.0.smithi006.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/c0e4a559e6a893f0a43da0996b739677775e7d3de1e86a9afdacd71c101f8d8c/merged 2024-05-09T04:58:11.488 INFO:tasks.workunit.client.0.smithi006.stderr: Output information may be incomplete. 2024-05-09T04:58:11.488 INFO:tasks.workunit.client.0.smithi006.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/192f823c8d2d603f9adbc4403f42856732e5d9d9ed62e28f0c0ccde44bd95469/merged 2024-05-09T04:58:11.488 INFO:tasks.workunit.client.0.smithi006.stderr: Output information may be incomplete. 2024-05-09T04:58:11.488 INFO:tasks.workunit.client.0.smithi006.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/66e7f3e6a903bb665418c10b577236bd2e13a7c0a801152b0262c658553f0560/merged 2024-05-09T04:58:11.488 INFO:tasks.workunit.client.0.smithi006.stderr: Output information may be incomplete. 2024-05-09T04:58:11.488 INFO:tasks.workunit.client.0.smithi006.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/dd808f3ff55e23849fb0a58df1c323d71c716751ffc408dfd83921997aa2c98d/merged 2024-05-09T04:58:11.488 INFO:tasks.workunit.client.0.smithi006.stderr: Output information may be incomplete. 2024-05-09T04:58:11.635 INFO:tasks.workunit.client.0.smithi006.stdout:COMMAND PID TID TASKCMD USER FD TYPE DEVICE SIZE/OFF NODE NAME 2024-05-09T04:58:11.635 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 1 root cwd unknown /proc/1/cwd (readlink: Permission denied) 2024-05-09T04:58:11.635 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 1 root rtd unknown /proc/1/root (readlink: Permission denied) 2024-05-09T04:58:11.635 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 1 root txt unknown /proc/1/exe (readlink: Permission denied) 2024-05-09T04:58:11.635 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 1 root NOFD /proc/1/fd (opendir: Permission denied) 2024-05-09T04:58:11.636 INFO:tasks.workunit.client.0.smithi006.stdout:kthreadd 2 root cwd unknown /proc/2/cwd (readlink: Permission denied) 2024-05-09T04:58:11.636 INFO:tasks.workunit.client.0.smithi006.stdout:kthreadd 2 root rtd unknown /proc/2/root (readlink: Permission denied) 2024-05-09T04:58:11.636 INFO:tasks.workunit.client.0.smithi006.stdout:kthreadd 2 root txt unknown /proc/2/exe (readlink: Permission denied) 2024-05-09T04:58:11.636 INFO:tasks.workunit.client.0.smithi006.stdout:kthreadd 2 root NOFD /proc/2/fd (opendir: Permission denied) 2024-05-09T04:58:11.636 INFO:tasks.workunit.client.0.smithi006.stdout:pool_work 3 root cwd unknown /proc/3/cwd (readlink: Permission denied) 2024-05-09T04:58:11.636 INFO:tasks.workunit.client.0.smithi006.stdout:pool_work 3 root rtd unknown /proc/3/root (readlink: Permission denied) 2024-05-09T04:58:11.636 INFO:tasks.workunit.client.0.smithi006.stdout:pool_work 3 root txt unknown /proc/3/exe (readlink: Permission denied) 2024-05-09T04:58:11.636 INFO:tasks.workunit.client.0.smithi006.stdout:pool_work 3 root NOFD /proc/3/fd (opendir: Permission denied) 2024-05-09T04:58:11.636 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 4 root cwd unknown /proc/4/cwd (readlink: Permission denied) 2024-05-09T04:58:11.636 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 4 root rtd unknown /proc/4/root (readlink: Permission denied) 2024-05-09T04:58:11.636 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 4 root txt unknown /proc/4/exe (readlink: Permission denied) 2024-05-09T04:58:11.636 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 4 root NOFD /proc/4/fd (opendir: Permission denied) 2024-05-09T04:58:11.636 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 5 root cwd unknown /proc/5/cwd (readlink: Permission denied) 2024-05-09T04:58:11.636 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 5 root rtd unknown /proc/5/root (readlink: Permission denied) 2024-05-09T04:58:11.637 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 5 root txt unknown /proc/5/exe (readlink: Permission denied) 2024-05-09T04:58:11.637 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 5 root NOFD /proc/5/fd (opendir: Permission denied) 2024-05-09T04:58:11.637 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 6 root cwd unknown /proc/6/cwd (readlink: Permission denied) 2024-05-09T04:58:11.637 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 6 root rtd unknown /proc/6/root (readlink: Permission denied) 2024-05-09T04:58:11.637 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 6 root txt unknown /proc/6/exe (readlink: Permission denied) 2024-05-09T04:58:11.637 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 6 root NOFD /proc/6/fd (opendir: Permission denied) 2024-05-09T04:58:11.637 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/0 9 root cwd unknown /proc/9/cwd (readlink: Permission denied) 2024-05-09T04:58:11.637 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/0 9 root rtd unknown /proc/9/root (readlink: Permission denied) 2024-05-09T04:58:11.637 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/0 9 root txt unknown /proc/9/exe (readlink: Permission denied) 2024-05-09T04:58:11.637 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/0 9 root NOFD /proc/9/fd (opendir: Permission denied) 2024-05-09T04:58:11.637 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/u 10 root cwd unknown /proc/10/cwd (readlink: Permission denied) 2024-05-09T04:58:11.637 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/u 10 root rtd unknown /proc/10/root (readlink: Permission denied) 2024-05-09T04:58:11.637 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/u 10 root txt unknown /proc/10/exe (readlink: Permission denied) 2024-05-09T04:58:11.637 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/u 10 root NOFD /proc/10/fd (opendir: Permission denied) 2024-05-09T04:58:11.637 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 11 root cwd unknown /proc/11/cwd (readlink: Permission denied) 2024-05-09T04:58:11.638 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 11 root rtd unknown /proc/11/root (readlink: Permission denied) 2024-05-09T04:58:11.638 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 11 root txt unknown /proc/11/exe (readlink: Permission denied) 2024-05-09T04:58:11.638 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 11 root NOFD /proc/11/fd (opendir: Permission denied) 2024-05-09T04:58:11.638 INFO:tasks.workunit.client.0.smithi006.stdout:rcu_tasks 12 root cwd unknown /proc/12/cwd (readlink: Permission denied) 2024-05-09T04:58:11.638 INFO:tasks.workunit.client.0.smithi006.stdout:rcu_tasks 12 root rtd unknown /proc/12/root (readlink: Permission denied) 2024-05-09T04:58:11.638 INFO:tasks.workunit.client.0.smithi006.stdout:rcu_tasks 12 root txt unknown /proc/12/exe (readlink: Permission denied) 2024-05-09T04:58:11.638 INFO:tasks.workunit.client.0.smithi006.stdout:rcu_tasks 12 root NOFD /proc/12/fd (opendir: Permission denied) 2024-05-09T04:58:11.638 INFO:tasks.workunit.client.0.smithi006.stdout:rcu_tasks 13 root cwd unknown /proc/13/cwd (readlink: Permission denied) 2024-05-09T04:58:11.638 INFO:tasks.workunit.client.0.smithi006.stdout:rcu_tasks 13 root rtd unknown /proc/13/root (readlink: Permission denied) 2024-05-09T04:58:11.638 INFO:tasks.workunit.client.0.smithi006.stdout:rcu_tasks 13 root txt unknown /proc/13/exe (readlink: Permission denied) 2024-05-09T04:58:11.638 INFO:tasks.workunit.client.0.smithi006.stdout:rcu_tasks 13 root NOFD /proc/13/fd (opendir: Permission denied) 2024-05-09T04:58:11.638 INFO:tasks.workunit.client.0.smithi006.stdout:rcu_tasks 14 root cwd unknown /proc/14/cwd (readlink: Permission denied) 2024-05-09T04:58:11.638 INFO:tasks.workunit.client.0.smithi006.stdout:rcu_tasks 14 root rtd unknown /proc/14/root (readlink: Permission denied) 2024-05-09T04:58:11.638 INFO:tasks.workunit.client.0.smithi006.stdout:rcu_tasks 14 root txt unknown /proc/14/exe (readlink: Permission denied) 2024-05-09T04:58:11.638 INFO:tasks.workunit.client.0.smithi006.stdout:rcu_tasks 14 root NOFD /proc/14/fd (opendir: Permission denied) 2024-05-09T04:58:11.639 INFO:tasks.workunit.client.0.smithi006.stdout:ksoftirqd 15 root cwd unknown /proc/15/cwd (readlink: Permission denied) 2024-05-09T04:58:11.639 INFO:tasks.workunit.client.0.smithi006.stdout:ksoftirqd 15 root rtd unknown /proc/15/root (readlink: Permission denied) 2024-05-09T04:58:11.639 INFO:tasks.workunit.client.0.smithi006.stdout:ksoftirqd 15 root txt unknown /proc/15/exe (readlink: Permission denied) 2024-05-09T04:58:11.639 INFO:tasks.workunit.client.0.smithi006.stdout:ksoftirqd 15 root NOFD /proc/15/fd (opendir: Permission denied) 2024-05-09T04:58:11.639 INFO:tasks.workunit.client.0.smithi006.stdout:rcu_preem 16 root cwd unknown /proc/16/cwd (readlink: Permission denied) 2024-05-09T04:58:11.639 INFO:tasks.workunit.client.0.smithi006.stdout:rcu_preem 16 root rtd unknown /proc/16/root (readlink: Permission denied) 2024-05-09T04:58:11.639 INFO:tasks.workunit.client.0.smithi006.stdout:rcu_preem 16 root txt unknown /proc/16/exe (readlink: Permission denied) 2024-05-09T04:58:11.639 INFO:tasks.workunit.client.0.smithi006.stdout:rcu_preem 16 root NOFD /proc/16/fd (opendir: Permission denied) 2024-05-09T04:58:11.639 INFO:tasks.workunit.client.0.smithi006.stdout:rcu_exp_p 17 root cwd unknown /proc/17/cwd (readlink: Permission denied) 2024-05-09T04:58:11.639 INFO:tasks.workunit.client.0.smithi006.stdout:rcu_exp_p 17 root rtd unknown /proc/17/root (readlink: Permission denied) 2024-05-09T04:58:11.639 INFO:tasks.workunit.client.0.smithi006.stdout:rcu_exp_p 17 root txt unknown /proc/17/exe (readlink: Permission denied) 2024-05-09T04:58:11.639 INFO:tasks.workunit.client.0.smithi006.stdout:rcu_exp_p 17 root NOFD /proc/17/fd (opendir: Permission denied) 2024-05-09T04:58:11.639 INFO:tasks.workunit.client.0.smithi006.stdout:rcu_exp_g 18 root cwd unknown /proc/18/cwd (readlink: Permission denied) 2024-05-09T04:58:11.639 INFO:tasks.workunit.client.0.smithi006.stdout:rcu_exp_g 18 root rtd unknown /proc/18/root (readlink: Permission denied) 2024-05-09T04:58:11.639 INFO:tasks.workunit.client.0.smithi006.stdout:rcu_exp_g 18 root txt unknown /proc/18/exe (readlink: Permission denied) 2024-05-09T04:58:11.640 INFO:tasks.workunit.client.0.smithi006.stdout:rcu_exp_g 18 root NOFD /proc/18/fd (opendir: Permission denied) 2024-05-09T04:58:11.640 INFO:tasks.workunit.client.0.smithi006.stdout:migration 19 root cwd unknown /proc/19/cwd (readlink: Permission denied) 2024-05-09T04:58:11.640 INFO:tasks.workunit.client.0.smithi006.stdout:migration 19 root rtd unknown /proc/19/root (readlink: Permission denied) 2024-05-09T04:58:11.640 INFO:tasks.workunit.client.0.smithi006.stdout:migration 19 root txt unknown /proc/19/exe (readlink: Permission denied) 2024-05-09T04:58:11.640 INFO:tasks.workunit.client.0.smithi006.stdout:migration 19 root NOFD /proc/19/fd (opendir: Permission denied) 2024-05-09T04:58:11.640 INFO:tasks.workunit.client.0.smithi006.stdout:cpuhp/0 20 root cwd unknown /proc/20/cwd (readlink: Permission denied) 2024-05-09T04:58:11.640 INFO:tasks.workunit.client.0.smithi006.stdout:cpuhp/0 20 root rtd unknown /proc/20/root (readlink: Permission denied) 2024-05-09T04:58:11.640 INFO:tasks.workunit.client.0.smithi006.stdout:cpuhp/0 20 root txt unknown /proc/20/exe (readlink: Permission denied) 2024-05-09T04:58:11.640 INFO:tasks.workunit.client.0.smithi006.stdout:cpuhp/0 20 root NOFD /proc/20/fd (opendir: Permission denied) 2024-05-09T04:58:11.640 INFO:tasks.workunit.client.0.smithi006.stdout:cpuhp/1 21 root cwd unknown /proc/21/cwd (readlink: Permission denied) 2024-05-09T04:58:11.640 INFO:tasks.workunit.client.0.smithi006.stdout:cpuhp/1 21 root rtd unknown /proc/21/root (readlink: Permission denied) 2024-05-09T04:58:11.640 INFO:tasks.workunit.client.0.smithi006.stdout:cpuhp/1 21 root txt unknown /proc/21/exe (readlink: Permission denied) 2024-05-09T04:58:11.640 INFO:tasks.workunit.client.0.smithi006.stdout:cpuhp/1 21 root NOFD /proc/21/fd (opendir: Permission denied) 2024-05-09T04:58:11.640 INFO:tasks.workunit.client.0.smithi006.stdout:migration 22 root cwd unknown /proc/22/cwd (readlink: Permission denied) 2024-05-09T04:58:11.640 INFO:tasks.workunit.client.0.smithi006.stdout:migration 22 root rtd unknown /proc/22/root (readlink: Permission denied) 2024-05-09T04:58:11.641 INFO:tasks.workunit.client.0.smithi006.stdout:migration 22 root txt unknown /proc/22/exe (readlink: Permission denied) 2024-05-09T04:58:11.641 INFO:tasks.workunit.client.0.smithi006.stdout:migration 22 root NOFD /proc/22/fd (opendir: Permission denied) 2024-05-09T04:58:11.641 INFO:tasks.workunit.client.0.smithi006.stdout:ksoftirqd 23 root cwd unknown /proc/23/cwd (readlink: Permission denied) 2024-05-09T04:58:11.641 INFO:tasks.workunit.client.0.smithi006.stdout:ksoftirqd 23 root rtd unknown /proc/23/root (readlink: Permission denied) 2024-05-09T04:58:11.641 INFO:tasks.workunit.client.0.smithi006.stdout:ksoftirqd 23 root txt unknown /proc/23/exe (readlink: Permission denied) 2024-05-09T04:58:11.641 INFO:tasks.workunit.client.0.smithi006.stdout:ksoftirqd 23 root NOFD /proc/23/fd (opendir: Permission denied) 2024-05-09T04:58:11.641 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/1 25 root cwd unknown /proc/25/cwd (readlink: Permission denied) 2024-05-09T04:58:11.641 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/1 25 root rtd unknown /proc/25/root (readlink: Permission denied) 2024-05-09T04:58:11.641 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/1 25 root txt unknown /proc/25/exe (readlink: Permission denied) 2024-05-09T04:58:11.641 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/1 25 root NOFD /proc/25/fd (opendir: Permission denied) 2024-05-09T04:58:11.641 INFO:tasks.workunit.client.0.smithi006.stdout:cpuhp/2 26 root cwd unknown /proc/26/cwd (readlink: Permission denied) 2024-05-09T04:58:11.641 INFO:tasks.workunit.client.0.smithi006.stdout:cpuhp/2 26 root rtd unknown /proc/26/root (readlink: Permission denied) 2024-05-09T04:58:11.641 INFO:tasks.workunit.client.0.smithi006.stdout:cpuhp/2 26 root txt unknown /proc/26/exe (readlink: Permission denied) 2024-05-09T04:58:11.642 INFO:tasks.workunit.client.0.smithi006.stdout:cpuhp/2 26 root NOFD /proc/26/fd (opendir: Permission denied) 2024-05-09T04:58:11.642 INFO:tasks.workunit.client.0.smithi006.stdout:migration 27 root cwd unknown /proc/27/cwd (readlink: Permission denied) 2024-05-09T04:58:11.642 INFO:tasks.workunit.client.0.smithi006.stdout:migration 27 root rtd unknown /proc/27/root (readlink: Permission denied) 2024-05-09T04:58:11.642 INFO:tasks.workunit.client.0.smithi006.stdout:migration 27 root txt unknown /proc/27/exe (readlink: Permission denied) 2024-05-09T04:58:11.642 INFO:tasks.workunit.client.0.smithi006.stdout:migration 27 root NOFD /proc/27/fd (opendir: Permission denied) 2024-05-09T04:58:11.642 INFO:tasks.workunit.client.0.smithi006.stdout:ksoftirqd 28 root cwd unknown /proc/28/cwd (readlink: Permission denied) 2024-05-09T04:58:11.642 INFO:tasks.workunit.client.0.smithi006.stdout:ksoftirqd 28 root rtd unknown /proc/28/root (readlink: Permission denied) 2024-05-09T04:58:11.642 INFO:tasks.workunit.client.0.smithi006.stdout:ksoftirqd 28 root txt unknown /proc/28/exe (readlink: Permission denied) 2024-05-09T04:58:11.642 INFO:tasks.workunit.client.0.smithi006.stdout:ksoftirqd 28 root NOFD /proc/28/fd (opendir: Permission denied) 2024-05-09T04:58:11.642 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/2 30 root cwd unknown /proc/30/cwd (readlink: Permission denied) 2024-05-09T04:58:11.642 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/2 30 root rtd unknown /proc/30/root (readlink: Permission denied) 2024-05-09T04:58:11.642 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/2 30 root txt unknown /proc/30/exe (readlink: Permission denied) 2024-05-09T04:58:11.642 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/2 30 root NOFD /proc/30/fd (opendir: Permission denied) 2024-05-09T04:58:11.642 INFO:tasks.workunit.client.0.smithi006.stdout:cpuhp/3 31 root cwd unknown /proc/31/cwd (readlink: Permission denied) 2024-05-09T04:58:11.643 INFO:tasks.workunit.client.0.smithi006.stdout:cpuhp/3 31 root rtd unknown /proc/31/root (readlink: Permission denied) 2024-05-09T04:58:11.643 INFO:tasks.workunit.client.0.smithi006.stdout:cpuhp/3 31 root txt unknown /proc/31/exe (readlink: Permission denied) 2024-05-09T04:58:11.643 INFO:tasks.workunit.client.0.smithi006.stdout:cpuhp/3 31 root NOFD /proc/31/fd (opendir: Permission denied) 2024-05-09T04:58:11.643 INFO:tasks.workunit.client.0.smithi006.stdout:migration 32 root cwd unknown /proc/32/cwd (readlink: Permission denied) 2024-05-09T04:58:11.643 INFO:tasks.workunit.client.0.smithi006.stdout:migration 32 root rtd unknown /proc/32/root (readlink: Permission denied) 2024-05-09T04:58:11.643 INFO:tasks.workunit.client.0.smithi006.stdout:migration 32 root txt unknown /proc/32/exe (readlink: Permission denied) 2024-05-09T04:58:11.643 INFO:tasks.workunit.client.0.smithi006.stdout:migration 32 root NOFD /proc/32/fd (opendir: Permission denied) 2024-05-09T04:58:11.643 INFO:tasks.workunit.client.0.smithi006.stdout:ksoftirqd 33 root cwd unknown /proc/33/cwd (readlink: Permission denied) 2024-05-09T04:58:11.643 INFO:tasks.workunit.client.0.smithi006.stdout:ksoftirqd 33 root rtd unknown /proc/33/root (readlink: Permission denied) 2024-05-09T04:58:11.643 INFO:tasks.workunit.client.0.smithi006.stdout:ksoftirqd 33 root txt unknown /proc/33/exe (readlink: Permission denied) 2024-05-09T04:58:11.643 INFO:tasks.workunit.client.0.smithi006.stdout:ksoftirqd 33 root NOFD /proc/33/fd (opendir: Permission denied) 2024-05-09T04:58:11.643 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/3 35 root cwd unknown /proc/35/cwd (readlink: Permission denied) 2024-05-09T04:58:11.643 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/3 35 root rtd unknown /proc/35/root (readlink: Permission denied) 2024-05-09T04:58:11.644 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/3 35 root txt unknown /proc/35/exe (readlink: Permission denied) 2024-05-09T04:58:11.644 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/3 35 root NOFD /proc/35/fd (opendir: Permission denied) 2024-05-09T04:58:11.644 INFO:tasks.workunit.client.0.smithi006.stdout:cpuhp/4 36 root cwd unknown /proc/36/cwd (readlink: Permission denied) 2024-05-09T04:58:11.644 INFO:tasks.workunit.client.0.smithi006.stdout:cpuhp/4 36 root rtd unknown /proc/36/root (readlink: Permission denied) 2024-05-09T04:58:11.644 INFO:tasks.workunit.client.0.smithi006.stdout:cpuhp/4 36 root txt unknown /proc/36/exe (readlink: Permission denied) 2024-05-09T04:58:11.644 INFO:tasks.workunit.client.0.smithi006.stdout:cpuhp/4 36 root NOFD /proc/36/fd (opendir: Permission denied) 2024-05-09T04:58:11.644 INFO:tasks.workunit.client.0.smithi006.stdout:migration 37 root cwd unknown /proc/37/cwd (readlink: Permission denied) 2024-05-09T04:58:11.644 INFO:tasks.workunit.client.0.smithi006.stdout:migration 37 root rtd unknown /proc/37/root (readlink: Permission denied) 2024-05-09T04:58:11.644 INFO:tasks.workunit.client.0.smithi006.stdout:migration 37 root txt unknown /proc/37/exe (readlink: Permission denied) 2024-05-09T04:58:11.644 INFO:tasks.workunit.client.0.smithi006.stdout:migration 37 root NOFD /proc/37/fd (opendir: Permission denied) 2024-05-09T04:58:11.644 INFO:tasks.workunit.client.0.smithi006.stdout:ksoftirqd 38 root cwd unknown /proc/38/cwd (readlink: Permission denied) 2024-05-09T04:58:11.644 INFO:tasks.workunit.client.0.smithi006.stdout:ksoftirqd 38 root rtd unknown /proc/38/root (readlink: Permission denied) 2024-05-09T04:58:11.644 INFO:tasks.workunit.client.0.smithi006.stdout:ksoftirqd 38 root txt unknown /proc/38/exe (readlink: Permission denied) 2024-05-09T04:58:11.645 INFO:tasks.workunit.client.0.smithi006.stdout:ksoftirqd 38 root NOFD /proc/38/fd (opendir: Permission denied) 2024-05-09T04:58:11.645 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/4 40 root cwd unknown /proc/40/cwd (readlink: Permission denied) 2024-05-09T04:58:11.645 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/4 40 root rtd unknown /proc/40/root (readlink: Permission denied) 2024-05-09T04:58:11.645 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/4 40 root txt unknown /proc/40/exe (readlink: Permission denied) 2024-05-09T04:58:11.645 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/4 40 root NOFD /proc/40/fd (opendir: Permission denied) 2024-05-09T04:58:11.645 INFO:tasks.workunit.client.0.smithi006.stdout:cpuhp/5 41 root cwd unknown /proc/41/cwd (readlink: Permission denied) 2024-05-09T04:58:11.645 INFO:tasks.workunit.client.0.smithi006.stdout:cpuhp/5 41 root rtd unknown /proc/41/root (readlink: Permission denied) 2024-05-09T04:58:11.645 INFO:tasks.workunit.client.0.smithi006.stdout:cpuhp/5 41 root txt unknown /proc/41/exe (readlink: Permission denied) 2024-05-09T04:58:11.645 INFO:tasks.workunit.client.0.smithi006.stdout:cpuhp/5 41 root NOFD /proc/41/fd (opendir: Permission denied) 2024-05-09T04:58:11.645 INFO:tasks.workunit.client.0.smithi006.stdout:migration 42 root cwd unknown /proc/42/cwd (readlink: Permission denied) 2024-05-09T04:58:11.645 INFO:tasks.workunit.client.0.smithi006.stdout:migration 42 root rtd unknown /proc/42/root (readlink: Permission denied) 2024-05-09T04:58:11.645 INFO:tasks.workunit.client.0.smithi006.stdout:migration 42 root txt unknown /proc/42/exe (readlink: Permission denied) 2024-05-09T04:58:11.645 INFO:tasks.workunit.client.0.smithi006.stdout:migration 42 root NOFD /proc/42/fd (opendir: Permission denied) 2024-05-09T04:58:11.645 INFO:tasks.workunit.client.0.smithi006.stdout:ksoftirqd 43 root cwd unknown /proc/43/cwd (readlink: Permission denied) 2024-05-09T04:58:11.646 INFO:tasks.workunit.client.0.smithi006.stdout:ksoftirqd 43 root rtd unknown /proc/43/root (readlink: Permission denied) 2024-05-09T04:58:11.646 INFO:tasks.workunit.client.0.smithi006.stdout:ksoftirqd 43 root txt unknown /proc/43/exe (readlink: Permission denied) 2024-05-09T04:58:11.646 INFO:tasks.workunit.client.0.smithi006.stdout:ksoftirqd 43 root NOFD /proc/43/fd (opendir: Permission denied) 2024-05-09T04:58:11.646 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/5 45 root cwd unknown /proc/45/cwd (readlink: Permission denied) 2024-05-09T04:58:11.646 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/5 45 root rtd unknown /proc/45/root (readlink: Permission denied) 2024-05-09T04:58:11.646 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/5 45 root txt unknown /proc/45/exe (readlink: Permission denied) 2024-05-09T04:58:11.646 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/5 45 root NOFD /proc/45/fd (opendir: Permission denied) 2024-05-09T04:58:11.646 INFO:tasks.workunit.client.0.smithi006.stdout:cpuhp/6 46 root cwd unknown /proc/46/cwd (readlink: Permission denied) 2024-05-09T04:58:11.646 INFO:tasks.workunit.client.0.smithi006.stdout:cpuhp/6 46 root rtd unknown /proc/46/root (readlink: Permission denied) 2024-05-09T04:58:11.646 INFO:tasks.workunit.client.0.smithi006.stdout:cpuhp/6 46 root txt unknown /proc/46/exe (readlink: Permission denied) 2024-05-09T04:58:11.646 INFO:tasks.workunit.client.0.smithi006.stdout:cpuhp/6 46 root NOFD /proc/46/fd (opendir: Permission denied) 2024-05-09T04:58:11.646 INFO:tasks.workunit.client.0.smithi006.stdout:migration 47 root cwd unknown /proc/47/cwd (readlink: Permission denied) 2024-05-09T04:58:11.646 INFO:tasks.workunit.client.0.smithi006.stdout:migration 47 root rtd unknown /proc/47/root (readlink: Permission denied) 2024-05-09T04:58:11.647 INFO:tasks.workunit.client.0.smithi006.stdout:migration 47 root txt unknown /proc/47/exe (readlink: Permission denied) 2024-05-09T04:58:11.647 INFO:tasks.workunit.client.0.smithi006.stdout:migration 47 root NOFD /proc/47/fd (opendir: Permission denied) 2024-05-09T04:58:11.647 INFO:tasks.workunit.client.0.smithi006.stdout:ksoftirqd 48 root cwd unknown /proc/48/cwd (readlink: Permission denied) 2024-05-09T04:58:11.647 INFO:tasks.workunit.client.0.smithi006.stdout:ksoftirqd 48 root rtd unknown /proc/48/root (readlink: Permission denied) 2024-05-09T04:58:11.647 INFO:tasks.workunit.client.0.smithi006.stdout:ksoftirqd 48 root txt unknown /proc/48/exe (readlink: Permission denied) 2024-05-09T04:58:11.647 INFO:tasks.workunit.client.0.smithi006.stdout:ksoftirqd 48 root NOFD /proc/48/fd (opendir: Permission denied) 2024-05-09T04:58:11.647 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/6 50 root cwd unknown /proc/50/cwd (readlink: Permission denied) 2024-05-09T04:58:11.647 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/6 50 root rtd unknown /proc/50/root (readlink: Permission denied) 2024-05-09T04:58:11.647 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/6 50 root txt unknown /proc/50/exe (readlink: Permission denied) 2024-05-09T04:58:11.647 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/6 50 root NOFD /proc/50/fd (opendir: Permission denied) 2024-05-09T04:58:11.647 INFO:tasks.workunit.client.0.smithi006.stdout:cpuhp/7 51 root cwd unknown /proc/51/cwd (readlink: Permission denied) 2024-05-09T04:58:11.647 INFO:tasks.workunit.client.0.smithi006.stdout:cpuhp/7 51 root rtd unknown /proc/51/root (readlink: Permission denied) 2024-05-09T04:58:11.648 INFO:tasks.workunit.client.0.smithi006.stdout:cpuhp/7 51 root txt unknown /proc/51/exe (readlink: Permission denied) 2024-05-09T04:58:11.648 INFO:tasks.workunit.client.0.smithi006.stdout:cpuhp/7 51 root NOFD /proc/51/fd (opendir: Permission denied) 2024-05-09T04:58:11.648 INFO:tasks.workunit.client.0.smithi006.stdout:migration 52 root cwd unknown /proc/52/cwd (readlink: Permission denied) 2024-05-09T04:58:11.648 INFO:tasks.workunit.client.0.smithi006.stdout:migration 52 root rtd unknown /proc/52/root (readlink: Permission denied) 2024-05-09T04:58:11.648 INFO:tasks.workunit.client.0.smithi006.stdout:migration 52 root txt unknown /proc/52/exe (readlink: Permission denied) 2024-05-09T04:58:11.648 INFO:tasks.workunit.client.0.smithi006.stdout:migration 52 root NOFD /proc/52/fd (opendir: Permission denied) 2024-05-09T04:58:11.648 INFO:tasks.workunit.client.0.smithi006.stdout:ksoftirqd 53 root cwd unknown /proc/53/cwd (readlink: Permission denied) 2024-05-09T04:58:11.648 INFO:tasks.workunit.client.0.smithi006.stdout:ksoftirqd 53 root rtd unknown /proc/53/root (readlink: Permission denied) 2024-05-09T04:58:11.648 INFO:tasks.workunit.client.0.smithi006.stdout:ksoftirqd 53 root txt unknown /proc/53/exe (readlink: Permission denied) 2024-05-09T04:58:11.648 INFO:tasks.workunit.client.0.smithi006.stdout:ksoftirqd 53 root NOFD /proc/53/fd (opendir: Permission denied) 2024-05-09T04:58:11.648 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/7 54 root cwd unknown /proc/54/cwd (readlink: Permission denied) 2024-05-09T04:58:11.648 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/7 54 root rtd unknown /proc/54/root (readlink: Permission denied) 2024-05-09T04:58:11.648 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/7 54 root txt unknown /proc/54/exe (readlink: Permission denied) 2024-05-09T04:58:11.648 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/7 54 root NOFD /proc/54/fd (opendir: Permission denied) 2024-05-09T04:58:11.649 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/7 55 root cwd unknown /proc/55/cwd (readlink: Permission denied) 2024-05-09T04:58:11.649 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/7 55 root rtd unknown /proc/55/root (readlink: Permission denied) 2024-05-09T04:58:11.649 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/7 55 root txt unknown /proc/55/exe (readlink: Permission denied) 2024-05-09T04:58:11.649 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/7 55 root NOFD /proc/55/fd (opendir: Permission denied) 2024-05-09T04:58:11.649 INFO:tasks.workunit.client.0.smithi006.stdout:kdevtmpfs 56 root cwd unknown /proc/56/cwd (readlink: Permission denied) 2024-05-09T04:58:11.649 INFO:tasks.workunit.client.0.smithi006.stdout:kdevtmpfs 56 root rtd unknown /proc/56/root (readlink: Permission denied) 2024-05-09T04:58:11.649 INFO:tasks.workunit.client.0.smithi006.stdout:kdevtmpfs 56 root txt unknown /proc/56/exe (readlink: Permission denied) 2024-05-09T04:58:11.649 INFO:tasks.workunit.client.0.smithi006.stdout:kdevtmpfs 56 root NOFD /proc/56/fd (opendir: Permission denied) 2024-05-09T04:58:11.649 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 57 root cwd unknown /proc/57/cwd (readlink: Permission denied) 2024-05-09T04:58:11.649 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 57 root rtd unknown /proc/57/root (readlink: Permission denied) 2024-05-09T04:58:11.649 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 57 root txt unknown /proc/57/exe (readlink: Permission denied) 2024-05-09T04:58:11.649 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 57 root NOFD /proc/57/fd (opendir: Permission denied) 2024-05-09T04:58:11.649 INFO:tasks.workunit.client.0.smithi006.stdout:kauditd 58 root cwd unknown /proc/58/cwd (readlink: Permission denied) 2024-05-09T04:58:11.650 INFO:tasks.workunit.client.0.smithi006.stdout:kauditd 58 root rtd unknown /proc/58/root (readlink: Permission denied) 2024-05-09T04:58:11.650 INFO:tasks.workunit.client.0.smithi006.stdout:kauditd 58 root txt unknown /proc/58/exe (readlink: Permission denied) 2024-05-09T04:58:11.650 INFO:tasks.workunit.client.0.smithi006.stdout:kauditd 58 root NOFD /proc/58/fd (opendir: Permission denied) 2024-05-09T04:58:11.650 INFO:tasks.workunit.client.0.smithi006.stdout:khungtask 59 root cwd unknown /proc/59/cwd (readlink: Permission denied) 2024-05-09T04:58:11.650 INFO:tasks.workunit.client.0.smithi006.stdout:khungtask 59 root rtd unknown /proc/59/root (readlink: Permission denied) 2024-05-09T04:58:11.650 INFO:tasks.workunit.client.0.smithi006.stdout:khungtask 59 root txt unknown /proc/59/exe (readlink: Permission denied) 2024-05-09T04:58:11.650 INFO:tasks.workunit.client.0.smithi006.stdout:khungtask 59 root NOFD /proc/59/fd (opendir: Permission denied) 2024-05-09T04:58:11.650 INFO:tasks.workunit.client.0.smithi006.stdout:oom_reape 61 root cwd unknown /proc/61/cwd (readlink: Permission denied) 2024-05-09T04:58:11.650 INFO:tasks.workunit.client.0.smithi006.stdout:oom_reape 61 root rtd unknown /proc/61/root (readlink: Permission denied) 2024-05-09T04:58:11.650 INFO:tasks.workunit.client.0.smithi006.stdout:oom_reape 61 root txt unknown /proc/61/exe (readlink: Permission denied) 2024-05-09T04:58:11.650 INFO:tasks.workunit.client.0.smithi006.stdout:oom_reape 61 root NOFD /proc/61/fd (opendir: Permission denied) 2024-05-09T04:58:11.650 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 63 root cwd unknown /proc/63/cwd (readlink: Permission denied) 2024-05-09T04:58:11.650 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 63 root rtd unknown /proc/63/root (readlink: Permission denied) 2024-05-09T04:58:11.650 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 63 root txt unknown /proc/63/exe (readlink: Permission denied) 2024-05-09T04:58:11.651 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 63 root NOFD /proc/63/fd (opendir: Permission denied) 2024-05-09T04:58:11.651 INFO:tasks.workunit.client.0.smithi006.stdout:kcompactd 64 root cwd unknown /proc/64/cwd (readlink: Permission denied) 2024-05-09T04:58:11.651 INFO:tasks.workunit.client.0.smithi006.stdout:kcompactd 64 root rtd unknown /proc/64/root (readlink: Permission denied) 2024-05-09T04:58:11.651 INFO:tasks.workunit.client.0.smithi006.stdout:kcompactd 64 root txt unknown /proc/64/exe (readlink: Permission denied) 2024-05-09T04:58:11.651 INFO:tasks.workunit.client.0.smithi006.stdout:kcompactd 64 root NOFD /proc/64/fd (opendir: Permission denied) 2024-05-09T04:58:11.651 INFO:tasks.workunit.client.0.smithi006.stdout:ksmd 65 root cwd unknown /proc/65/cwd (readlink: Permission denied) 2024-05-09T04:58:11.651 INFO:tasks.workunit.client.0.smithi006.stdout:ksmd 65 root rtd unknown /proc/65/root (readlink: Permission denied) 2024-05-09T04:58:11.651 INFO:tasks.workunit.client.0.smithi006.stdout:ksmd 65 root txt unknown /proc/65/exe (readlink: Permission denied) 2024-05-09T04:58:11.651 INFO:tasks.workunit.client.0.smithi006.stdout:ksmd 65 root NOFD /proc/65/fd (opendir: Permission denied) 2024-05-09T04:58:11.651 INFO:tasks.workunit.client.0.smithi006.stdout:khugepage 66 root cwd unknown /proc/66/cwd (readlink: Permission denied) 2024-05-09T04:58:11.651 INFO:tasks.workunit.client.0.smithi006.stdout:khugepage 66 root rtd unknown /proc/66/root (readlink: Permission denied) 2024-05-09T04:58:11.651 INFO:tasks.workunit.client.0.smithi006.stdout:khugepage 66 root txt unknown /proc/66/exe (readlink: Permission denied) 2024-05-09T04:58:11.651 INFO:tasks.workunit.client.0.smithi006.stdout:khugepage 66 root NOFD /proc/66/fd (opendir: Permission denied) 2024-05-09T04:58:11.652 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 67 root cwd unknown /proc/67/cwd (readlink: Permission denied) 2024-05-09T04:58:11.652 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 67 root rtd unknown /proc/67/root (readlink: Permission denied) 2024-05-09T04:58:11.652 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 67 root txt unknown /proc/67/exe (readlink: Permission denied) 2024-05-09T04:58:11.652 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 67 root NOFD /proc/67/fd (opendir: Permission denied) 2024-05-09T04:58:11.652 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 68 root cwd unknown /proc/68/cwd (readlink: Permission denied) 2024-05-09T04:58:11.652 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 68 root rtd unknown /proc/68/root (readlink: Permission denied) 2024-05-09T04:58:11.652 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 68 root txt unknown /proc/68/exe (readlink: Permission denied) 2024-05-09T04:58:11.652 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 68 root NOFD /proc/68/fd (opendir: Permission denied) 2024-05-09T04:58:11.652 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 69 root cwd unknown /proc/69/cwd (readlink: Permission denied) 2024-05-09T04:58:11.652 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 69 root rtd unknown /proc/69/root (readlink: Permission denied) 2024-05-09T04:58:11.652 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 69 root txt unknown /proc/69/exe (readlink: Permission denied) 2024-05-09T04:58:11.652 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 69 root NOFD /proc/69/fd (opendir: Permission denied) 2024-05-09T04:58:11.652 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 70 root cwd unknown /proc/70/cwd (readlink: Permission denied) 2024-05-09T04:58:11.652 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 70 root rtd unknown /proc/70/root (readlink: Permission denied) 2024-05-09T04:58:11.653 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 70 root txt unknown /proc/70/exe (readlink: Permission denied) 2024-05-09T04:58:11.653 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 70 root NOFD /proc/70/fd (opendir: Permission denied) 2024-05-09T04:58:11.653 INFO:tasks.workunit.client.0.smithi006.stdout:irq/9-acp 78 root cwd unknown /proc/78/cwd (readlink: Permission denied) 2024-05-09T04:58:11.653 INFO:tasks.workunit.client.0.smithi006.stdout:irq/9-acp 78 root rtd unknown /proc/78/root (readlink: Permission denied) 2024-05-09T04:58:11.653 INFO:tasks.workunit.client.0.smithi006.stdout:irq/9-acp 78 root txt unknown /proc/78/exe (readlink: Permission denied) 2024-05-09T04:58:11.653 INFO:tasks.workunit.client.0.smithi006.stdout:irq/9-acp 78 root NOFD /proc/78/fd (opendir: Permission denied) 2024-05-09T04:58:11.653 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 79 root cwd unknown /proc/79/cwd (readlink: Permission denied) 2024-05-09T04:58:11.653 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 79 root rtd unknown /proc/79/root (readlink: Permission denied) 2024-05-09T04:58:11.653 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 79 root txt unknown /proc/79/exe (readlink: Permission denied) 2024-05-09T04:58:11.653 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 79 root NOFD /proc/79/fd (opendir: Permission denied) 2024-05-09T04:58:11.653 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 80 root cwd unknown /proc/80/cwd (readlink: Permission denied) 2024-05-09T04:58:11.653 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 80 root rtd unknown /proc/80/root (readlink: Permission denied) 2024-05-09T04:58:11.653 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 80 root txt unknown /proc/80/exe (readlink: Permission denied) 2024-05-09T04:58:11.654 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 80 root NOFD /proc/80/fd (opendir: Permission denied) 2024-05-09T04:58:11.654 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 81 root cwd unknown /proc/81/cwd (readlink: Permission denied) 2024-05-09T04:58:11.654 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 81 root rtd unknown /proc/81/root (readlink: Permission denied) 2024-05-09T04:58:11.654 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 81 root txt unknown /proc/81/exe (readlink: Permission denied) 2024-05-09T04:58:11.654 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 81 root NOFD /proc/81/fd (opendir: Permission denied) 2024-05-09T04:58:11.654 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 82 root cwd unknown /proc/82/cwd (readlink: Permission denied) 2024-05-09T04:58:11.654 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 82 root rtd unknown /proc/82/root (readlink: Permission denied) 2024-05-09T04:58:11.654 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 82 root txt unknown /proc/82/exe (readlink: Permission denied) 2024-05-09T04:58:11.654 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 82 root NOFD /proc/82/fd (opendir: Permission denied) 2024-05-09T04:58:11.654 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 83 root cwd unknown /proc/83/cwd (readlink: Permission denied) 2024-05-09T04:58:11.654 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 83 root rtd unknown /proc/83/root (readlink: Permission denied) 2024-05-09T04:58:11.654 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 83 root txt unknown /proc/83/exe (readlink: Permission denied) 2024-05-09T04:58:11.654 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 83 root NOFD /proc/83/fd (opendir: Permission denied) 2024-05-09T04:58:11.654 INFO:tasks.workunit.client.0.smithi006.stdout:watchdogd 84 root cwd unknown /proc/84/cwd (readlink: Permission denied) 2024-05-09T04:58:11.655 INFO:tasks.workunit.client.0.smithi006.stdout:watchdogd 84 root rtd unknown /proc/84/root (readlink: Permission denied) 2024-05-09T04:58:11.655 INFO:tasks.workunit.client.0.smithi006.stdout:watchdogd 84 root txt unknown /proc/84/exe (readlink: Permission denied) 2024-05-09T04:58:11.655 INFO:tasks.workunit.client.0.smithi006.stdout:watchdogd 84 root NOFD /proc/84/fd (opendir: Permission denied) 2024-05-09T04:58:11.655 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/0 85 root cwd unknown /proc/85/cwd (readlink: Permission denied) 2024-05-09T04:58:11.655 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/0 85 root rtd unknown /proc/85/root (readlink: Permission denied) 2024-05-09T04:58:11.655 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/0 85 root txt unknown /proc/85/exe (readlink: Permission denied) 2024-05-09T04:58:11.655 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/0 85 root NOFD /proc/85/fd (opendir: Permission denied) 2024-05-09T04:58:11.655 INFO:tasks.workunit.client.0.smithi006.stdout:kswapd0 86 root cwd unknown /proc/86/cwd (readlink: Permission denied) 2024-05-09T04:58:11.655 INFO:tasks.workunit.client.0.smithi006.stdout:kswapd0 86 root rtd unknown /proc/86/root (readlink: Permission denied) 2024-05-09T04:58:11.655 INFO:tasks.workunit.client.0.smithi006.stdout:kswapd0 86 root txt unknown /proc/86/exe (readlink: Permission denied) 2024-05-09T04:58:11.655 INFO:tasks.workunit.client.0.smithi006.stdout:kswapd0 86 root NOFD /proc/86/fd (opendir: Permission denied) 2024-05-09T04:58:11.655 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 88 root cwd unknown /proc/88/cwd (readlink: Permission denied) 2024-05-09T04:58:11.655 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 88 root rtd unknown /proc/88/root (readlink: Permission denied) 2024-05-09T04:58:11.656 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 88 root txt unknown /proc/88/exe (readlink: Permission denied) 2024-05-09T04:58:11.656 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 88 root NOFD /proc/88/fd (opendir: Permission denied) 2024-05-09T04:58:11.656 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 91 root cwd unknown /proc/91/cwd (readlink: Permission denied) 2024-05-09T04:58:11.656 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 91 root rtd unknown /proc/91/root (readlink: Permission denied) 2024-05-09T04:58:11.656 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 91 root txt unknown /proc/91/exe (readlink: Permission denied) 2024-05-09T04:58:11.656 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 91 root NOFD /proc/91/fd (opendir: Permission denied) 2024-05-09T04:58:11.656 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/6 92 root cwd unknown /proc/92/cwd (readlink: Permission denied) 2024-05-09T04:58:11.656 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/6 92 root rtd unknown /proc/92/root (readlink: Permission denied) 2024-05-09T04:58:11.656 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/6 92 root txt unknown /proc/92/exe (readlink: Permission denied) 2024-05-09T04:58:11.656 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/6 92 root NOFD /proc/92/fd (opendir: Permission denied) 2024-05-09T04:58:11.656 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_0 93 root cwd unknown /proc/93/cwd (readlink: Permission denied) 2024-05-09T04:58:11.656 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_0 93 root rtd unknown /proc/93/root (readlink: Permission denied) 2024-05-09T04:58:11.656 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_0 93 root txt unknown /proc/93/exe (readlink: Permission denied) 2024-05-09T04:58:11.657 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_0 93 root NOFD /proc/93/fd (opendir: Permission denied) 2024-05-09T04:58:11.657 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 94 root cwd unknown /proc/94/cwd (readlink: Permission denied) 2024-05-09T04:58:11.657 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 94 root rtd unknown /proc/94/root (readlink: Permission denied) 2024-05-09T04:58:11.657 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 94 root txt unknown /proc/94/exe (readlink: Permission denied) 2024-05-09T04:58:11.657 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 94 root NOFD /proc/94/fd (opendir: Permission denied) 2024-05-09T04:58:11.657 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_1 95 root cwd unknown /proc/95/cwd (readlink: Permission denied) 2024-05-09T04:58:11.657 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_1 95 root rtd unknown /proc/95/root (readlink: Permission denied) 2024-05-09T04:58:11.657 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_1 95 root txt unknown /proc/95/exe (readlink: Permission denied) 2024-05-09T04:58:11.657 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_1 95 root NOFD /proc/95/fd (opendir: Permission denied) 2024-05-09T04:58:11.657 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 96 root cwd unknown /proc/96/cwd (readlink: Permission denied) 2024-05-09T04:58:11.657 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 96 root rtd unknown /proc/96/root (readlink: Permission denied) 2024-05-09T04:58:11.657 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 96 root txt unknown /proc/96/exe (readlink: Permission denied) 2024-05-09T04:58:11.657 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 96 root NOFD /proc/96/fd (opendir: Permission denied) 2024-05-09T04:58:11.657 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_2 97 root cwd unknown /proc/97/cwd (readlink: Permission denied) 2024-05-09T04:58:11.657 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_2 97 root rtd unknown /proc/97/root (readlink: Permission denied) 2024-05-09T04:58:11.658 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_2 97 root txt unknown /proc/97/exe (readlink: Permission denied) 2024-05-09T04:58:11.658 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_2 97 root NOFD /proc/97/fd (opendir: Permission denied) 2024-05-09T04:58:11.658 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 98 root cwd unknown /proc/98/cwd (readlink: Permission denied) 2024-05-09T04:58:11.658 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 98 root rtd unknown /proc/98/root (readlink: Permission denied) 2024-05-09T04:58:11.658 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 98 root txt unknown /proc/98/exe (readlink: Permission denied) 2024-05-09T04:58:11.658 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 98 root NOFD /proc/98/fd (opendir: Permission denied) 2024-05-09T04:58:11.658 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_3 99 root cwd unknown /proc/99/cwd (readlink: Permission denied) 2024-05-09T04:58:11.658 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_3 99 root rtd unknown /proc/99/root (readlink: Permission denied) 2024-05-09T04:58:11.658 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_3 99 root txt unknown /proc/99/exe (readlink: Permission denied) 2024-05-09T04:58:11.658 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_3 99 root NOFD /proc/99/fd (opendir: Permission denied) 2024-05-09T04:58:11.658 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 100 root cwd unknown /proc/100/cwd (readlink: Permission denied) 2024-05-09T04:58:11.658 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 100 root rtd unknown /proc/100/root (readlink: Permission denied) 2024-05-09T04:58:11.658 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 100 root txt unknown /proc/100/exe (readlink: Permission denied) 2024-05-09T04:58:11.658 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 100 root NOFD /proc/100/fd (opendir: Permission denied) 2024-05-09T04:58:11.659 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/u 103 root cwd unknown /proc/103/cwd (readlink: Permission denied) 2024-05-09T04:58:11.659 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/u 103 root rtd unknown /proc/103/root (readlink: Permission denied) 2024-05-09T04:58:11.659 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/u 103 root txt unknown /proc/103/exe (readlink: Permission denied) 2024-05-09T04:58:11.659 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/u 103 root NOFD /proc/103/fd (opendir: Permission denied) 2024-05-09T04:58:11.659 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_4 104 root cwd unknown /proc/104/cwd (readlink: Permission denied) 2024-05-09T04:58:11.659 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_4 104 root rtd unknown /proc/104/root (readlink: Permission denied) 2024-05-09T04:58:11.659 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_4 104 root txt unknown /proc/104/exe (readlink: Permission denied) 2024-05-09T04:58:11.659 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_4 104 root NOFD /proc/104/fd (opendir: Permission denied) 2024-05-09T04:58:11.659 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 105 root cwd unknown /proc/105/cwd (readlink: Permission denied) 2024-05-09T04:58:11.659 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 105 root rtd unknown /proc/105/root (readlink: Permission denied) 2024-05-09T04:58:11.659 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 105 root txt unknown /proc/105/exe (readlink: Permission denied) 2024-05-09T04:58:11.659 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 105 root NOFD /proc/105/fd (opendir: Permission denied) 2024-05-09T04:58:11.659 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_5 106 root cwd unknown /proc/106/cwd (readlink: Permission denied) 2024-05-09T04:58:11.659 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_5 106 root rtd unknown /proc/106/root (readlink: Permission denied) 2024-05-09T04:58:11.659 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_5 106 root txt unknown /proc/106/exe (readlink: Permission denied) 2024-05-09T04:58:11.660 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_5 106 root NOFD /proc/106/fd (opendir: Permission denied) 2024-05-09T04:58:11.660 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 107 root cwd unknown /proc/107/cwd (readlink: Permission denied) 2024-05-09T04:58:11.660 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 107 root rtd unknown /proc/107/root (readlink: Permission denied) 2024-05-09T04:58:11.660 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 107 root txt unknown /proc/107/exe (readlink: Permission denied) 2024-05-09T04:58:11.660 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 107 root NOFD /proc/107/fd (opendir: Permission denied) 2024-05-09T04:58:11.660 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_6 108 root cwd unknown /proc/108/cwd (readlink: Permission denied) 2024-05-09T04:58:11.660 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_6 108 root rtd unknown /proc/108/root (readlink: Permission denied) 2024-05-09T04:58:11.660 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_6 108 root txt unknown /proc/108/exe (readlink: Permission denied) 2024-05-09T04:58:11.660 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_6 108 root NOFD /proc/108/fd (opendir: Permission denied) 2024-05-09T04:58:11.660 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 109 root cwd unknown /proc/109/cwd (readlink: Permission denied) 2024-05-09T04:58:11.660 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 109 root rtd unknown /proc/109/root (readlink: Permission denied) 2024-05-09T04:58:11.660 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 109 root txt unknown /proc/109/exe (readlink: Permission denied) 2024-05-09T04:58:11.660 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 109 root NOFD /proc/109/fd (opendir: Permission denied) 2024-05-09T04:58:11.660 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_7 110 root cwd unknown /proc/110/cwd (readlink: Permission denied) 2024-05-09T04:58:11.660 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_7 110 root rtd unknown /proc/110/root (readlink: Permission denied) 2024-05-09T04:58:11.660 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_7 110 root txt unknown /proc/110/exe (readlink: Permission denied) 2024-05-09T04:58:11.661 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_7 110 root NOFD /proc/110/fd (opendir: Permission denied) 2024-05-09T04:58:11.661 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 111 root cwd unknown /proc/111/cwd (readlink: Permission denied) 2024-05-09T04:58:11.661 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 111 root rtd unknown /proc/111/root (readlink: Permission denied) 2024-05-09T04:58:11.661 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 111 root txt unknown /proc/111/exe (readlink: Permission denied) 2024-05-09T04:58:11.661 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 111 root NOFD /proc/111/fd (opendir: Permission denied) 2024-05-09T04:58:11.661 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_8 112 root cwd unknown /proc/112/cwd (readlink: Permission denied) 2024-05-09T04:58:11.661 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_8 112 root rtd unknown /proc/112/root (readlink: Permission denied) 2024-05-09T04:58:11.661 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_8 112 root txt unknown /proc/112/exe (readlink: Permission denied) 2024-05-09T04:58:11.661 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_8 112 root NOFD /proc/112/fd (opendir: Permission denied) 2024-05-09T04:58:11.661 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 113 root cwd unknown /proc/113/cwd (readlink: Permission denied) 2024-05-09T04:58:11.661 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 113 root rtd unknown /proc/113/root (readlink: Permission denied) 2024-05-09T04:58:11.661 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 113 root txt unknown /proc/113/exe (readlink: Permission denied) 2024-05-09T04:58:11.661 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 113 root NOFD /proc/113/fd (opendir: Permission denied) 2024-05-09T04:58:11.661 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_9 114 root cwd unknown /proc/114/cwd (readlink: Permission denied) 2024-05-09T04:58:11.662 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_9 114 root rtd unknown /proc/114/root (readlink: Permission denied) 2024-05-09T04:58:11.662 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_9 114 root txt unknown /proc/114/exe (readlink: Permission denied) 2024-05-09T04:58:11.662 INFO:tasks.workunit.client.0.smithi006.stdout:scsi_eh_9 114 root NOFD /proc/114/fd (opendir: Permission denied) 2024-05-09T04:58:11.662 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 115 root cwd unknown /proc/115/cwd (readlink: Permission denied) 2024-05-09T04:58:11.662 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 115 root rtd unknown /proc/115/root (readlink: Permission denied) 2024-05-09T04:58:11.662 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 115 root txt unknown /proc/115/exe (readlink: Permission denied) 2024-05-09T04:58:11.662 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 115 root NOFD /proc/115/fd (opendir: Permission denied) 2024-05-09T04:58:11.662 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/u 117 root cwd unknown /proc/117/cwd (readlink: Permission denied) 2024-05-09T04:58:11.662 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/u 117 root rtd unknown /proc/117/root (readlink: Permission denied) 2024-05-09T04:58:11.662 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/u 117 root txt unknown /proc/117/exe (readlink: Permission denied) 2024-05-09T04:58:11.662 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/u 117 root NOFD /proc/117/fd (opendir: Permission denied) 2024-05-09T04:58:11.662 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/u 118 root cwd unknown /proc/118/cwd (readlink: Permission denied) 2024-05-09T04:58:11.662 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/u 118 root rtd unknown /proc/118/root (readlink: Permission denied) 2024-05-09T04:58:11.662 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/u 118 root txt unknown /proc/118/exe (readlink: Permission denied) 2024-05-09T04:58:11.662 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/u 118 root NOFD /proc/118/fd (opendir: Permission denied) 2024-05-09T04:58:11.663 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 128 root cwd unknown /proc/128/cwd (readlink: Permission denied) 2024-05-09T04:58:11.663 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 128 root rtd unknown /proc/128/root (readlink: Permission denied) 2024-05-09T04:58:11.663 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 128 root txt unknown /proc/128/exe (readlink: Permission denied) 2024-05-09T04:58:11.663 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 128 root NOFD /proc/128/fd (opendir: Permission denied) 2024-05-09T04:58:11.663 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/2 129 root cwd unknown /proc/129/cwd (readlink: Permission denied) 2024-05-09T04:58:11.663 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/2 129 root rtd unknown /proc/129/root (readlink: Permission denied) 2024-05-09T04:58:11.663 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/2 129 root txt unknown /proc/129/exe (readlink: Permission denied) 2024-05-09T04:58:11.663 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/2 129 root NOFD /proc/129/fd (opendir: Permission denied) 2024-05-09T04:58:11.663 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 130 root cwd unknown /proc/130/cwd (readlink: Permission denied) 2024-05-09T04:58:11.663 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 130 root rtd unknown /proc/130/root (readlink: Permission denied) 2024-05-09T04:58:11.663 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 130 root txt unknown /proc/130/exe (readlink: Permission denied) 2024-05-09T04:58:11.663 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 130 root NOFD /proc/130/fd (opendir: Permission denied) 2024-05-09T04:58:11.663 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 131 root cwd unknown /proc/131/cwd (readlink: Permission denied) 2024-05-09T04:58:11.663 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 131 root rtd unknown /proc/131/root (readlink: Permission denied) 2024-05-09T04:58:11.664 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 131 root txt unknown /proc/131/exe (readlink: Permission denied) 2024-05-09T04:58:11.664 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 131 root NOFD /proc/131/fd (opendir: Permission denied) 2024-05-09T04:58:11.664 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/u 139 root cwd unknown /proc/139/cwd (readlink: Permission denied) 2024-05-09T04:58:11.664 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/u 139 root rtd unknown /proc/139/root (readlink: Permission denied) 2024-05-09T04:58:11.664 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/u 139 root txt unknown /proc/139/exe (readlink: Permission denied) 2024-05-09T04:58:11.664 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/u 139 root NOFD /proc/139/fd (opendir: Permission denied) 2024-05-09T04:58:11.664 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/6 248 root cwd unknown /proc/248/cwd (readlink: Permission denied) 2024-05-09T04:58:11.664 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/6 248 root rtd unknown /proc/248/root (readlink: Permission denied) 2024-05-09T04:58:11.664 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/6 248 root txt unknown /proc/248/exe (readlink: Permission denied) 2024-05-09T04:58:11.664 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/6 248 root NOFD /proc/248/fd (opendir: Permission denied) 2024-05-09T04:58:11.664 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/1 250 root cwd unknown /proc/250/cwd (readlink: Permission denied) 2024-05-09T04:58:11.664 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/1 250 root rtd unknown /proc/250/root (readlink: Permission denied) 2024-05-09T04:58:11.664 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/1 250 root txt unknown /proc/250/exe (readlink: Permission denied) 2024-05-09T04:58:11.665 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/1 250 root NOFD /proc/250/fd (opendir: Permission denied) 2024-05-09T04:58:11.665 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/7 252 root cwd unknown /proc/252/cwd (readlink: Permission denied) 2024-05-09T04:58:11.665 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/7 252 root rtd unknown /proc/252/root (readlink: Permission denied) 2024-05-09T04:58:11.665 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/7 252 root txt unknown /proc/252/exe (readlink: Permission denied) 2024-05-09T04:58:11.665 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/7 252 root NOFD /proc/252/fd (opendir: Permission denied) 2024-05-09T04:58:11.665 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/2 264 root cwd unknown /proc/264/cwd (readlink: Permission denied) 2024-05-09T04:58:11.665 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/2 264 root rtd unknown /proc/264/root (readlink: Permission denied) 2024-05-09T04:58:11.665 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/2 264 root txt unknown /proc/264/exe (readlink: Permission denied) 2024-05-09T04:58:11.665 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/2 264 root NOFD /proc/264/fd (opendir: Permission denied) 2024-05-09T04:58:11.665 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/3 268 root cwd unknown /proc/268/cwd (readlink: Permission denied) 2024-05-09T04:58:11.665 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/3 268 root rtd unknown /proc/268/root (readlink: Permission denied) 2024-05-09T04:58:11.665 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/3 268 root txt unknown /proc/268/exe (readlink: Permission denied) 2024-05-09T04:58:11.665 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/3 268 root NOFD /proc/268/fd (opendir: Permission denied) 2024-05-09T04:58:11.665 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/4 330 root cwd unknown /proc/330/cwd (readlink: Permission denied) 2024-05-09T04:58:11.666 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/4 330 root rtd unknown /proc/330/root (readlink: Permission denied) 2024-05-09T04:58:11.666 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/4 330 root txt unknown /proc/330/exe (readlink: Permission denied) 2024-05-09T04:58:11.666 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/4 330 root NOFD /proc/330/fd (opendir: Permission denied) 2024-05-09T04:58:11.666 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/5 359 root cwd unknown /proc/359/cwd (readlink: Permission denied) 2024-05-09T04:58:11.666 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/5 359 root rtd unknown /proc/359/root (readlink: Permission denied) 2024-05-09T04:58:11.666 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/5 359 root txt unknown /proc/359/exe (readlink: Permission denied) 2024-05-09T04:58:11.666 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/5 359 root NOFD /proc/359/fd (opendir: Permission denied) 2024-05-09T04:58:11.666 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 401 root cwd unknown /proc/401/cwd (readlink: Permission denied) 2024-05-09T04:58:11.666 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 401 root rtd unknown /proc/401/root (readlink: Permission denied) 2024-05-09T04:58:11.666 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 401 root txt unknown /proc/401/exe (readlink: Permission denied) 2024-05-09T04:58:11.666 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 401 root NOFD /proc/401/fd (opendir: Permission denied) 2024-05-09T04:58:11.666 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 402 root cwd unknown /proc/402/cwd (readlink: Permission denied) 2024-05-09T04:58:11.666 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 402 root rtd unknown /proc/402/root (readlink: Permission denied) 2024-05-09T04:58:11.666 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 402 root txt unknown /proc/402/exe (readlink: Permission denied) 2024-05-09T04:58:11.666 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 402 root NOFD /proc/402/fd (opendir: Permission denied) 2024-05-09T04:58:11.667 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 410 root cwd unknown /proc/410/cwd (readlink: Permission denied) 2024-05-09T04:58:11.667 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 410 root rtd unknown /proc/410/root (readlink: Permission denied) 2024-05-09T04:58:11.667 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 410 root txt unknown /proc/410/exe (readlink: Permission denied) 2024-05-09T04:58:11.667 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 410 root NOFD /proc/410/fd (opendir: Permission denied) 2024-05-09T04:58:11.667 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 432 root cwd unknown /proc/432/cwd (readlink: Permission denied) 2024-05-09T04:58:11.667 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 432 root rtd unknown /proc/432/root (readlink: Permission denied) 2024-05-09T04:58:11.667 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 432 root txt unknown /proc/432/exe (readlink: Permission denied) 2024-05-09T04:58:11.667 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 432 root NOFD /proc/432/fd (opendir: Permission denied) 2024-05-09T04:58:11.667 INFO:tasks.workunit.client.0.smithi006.stdout:jbd2/sda1 445 root cwd unknown /proc/445/cwd (readlink: Permission denied) 2024-05-09T04:58:11.667 INFO:tasks.workunit.client.0.smithi006.stdout:jbd2/sda1 445 root rtd unknown /proc/445/root (readlink: Permission denied) 2024-05-09T04:58:11.667 INFO:tasks.workunit.client.0.smithi006.stdout:jbd2/sda1 445 root txt unknown /proc/445/exe (readlink: Permission denied) 2024-05-09T04:58:11.667 INFO:tasks.workunit.client.0.smithi006.stdout:jbd2/sda1 445 root NOFD /proc/445/fd (opendir: Permission denied) 2024-05-09T04:58:11.667 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 446 root cwd unknown /proc/446/cwd (readlink: Permission denied) 2024-05-09T04:58:11.667 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 446 root rtd unknown /proc/446/root (readlink: Permission denied) 2024-05-09T04:58:11.668 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 446 root txt unknown /proc/446/exe (readlink: Permission denied) 2024-05-09T04:58:11.668 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 446 root NOFD /proc/446/fd (opendir: Permission denied) 2024-05-09T04:58:11.668 INFO:tasks.workunit.client.0.smithi006.stdout:systemd-j 522 root cwd unknown /proc/522/cwd (readlink: Permission denied) 2024-05-09T04:58:11.668 INFO:tasks.workunit.client.0.smithi006.stdout:systemd-j 522 root rtd unknown /proc/522/root (readlink: Permission denied) 2024-05-09T04:58:11.668 INFO:tasks.workunit.client.0.smithi006.stdout:systemd-j 522 root txt unknown /proc/522/exe (readlink: Permission denied) 2024-05-09T04:58:11.668 INFO:tasks.workunit.client.0.smithi006.stdout:systemd-j 522 root NOFD /proc/522/fd (opendir: Permission denied) 2024-05-09T04:58:11.668 INFO:tasks.workunit.client.0.smithi006.stdout:ext4lazyi 529 root cwd unknown /proc/529/cwd (readlink: Permission denied) 2024-05-09T04:58:11.668 INFO:tasks.workunit.client.0.smithi006.stdout:ext4lazyi 529 root rtd unknown /proc/529/root (readlink: Permission denied) 2024-05-09T04:58:11.668 INFO:tasks.workunit.client.0.smithi006.stdout:ext4lazyi 529 root txt unknown /proc/529/exe (readlink: Permission denied) 2024-05-09T04:58:11.668 INFO:tasks.workunit.client.0.smithi006.stdout:ext4lazyi 529 root NOFD /proc/529/fd (opendir: Permission denied) 2024-05-09T04:58:11.668 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/1 539 root cwd unknown /proc/539/cwd (readlink: Permission denied) 2024-05-09T04:58:11.668 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/1 539 root rtd unknown /proc/539/root (readlink: Permission denied) 2024-05-09T04:58:11.668 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/1 539 root txt unknown /proc/539/exe (readlink: Permission denied) 2024-05-09T04:58:11.668 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/1 539 root NOFD /proc/539/fd (opendir: Permission denied) 2024-05-09T04:58:11.668 INFO:tasks.workunit.client.0.smithi006.stdout:systemd-u 540 root cwd unknown /proc/540/cwd (readlink: Permission denied) 2024-05-09T04:58:11.668 INFO:tasks.workunit.client.0.smithi006.stdout:systemd-u 540 root rtd unknown /proc/540/root (readlink: Permission denied) 2024-05-09T04:58:11.669 INFO:tasks.workunit.client.0.smithi006.stdout:systemd-u 540 root txt unknown /proc/540/exe (readlink: Permission denied) 2024-05-09T04:58:11.669 INFO:tasks.workunit.client.0.smithi006.stdout:systemd-u 540 root NOFD /proc/540/fd (opendir: Permission denied) 2024-05-09T04:58:11.669 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 591 root cwd unknown /proc/591/cwd (readlink: Permission denied) 2024-05-09T04:58:11.669 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 591 root rtd unknown /proc/591/root (readlink: Permission denied) 2024-05-09T04:58:11.669 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 591 root txt unknown /proc/591/exe (readlink: Permission denied) 2024-05-09T04:58:11.669 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 591 root NOFD /proc/591/fd (opendir: Permission denied) 2024-05-09T04:58:11.669 INFO:tasks.workunit.client.0.smithi006.stdout:rpcbind 597 rpc cwd unknown /proc/597/cwd (readlink: Permission denied) 2024-05-09T04:58:11.669 INFO:tasks.workunit.client.0.smithi006.stdout:rpcbind 597 rpc rtd unknown /proc/597/root (readlink: Permission denied) 2024-05-09T04:58:11.669 INFO:tasks.workunit.client.0.smithi006.stdout:rpcbind 597 rpc txt unknown /proc/597/exe (readlink: Permission denied) 2024-05-09T04:58:11.669 INFO:tasks.workunit.client.0.smithi006.stdout:rpcbind 597 rpc NOFD /proc/597/fd (opendir: Permission denied) 2024-05-09T04:58:11.669 INFO:tasks.workunit.client.0.smithi006.stdout:kipmi0 612 root cwd unknown /proc/612/cwd (readlink: Permission denied) 2024-05-09T04:58:11.669 INFO:tasks.workunit.client.0.smithi006.stdout:kipmi0 612 root rtd unknown /proc/612/root (readlink: Permission denied) 2024-05-09T04:58:11.669 INFO:tasks.workunit.client.0.smithi006.stdout:kipmi0 612 root txt unknown /proc/612/exe (readlink: Permission denied) 2024-05-09T04:58:11.670 INFO:tasks.workunit.client.0.smithi006.stdout:kipmi0 612 root NOFD /proc/612/fd (opendir: Permission denied) 2024-05-09T04:58:11.670 INFO:tasks.workunit.client.0.smithi006.stdout:auditd 615 root cwd unknown /proc/615/cwd (readlink: Permission denied) 2024-05-09T04:58:11.670 INFO:tasks.workunit.client.0.smithi006.stdout:auditd 615 root rtd unknown /proc/615/root (readlink: Permission denied) 2024-05-09T04:58:11.670 INFO:tasks.workunit.client.0.smithi006.stdout:auditd 615 root txt unknown /proc/615/exe (readlink: Permission denied) 2024-05-09T04:58:11.670 INFO:tasks.workunit.client.0.smithi006.stdout:auditd 615 root NOFD /proc/615/fd (opendir: Permission denied) 2024-05-09T04:58:11.670 INFO:tasks.workunit.client.0.smithi006.stdout:auditd 615 616 auditd root cwd unknown /proc/615/task/616/cwd (readlink: Permission denied) 2024-05-09T04:58:11.670 INFO:tasks.workunit.client.0.smithi006.stdout:auditd 615 616 auditd root rtd unknown /proc/615/task/616/root (readlink: Permission denied) 2024-05-09T04:58:11.670 INFO:tasks.workunit.client.0.smithi006.stdout:auditd 615 616 auditd root txt unknown /proc/615/task/616/exe (readlink: Permission denied) 2024-05-09T04:58:11.670 INFO:tasks.workunit.client.0.smithi006.stdout:auditd 615 616 auditd root NOFD /proc/615/task/616/fd (opendir: Permission denied) 2024-05-09T04:58:11.670 INFO:tasks.workunit.client.0.smithi006.stdout:auditd 615 618 auditd root cwd unknown /proc/615/task/618/cwd (readlink: Permission denied) 2024-05-09T04:58:11.670 INFO:tasks.workunit.client.0.smithi006.stdout:auditd 615 618 auditd root rtd unknown /proc/615/task/618/root (readlink: Permission denied) 2024-05-09T04:58:11.670 INFO:tasks.workunit.client.0.smithi006.stdout:auditd 615 618 auditd root txt unknown /proc/615/task/618/exe (readlink: Permission denied) 2024-05-09T04:58:11.670 INFO:tasks.workunit.client.0.smithi006.stdout:auditd 615 618 auditd root NOFD /proc/615/task/618/fd (opendir: Permission denied) 2024-05-09T04:58:11.670 INFO:tasks.workunit.client.0.smithi006.stdout:sedispatc 617 root cwd unknown /proc/617/cwd (readlink: Permission denied) 2024-05-09T04:58:11.671 INFO:tasks.workunit.client.0.smithi006.stdout:sedispatc 617 root rtd unknown /proc/617/root (readlink: Permission denied) 2024-05-09T04:58:11.671 INFO:tasks.workunit.client.0.smithi006.stdout:sedispatc 617 root txt unknown /proc/617/exe (readlink: Permission denied) 2024-05-09T04:58:11.671 INFO:tasks.workunit.client.0.smithi006.stdout:sedispatc 617 root NOFD /proc/617/fd (opendir: Permission denied) 2024-05-09T04:58:11.671 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 621 root cwd unknown /proc/621/cwd (readlink: Permission denied) 2024-05-09T04:58:11.671 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 621 root rtd unknown /proc/621/root (readlink: Permission denied) 2024-05-09T04:58:11.671 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 621 root txt unknown /proc/621/exe (readlink: Permission denied) 2024-05-09T04:58:11.671 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 621 root NOFD /proc/621/fd (opendir: Permission denied) 2024-05-09T04:58:11.671 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 622 root cwd unknown /proc/622/cwd (readlink: Permission denied) 2024-05-09T04:58:11.671 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 622 root rtd unknown /proc/622/root (readlink: Permission denied) 2024-05-09T04:58:11.671 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 622 root txt unknown /proc/622/exe (readlink: Permission denied) 2024-05-09T04:58:11.671 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 622 root NOFD /proc/622/fd (opendir: Permission denied) 2024-05-09T04:58:11.671 INFO:tasks.workunit.client.0.smithi006.stdout:dbus-brok 657 dbus cwd unknown /proc/657/cwd (readlink: Permission denied) 2024-05-09T04:58:11.671 INFO:tasks.workunit.client.0.smithi006.stdout:dbus-brok 657 dbus rtd unknown /proc/657/root (readlink: Permission denied) 2024-05-09T04:58:11.671 INFO:tasks.workunit.client.0.smithi006.stdout:dbus-brok 657 dbus txt unknown /proc/657/exe (readlink: Permission denied) 2024-05-09T04:58:11.672 INFO:tasks.workunit.client.0.smithi006.stdout:dbus-brok 657 dbus NOFD /proc/657/fd (opendir: Permission denied) 2024-05-09T04:58:11.672 INFO:tasks.workunit.client.0.smithi006.stdout:dbus-brok 658 dbus cwd unknown /proc/658/cwd (readlink: Permission denied) 2024-05-09T04:58:11.672 INFO:tasks.workunit.client.0.smithi006.stdout:dbus-brok 658 dbus rtd unknown /proc/658/root (readlink: Permission denied) 2024-05-09T04:58:11.672 INFO:tasks.workunit.client.0.smithi006.stdout:dbus-brok 658 dbus txt unknown /proc/658/exe (readlink: Permission denied) 2024-05-09T04:58:11.672 INFO:tasks.workunit.client.0.smithi006.stdout:dbus-brok 658 dbus NOFD /proc/658/fd (opendir: Permission denied) 2024-05-09T04:58:11.672 INFO:tasks.workunit.client.0.smithi006.stdout:NetworkMa 659 root cwd unknown /proc/659/cwd (readlink: Permission denied) 2024-05-09T04:58:11.672 INFO:tasks.workunit.client.0.smithi006.stdout:NetworkMa 659 root rtd unknown /proc/659/root (readlink: Permission denied) 2024-05-09T04:58:11.672 INFO:tasks.workunit.client.0.smithi006.stdout:NetworkMa 659 root txt unknown /proc/659/exe (readlink: Permission denied) 2024-05-09T04:58:11.672 INFO:tasks.workunit.client.0.smithi006.stdout:NetworkMa 659 root NOFD /proc/659/fd (opendir: Permission denied) 2024-05-09T04:58:11.672 INFO:tasks.workunit.client.0.smithi006.stdout:NetworkMa 659 674 gmain root cwd unknown /proc/659/task/674/cwd (readlink: Permission denied) 2024-05-09T04:58:11.672 INFO:tasks.workunit.client.0.smithi006.stdout:NetworkMa 659 674 gmain root rtd unknown /proc/659/task/674/root (readlink: Permission denied) 2024-05-09T04:58:11.672 INFO:tasks.workunit.client.0.smithi006.stdout:NetworkMa 659 674 gmain root txt unknown /proc/659/task/674/exe (readlink: Permission denied) 2024-05-09T04:58:11.672 INFO:tasks.workunit.client.0.smithi006.stdout:NetworkMa 659 674 gmain root NOFD /proc/659/task/674/fd (opendir: Permission denied) 2024-05-09T04:58:11.672 INFO:tasks.workunit.client.0.smithi006.stdout:NetworkMa 659 675 gdbus root cwd unknown /proc/659/task/675/cwd (readlink: Permission denied) 2024-05-09T04:58:11.673 INFO:tasks.workunit.client.0.smithi006.stdout:NetworkMa 659 675 gdbus root rtd unknown /proc/659/task/675/root (readlink: Permission denied) 2024-05-09T04:58:11.673 INFO:tasks.workunit.client.0.smithi006.stdout:NetworkMa 659 675 gdbus root txt unknown /proc/659/task/675/exe (readlink: Permission denied) 2024-05-09T04:58:11.673 INFO:tasks.workunit.client.0.smithi006.stdout:NetworkMa 659 675 gdbus root NOFD /proc/659/task/675/fd (opendir: Permission denied) 2024-05-09T04:58:11.673 INFO:tasks.workunit.client.0.smithi006.stdout:irqbalanc 662 root cwd unknown /proc/662/cwd (readlink: Permission denied) 2024-05-09T04:58:11.673 INFO:tasks.workunit.client.0.smithi006.stdout:irqbalanc 662 root rtd unknown /proc/662/root (readlink: Permission denied) 2024-05-09T04:58:11.673 INFO:tasks.workunit.client.0.smithi006.stdout:irqbalanc 662 root txt unknown /proc/662/exe (readlink: Permission denied) 2024-05-09T04:58:11.673 INFO:tasks.workunit.client.0.smithi006.stdout:irqbalanc 662 root NOFD /proc/662/fd (opendir: Permission denied) 2024-05-09T04:58:11.673 INFO:tasks.workunit.client.0.smithi006.stdout:irqbalanc 662 667 gmain root cwd unknown /proc/662/task/667/cwd (readlink: Permission denied) 2024-05-09T04:58:11.673 INFO:tasks.workunit.client.0.smithi006.stdout:irqbalanc 662 667 gmain root rtd unknown /proc/662/task/667/root (readlink: Permission denied) 2024-05-09T04:58:11.673 INFO:tasks.workunit.client.0.smithi006.stdout:irqbalanc 662 667 gmain root txt unknown /proc/662/task/667/exe (readlink: Permission denied) 2024-05-09T04:58:11.673 INFO:tasks.workunit.client.0.smithi006.stdout:irqbalanc 662 667 gmain root NOFD /proc/662/task/667/fd (opendir: Permission denied) 2024-05-09T04:58:11.673 INFO:tasks.workunit.client.0.smithi006.stdout:lsmd 663 libstoragemgmt cwd unknown /proc/663/cwd (readlink: Permission denied) 2024-05-09T04:58:11.673 INFO:tasks.workunit.client.0.smithi006.stdout:lsmd 663 libstoragemgmt rtd unknown /proc/663/root (readlink: Permission denied) 2024-05-09T04:58:11.673 INFO:tasks.workunit.client.0.smithi006.stdout:lsmd 663 libstoragemgmt txt unknown /proc/663/exe (readlink: Permission denied) 2024-05-09T04:58:11.674 INFO:tasks.workunit.client.0.smithi006.stdout:lsmd 663 libstoragemgmt NOFD /proc/663/fd (opendir: Permission denied) 2024-05-09T04:58:11.674 INFO:tasks.workunit.client.0.smithi006.stdout:smartd 664 root cwd unknown /proc/664/cwd (readlink: Permission denied) 2024-05-09T04:58:11.674 INFO:tasks.workunit.client.0.smithi006.stdout:smartd 664 root rtd unknown /proc/664/root (readlink: Permission denied) 2024-05-09T04:58:11.674 INFO:tasks.workunit.client.0.smithi006.stdout:smartd 664 root txt unknown /proc/664/exe (readlink: Permission denied) 2024-05-09T04:58:11.674 INFO:tasks.workunit.client.0.smithi006.stdout:smartd 664 root NOFD /proc/664/fd (opendir: Permission denied) 2024-05-09T04:58:11.674 INFO:tasks.workunit.client.0.smithi006.stdout:systemd-l 666 root cwd unknown /proc/666/cwd (readlink: Permission denied) 2024-05-09T04:58:11.674 INFO:tasks.workunit.client.0.smithi006.stdout:systemd-l 666 root rtd unknown /proc/666/root (readlink: Permission denied) 2024-05-09T04:58:11.674 INFO:tasks.workunit.client.0.smithi006.stdout:systemd-l 666 root txt unknown /proc/666/exe (readlink: Permission denied) 2024-05-09T04:58:11.674 INFO:tasks.workunit.client.0.smithi006.stdout:systemd-l 666 root NOFD /proc/666/fd (opendir: Permission denied) 2024-05-09T04:58:11.674 INFO:tasks.workunit.client.0.smithi006.stdout:sshd 722 root cwd unknown /proc/722/cwd (readlink: Permission denied) 2024-05-09T04:58:11.674 INFO:tasks.workunit.client.0.smithi006.stdout:sshd 722 root rtd unknown /proc/722/root (readlink: Permission denied) 2024-05-09T04:58:11.674 INFO:tasks.workunit.client.0.smithi006.stdout:sshd 722 root txt unknown /proc/722/exe (readlink: Permission denied) 2024-05-09T04:58:11.674 INFO:tasks.workunit.client.0.smithi006.stdout:sshd 722 root NOFD /proc/722/fd (opendir: Permission denied) 2024-05-09T04:58:11.674 INFO:tasks.workunit.client.0.smithi006.stdout:gssproxy 731 root cwd unknown /proc/731/cwd (readlink: Permission denied) 2024-05-09T04:58:11.674 INFO:tasks.workunit.client.0.smithi006.stdout:gssproxy 731 root rtd unknown /proc/731/root (readlink: Permission denied) 2024-05-09T04:58:11.675 INFO:tasks.workunit.client.0.smithi006.stdout:gssproxy 731 root txt unknown /proc/731/exe (readlink: Permission denied) 2024-05-09T04:58:11.675 INFO:tasks.workunit.client.0.smithi006.stdout:gssproxy 731 root NOFD /proc/731/fd (opendir: Permission denied) 2024-05-09T04:58:11.675 INFO:tasks.workunit.client.0.smithi006.stdout:gssproxy 731 732 gssproxy root cwd unknown /proc/731/task/732/cwd (readlink: Permission denied) 2024-05-09T04:58:11.675 INFO:tasks.workunit.client.0.smithi006.stdout:gssproxy 731 732 gssproxy root rtd unknown /proc/731/task/732/root (readlink: Permission denied) 2024-05-09T04:58:11.675 INFO:tasks.workunit.client.0.smithi006.stdout:gssproxy 731 732 gssproxy root txt unknown /proc/731/task/732/exe (readlink: Permission denied) 2024-05-09T04:58:11.675 INFO:tasks.workunit.client.0.smithi006.stdout:gssproxy 731 732 gssproxy root NOFD /proc/731/task/732/fd (opendir: Permission denied) 2024-05-09T04:58:11.675 INFO:tasks.workunit.client.0.smithi006.stdout:gssproxy 731 733 gssproxy root cwd unknown /proc/731/task/733/cwd (readlink: Permission denied) 2024-05-09T04:58:11.675 INFO:tasks.workunit.client.0.smithi006.stdout:gssproxy 731 733 gssproxy root rtd unknown /proc/731/task/733/root (readlink: Permission denied) 2024-05-09T04:58:11.675 INFO:tasks.workunit.client.0.smithi006.stdout:gssproxy 731 733 gssproxy root txt unknown /proc/731/task/733/exe (readlink: Permission denied) 2024-05-09T04:58:11.675 INFO:tasks.workunit.client.0.smithi006.stdout:gssproxy 731 733 gssproxy root NOFD /proc/731/task/733/fd (opendir: Permission denied) 2024-05-09T04:58:11.675 INFO:tasks.workunit.client.0.smithi006.stdout:gssproxy 731 734 gssproxy root cwd unknown /proc/731/task/734/cwd (readlink: Permission denied) 2024-05-09T04:58:11.675 INFO:tasks.workunit.client.0.smithi006.stdout:gssproxy 731 734 gssproxy root rtd unknown /proc/731/task/734/root (readlink: Permission denied) 2024-05-09T04:58:11.675 INFO:tasks.workunit.client.0.smithi006.stdout:gssproxy 731 734 gssproxy root txt unknown /proc/731/task/734/exe (readlink: Permission denied) 2024-05-09T04:58:11.675 INFO:tasks.workunit.client.0.smithi006.stdout:gssproxy 731 734 gssproxy root NOFD /proc/731/task/734/fd (opendir: Permission denied) 2024-05-09T04:58:11.676 INFO:tasks.workunit.client.0.smithi006.stdout:gssproxy 731 735 gssproxy root cwd unknown /proc/731/task/735/cwd (readlink: Permission denied) 2024-05-09T04:58:11.676 INFO:tasks.workunit.client.0.smithi006.stdout:gssproxy 731 735 gssproxy root rtd unknown /proc/731/task/735/root (readlink: Permission denied) 2024-05-09T04:58:11.676 INFO:tasks.workunit.client.0.smithi006.stdout:gssproxy 731 735 gssproxy root txt unknown /proc/731/task/735/exe (readlink: Permission denied) 2024-05-09T04:58:11.676 INFO:tasks.workunit.client.0.smithi006.stdout:gssproxy 731 735 gssproxy root NOFD /proc/731/task/735/fd (opendir: Permission denied) 2024-05-09T04:58:11.676 INFO:tasks.workunit.client.0.smithi006.stdout:gssproxy 731 736 gssproxy root cwd unknown /proc/731/task/736/cwd (readlink: Permission denied) 2024-05-09T04:58:11.676 INFO:tasks.workunit.client.0.smithi006.stdout:gssproxy 731 736 gssproxy root rtd unknown /proc/731/task/736/root (readlink: Permission denied) 2024-05-09T04:58:11.676 INFO:tasks.workunit.client.0.smithi006.stdout:gssproxy 731 736 gssproxy root txt unknown /proc/731/task/736/exe (readlink: Permission denied) 2024-05-09T04:58:11.676 INFO:tasks.workunit.client.0.smithi006.stdout:gssproxy 731 736 gssproxy root NOFD /proc/731/task/736/fd (opendir: Permission denied) 2024-05-09T04:58:11.676 INFO:tasks.workunit.client.0.smithi006.stdout:atd 738 root cwd unknown /proc/738/cwd (readlink: Permission denied) 2024-05-09T04:58:11.676 INFO:tasks.workunit.client.0.smithi006.stdout:atd 738 root rtd unknown /proc/738/root (readlink: Permission denied) 2024-05-09T04:58:11.676 INFO:tasks.workunit.client.0.smithi006.stdout:atd 738 root txt unknown /proc/738/exe (readlink: Permission denied) 2024-05-09T04:58:11.676 INFO:tasks.workunit.client.0.smithi006.stdout:atd 738 root NOFD /proc/738/fd (opendir: Permission denied) 2024-05-09T04:58:11.676 INFO:tasks.workunit.client.0.smithi006.stdout:crond 739 root cwd unknown /proc/739/cwd (readlink: Permission denied) 2024-05-09T04:58:11.676 INFO:tasks.workunit.client.0.smithi006.stdout:crond 739 root rtd unknown /proc/739/root (readlink: Permission denied) 2024-05-09T04:58:11.677 INFO:tasks.workunit.client.0.smithi006.stdout:crond 739 root txt unknown /proc/739/exe (readlink: Permission denied) 2024-05-09T04:58:11.677 INFO:tasks.workunit.client.0.smithi006.stdout:crond 739 root NOFD /proc/739/fd (opendir: Permission denied) 2024-05-09T04:58:11.677 INFO:tasks.workunit.client.0.smithi006.stdout:nrpe 763 nrpe cwd unknown /proc/763/cwd (readlink: Permission denied) 2024-05-09T04:58:11.677 INFO:tasks.workunit.client.0.smithi006.stdout:nrpe 763 nrpe rtd unknown /proc/763/root (readlink: Permission denied) 2024-05-09T04:58:11.677 INFO:tasks.workunit.client.0.smithi006.stdout:nrpe 763 nrpe txt unknown /proc/763/exe (readlink: Permission denied) 2024-05-09T04:58:11.677 INFO:tasks.workunit.client.0.smithi006.stdout:nrpe 763 nrpe NOFD /proc/763/fd (opendir: Permission denied) 2024-05-09T04:58:11.677 INFO:tasks.workunit.client.0.smithi006.stdout:agetty 1085 root cwd unknown /proc/1085/cwd (readlink: Permission denied) 2024-05-09T04:58:11.677 INFO:tasks.workunit.client.0.smithi006.stdout:agetty 1085 root rtd unknown /proc/1085/root (readlink: Permission denied) 2024-05-09T04:58:11.677 INFO:tasks.workunit.client.0.smithi006.stdout:agetty 1085 root txt unknown /proc/1085/exe (readlink: Permission denied) 2024-05-09T04:58:11.677 INFO:tasks.workunit.client.0.smithi006.stdout:agetty 1085 root NOFD /proc/1085/fd (opendir: Permission denied) 2024-05-09T04:58:11.677 INFO:tasks.workunit.client.0.smithi006.stdout:agetty 1086 root cwd unknown /proc/1086/cwd (readlink: Permission denied) 2024-05-09T04:58:11.677 INFO:tasks.workunit.client.0.smithi006.stdout:agetty 1086 root rtd unknown /proc/1086/root (readlink: Permission denied) 2024-05-09T04:58:11.677 INFO:tasks.workunit.client.0.smithi006.stdout:agetty 1086 root txt unknown /proc/1086/exe (readlink: Permission denied) 2024-05-09T04:58:11.677 INFO:tasks.workunit.client.0.smithi006.stdout:agetty 1086 root NOFD /proc/1086/fd (opendir: Permission denied) 2024-05-09T04:58:11.678 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu cwd DIR 8,1 4096 2 / 2024-05-09T04:58:11.678 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu rtd DIR 8,1 4096 2 / 2024-05-09T04:58:11.678 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu txt REG 8,1 98216 18583 /usr/lib/systemd/systemd 2024-05-09T04:58:11.678 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu DEL REG 8,1 849 /etc/selinux/targeted/contexts/files/file_contexts.bin 2024-05-09T04:58:11.678 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-05-09T04:58:11.678 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-05-09T04:58:11.678 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-05-09T04:58:11.678 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu DEL REG 8,1 7490 /usr/lib64/libcrypto.so.3.0.7 2024-05-09T04:58:11.678 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-05-09T04:58:11.678 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu mem REG 8,1 1714208 7271 /usr/lib64/libp11-kit.so.0.3.1 2024-05-09T04:58:11.678 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-05-09T04:58:11.678 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu DEL REG 8,1 51505 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2024-05-09T04:58:11.678 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-05-09T04:58:11.678 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu mem REG 8,1 28552 7287 /usr/lib64/libattr.so.1.1.2501 2024-05-09T04:58:11.679 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-05-09T04:58:11.679 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-05-09T04:58:11.679 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu mem REG 8,1 40424 8439 /usr/lib64/libeconf.so.0.4.1 2024-05-09T04:58:11.679 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-05-09T04:58:11.679 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-05-09T04:58:11.679 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-05-09T04:58:11.679 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu mem REG 8,1 3785856 46432 /usr/lib64/systemd/libsystemd-shared-252.so 2024-05-09T04:58:11.679 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-05-09T04:58:11.679 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-05-09T04:58:11.679 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-05-09T04:58:11.679 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu mem REG 8,1 40496 7288 /usr/lib64/libacl.so.1.1.2301 2024-05-09T04:58:11.679 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu mem REG 8,1 2157904 46429 /usr/lib64/systemd/libsystemd-core-252.so 2024-05-09T04:58:11.679 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu mem REG 8,1 291512 7639 /usr/lib64/libmount.so.1.1.0 2024-05-09T04:58:11.679 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-05-09T04:58:11.680 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu mem REG 8,1 107424 7663 /usr/lib64/libkmod.so.2.3.6 2024-05-09T04:58:11.680 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-05-09T04:58:11.680 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu mem REG 8,1 69592 8448 /usr/lib64/libpam.so.0.85.1 2024-05-09T04:58:11.680 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu mem REG 8,1 127680 7455 /usr/lib64/libseccomp.so.2.5.2 2024-05-09T04:58:11.680 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-05-09T04:58:11.680 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-05-09T04:58:11.680 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu mem REG 0,20 0 19 /sys/fs/selinux/status 2024-05-09T04:58:11.680 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu 0r CHR 1,3 0t0 4 /dev/null 2024-05-09T04:58:11.680 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu 1u unix 0x0000000000000000 0t0 2782 type=STREAM (CONNECTED) 2024-05-09T04:58:11.680 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu 2u unix 0x0000000000000000 0t0 2782 type=STREAM (CONNECTED) 2024-05-09T04:58:11.680 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu 3u unix 0x0000000000000000 0t0 2811 type=DGRAM (CONNECTED) 2024-05-09T04:58:11.680 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu 4u a_inode 0,15 0 50 [eventpoll:5,6,8,9,11,12,14,15,16,17,19,20,21] 2024-05-09T04:58:11.680 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu 5u a_inode 0,15 0 50 [signalfd] 2024-05-09T04:58:11.680 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu 6r a_inode 0,15 0 50 inotify 2024-05-09T04:58:11.680 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu 7r DIR 0,26 0 3581 /sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service 2024-05-09T04:58:11.681 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu 8u a_inode 0,15 0 50 [timerfd] 2024-05-09T04:58:11.681 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu 9u a_inode 0,15 0 50 [eventpoll:10,13] 2024-05-09T04:58:11.681 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu 10r REG 0,21 0 2800 /proc/4219/mountinfo 2024-05-09T04:58:11.681 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu 11r a_inode 0,15 0 50 inotify 2024-05-09T04:58:11.681 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu 12u unix 0x0000000000000000 0t0 2833 /run/user/1000/bus type=STREAM (LISTEN) 2024-05-09T04:58:11.681 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu 13r a_inode 0,15 0 50 inotify 2024-05-09T04:58:11.681 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu 14r REG 0,21 0 4026532078 /proc/swaps 2024-05-09T04:58:11.681 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu 15u netlink 0t0 2815 KOBJECT_UEVENT 2024-05-09T04:58:11.681 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu 16u unix 0x0000000000000000 0t0 2820 /run/user/1000/systemd/notify type=DGRAM (UNCONNECTED) 2024-05-09T04:58:11.681 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu 17u unix 0x0000000000000000 0t0 2821 type=DGRAM (CONNECTED) 2024-05-09T04:58:11.681 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu 18u unix 0x0000000000000000 0t0 2822 type=DGRAM (CONNECTED) 2024-05-09T04:58:11.681 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu 19u unix 0x0000000000000000 0t0 2823 /run/user/1000/systemd/private type=STREAM (LISTEN) 2024-05-09T04:58:11.681 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu 20u unix 0x0000000000000000 0t0 2825 type=STREAM (CONNECTED) 2024-05-09T04:58:11.681 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 4219 ubuntu 21u a_inode 0,15 0 50 [timerfd] 2024-05-09T04:58:11.681 INFO:tasks.workunit.client.0.smithi006.stdout:(sd-pam) 4221 ubuntu cwd unknown /proc/4221/cwd (readlink: Permission denied) 2024-05-09T04:58:11.682 INFO:tasks.workunit.client.0.smithi006.stdout:(sd-pam) 4221 ubuntu rtd unknown /proc/4221/root (readlink: Permission denied) 2024-05-09T04:58:11.682 INFO:tasks.workunit.client.0.smithi006.stdout:(sd-pam) 4221 ubuntu txt unknown /proc/4221/exe (readlink: Permission denied) 2024-05-09T04:58:11.682 INFO:tasks.workunit.client.0.smithi006.stdout:(sd-pam) 4221 ubuntu NOFD /proc/4221/fd (opendir: Permission denied) 2024-05-09T04:58:11.682 INFO:tasks.workunit.client.0.smithi006.stdout:rsyslogd 4464 root cwd unknown /proc/4464/cwd (readlink: Permission denied) 2024-05-09T04:58:11.682 INFO:tasks.workunit.client.0.smithi006.stdout:rsyslogd 4464 root rtd unknown /proc/4464/root (readlink: Permission denied) 2024-05-09T04:58:11.682 INFO:tasks.workunit.client.0.smithi006.stdout:rsyslogd 4464 root txt unknown /proc/4464/exe (readlink: Permission denied) 2024-05-09T04:58:11.682 INFO:tasks.workunit.client.0.smithi006.stdout:rsyslogd 4464 root NOFD /proc/4464/fd (opendir: Permission denied) 2024-05-09T04:58:11.682 INFO:tasks.workunit.client.0.smithi006.stdout:rsyslogd 4464 4466 in:imjour root cwd unknown /proc/4464/task/4466/cwd (readlink: Permission denied) 2024-05-09T04:58:11.682 INFO:tasks.workunit.client.0.smithi006.stdout:rsyslogd 4464 4466 in:imjour root rtd unknown /proc/4464/task/4466/root (readlink: Permission denied) 2024-05-09T04:58:11.682 INFO:tasks.workunit.client.0.smithi006.stdout:rsyslogd 4464 4466 in:imjour root txt unknown /proc/4464/task/4466/exe (readlink: Permission denied) 2024-05-09T04:58:11.682 INFO:tasks.workunit.client.0.smithi006.stdout:rsyslogd 4464 4466 in:imjour root NOFD /proc/4464/task/4466/fd (opendir: Permission denied) 2024-05-09T04:58:11.682 INFO:tasks.workunit.client.0.smithi006.stdout:rsyslogd 4464 4467 rs:main root cwd unknown /proc/4464/task/4467/cwd (readlink: Permission denied) 2024-05-09T04:58:11.682 INFO:tasks.workunit.client.0.smithi006.stdout:rsyslogd 4464 4467 rs:main root rtd unknown /proc/4464/task/4467/root (readlink: Permission denied) 2024-05-09T04:58:11.682 INFO:tasks.workunit.client.0.smithi006.stdout:rsyslogd 4464 4467 rs:main root txt unknown /proc/4464/task/4467/exe (readlink: Permission denied) 2024-05-09T04:58:11.682 INFO:tasks.workunit.client.0.smithi006.stdout:rsyslogd 4464 4467 rs:main root NOFD /proc/4464/task/4467/fd (opendir: Permission denied) 2024-05-09T04:58:11.683 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 polkitd cwd unknown /proc/15944/cwd (readlink: Permission denied) 2024-05-09T04:58:11.683 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 polkitd rtd unknown /proc/15944/root (readlink: Permission denied) 2024-05-09T04:58:11.683 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 polkitd txt unknown /proc/15944/exe (readlink: Permission denied) 2024-05-09T04:58:11.683 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 polkitd NOFD /proc/15944/fd (opendir: Permission denied) 2024-05-09T04:58:11.683 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15946 gmain polkitd cwd unknown /proc/15944/task/15946/cwd (readlink: Permission denied) 2024-05-09T04:58:11.683 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15946 gmain polkitd rtd unknown /proc/15944/task/15946/root (readlink: Permission denied) 2024-05-09T04:58:11.683 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15946 gmain polkitd txt unknown /proc/15944/task/15946/exe (readlink: Permission denied) 2024-05-09T04:58:11.683 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15946 gmain polkitd NOFD /proc/15944/task/15946/fd (opendir: Permission denied) 2024-05-09T04:58:11.683 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15947 gdbus polkitd cwd unknown /proc/15944/task/15947/cwd (readlink: Permission denied) 2024-05-09T04:58:11.683 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15947 gdbus polkitd rtd unknown /proc/15944/task/15947/root (readlink: Permission denied) 2024-05-09T04:58:11.683 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15947 gdbus polkitd txt unknown /proc/15944/task/15947/exe (readlink: Permission denied) 2024-05-09T04:58:11.683 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15947 gdbus polkitd NOFD /proc/15944/task/15947/fd (opendir: Permission denied) 2024-05-09T04:58:11.683 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15948 JS\x20Hel polkitd cwd unknown /proc/15944/task/15948/cwd (readlink: Permission denied) 2024-05-09T04:58:11.683 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15948 JS\x20Hel polkitd rtd unknown /proc/15944/task/15948/root (readlink: Permission denied) 2024-05-09T04:58:11.683 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15948 JS\x20Hel polkitd txt unknown /proc/15944/task/15948/exe (readlink: Permission denied) 2024-05-09T04:58:11.684 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15948 JS\x20Hel polkitd NOFD /proc/15944/task/15948/fd (opendir: Permission denied) 2024-05-09T04:58:11.684 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15949 JS\x20Hel polkitd cwd unknown /proc/15944/task/15949/cwd (readlink: Permission denied) 2024-05-09T04:58:11.684 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15949 JS\x20Hel polkitd rtd unknown /proc/15944/task/15949/root (readlink: Permission denied) 2024-05-09T04:58:11.684 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15949 JS\x20Hel polkitd txt unknown /proc/15944/task/15949/exe (readlink: Permission denied) 2024-05-09T04:58:11.684 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15949 JS\x20Hel polkitd NOFD /proc/15944/task/15949/fd (opendir: Permission denied) 2024-05-09T04:58:11.684 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15950 JS\x20Hel polkitd cwd unknown /proc/15944/task/15950/cwd (readlink: Permission denied) 2024-05-09T04:58:11.684 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15950 JS\x20Hel polkitd rtd unknown /proc/15944/task/15950/root (readlink: Permission denied) 2024-05-09T04:58:11.684 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15950 JS\x20Hel polkitd txt unknown /proc/15944/task/15950/exe (readlink: Permission denied) 2024-05-09T04:58:11.684 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15950 JS\x20Hel polkitd NOFD /proc/15944/task/15950/fd (opendir: Permission denied) 2024-05-09T04:58:11.684 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15951 JS\x20Hel polkitd cwd unknown /proc/15944/task/15951/cwd (readlink: Permission denied) 2024-05-09T04:58:11.684 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15951 JS\x20Hel polkitd rtd unknown /proc/15944/task/15951/root (readlink: Permission denied) 2024-05-09T04:58:11.684 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15951 JS\x20Hel polkitd txt unknown /proc/15944/task/15951/exe (readlink: Permission denied) 2024-05-09T04:58:11.684 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15951 JS\x20Hel polkitd NOFD /proc/15944/task/15951/fd (opendir: Permission denied) 2024-05-09T04:58:11.684 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15952 JS\x20Hel polkitd cwd unknown /proc/15944/task/15952/cwd (readlink: Permission denied) 2024-05-09T04:58:11.684 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15952 JS\x20Hel polkitd rtd unknown /proc/15944/task/15952/root (readlink: Permission denied) 2024-05-09T04:58:11.685 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15952 JS\x20Hel polkitd txt unknown /proc/15944/task/15952/exe (readlink: Permission denied) 2024-05-09T04:58:11.685 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15952 JS\x20Hel polkitd NOFD /proc/15944/task/15952/fd (opendir: Permission denied) 2024-05-09T04:58:11.685 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15953 JS\x20Hel polkitd cwd unknown /proc/15944/task/15953/cwd (readlink: Permission denied) 2024-05-09T04:58:11.685 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15953 JS\x20Hel polkitd rtd unknown /proc/15944/task/15953/root (readlink: Permission denied) 2024-05-09T04:58:11.685 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15953 JS\x20Hel polkitd txt unknown /proc/15944/task/15953/exe (readlink: Permission denied) 2024-05-09T04:58:11.685 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15953 JS\x20Hel polkitd NOFD /proc/15944/task/15953/fd (opendir: Permission denied) 2024-05-09T04:58:11.685 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15954 JS\x20Hel polkitd cwd unknown /proc/15944/task/15954/cwd (readlink: Permission denied) 2024-05-09T04:58:11.685 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15954 JS\x20Hel polkitd rtd unknown /proc/15944/task/15954/root (readlink: Permission denied) 2024-05-09T04:58:11.685 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15954 JS\x20Hel polkitd txt unknown /proc/15944/task/15954/exe (readlink: Permission denied) 2024-05-09T04:58:11.685 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15954 JS\x20Hel polkitd NOFD /proc/15944/task/15954/fd (opendir: Permission denied) 2024-05-09T04:58:11.685 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15955 JS\x20Hel polkitd cwd unknown /proc/15944/task/15955/cwd (readlink: Permission denied) 2024-05-09T04:58:11.685 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15955 JS\x20Hel polkitd rtd unknown /proc/15944/task/15955/root (readlink: Permission denied) 2024-05-09T04:58:11.685 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15955 JS\x20Hel polkitd txt unknown /proc/15944/task/15955/exe (readlink: Permission denied) 2024-05-09T04:58:11.685 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15955 JS\x20Hel polkitd NOFD /proc/15944/task/15955/fd (opendir: Permission denied) 2024-05-09T04:58:11.685 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15956 polkitd polkitd cwd unknown /proc/15944/task/15956/cwd (readlink: Permission denied) 2024-05-09T04:58:11.685 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15956 polkitd polkitd rtd unknown /proc/15944/task/15956/root (readlink: Permission denied) 2024-05-09T04:58:11.686 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15956 polkitd polkitd txt unknown /proc/15944/task/15956/exe (readlink: Permission denied) 2024-05-09T04:58:11.686 INFO:tasks.workunit.client.0.smithi006.stdout:polkitd 15944 15956 polkitd polkitd NOFD /proc/15944/task/15956/fd (opendir: Permission denied) 2024-05-09T04:58:11.686 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/4 17993 root cwd unknown /proc/17993/cwd (readlink: Permission denied) 2024-05-09T04:58:11.686 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/4 17993 root rtd unknown /proc/17993/root (readlink: Permission denied) 2024-05-09T04:58:11.686 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/4 17993 root txt unknown /proc/17993/exe (readlink: Permission denied) 2024-05-09T04:58:11.686 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/4 17993 root NOFD /proc/17993/fd (opendir: Permission denied) 2024-05-09T04:58:11.686 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 19785 root cwd unknown /proc/19785/cwd (readlink: Permission denied) 2024-05-09T04:58:11.686 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 19785 root rtd unknown /proc/19785/root (readlink: Permission denied) 2024-05-09T04:58:11.686 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 19785 root txt unknown /proc/19785/exe (readlink: Permission denied) 2024-05-09T04:58:11.686 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 19785 root NOFD /proc/19785/fd (opendir: Permission denied) 2024-05-09T04:58:11.686 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 19815 root cwd unknown /proc/19815/cwd (readlink: Permission denied) 2024-05-09T04:58:11.686 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 19815 root rtd unknown /proc/19815/root (readlink: Permission denied) 2024-05-09T04:58:11.686 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 19815 root txt unknown /proc/19815/exe (readlink: Permission denied) 2024-05-09T04:58:11.686 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 19815 root NOFD /proc/19815/fd (opendir: Permission denied) 2024-05-09T04:58:11.687 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 19844 root cwd unknown /proc/19844/cwd (readlink: Permission denied) 2024-05-09T04:58:11.687 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 19844 root rtd unknown /proc/19844/root (readlink: Permission denied) 2024-05-09T04:58:11.687 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 19844 root txt unknown /proc/19844/exe (readlink: Permission denied) 2024-05-09T04:58:11.687 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 19844 root NOFD /proc/19844/fd (opendir: Permission denied) 2024-05-09T04:58:11.687 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 19873 root cwd unknown /proc/19873/cwd (readlink: Permission denied) 2024-05-09T04:58:11.687 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 19873 root rtd unknown /proc/19873/root (readlink: Permission denied) 2024-05-09T04:58:11.687 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 19873 root txt unknown /proc/19873/exe (readlink: Permission denied) 2024-05-09T04:58:11.687 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 19873 root NOFD /proc/19873/fd (opendir: Permission denied) 2024-05-09T04:58:11.687 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 19902 root cwd unknown /proc/19902/cwd (readlink: Permission denied) 2024-05-09T04:58:11.687 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 19902 root rtd unknown /proc/19902/root (readlink: Permission denied) 2024-05-09T04:58:11.687 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 19902 root txt unknown /proc/19902/exe (readlink: Permission denied) 2024-05-09T04:58:11.687 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 19902 root NOFD /proc/19902/fd (opendir: Permission denied) 2024-05-09T04:58:11.687 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20066 root cwd unknown /proc/20066/cwd (readlink: Permission denied) 2024-05-09T04:58:11.687 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20066 root rtd unknown /proc/20066/root (readlink: Permission denied) 2024-05-09T04:58:11.687 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20066 root txt unknown /proc/20066/exe (readlink: Permission denied) 2024-05-09T04:58:11.687 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20066 root NOFD /proc/20066/fd (opendir: Permission denied) 2024-05-09T04:58:11.688 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20067 root cwd unknown /proc/20067/cwd (readlink: Permission denied) 2024-05-09T04:58:11.688 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20067 root rtd unknown /proc/20067/root (readlink: Permission denied) 2024-05-09T04:58:11.688 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20067 root txt unknown /proc/20067/exe (readlink: Permission denied) 2024-05-09T04:58:11.688 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20067 root NOFD /proc/20067/fd (opendir: Permission denied) 2024-05-09T04:58:11.688 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20068 root cwd unknown /proc/20068/cwd (readlink: Permission denied) 2024-05-09T04:58:11.688 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20068 root rtd unknown /proc/20068/root (readlink: Permission denied) 2024-05-09T04:58:11.688 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20068 root txt unknown /proc/20068/exe (readlink: Permission denied) 2024-05-09T04:58:11.688 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20068 root NOFD /proc/20068/fd (opendir: Permission denied) 2024-05-09T04:58:11.688 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20069 root cwd unknown /proc/20069/cwd (readlink: Permission denied) 2024-05-09T04:58:11.688 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20069 root rtd unknown /proc/20069/root (readlink: Permission denied) 2024-05-09T04:58:11.688 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20069 root txt unknown /proc/20069/exe (readlink: Permission denied) 2024-05-09T04:58:11.688 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20069 root NOFD /proc/20069/fd (opendir: Permission denied) 2024-05-09T04:58:11.688 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20070 root cwd unknown /proc/20070/cwd (readlink: Permission denied) 2024-05-09T04:58:11.688 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20070 root rtd unknown /proc/20070/root (readlink: Permission denied) 2024-05-09T04:58:11.688 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20070 root txt unknown /proc/20070/exe (readlink: Permission denied) 2024-05-09T04:58:11.688 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20070 root NOFD /proc/20070/fd (opendir: Permission denied) 2024-05-09T04:58:11.689 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20071 root cwd unknown /proc/20071/cwd (readlink: Permission denied) 2024-05-09T04:58:11.689 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20071 root rtd unknown /proc/20071/root (readlink: Permission denied) 2024-05-09T04:58:11.689 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20071 root txt unknown /proc/20071/exe (readlink: Permission denied) 2024-05-09T04:58:11.689 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20071 root NOFD /proc/20071/fd (opendir: Permission denied) 2024-05-09T04:58:11.689 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20072 root cwd unknown /proc/20072/cwd (readlink: Permission denied) 2024-05-09T04:58:11.689 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20072 root rtd unknown /proc/20072/root (readlink: Permission denied) 2024-05-09T04:58:11.689 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20072 root txt unknown /proc/20072/exe (readlink: Permission denied) 2024-05-09T04:58:11.689 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20072 root NOFD /proc/20072/fd (opendir: Permission denied) 2024-05-09T04:58:11.689 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20073 root cwd unknown /proc/20073/cwd (readlink: Permission denied) 2024-05-09T04:58:11.689 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20073 root rtd unknown /proc/20073/root (readlink: Permission denied) 2024-05-09T04:58:11.689 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20073 root txt unknown /proc/20073/exe (readlink: Permission denied) 2024-05-09T04:58:11.689 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20073 root NOFD /proc/20073/fd (opendir: Permission denied) 2024-05-09T04:58:11.689 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20074 root cwd unknown /proc/20074/cwd (readlink: Permission denied) 2024-05-09T04:58:11.689 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20074 root rtd unknown /proc/20074/root (readlink: Permission denied) 2024-05-09T04:58:11.689 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20074 root txt unknown /proc/20074/exe (readlink: Permission denied) 2024-05-09T04:58:11.689 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 20074 root NOFD /proc/20074/fd (opendir: Permission denied) 2024-05-09T04:58:11.689 INFO:tasks.workunit.client.0.smithi006.stdout:xfsaild/d 20075 root cwd unknown /proc/20075/cwd (readlink: Permission denied) 2024-05-09T04:58:11.690 INFO:tasks.workunit.client.0.smithi006.stdout:xfsaild/d 20075 root rtd unknown /proc/20075/root (readlink: Permission denied) 2024-05-09T04:58:11.690 INFO:tasks.workunit.client.0.smithi006.stdout:xfsaild/d 20075 root txt unknown /proc/20075/exe (readlink: Permission denied) 2024-05-09T04:58:11.690 INFO:tasks.workunit.client.0.smithi006.stdout:xfsaild/d 20075 root NOFD /proc/20075/fd (opendir: Permission denied) 2024-05-09T04:58:11.690 INFO:tasks.workunit.client.0.smithi006.stdout:sshd 20994 root cwd unknown /proc/20994/cwd (readlink: Permission denied) 2024-05-09T04:58:11.690 INFO:tasks.workunit.client.0.smithi006.stdout:sshd 20994 root rtd unknown /proc/20994/root (readlink: Permission denied) 2024-05-09T04:58:11.690 INFO:tasks.workunit.client.0.smithi006.stdout:sshd 20994 root txt unknown /proc/20994/exe (readlink: Permission denied) 2024-05-09T04:58:11.690 INFO:tasks.workunit.client.0.smithi006.stdout:sshd 20994 root NOFD /proc/20994/fd (opendir: Permission denied) 2024-05-09T04:58:11.690 INFO:tasks.workunit.client.0.smithi006.stdout:sshd 20997 ubuntu cwd unknown /proc/20997/cwd (readlink: Permission denied) 2024-05-09T04:58:11.690 INFO:tasks.workunit.client.0.smithi006.stdout:sshd 20997 ubuntu rtd unknown /proc/20997/root (readlink: Permission denied) 2024-05-09T04:58:11.690 INFO:tasks.workunit.client.0.smithi006.stdout:sshd 20997 ubuntu txt unknown /proc/20997/exe (readlink: Permission denied) 2024-05-09T04:58:11.690 INFO:tasks.workunit.client.0.smithi006.stdout:sshd 20997 ubuntu NOFD /proc/20997/fd (opendir: Permission denied) 2024-05-09T04:58:11.690 INFO:tasks.workunit.client.0.smithi006.stdout:chronyd 21057 chrony cwd unknown /proc/21057/cwd (readlink: Permission denied) 2024-05-09T04:58:11.690 INFO:tasks.workunit.client.0.smithi006.stdout:chronyd 21057 chrony rtd unknown /proc/21057/root (readlink: Permission denied) 2024-05-09T04:58:11.690 INFO:tasks.workunit.client.0.smithi006.stdout:chronyd 21057 chrony txt unknown /proc/21057/exe (readlink: Permission denied) 2024-05-09T04:58:11.691 INFO:tasks.workunit.client.0.smithi006.stdout:chronyd 21057 chrony NOFD /proc/21057/fd (opendir: Permission denied) 2024-05-09T04:58:11.691 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-cras 21412 ceph cwd unknown /proc/21412/cwd (readlink: Permission denied) 2024-05-09T04:58:11.691 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-cras 21412 ceph rtd unknown /proc/21412/root (readlink: Permission denied) 2024-05-09T04:58:11.691 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-cras 21412 ceph txt unknown /proc/21412/exe (readlink: Permission denied) 2024-05-09T04:58:11.691 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-cras 21412 ceph NOFD /proc/21412/fd (opendir: Permission denied) 2024-05-09T04:58:11.691 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/3 21937 root cwd unknown /proc/21937/cwd (readlink: Permission denied) 2024-05-09T04:58:11.691 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/3 21937 root rtd unknown /proc/21937/root (readlink: Permission denied) 2024-05-09T04:58:11.691 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/3 21937 root txt unknown /proc/21937/exe (readlink: Permission denied) 2024-05-09T04:58:11.691 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/3 21937 root NOFD /proc/21937/fd (opendir: Permission denied) 2024-05-09T04:58:11.691 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/0 23061 root cwd unknown /proc/23061/cwd (readlink: Permission denied) 2024-05-09T04:58:11.691 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/0 23061 root rtd unknown /proc/23061/root (readlink: Permission denied) 2024-05-09T04:58:11.691 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/0 23061 root txt unknown /proc/23061/exe (readlink: Permission denied) 2024-05-09T04:58:11.691 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/0 23061 root NOFD /proc/23061/fd (opendir: Permission denied) 2024-05-09T04:58:11.691 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 28828 root cwd unknown /proc/28828/cwd (readlink: Permission denied) 2024-05-09T04:58:11.691 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 28828 root rtd unknown /proc/28828/root (readlink: Permission denied) 2024-05-09T04:58:11.692 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 28828 root txt unknown /proc/28828/exe (readlink: Permission denied) 2024-05-09T04:58:11.692 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 28828 root NOFD /proc/28828/fd (opendir: Permission denied) 2024-05-09T04:58:11.692 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 28844 root cwd unknown /proc/28844/cwd (readlink: Permission denied) 2024-05-09T04:58:11.692 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 28844 root rtd unknown /proc/28844/root (readlink: Permission denied) 2024-05-09T04:58:11.692 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 28844 root txt unknown /proc/28844/exe (readlink: Permission denied) 2024-05-09T04:58:11.692 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 28844 root NOFD /proc/28844/fd (opendir: Permission denied) 2024-05-09T04:58:11.692 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 28858 root cwd unknown /proc/28858/cwd (readlink: Permission denied) 2024-05-09T04:58:11.692 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 28858 root rtd unknown /proc/28858/root (readlink: Permission denied) 2024-05-09T04:58:11.692 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 28858 root txt unknown /proc/28858/exe (readlink: Permission denied) 2024-05-09T04:58:11.692 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 28858 root NOFD /proc/28858/fd (opendir: Permission denied) 2024-05-09T04:58:11.692 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 28874 root cwd unknown /proc/28874/cwd (readlink: Permission denied) 2024-05-09T04:58:11.692 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 28874 root rtd unknown /proc/28874/root (readlink: Permission denied) 2024-05-09T04:58:11.692 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 28874 root txt unknown /proc/28874/exe (readlink: Permission denied) 2024-05-09T04:58:11.692 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 28874 root NOFD /proc/28874/fd (opendir: Permission denied) 2024-05-09T04:58:11.692 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 29831 root cwd unknown /proc/29831/cwd (readlink: Permission denied) 2024-05-09T04:58:11.692 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 29831 root rtd unknown /proc/29831/root (readlink: Permission denied) 2024-05-09T04:58:11.693 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 29831 root txt unknown /proc/29831/exe (readlink: Permission denied) 2024-05-09T04:58:11.693 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 29831 root NOFD /proc/29831/fd (opendir: Permission denied) 2024-05-09T04:58:11.693 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 29833 root cwd unknown /proc/29833/cwd (readlink: Permission denied) 2024-05-09T04:58:11.693 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 29833 root rtd unknown /proc/29833/root (readlink: Permission denied) 2024-05-09T04:58:11.693 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 29833 root txt unknown /proc/29833/exe (readlink: Permission denied) 2024-05-09T04:58:11.693 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 29833 root NOFD /proc/29833/fd (opendir: Permission denied) 2024-05-09T04:58:11.693 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 ceph cwd unknown /proc/29835/cwd (readlink: Permission denied) 2024-05-09T04:58:11.693 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 ceph rtd unknown /proc/29835/root (readlink: Permission denied) 2024-05-09T04:58:11.693 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 ceph txt unknown /proc/29835/exe (readlink: Permission denied) 2024-05-09T04:58:11.693 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 ceph NOFD /proc/29835/fd (opendir: Permission denied) 2024-05-09T04:58:11.693 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29843 log ceph cwd unknown /proc/29835/task/29843/cwd (readlink: Permission denied) 2024-05-09T04:58:11.693 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29843 log ceph rtd unknown /proc/29835/task/29843/root (readlink: Permission denied) 2024-05-09T04:58:11.693 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29843 log ceph txt unknown /proc/29835/task/29843/exe (readlink: Permission denied) 2024-05-09T04:58:11.693 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29843 log ceph NOFD /proc/29835/task/29843/fd (opendir: Permission denied) 2024-05-09T04:58:11.693 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29844 service ceph cwd unknown /proc/29835/task/29844/cwd (readlink: Permission denied) 2024-05-09T04:58:11.694 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29844 service ceph rtd unknown /proc/29835/task/29844/root (readlink: Permission denied) 2024-05-09T04:58:11.694 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29844 service ceph txt unknown /proc/29835/task/29844/exe (readlink: Permission denied) 2024-05-09T04:58:11.694 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29844 service ceph NOFD /proc/29835/task/29844/fd (opendir: Permission denied) 2024-05-09T04:58:11.694 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29845 admin_soc ceph cwd unknown /proc/29835/task/29845/cwd (readlink: Permission denied) 2024-05-09T04:58:11.694 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29845 admin_soc ceph rtd unknown /proc/29835/task/29845/root (readlink: Permission denied) 2024-05-09T04:58:11.694 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29845 admin_soc ceph txt unknown /proc/29835/task/29845/exe (readlink: Permission denied) 2024-05-09T04:58:11.694 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29845 admin_soc ceph NOFD /proc/29835/task/29845/fd (opendir: Permission denied) 2024-05-09T04:58:11.694 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29849 signal_ha ceph cwd unknown /proc/29835/task/29849/cwd (readlink: Permission denied) 2024-05-09T04:58:11.694 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29849 signal_ha ceph rtd unknown /proc/29835/task/29849/root (readlink: Permission denied) 2024-05-09T04:58:11.694 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29849 signal_ha ceph txt unknown /proc/29835/task/29849/exe (readlink: Permission denied) 2024-05-09T04:58:11.694 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29849 signal_ha ceph NOFD /proc/29835/task/29849/fd (opendir: Permission denied) 2024-05-09T04:58:11.694 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29850 rocksdb:l ceph cwd unknown /proc/29835/task/29850/cwd (readlink: Permission denied) 2024-05-09T04:58:11.694 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29850 rocksdb:l ceph rtd unknown /proc/29835/task/29850/root (readlink: Permission denied) 2024-05-09T04:58:11.694 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29850 rocksdb:l ceph txt unknown /proc/29835/task/29850/exe (readlink: Permission denied) 2024-05-09T04:58:11.695 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29850 rocksdb:l ceph NOFD /proc/29835/task/29850/fd (opendir: Permission denied) 2024-05-09T04:58:11.695 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29851 rocksdb:h ceph cwd unknown /proc/29835/task/29851/cwd (readlink: Permission denied) 2024-05-09T04:58:11.695 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29851 rocksdb:h ceph rtd unknown /proc/29835/task/29851/root (readlink: Permission denied) 2024-05-09T04:58:11.695 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29851 rocksdb:h ceph txt unknown /proc/29835/task/29851/exe (readlink: Permission denied) 2024-05-09T04:58:11.695 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29851 rocksdb:h ceph NOFD /proc/29835/task/29851/fd (opendir: Permission denied) 2024-05-09T04:58:11.695 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29867 ceph-mon ceph cwd unknown /proc/29835/task/29867/cwd (readlink: Permission denied) 2024-05-09T04:58:11.695 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29867 ceph-mon ceph rtd unknown /proc/29835/task/29867/root (readlink: Permission denied) 2024-05-09T04:58:11.695 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29867 ceph-mon ceph txt unknown /proc/29835/task/29867/exe (readlink: Permission denied) 2024-05-09T04:58:11.695 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29867 ceph-mon ceph NOFD /proc/29835/task/29867/fd (opendir: Permission denied) 2024-05-09T04:58:11.695 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29868 fn_monsto ceph cwd unknown /proc/29835/task/29868/cwd (readlink: Permission denied) 2024-05-09T04:58:11.695 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29868 fn_monsto ceph rtd unknown /proc/29835/task/29868/root (readlink: Permission denied) 2024-05-09T04:58:11.695 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29868 fn_monsto ceph txt unknown /proc/29835/task/29868/exe (readlink: Permission denied) 2024-05-09T04:58:11.695 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29868 fn_monsto ceph NOFD /proc/29835/task/29868/fd (opendir: Permission denied) 2024-05-09T04:58:11.695 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29869 msgr-work ceph cwd unknown /proc/29835/task/29869/cwd (readlink: Permission denied) 2024-05-09T04:58:11.695 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29869 msgr-work ceph rtd unknown /proc/29835/task/29869/root (readlink: Permission denied) 2024-05-09T04:58:11.695 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29869 msgr-work ceph txt unknown /proc/29835/task/29869/exe (readlink: Permission denied) 2024-05-09T04:58:11.696 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29869 msgr-work ceph NOFD /proc/29835/task/29869/fd (opendir: Permission denied) 2024-05-09T04:58:11.696 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29870 msgr-work ceph cwd unknown /proc/29835/task/29870/cwd (readlink: Permission denied) 2024-05-09T04:58:11.696 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29870 msgr-work ceph rtd unknown /proc/29835/task/29870/root (readlink: Permission denied) 2024-05-09T04:58:11.696 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29870 msgr-work ceph txt unknown /proc/29835/task/29870/exe (readlink: Permission denied) 2024-05-09T04:58:11.696 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29870 msgr-work ceph NOFD /proc/29835/task/29870/fd (opendir: Permission denied) 2024-05-09T04:58:11.696 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29871 msgr-work ceph cwd unknown /proc/29835/task/29871/cwd (readlink: Permission denied) 2024-05-09T04:58:11.696 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29871 msgr-work ceph rtd unknown /proc/29835/task/29871/root (readlink: Permission denied) 2024-05-09T04:58:11.696 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29871 msgr-work ceph txt unknown /proc/29835/task/29871/exe (readlink: Permission denied) 2024-05-09T04:58:11.696 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29871 msgr-work ceph NOFD /proc/29835/task/29871/fd (opendir: Permission denied) 2024-05-09T04:58:11.696 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29872 OpHistory ceph cwd unknown /proc/29835/task/29872/cwd (readlink: Permission denied) 2024-05-09T04:58:11.696 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29872 OpHistory ceph rtd unknown /proc/29835/task/29872/root (readlink: Permission denied) 2024-05-09T04:58:11.696 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29872 OpHistory ceph txt unknown /proc/29835/task/29872/exe (readlink: Permission denied) 2024-05-09T04:58:11.696 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29872 OpHistory ceph NOFD /proc/29835/task/29872/fd (opendir: Permission denied) 2024-05-09T04:58:11.696 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29876 fin ceph cwd unknown /proc/29835/task/29876/cwd (readlink: Permission denied) 2024-05-09T04:58:11.696 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29876 fin ceph rtd unknown /proc/29835/task/29876/root (readlink: Permission denied) 2024-05-09T04:58:11.697 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29876 fin ceph txt unknown /proc/29835/task/29876/exe (readlink: Permission denied) 2024-05-09T04:58:11.697 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29876 fin ceph NOFD /proc/29835/task/29876/fd (opendir: Permission denied) 2024-05-09T04:58:11.697 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29877 safe_time ceph cwd unknown /proc/29835/task/29877/cwd (readlink: Permission denied) 2024-05-09T04:58:11.697 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29877 safe_time ceph rtd unknown /proc/29835/task/29877/root (readlink: Permission denied) 2024-05-09T04:58:11.697 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29877 safe_time ceph txt unknown /proc/29835/task/29877/exe (readlink: Permission denied) 2024-05-09T04:58:11.697 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29877 safe_time ceph NOFD /proc/29835/task/29877/fd (opendir: Permission denied) 2024-05-09T04:58:11.697 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29878 cpu_tp ceph cwd unknown /proc/29835/task/29878/cwd (readlink: Permission denied) 2024-05-09T04:58:11.697 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29878 cpu_tp ceph rtd unknown /proc/29835/task/29878/root (readlink: Permission denied) 2024-05-09T04:58:11.697 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29878 cpu_tp ceph txt unknown /proc/29835/task/29878/exe (readlink: Permission denied) 2024-05-09T04:58:11.697 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29878 cpu_tp ceph NOFD /proc/29835/task/29878/fd (opendir: Permission denied) 2024-05-09T04:58:11.697 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29879 cpu_tp ceph cwd unknown /proc/29835/task/29879/cwd (readlink: Permission denied) 2024-05-09T04:58:11.697 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29879 cpu_tp ceph rtd unknown /proc/29835/task/29879/root (readlink: Permission denied) 2024-05-09T04:58:11.697 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29879 cpu_tp ceph txt unknown /proc/29835/task/29879/exe (readlink: Permission denied) 2024-05-09T04:58:11.697 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29879 cpu_tp ceph NOFD /proc/29835/task/29879/fd (opendir: Permission denied) 2024-05-09T04:58:11.697 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29880 cpu_tp ceph cwd unknown /proc/29835/task/29880/cwd (readlink: Permission denied) 2024-05-09T04:58:11.698 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29880 cpu_tp ceph rtd unknown /proc/29835/task/29880/root (readlink: Permission denied) 2024-05-09T04:58:11.698 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29880 cpu_tp ceph txt unknown /proc/29835/task/29880/exe (readlink: Permission denied) 2024-05-09T04:58:11.698 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29880 cpu_tp ceph NOFD /proc/29835/task/29880/fd (opendir: Permission denied) 2024-05-09T04:58:11.698 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29881 cpu_tp ceph cwd unknown /proc/29835/task/29881/cwd (readlink: Permission denied) 2024-05-09T04:58:11.698 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29881 cpu_tp ceph rtd unknown /proc/29835/task/29881/root (readlink: Permission denied) 2024-05-09T04:58:11.698 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29881 cpu_tp ceph txt unknown /proc/29835/task/29881/exe (readlink: Permission denied) 2024-05-09T04:58:11.698 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29881 cpu_tp ceph NOFD /proc/29835/task/29881/fd (opendir: Permission denied) 2024-05-09T04:58:11.698 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29882 ms_dispat ceph cwd unknown /proc/29835/task/29882/cwd (readlink: Permission denied) 2024-05-09T04:58:11.698 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29882 ms_dispat ceph rtd unknown /proc/29835/task/29882/root (readlink: Permission denied) 2024-05-09T04:58:11.698 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29882 ms_dispat ceph txt unknown /proc/29835/task/29882/exe (readlink: Permission denied) 2024-05-09T04:58:11.698 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29882 ms_dispat ceph NOFD /proc/29835/task/29882/fd (opendir: Permission denied) 2024-05-09T04:58:11.698 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29883 ms_local ceph cwd unknown /proc/29835/task/29883/cwd (readlink: Permission denied) 2024-05-09T04:58:11.698 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29883 ms_local ceph rtd unknown /proc/29835/task/29883/root (readlink: Permission denied) 2024-05-09T04:58:11.698 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29883 ms_local ceph txt unknown /proc/29835/task/29883/exe (readlink: Permission denied) 2024-05-09T04:58:11.698 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29883 ms_local ceph NOFD /proc/29835/task/29883/fd (opendir: Permission denied) 2024-05-09T04:58:11.698 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29884 safe_time ceph cwd unknown /proc/29835/task/29884/cwd (readlink: Permission denied) 2024-05-09T04:58:11.699 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29884 safe_time ceph rtd unknown /proc/29835/task/29884/root (readlink: Permission denied) 2024-05-09T04:58:11.699 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29884 safe_time ceph txt unknown /proc/29835/task/29884/exe (readlink: Permission denied) 2024-05-09T04:58:11.699 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29884 safe_time ceph NOFD /proc/29835/task/29884/fd (opendir: Permission denied) 2024-05-09T04:58:11.699 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29885 ms_dispat ceph cwd unknown /proc/29835/task/29885/cwd (readlink: Permission denied) 2024-05-09T04:58:11.699 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29885 ms_dispat ceph rtd unknown /proc/29835/task/29885/root (readlink: Permission denied) 2024-05-09T04:58:11.699 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29885 ms_dispat ceph txt unknown /proc/29835/task/29885/exe (readlink: Permission denied) 2024-05-09T04:58:11.699 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29885 ms_dispat ceph NOFD /proc/29835/task/29885/fd (opendir: Permission denied) 2024-05-09T04:58:11.699 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29886 ms_local ceph cwd unknown /proc/29835/task/29886/cwd (readlink: Permission denied) 2024-05-09T04:58:11.699 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29886 ms_local ceph rtd unknown /proc/29835/task/29886/root (readlink: Permission denied) 2024-05-09T04:58:11.699 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29886 ms_local ceph txt unknown /proc/29835/task/29886/exe (readlink: Permission denied) 2024-05-09T04:58:11.699 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 29886 ms_local ceph NOFD /proc/29835/task/29886/fd (opendir: Permission denied) 2024-05-09T04:58:11.699 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 62341 rstore_co ceph cwd unknown /proc/29835/task/62341/cwd (readlink: Permission denied) 2024-05-09T04:58:11.699 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 62341 rstore_co ceph rtd unknown /proc/29835/task/62341/root (readlink: Permission denied) 2024-05-09T04:58:11.699 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 62341 rstore_co ceph txt unknown /proc/29835/task/62341/exe (readlink: Permission denied) 2024-05-09T04:58:11.699 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mon 29835 62341 rstore_co ceph NOFD /proc/29835/task/62341/fd (opendir: Permission denied) 2024-05-09T04:58:11.699 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 30063 root cwd unknown /proc/30063/cwd (readlink: Permission denied) 2024-05-09T04:58:11.700 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 30063 root rtd unknown /proc/30063/root (readlink: Permission denied) 2024-05-09T04:58:11.700 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 30063 root txt unknown /proc/30063/exe (readlink: Permission denied) 2024-05-09T04:58:11.700 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 30063 root NOFD /proc/30063/fd (opendir: Permission denied) 2024-05-09T04:58:11.700 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 30065 root cwd unknown /proc/30065/cwd (readlink: Permission denied) 2024-05-09T04:58:11.700 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 30065 root rtd unknown /proc/30065/root (readlink: Permission denied) 2024-05-09T04:58:11.700 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 30065 root txt unknown /proc/30065/exe (readlink: Permission denied) 2024-05-09T04:58:11.700 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 30065 root NOFD /proc/30065/fd (opendir: Permission denied) 2024-05-09T04:58:11.700 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 ceph cwd unknown /proc/30067/cwd (readlink: Permission denied) 2024-05-09T04:58:11.700 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 ceph rtd unknown /proc/30067/root (readlink: Permission denied) 2024-05-09T04:58:11.700 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 ceph txt unknown /proc/30067/exe (readlink: Permission denied) 2024-05-09T04:58:11.700 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 ceph NOFD /proc/30067/fd (opendir: Permission denied) 2024-05-09T04:58:11.700 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32445 log ceph cwd unknown /proc/30067/task/32445/cwd (readlink: Permission denied) 2024-05-09T04:58:11.700 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32445 log ceph rtd unknown /proc/30067/task/32445/root (readlink: Permission denied) 2024-05-09T04:58:11.700 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32445 log ceph txt unknown /proc/30067/task/32445/exe (readlink: Permission denied) 2024-05-09T04:58:11.700 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32445 log ceph NOFD /proc/30067/task/32445/fd (opendir: Permission denied) 2024-05-09T04:58:11.701 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32447 msgr-work ceph cwd unknown /proc/30067/task/32447/cwd (readlink: Permission denied) 2024-05-09T04:58:11.701 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32447 msgr-work ceph rtd unknown /proc/30067/task/32447/root (readlink: Permission denied) 2024-05-09T04:58:11.701 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32447 msgr-work ceph txt unknown /proc/30067/task/32447/exe (readlink: Permission denied) 2024-05-09T04:58:11.701 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32447 msgr-work ceph NOFD /proc/30067/task/32447/fd (opendir: Permission denied) 2024-05-09T04:58:11.701 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32448 msgr-work ceph cwd unknown /proc/30067/task/32448/cwd (readlink: Permission denied) 2024-05-09T04:58:11.701 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32448 msgr-work ceph rtd unknown /proc/30067/task/32448/root (readlink: Permission denied) 2024-05-09T04:58:11.701 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32448 msgr-work ceph txt unknown /proc/30067/task/32448/exe (readlink: Permission denied) 2024-05-09T04:58:11.701 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32448 msgr-work ceph NOFD /proc/30067/task/32448/fd (opendir: Permission denied) 2024-05-09T04:58:11.701 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32449 msgr-work ceph cwd unknown /proc/30067/task/32449/cwd (readlink: Permission denied) 2024-05-09T04:58:11.701 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32449 msgr-work ceph rtd unknown /proc/30067/task/32449/root (readlink: Permission denied) 2024-05-09T04:58:11.701 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32449 msgr-work ceph txt unknown /proc/30067/task/32449/exe (readlink: Permission denied) 2024-05-09T04:58:11.701 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32449 msgr-work ceph NOFD /proc/30067/task/32449/fd (opendir: Permission denied) 2024-05-09T04:58:11.701 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32453 service ceph cwd unknown /proc/30067/task/32453/cwd (readlink: Permission denied) 2024-05-09T04:58:11.701 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32453 service ceph rtd unknown /proc/30067/task/32453/root (readlink: Permission denied) 2024-05-09T04:58:11.701 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32453 service ceph txt unknown /proc/30067/task/32453/exe (readlink: Permission denied) 2024-05-09T04:58:11.702 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32453 service ceph NOFD /proc/30067/task/32453/fd (opendir: Permission denied) 2024-05-09T04:58:11.702 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32454 admin_soc ceph cwd unknown /proc/30067/task/32454/cwd (readlink: Permission denied) 2024-05-09T04:58:11.702 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32454 admin_soc ceph rtd unknown /proc/30067/task/32454/root (readlink: Permission denied) 2024-05-09T04:58:11.702 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32454 admin_soc ceph txt unknown /proc/30067/task/32454/exe (readlink: Permission denied) 2024-05-09T04:58:11.702 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32454 admin_soc ceph NOFD /proc/30067/task/32454/fd (opendir: Permission denied) 2024-05-09T04:58:11.702 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32455 ceph_time ceph cwd unknown /proc/30067/task/32455/cwd (readlink: Permission denied) 2024-05-09T04:58:11.702 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32455 ceph_time ceph rtd unknown /proc/30067/task/32455/root (readlink: Permission denied) 2024-05-09T04:58:11.702 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32455 ceph_time ceph txt unknown /proc/30067/task/32455/exe (readlink: Permission denied) 2024-05-09T04:58:11.702 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32455 ceph_time ceph NOFD /proc/30067/task/32455/fd (opendir: Permission denied) 2024-05-09T04:58:11.702 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32456 fn_anonym ceph cwd unknown /proc/30067/task/32456/cwd (readlink: Permission denied) 2024-05-09T04:58:11.702 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32456 fn_anonym ceph rtd unknown /proc/30067/task/32456/root (readlink: Permission denied) 2024-05-09T04:58:11.702 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32456 fn_anonym ceph txt unknown /proc/30067/task/32456/exe (readlink: Permission denied) 2024-05-09T04:58:11.702 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32456 fn_anonym ceph NOFD /proc/30067/task/32456/fd (opendir: Permission denied) 2024-05-09T04:58:11.702 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32457 signal_ha ceph cwd unknown /proc/30067/task/32457/cwd (readlink: Permission denied) 2024-05-09T04:58:11.702 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32457 signal_ha ceph rtd unknown /proc/30067/task/32457/root (readlink: Permission denied) 2024-05-09T04:58:11.703 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32457 signal_ha ceph txt unknown /proc/30067/task/32457/exe (readlink: Permission denied) 2024-05-09T04:58:11.703 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32457 signal_ha ceph NOFD /proc/30067/task/32457/fd (opendir: Permission denied) 2024-05-09T04:58:11.703 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32458 mgrsb-fin ceph cwd unknown /proc/30067/task/32458/cwd (readlink: Permission denied) 2024-05-09T04:58:11.703 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32458 mgrsb-fin ceph rtd unknown /proc/30067/task/32458/root (readlink: Permission denied) 2024-05-09T04:58:11.703 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32458 mgrsb-fin ceph txt unknown /proc/30067/task/32458/exe (readlink: Permission denied) 2024-05-09T04:58:11.703 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32458 mgrsb-fin ceph NOFD /proc/30067/task/32458/fd (opendir: Permission denied) 2024-05-09T04:58:11.703 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32459 ms_dispat ceph cwd unknown /proc/30067/task/32459/cwd (readlink: Permission denied) 2024-05-09T04:58:11.703 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32459 ms_dispat ceph rtd unknown /proc/30067/task/32459/root (readlink: Permission denied) 2024-05-09T04:58:11.703 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32459 ms_dispat ceph txt unknown /proc/30067/task/32459/exe (readlink: Permission denied) 2024-05-09T04:58:11.703 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32459 ms_dispat ceph NOFD /proc/30067/task/32459/fd (opendir: Permission denied) 2024-05-09T04:58:11.703 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32460 ms_local ceph cwd unknown /proc/30067/task/32460/cwd (readlink: Permission denied) 2024-05-09T04:58:11.703 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32460 ms_local ceph rtd unknown /proc/30067/task/32460/root (readlink: Permission denied) 2024-05-09T04:58:11.703 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32460 ms_local ceph txt unknown /proc/30067/task/32460/exe (readlink: Permission denied) 2024-05-09T04:58:11.703 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32460 ms_local ceph NOFD /proc/30067/task/32460/fd (opendir: Permission denied) 2024-05-09T04:58:11.703 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32461 io_contex ceph cwd unknown /proc/30067/task/32461/cwd (readlink: Permission denied) 2024-05-09T04:58:11.704 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32461 io_contex ceph rtd unknown /proc/30067/task/32461/root (readlink: Permission denied) 2024-05-09T04:58:11.704 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32461 io_contex ceph txt unknown /proc/30067/task/32461/exe (readlink: Permission denied) 2024-05-09T04:58:11.704 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32461 io_contex ceph NOFD /proc/30067/task/32461/fd (opendir: Permission denied) 2024-05-09T04:58:11.704 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32462 io_contex ceph cwd unknown /proc/30067/task/32462/cwd (readlink: Permission denied) 2024-05-09T04:58:11.704 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32462 io_contex ceph rtd unknown /proc/30067/task/32462/root (readlink: Permission denied) 2024-05-09T04:58:11.704 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32462 io_contex ceph txt unknown /proc/30067/task/32462/exe (readlink: Permission denied) 2024-05-09T04:58:11.704 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32462 io_contex ceph NOFD /proc/30067/task/32462/fd (opendir: Permission denied) 2024-05-09T04:58:11.704 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32463 safe_time ceph cwd unknown /proc/30067/task/32463/cwd (readlink: Permission denied) 2024-05-09T04:58:11.704 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32463 safe_time ceph rtd unknown /proc/30067/task/32463/root (readlink: Permission denied) 2024-05-09T04:58:11.704 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32463 safe_time ceph txt unknown /proc/30067/task/32463/exe (readlink: Permission denied) 2024-05-09T04:58:11.704 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32463 safe_time ceph NOFD /proc/30067/task/32463/fd (opendir: Permission denied) 2024-05-09T04:58:11.704 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32464 safe_time ceph cwd unknown /proc/30067/task/32464/cwd (readlink: Permission denied) 2024-05-09T04:58:11.704 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32464 safe_time ceph rtd unknown /proc/30067/task/32464/root (readlink: Permission denied) 2024-05-09T04:58:11.704 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32464 safe_time ceph txt unknown /proc/30067/task/32464/exe (readlink: Permission denied) 2024-05-09T04:58:11.704 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32464 safe_time ceph NOFD /proc/30067/task/32464/fd (opendir: Permission denied) 2024-05-09T04:58:11.705 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32465 safe_time ceph cwd unknown /proc/30067/task/32465/cwd (readlink: Permission denied) 2024-05-09T04:58:11.705 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32465 safe_time ceph rtd unknown /proc/30067/task/32465/root (readlink: Permission denied) 2024-05-09T04:58:11.705 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32465 safe_time ceph txt unknown /proc/30067/task/32465/exe (readlink: Permission denied) 2024-05-09T04:58:11.705 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32465 safe_time ceph NOFD /proc/30067/task/32465/fd (opendir: Permission denied) 2024-05-09T04:58:11.705 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32466 fn_anonym ceph cwd unknown /proc/30067/task/32466/cwd (readlink: Permission denied) 2024-05-09T04:58:11.705 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32466 fn_anonym ceph rtd unknown /proc/30067/task/32466/root (readlink: Permission denied) 2024-05-09T04:58:11.705 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32466 fn_anonym ceph txt unknown /proc/30067/task/32466/exe (readlink: Permission denied) 2024-05-09T04:58:11.705 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32466 fn_anonym ceph NOFD /proc/30067/task/32466/fd (opendir: Permission denied) 2024-05-09T04:58:11.705 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32467 flusher ceph cwd unknown /proc/30067/task/32467/cwd (readlink: Permission denied) 2024-05-09T04:58:11.705 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32467 flusher ceph rtd unknown /proc/30067/task/32467/root (readlink: Permission denied) 2024-05-09T04:58:11.705 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32467 flusher ceph txt unknown /proc/30067/task/32467/exe (readlink: Permission denied) 2024-05-09T04:58:11.705 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32467 flusher ceph NOFD /proc/30067/task/32467/fd (opendir: Permission denied) 2024-05-09T04:58:11.705 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32468 safe_time ceph cwd unknown /proc/30067/task/32468/cwd (readlink: Permission denied) 2024-05-09T04:58:11.705 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32468 safe_time ceph rtd unknown /proc/30067/task/32468/root (readlink: Permission denied) 2024-05-09T04:58:11.706 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32468 safe_time ceph txt unknown /proc/30067/task/32468/exe (readlink: Permission denied) 2024-05-09T04:58:11.706 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32468 safe_time ceph NOFD /proc/30067/task/32468/fd (opendir: Permission denied) 2024-05-09T04:58:11.706 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32647 OpHistory ceph cwd unknown /proc/30067/task/32647/cwd (readlink: Permission denied) 2024-05-09T04:58:11.706 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32647 OpHistory ceph rtd unknown /proc/30067/task/32647/root (readlink: Permission denied) 2024-05-09T04:58:11.706 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32647 OpHistory ceph txt unknown /proc/30067/task/32647/exe (readlink: Permission denied) 2024-05-09T04:58:11.706 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32647 OpHistory ceph NOFD /proc/30067/task/32647/fd (opendir: Permission denied) 2024-05-09T04:58:11.706 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32648 mgr-fin ceph cwd unknown /proc/30067/task/32648/cwd (readlink: Permission denied) 2024-05-09T04:58:11.706 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32648 mgr-fin ceph rtd unknown /proc/30067/task/32648/root (readlink: Permission denied) 2024-05-09T04:58:11.706 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32648 mgr-fin ceph txt unknown /proc/30067/task/32648/exe (readlink: Permission denied) 2024-05-09T04:58:11.706 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32648 mgr-fin ceph NOFD /proc/30067/task/32648/fd (opendir: Permission denied) 2024-05-09T04:58:11.706 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32649 ms_dispat ceph cwd unknown /proc/30067/task/32649/cwd (readlink: Permission denied) 2024-05-09T04:58:11.706 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32649 ms_dispat ceph rtd unknown /proc/30067/task/32649/root (readlink: Permission denied) 2024-05-09T04:58:11.706 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32649 ms_dispat ceph txt unknown /proc/30067/task/32649/exe (readlink: Permission denied) 2024-05-09T04:58:11.706 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32649 ms_dispat ceph NOFD /proc/30067/task/32649/fd (opendir: Permission denied) 2024-05-09T04:58:11.706 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32650 ms_local ceph cwd unknown /proc/30067/task/32650/cwd (readlink: Permission denied) 2024-05-09T04:58:11.707 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32650 ms_local ceph rtd unknown /proc/30067/task/32650/root (readlink: Permission denied) 2024-05-09T04:58:11.707 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32650 ms_local ceph txt unknown /proc/30067/task/32650/exe (readlink: Permission denied) 2024-05-09T04:58:11.707 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32650 ms_local ceph NOFD /proc/30067/task/32650/fd (opendir: Permission denied) 2024-05-09T04:58:11.707 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32651 safe_time ceph cwd unknown /proc/30067/task/32651/cwd (readlink: Permission denied) 2024-05-09T04:58:11.707 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32651 safe_time ceph rtd unknown /proc/30067/task/32651/root (readlink: Permission denied) 2024-05-09T04:58:11.707 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32651 safe_time ceph txt unknown /proc/30067/task/32651/exe (readlink: Permission denied) 2024-05-09T04:58:11.707 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32651 safe_time ceph NOFD /proc/30067/task/32651/fd (opendir: Permission denied) 2024-05-09T04:58:11.707 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32652 cmdfin ceph cwd unknown /proc/30067/task/32652/cwd (readlink: Permission denied) 2024-05-09T04:58:11.707 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32652 cmdfin ceph rtd unknown /proc/30067/task/32652/root (readlink: Permission denied) 2024-05-09T04:58:11.707 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32652 cmdfin ceph txt unknown /proc/30067/task/32652/exe (readlink: Permission denied) 2024-05-09T04:58:11.707 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32652 cmdfin ceph NOFD /proc/30067/task/32652/fd (opendir: Permission denied) 2024-05-09T04:58:11.707 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32656 io_contex ceph cwd unknown /proc/30067/task/32656/cwd (readlink: Permission denied) 2024-05-09T04:58:11.707 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32656 io_contex ceph rtd unknown /proc/30067/task/32656/root (readlink: Permission denied) 2024-05-09T04:58:11.707 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32656 io_contex ceph txt unknown /proc/30067/task/32656/exe (readlink: Permission denied) 2024-05-09T04:58:11.708 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32656 io_contex ceph NOFD /proc/30067/task/32656/fd (opendir: Permission denied) 2024-05-09T04:58:11.708 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32657 io_contex ceph cwd unknown /proc/30067/task/32657/cwd (readlink: Permission denied) 2024-05-09T04:58:11.708 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32657 io_contex ceph rtd unknown /proc/30067/task/32657/root (readlink: Permission denied) 2024-05-09T04:58:11.708 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32657 io_contex ceph txt unknown /proc/30067/task/32657/exe (readlink: Permission denied) 2024-05-09T04:58:11.708 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32657 io_contex ceph NOFD /proc/30067/task/32657/fd (opendir: Permission denied) 2024-05-09T04:58:11.708 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32658 ceph_time ceph cwd unknown /proc/30067/task/32658/cwd (readlink: Permission denied) 2024-05-09T04:58:11.708 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32658 ceph_time ceph rtd unknown /proc/30067/task/32658/root (readlink: Permission denied) 2024-05-09T04:58:11.708 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32658 ceph_time ceph txt unknown /proc/30067/task/32658/exe (readlink: Permission denied) 2024-05-09T04:58:11.708 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32658 ceph_time ceph NOFD /proc/30067/task/32658/fd (opendir: Permission denied) 2024-05-09T04:58:11.708 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32659 ms_dispat ceph cwd unknown /proc/30067/task/32659/cwd (readlink: Permission denied) 2024-05-09T04:58:11.708 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32659 ms_dispat ceph rtd unknown /proc/30067/task/32659/root (readlink: Permission denied) 2024-05-09T04:58:11.708 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32659 ms_dispat ceph txt unknown /proc/30067/task/32659/exe (readlink: Permission denied) 2024-05-09T04:58:11.708 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32659 ms_dispat ceph NOFD /proc/30067/task/32659/fd (opendir: Permission denied) 2024-05-09T04:58:11.708 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32660 ms_local ceph cwd unknown /proc/30067/task/32660/cwd (readlink: Permission denied) 2024-05-09T04:58:11.708 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32660 ms_local ceph rtd unknown /proc/30067/task/32660/root (readlink: Permission denied) 2024-05-09T04:58:11.708 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32660 ms_local ceph txt unknown /proc/30067/task/32660/exe (readlink: Permission denied) 2024-05-09T04:58:11.709 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32660 ms_local ceph NOFD /proc/30067/task/32660/fd (opendir: Permission denied) 2024-05-09T04:58:11.709 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32661 safe_time ceph cwd unknown /proc/30067/task/32661/cwd (readlink: Permission denied) 2024-05-09T04:58:11.709 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32661 safe_time ceph rtd unknown /proc/30067/task/32661/root (readlink: Permission denied) 2024-05-09T04:58:11.709 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32661 safe_time ceph txt unknown /proc/30067/task/32661/exe (readlink: Permission denied) 2024-05-09T04:58:11.709 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32661 safe_time ceph NOFD /proc/30067/task/32661/fd (opendir: Permission denied) 2024-05-09T04:58:11.709 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32662 safe_time ceph cwd unknown /proc/30067/task/32662/cwd (readlink: Permission denied) 2024-05-09T04:58:11.709 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32662 safe_time ceph rtd unknown /proc/30067/task/32662/root (readlink: Permission denied) 2024-05-09T04:58:11.709 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32662 safe_time ceph txt unknown /proc/30067/task/32662/exe (readlink: Permission denied) 2024-05-09T04:58:11.709 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32662 safe_time ceph NOFD /proc/30067/task/32662/fd (opendir: Permission denied) 2024-05-09T04:58:11.709 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32663 balancer ceph cwd unknown /proc/30067/task/32663/cwd (readlink: Permission denied) 2024-05-09T04:58:11.709 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32663 balancer ceph rtd unknown /proc/30067/task/32663/root (readlink: Permission denied) 2024-05-09T04:58:11.709 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32663 balancer ceph txt unknown /proc/30067/task/32663/exe (readlink: Permission denied) 2024-05-09T04:58:11.710 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32663 balancer ceph NOFD /proc/30067/task/32663/fd (opendir: Permission denied) 2024-05-09T04:58:11.710 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32664 m-fin-bal ceph cwd unknown /proc/30067/task/32664/cwd (readlink: Permission denied) 2024-05-09T04:58:11.710 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32664 m-fin-bal ceph rtd unknown /proc/30067/task/32664/root (readlink: Permission denied) 2024-05-09T04:58:11.710 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32664 m-fin-bal ceph txt unknown /proc/30067/task/32664/exe (readlink: Permission denied) 2024-05-09T04:58:11.710 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32664 m-fin-bal ceph NOFD /proc/30067/task/32664/fd (opendir: Permission denied) 2024-05-09T04:58:11.710 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32665 mgr-fin ceph cwd unknown /proc/30067/task/32665/cwd (readlink: Permission denied) 2024-05-09T04:58:11.710 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32665 mgr-fin ceph rtd unknown /proc/30067/task/32665/root (readlink: Permission denied) 2024-05-09T04:58:11.710 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32665 mgr-fin ceph txt unknown /proc/30067/task/32665/exe (readlink: Permission denied) 2024-05-09T04:58:11.710 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32665 mgr-fin ceph NOFD /proc/30067/task/32665/fd (opendir: Permission denied) 2024-05-09T04:58:11.710 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32666 mgr-fin ceph cwd unknown /proc/30067/task/32666/cwd (readlink: Permission denied) 2024-05-09T04:58:11.710 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32666 mgr-fin ceph rtd unknown /proc/30067/task/32666/root (readlink: Permission denied) 2024-05-09T04:58:11.710 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32666 mgr-fin ceph txt unknown /proc/30067/task/32666/exe (readlink: Permission denied) 2024-05-09T04:58:11.710 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32666 mgr-fin ceph NOFD /proc/30067/task/32666/fd (opendir: Permission denied) 2024-05-09T04:58:11.710 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32667 mgr-fin ceph cwd unknown /proc/30067/task/32667/cwd (readlink: Permission denied) 2024-05-09T04:58:11.711 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32667 mgr-fin ceph rtd unknown /proc/30067/task/32667/root (readlink: Permission denied) 2024-05-09T04:58:11.711 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32667 mgr-fin ceph txt unknown /proc/30067/task/32667/exe (readlink: Permission denied) 2024-05-09T04:58:11.711 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32667 mgr-fin ceph NOFD /proc/30067/task/32667/fd (opendir: Permission denied) 2024-05-09T04:58:11.711 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32668 mgr-fin ceph cwd unknown /proc/30067/task/32668/cwd (readlink: Permission denied) 2024-05-09T04:58:11.711 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32668 mgr-fin ceph rtd unknown /proc/30067/task/32668/root (readlink: Permission denied) 2024-05-09T04:58:11.711 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32668 mgr-fin ceph txt unknown /proc/30067/task/32668/exe (readlink: Permission denied) 2024-05-09T04:58:11.711 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32668 mgr-fin ceph NOFD /proc/30067/task/32668/fd (opendir: Permission denied) 2024-05-09T04:58:11.711 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32669 mgr-fin ceph cwd unknown /proc/30067/task/32669/cwd (readlink: Permission denied) 2024-05-09T04:58:11.711 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32669 mgr-fin ceph rtd unknown /proc/30067/task/32669/root (readlink: Permission denied) 2024-05-09T04:58:11.711 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32669 mgr-fin ceph txt unknown /proc/30067/task/32669/exe (readlink: Permission denied) 2024-05-09T04:58:11.711 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32669 mgr-fin ceph NOFD /proc/30067/task/32669/fd (opendir: Permission denied) 2024-05-09T04:58:11.711 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32670 mgr-fin ceph cwd unknown /proc/30067/task/32670/cwd (readlink: Permission denied) 2024-05-09T04:58:11.711 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32670 mgr-fin ceph rtd unknown /proc/30067/task/32670/root (readlink: Permission denied) 2024-05-09T04:58:11.711 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32670 mgr-fin ceph txt unknown /proc/30067/task/32670/exe (readlink: Permission denied) 2024-05-09T04:58:11.712 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32670 mgr-fin ceph NOFD /proc/30067/task/32670/fd (opendir: Permission denied) 2024-05-09T04:58:11.712 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32671 mgr-fin ceph cwd unknown /proc/30067/task/32671/cwd (readlink: Permission denied) 2024-05-09T04:58:11.712 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32671 mgr-fin ceph rtd unknown /proc/30067/task/32671/root (readlink: Permission denied) 2024-05-09T04:58:11.712 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32671 mgr-fin ceph txt unknown /proc/30067/task/32671/exe (readlink: Permission denied) 2024-05-09T04:58:11.712 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32671 mgr-fin ceph NOFD /proc/30067/task/32671/fd (opendir: Permission denied) 2024-05-09T04:58:11.712 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32672 mgr-fin ceph cwd unknown /proc/30067/task/32672/cwd (readlink: Permission denied) 2024-05-09T04:58:11.712 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32672 mgr-fin ceph rtd unknown /proc/30067/task/32672/root (readlink: Permission denied) 2024-05-09T04:58:11.712 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32672 mgr-fin ceph txt unknown /proc/30067/task/32672/exe (readlink: Permission denied) 2024-05-09T04:58:11.712 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32672 mgr-fin ceph NOFD /proc/30067/task/32672/fd (opendir: Permission denied) 2024-05-09T04:58:11.712 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32673 mgr-fin ceph cwd unknown /proc/30067/task/32673/cwd (readlink: Permission denied) 2024-05-09T04:58:11.712 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32673 mgr-fin ceph rtd unknown /proc/30067/task/32673/root (readlink: Permission denied) 2024-05-09T04:58:11.712 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32673 mgr-fin ceph txt unknown /proc/30067/task/32673/exe (readlink: Permission denied) 2024-05-09T04:58:11.712 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32673 mgr-fin ceph NOFD /proc/30067/task/32673/fd (opendir: Permission denied) 2024-05-09T04:58:11.712 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32674 mgr-fin ceph cwd unknown /proc/30067/task/32674/cwd (readlink: Permission denied) 2024-05-09T04:58:11.713 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32674 mgr-fin ceph rtd unknown /proc/30067/task/32674/root (readlink: Permission denied) 2024-05-09T04:58:11.713 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32674 mgr-fin ceph txt unknown /proc/30067/task/32674/exe (readlink: Permission denied) 2024-05-09T04:58:11.713 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32674 mgr-fin ceph NOFD /proc/30067/task/32674/fd (opendir: Permission denied) 2024-05-09T04:58:11.713 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32675 mgr-fin ceph cwd unknown /proc/30067/task/32675/cwd (readlink: Permission denied) 2024-05-09T04:58:11.713 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32675 mgr-fin ceph rtd unknown /proc/30067/task/32675/root (readlink: Permission denied) 2024-05-09T04:58:11.713 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32675 mgr-fin ceph txt unknown /proc/30067/task/32675/exe (readlink: Permission denied) 2024-05-09T04:58:11.713 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32675 mgr-fin ceph NOFD /proc/30067/task/32675/fd (opendir: Permission denied) 2024-05-09T04:58:11.713 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32676 mgr-fin ceph cwd unknown /proc/30067/task/32676/cwd (readlink: Permission denied) 2024-05-09T04:58:11.713 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32676 mgr-fin ceph rtd unknown /proc/30067/task/32676/root (readlink: Permission denied) 2024-05-09T04:58:11.713 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32676 mgr-fin ceph txt unknown /proc/30067/task/32676/exe (readlink: Permission denied) 2024-05-09T04:58:11.713 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32676 mgr-fin ceph NOFD /proc/30067/task/32676/fd (opendir: Permission denied) 2024-05-09T04:58:11.713 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32677 mgr-fin ceph cwd unknown /proc/30067/task/32677/cwd (readlink: Permission denied) 2024-05-09T04:58:11.713 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32677 mgr-fin ceph rtd unknown /proc/30067/task/32677/root (readlink: Permission denied) 2024-05-09T04:58:11.713 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32677 mgr-fin ceph txt unknown /proc/30067/task/32677/exe (readlink: Permission denied) 2024-05-09T04:58:11.714 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32677 mgr-fin ceph NOFD /proc/30067/task/32677/fd (opendir: Permission denied) 2024-05-09T04:58:11.714 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32678 mgr-fin ceph cwd unknown /proc/30067/task/32678/cwd (readlink: Permission denied) 2024-05-09T04:58:11.714 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32678 mgr-fin ceph rtd unknown /proc/30067/task/32678/root (readlink: Permission denied) 2024-05-09T04:58:11.714 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32678 mgr-fin ceph txt unknown /proc/30067/task/32678/exe (readlink: Permission denied) 2024-05-09T04:58:11.714 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32678 mgr-fin ceph NOFD /proc/30067/task/32678/fd (opendir: Permission denied) 2024-05-09T04:58:11.714 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32679 mgr-fin ceph cwd unknown /proc/30067/task/32679/cwd (readlink: Permission denied) 2024-05-09T04:58:11.714 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32679 mgr-fin ceph rtd unknown /proc/30067/task/32679/root (readlink: Permission denied) 2024-05-09T04:58:11.714 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32679 mgr-fin ceph txt unknown /proc/30067/task/32679/exe (readlink: Permission denied) 2024-05-09T04:58:11.714 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32679 mgr-fin ceph NOFD /proc/30067/task/32679/fd (opendir: Permission denied) 2024-05-09T04:58:11.714 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32680 mgr-fin ceph cwd unknown /proc/30067/task/32680/cwd (readlink: Permission denied) 2024-05-09T04:58:11.714 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32680 mgr-fin ceph rtd unknown /proc/30067/task/32680/root (readlink: Permission denied) 2024-05-09T04:58:11.714 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32680 mgr-fin ceph txt unknown /proc/30067/task/32680/exe (readlink: Permission denied) 2024-05-09T04:58:11.714 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32680 mgr-fin ceph NOFD /proc/30067/task/32680/fd (opendir: Permission denied) 2024-05-09T04:58:11.714 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32681 cephadm ceph cwd unknown /proc/30067/task/32681/cwd (readlink: Permission denied) 2024-05-09T04:58:11.715 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32681 cephadm ceph rtd unknown /proc/30067/task/32681/root (readlink: Permission denied) 2024-05-09T04:58:11.715 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32681 cephadm ceph txt unknown /proc/30067/task/32681/exe (readlink: Permission denied) 2024-05-09T04:58:11.715 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32681 cephadm ceph NOFD /proc/30067/task/32681/fd (opendir: Permission denied) 2024-05-09T04:58:11.715 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32682 m-fin-cep ceph cwd unknown /proc/30067/task/32682/cwd (readlink: Permission denied) 2024-05-09T04:58:11.715 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32682 m-fin-cep ceph rtd unknown /proc/30067/task/32682/root (readlink: Permission denied) 2024-05-09T04:58:11.715 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32682 m-fin-cep ceph txt unknown /proc/30067/task/32682/exe (readlink: Permission denied) 2024-05-09T04:58:11.715 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32682 m-fin-cep ceph NOFD /proc/30067/task/32682/fd (opendir: Permission denied) 2024-05-09T04:58:11.715 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32683 mgr-fin ceph cwd unknown /proc/30067/task/32683/cwd (readlink: Permission denied) 2024-05-09T04:58:11.715 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32683 mgr-fin ceph rtd unknown /proc/30067/task/32683/root (readlink: Permission denied) 2024-05-09T04:58:11.715 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32683 mgr-fin ceph txt unknown /proc/30067/task/32683/exe (readlink: Permission denied) 2024-05-09T04:58:11.715 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32683 mgr-fin ceph NOFD /proc/30067/task/32683/fd (opendir: Permission denied) 2024-05-09T04:58:11.715 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32684 crash ceph cwd unknown /proc/30067/task/32684/cwd (readlink: Permission denied) 2024-05-09T04:58:11.715 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32684 crash ceph rtd unknown /proc/30067/task/32684/root (readlink: Permission denied) 2024-05-09T04:58:11.715 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32684 crash ceph txt unknown /proc/30067/task/32684/exe (readlink: Permission denied) 2024-05-09T04:58:11.716 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32684 crash ceph NOFD /proc/30067/task/32684/fd (opendir: Permission denied) 2024-05-09T04:58:11.716 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32685 m-fin-cra ceph cwd unknown /proc/30067/task/32685/cwd (readlink: Permission denied) 2024-05-09T04:58:11.716 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32685 m-fin-cra ceph rtd unknown /proc/30067/task/32685/root (readlink: Permission denied) 2024-05-09T04:58:11.716 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32685 m-fin-cra ceph txt unknown /proc/30067/task/32685/exe (readlink: Permission denied) 2024-05-09T04:58:11.716 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32685 m-fin-cra ceph NOFD /proc/30067/task/32685/fd (opendir: Permission denied) 2024-05-09T04:58:11.716 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32686 mgr-fin ceph cwd unknown /proc/30067/task/32686/cwd (readlink: Permission denied) 2024-05-09T04:58:11.716 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32686 mgr-fin ceph rtd unknown /proc/30067/task/32686/root (readlink: Permission denied) 2024-05-09T04:58:11.716 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32686 mgr-fin ceph txt unknown /proc/30067/task/32686/exe (readlink: Permission denied) 2024-05-09T04:58:11.716 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32686 mgr-fin ceph NOFD /proc/30067/task/32686/fd (opendir: Permission denied) 2024-05-09T04:58:11.718 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32687 dashboard ceph cwd unknown /proc/30067/task/32687/cwd (readlink: Permission denied) 2024-05-09T04:58:11.718 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32687 dashboard ceph rtd unknown /proc/30067/task/32687/root (readlink: Permission denied) 2024-05-09T04:58:11.718 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32687 dashboard ceph txt unknown /proc/30067/task/32687/exe (readlink: Permission denied) 2024-05-09T04:58:11.718 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32687 dashboard ceph NOFD /proc/30067/task/32687/fd (opendir: Permission denied) 2024-05-09T04:58:11.718 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32688 m-fin-das ceph cwd unknown /proc/30067/task/32688/cwd (readlink: Permission denied) 2024-05-09T04:58:11.718 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32688 m-fin-das ceph rtd unknown /proc/30067/task/32688/root (readlink: Permission denied) 2024-05-09T04:58:11.718 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32688 m-fin-das ceph txt unknown /proc/30067/task/32688/exe (readlink: Permission denied) 2024-05-09T04:58:11.718 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32688 m-fin-das ceph NOFD /proc/30067/task/32688/fd (opendir: Permission denied) 2024-05-09T04:58:11.718 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32689 cephadm ceph cwd unknown /proc/30067/task/32689/cwd (readlink: Permission denied) 2024-05-09T04:58:11.718 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32689 cephadm ceph rtd unknown /proc/30067/task/32689/root (readlink: Permission denied) 2024-05-09T04:58:11.718 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32689 cephadm ceph txt unknown /proc/30067/task/32689/exe (readlink: Permission denied) 2024-05-09T04:58:11.718 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32689 cephadm ceph NOFD /proc/30067/task/32689/fd (opendir: Permission denied) 2024-05-09T04:58:11.718 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32690 mgr-fin ceph cwd unknown /proc/30067/task/32690/cwd (readlink: Permission denied) 2024-05-09T04:58:11.719 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32690 mgr-fin ceph rtd unknown /proc/30067/task/32690/root (readlink: Permission denied) 2024-05-09T04:58:11.719 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32690 mgr-fin ceph txt unknown /proc/30067/task/32690/exe (readlink: Permission denied) 2024-05-09T04:58:11.719 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32690 mgr-fin ceph NOFD /proc/30067/task/32690/fd (opendir: Permission denied) 2024-05-09T04:58:11.719 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32691 mgr-fin ceph cwd unknown /proc/30067/task/32691/cwd (readlink: Permission denied) 2024-05-09T04:58:11.719 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32691 mgr-fin ceph rtd unknown /proc/30067/task/32691/root (readlink: Permission denied) 2024-05-09T04:58:11.719 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32691 mgr-fin ceph txt unknown /proc/30067/task/32691/exe (readlink: Permission denied) 2024-05-09T04:58:11.719 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32691 mgr-fin ceph NOFD /proc/30067/task/32691/fd (opendir: Permission denied) 2024-05-09T04:58:11.719 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32692 devicehea ceph cwd unknown /proc/30067/task/32692/cwd (readlink: Permission denied) 2024-05-09T04:58:11.719 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32692 devicehea ceph rtd unknown /proc/30067/task/32692/root (readlink: Permission denied) 2024-05-09T04:58:11.719 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32692 devicehea ceph txt unknown /proc/30067/task/32692/exe (readlink: Permission denied) 2024-05-09T04:58:11.719 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32692 devicehea ceph NOFD /proc/30067/task/32692/fd (opendir: Permission denied) 2024-05-09T04:58:11.719 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32693 mgr-fin ceph cwd unknown /proc/30067/task/32693/cwd (readlink: Permission denied) 2024-05-09T04:58:11.719 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32693 mgr-fin ceph rtd unknown /proc/30067/task/32693/root (readlink: Permission denied) 2024-05-09T04:58:11.719 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32693 mgr-fin ceph txt unknown /proc/30067/task/32693/exe (readlink: Permission denied) 2024-05-09T04:58:11.720 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32693 mgr-fin ceph NOFD /proc/30067/task/32693/fd (opendir: Permission denied) 2024-05-09T04:58:11.720 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32694 m-fin-dev ceph cwd unknown /proc/30067/task/32694/cwd (readlink: Permission denied) 2024-05-09T04:58:11.720 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32694 m-fin-dev ceph rtd unknown /proc/30067/task/32694/root (readlink: Permission denied) 2024-05-09T04:58:11.720 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32694 m-fin-dev ceph txt unknown /proc/30067/task/32694/exe (readlink: Permission denied) 2024-05-09T04:58:11.720 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32694 m-fin-dev ceph NOFD /proc/30067/task/32694/fd (opendir: Permission denied) 2024-05-09T04:58:11.720 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32695 mgr-fin ceph cwd unknown /proc/30067/task/32695/cwd (readlink: Permission denied) 2024-05-09T04:58:11.720 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32695 mgr-fin ceph rtd unknown /proc/30067/task/32695/root (readlink: Permission denied) 2024-05-09T04:58:11.720 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32695 mgr-fin ceph txt unknown /proc/30067/task/32695/exe (readlink: Permission denied) 2024-05-09T04:58:11.720 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32695 mgr-fin ceph NOFD /proc/30067/task/32695/fd (opendir: Permission denied) 2024-05-09T04:58:11.720 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32697 m-fin-ios ceph cwd unknown /proc/30067/task/32697/cwd (readlink: Permission denied) 2024-05-09T04:58:11.720 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32697 m-fin-ios ceph rtd unknown /proc/30067/task/32697/root (readlink: Permission denied) 2024-05-09T04:58:11.720 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32697 m-fin-ios ceph txt unknown /proc/30067/task/32697/exe (readlink: Permission denied) 2024-05-09T04:58:11.720 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32697 m-fin-ios ceph NOFD /proc/30067/task/32697/fd (opendir: Permission denied) 2024-05-09T04:58:11.720 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32698 mgr-fin ceph cwd unknown /proc/30067/task/32698/cwd (readlink: Permission denied) 2024-05-09T04:58:11.720 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32698 mgr-fin ceph rtd unknown /proc/30067/task/32698/root (readlink: Permission denied) 2024-05-09T04:58:11.721 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32698 mgr-fin ceph txt unknown /proc/30067/task/32698/exe (readlink: Permission denied) 2024-05-09T04:58:11.721 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32698 mgr-fin ceph NOFD /proc/30067/task/32698/fd (opendir: Permission denied) 2024-05-09T04:58:11.721 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32699 mgr-fin ceph cwd unknown /proc/30067/task/32699/cwd (readlink: Permission denied) 2024-05-09T04:58:11.721 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32699 mgr-fin ceph rtd unknown /proc/30067/task/32699/root (readlink: Permission denied) 2024-05-09T04:58:11.721 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32699 mgr-fin ceph txt unknown /proc/30067/task/32699/exe (readlink: Permission denied) 2024-05-09T04:58:11.721 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32699 mgr-fin ceph NOFD /proc/30067/task/32699/fd (opendir: Permission denied) 2024-05-09T04:58:11.721 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32701 m-fin-nfs ceph cwd unknown /proc/30067/task/32701/cwd (readlink: Permission denied) 2024-05-09T04:58:11.721 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32701 m-fin-nfs ceph rtd unknown /proc/30067/task/32701/root (readlink: Permission denied) 2024-05-09T04:58:11.721 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32701 m-fin-nfs ceph txt unknown /proc/30067/task/32701/exe (readlink: Permission denied) 2024-05-09T04:58:11.721 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32701 m-fin-nfs ceph NOFD /proc/30067/task/32701/fd (opendir: Permission denied) 2024-05-09T04:58:11.721 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32702 mgr-fin ceph cwd unknown /proc/30067/task/32702/cwd (readlink: Permission denied) 2024-05-09T04:58:11.721 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32702 mgr-fin ceph rtd unknown /proc/30067/task/32702/root (readlink: Permission denied) 2024-05-09T04:58:11.721 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32702 mgr-fin ceph txt unknown /proc/30067/task/32702/exe (readlink: Permission denied) 2024-05-09T04:58:11.721 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32702 mgr-fin ceph NOFD /proc/30067/task/32702/fd (opendir: Permission denied) 2024-05-09T04:58:11.722 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32703 mgr-fin ceph cwd unknown /proc/30067/task/32703/cwd (readlink: Permission denied) 2024-05-09T04:58:11.722 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32703 mgr-fin ceph rtd unknown /proc/30067/task/32703/root (readlink: Permission denied) 2024-05-09T04:58:11.722 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32703 mgr-fin ceph txt unknown /proc/30067/task/32703/exe (readlink: Permission denied) 2024-05-09T04:58:11.722 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32703 mgr-fin ceph NOFD /proc/30067/task/32703/fd (opendir: Permission denied) 2024-05-09T04:58:11.722 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32705 mgr-fin ceph cwd unknown /proc/30067/task/32705/cwd (readlink: Permission denied) 2024-05-09T04:58:11.722 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32705 mgr-fin ceph rtd unknown /proc/30067/task/32705/root (readlink: Permission denied) 2024-05-09T04:58:11.722 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32705 mgr-fin ceph txt unknown /proc/30067/task/32705/exe (readlink: Permission denied) 2024-05-09T04:58:11.722 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32705 mgr-fin ceph NOFD /proc/30067/task/32705/fd (opendir: Permission denied) 2024-05-09T04:58:11.722 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32706 m-fin-orc ceph cwd unknown /proc/30067/task/32706/cwd (readlink: Permission denied) 2024-05-09T04:58:11.722 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32706 m-fin-orc ceph rtd unknown /proc/30067/task/32706/root (readlink: Permission denied) 2024-05-09T04:58:11.722 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32706 m-fin-orc ceph txt unknown /proc/30067/task/32706/exe (readlink: Permission denied) 2024-05-09T04:58:11.722 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32706 m-fin-orc ceph NOFD /proc/30067/task/32706/fd (opendir: Permission denied) 2024-05-09T04:58:11.722 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32707 mgr-fin ceph cwd unknown /proc/30067/task/32707/cwd (readlink: Permission denied) 2024-05-09T04:58:11.722 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32707 mgr-fin ceph rtd unknown /proc/30067/task/32707/root (readlink: Permission denied) 2024-05-09T04:58:11.723 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32707 mgr-fin ceph txt unknown /proc/30067/task/32707/exe (readlink: Permission denied) 2024-05-09T04:58:11.723 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32707 mgr-fin ceph NOFD /proc/30067/task/32707/fd (opendir: Permission denied) 2024-05-09T04:58:11.723 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32708 pg_autosc ceph cwd unknown /proc/30067/task/32708/cwd (readlink: Permission denied) 2024-05-09T04:58:11.723 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32708 pg_autosc ceph rtd unknown /proc/30067/task/32708/root (readlink: Permission denied) 2024-05-09T04:58:11.723 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32708 pg_autosc ceph txt unknown /proc/30067/task/32708/exe (readlink: Permission denied) 2024-05-09T04:58:11.723 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32708 pg_autosc ceph NOFD /proc/30067/task/32708/fd (opendir: Permission denied) 2024-05-09T04:58:11.723 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32709 m-fin-pg_ ceph cwd unknown /proc/30067/task/32709/cwd (readlink: Permission denied) 2024-05-09T04:58:11.723 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32709 m-fin-pg_ ceph rtd unknown /proc/30067/task/32709/root (readlink: Permission denied) 2024-05-09T04:58:11.723 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32709 m-fin-pg_ ceph txt unknown /proc/30067/task/32709/exe (readlink: Permission denied) 2024-05-09T04:58:11.723 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32709 m-fin-pg_ ceph NOFD /proc/30067/task/32709/fd (opendir: Permission denied) 2024-05-09T04:58:11.723 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32710 progress ceph cwd unknown /proc/30067/task/32710/cwd (readlink: Permission denied) 2024-05-09T04:58:11.723 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32710 progress ceph rtd unknown /proc/30067/task/32710/root (readlink: Permission denied) 2024-05-09T04:58:11.723 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32710 progress ceph txt unknown /proc/30067/task/32710/exe (readlink: Permission denied) 2024-05-09T04:58:11.723 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32710 progress ceph NOFD /proc/30067/task/32710/fd (opendir: Permission denied) 2024-05-09T04:58:11.724 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32711 m-fin-pro ceph cwd unknown /proc/30067/task/32711/cwd (readlink: Permission denied) 2024-05-09T04:58:11.724 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32711 m-fin-pro ceph rtd unknown /proc/30067/task/32711/root (readlink: Permission denied) 2024-05-09T04:58:11.724 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32711 m-fin-pro ceph txt unknown /proc/30067/task/32711/exe (readlink: Permission denied) 2024-05-09T04:58:11.724 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32711 m-fin-pro ceph NOFD /proc/30067/task/32711/fd (opendir: Permission denied) 2024-05-09T04:58:11.724 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32715 io_contex ceph cwd unknown /proc/30067/task/32715/cwd (readlink: Permission denied) 2024-05-09T04:58:11.724 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32715 io_contex ceph rtd unknown /proc/30067/task/32715/root (readlink: Permission denied) 2024-05-09T04:58:11.724 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32715 io_contex ceph txt unknown /proc/30067/task/32715/exe (readlink: Permission denied) 2024-05-09T04:58:11.724 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32715 io_contex ceph NOFD /proc/30067/task/32715/fd (opendir: Permission denied) 2024-05-09T04:58:11.724 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32716 io_contex ceph cwd unknown /proc/30067/task/32716/cwd (readlink: Permission denied) 2024-05-09T04:58:11.724 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32716 io_contex ceph rtd unknown /proc/30067/task/32716/root (readlink: Permission denied) 2024-05-09T04:58:11.724 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32716 io_contex ceph txt unknown /proc/30067/task/32716/exe (readlink: Permission denied) 2024-05-09T04:58:11.724 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32716 io_contex ceph NOFD /proc/30067/task/32716/fd (opendir: Permission denied) 2024-05-09T04:58:11.724 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32717 ceph_time ceph cwd unknown /proc/30067/task/32717/cwd (readlink: Permission denied) 2024-05-09T04:58:11.724 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32717 ceph_time ceph rtd unknown /proc/30067/task/32717/root (readlink: Permission denied) 2024-05-09T04:58:11.725 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32717 ceph_time ceph txt unknown /proc/30067/task/32717/exe (readlink: Permission denied) 2024-05-09T04:58:11.725 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32717 ceph_time ceph NOFD /proc/30067/task/32717/fd (opendir: Permission denied) 2024-05-09T04:58:11.725 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32718 ms_dispat ceph cwd unknown /proc/30067/task/32718/cwd (readlink: Permission denied) 2024-05-09T04:58:11.725 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32718 ms_dispat ceph rtd unknown /proc/30067/task/32718/root (readlink: Permission denied) 2024-05-09T04:58:11.725 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32718 ms_dispat ceph txt unknown /proc/30067/task/32718/exe (readlink: Permission denied) 2024-05-09T04:58:11.725 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32718 ms_dispat ceph NOFD /proc/30067/task/32718/fd (opendir: Permission denied) 2024-05-09T04:58:11.725 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32719 ms_local ceph cwd unknown /proc/30067/task/32719/cwd (readlink: Permission denied) 2024-05-09T04:58:11.725 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32719 ms_local ceph rtd unknown /proc/30067/task/32719/root (readlink: Permission denied) 2024-05-09T04:58:11.725 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32719 ms_local ceph txt unknown /proc/30067/task/32719/exe (readlink: Permission denied) 2024-05-09T04:58:11.725 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32719 ms_local ceph NOFD /proc/30067/task/32719/fd (opendir: Permission denied) 2024-05-09T04:58:11.725 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32720 safe_time ceph cwd unknown /proc/30067/task/32720/cwd (readlink: Permission denied) 2024-05-09T04:58:11.725 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32720 safe_time ceph rtd unknown /proc/30067/task/32720/root (readlink: Permission denied) 2024-05-09T04:58:11.725 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32720 safe_time ceph txt unknown /proc/30067/task/32720/exe (readlink: Permission denied) 2024-05-09T04:58:11.725 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32720 safe_time ceph NOFD /proc/30067/task/32720/fd (opendir: Permission denied) 2024-05-09T04:58:11.726 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32721 safe_time ceph cwd unknown /proc/30067/task/32721/cwd (readlink: Permission denied) 2024-05-09T04:58:11.726 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32721 safe_time ceph rtd unknown /proc/30067/task/32721/root (readlink: Permission denied) 2024-05-09T04:58:11.726 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32721 safe_time ceph txt unknown /proc/30067/task/32721/exe (readlink: Permission denied) 2024-05-09T04:58:11.726 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32721 safe_time ceph NOFD /proc/30067/task/32721/fd (opendir: Permission denied) 2024-05-09T04:58:11.726 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32722 mgr-fin ceph cwd unknown /proc/30067/task/32722/cwd (readlink: Permission denied) 2024-05-09T04:58:11.726 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32722 mgr-fin ceph rtd unknown /proc/30067/task/32722/root (readlink: Permission denied) 2024-05-09T04:58:11.726 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32722 mgr-fin ceph txt unknown /proc/30067/task/32722/exe (readlink: Permission denied) 2024-05-09T04:58:11.726 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32722 mgr-fin ceph NOFD /proc/30067/task/32722/fd (opendir: Permission denied) 2024-05-09T04:58:11.726 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32724 m-fin-rbd ceph cwd unknown /proc/30067/task/32724/cwd (readlink: Permission denied) 2024-05-09T04:58:11.726 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32724 m-fin-rbd ceph rtd unknown /proc/30067/task/32724/root (readlink: Permission denied) 2024-05-09T04:58:11.726 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32724 m-fin-rbd ceph txt unknown /proc/30067/task/32724/exe (readlink: Permission denied) 2024-05-09T04:58:11.726 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32724 m-fin-rbd ceph NOFD /proc/30067/task/32724/fd (opendir: Permission denied) 2024-05-09T04:58:11.726 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32725 restful ceph cwd unknown /proc/30067/task/32725/cwd (readlink: Permission denied) 2024-05-09T04:58:11.726 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32725 restful ceph rtd unknown /proc/30067/task/32725/root (readlink: Permission denied) 2024-05-09T04:58:11.727 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32725 restful ceph txt unknown /proc/30067/task/32725/exe (readlink: Permission denied) 2024-05-09T04:58:11.727 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32725 restful ceph NOFD /proc/30067/task/32725/fd (opendir: Permission denied) 2024-05-09T04:58:11.727 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32726 m-fin-res ceph cwd unknown /proc/30067/task/32726/cwd (readlink: Permission denied) 2024-05-09T04:58:11.727 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32726 m-fin-res ceph rtd unknown /proc/30067/task/32726/root (readlink: Permission denied) 2024-05-09T04:58:11.727 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32726 m-fin-res ceph txt unknown /proc/30067/task/32726/exe (readlink: Permission denied) 2024-05-09T04:58:11.727 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32726 m-fin-res ceph NOFD /proc/30067/task/32726/fd (opendir: Permission denied) 2024-05-09T04:58:11.727 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32728 m-fin-sta ceph cwd unknown /proc/30067/task/32728/cwd (readlink: Permission denied) 2024-05-09T04:58:11.727 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32728 m-fin-sta ceph rtd unknown /proc/30067/task/32728/root (readlink: Permission denied) 2024-05-09T04:58:11.727 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32728 m-fin-sta ceph txt unknown /proc/30067/task/32728/exe (readlink: Permission denied) 2024-05-09T04:58:11.727 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32728 m-fin-sta ceph NOFD /proc/30067/task/32728/fd (opendir: Permission denied) 2024-05-09T04:58:11.727 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32729 telemetry ceph cwd unknown /proc/30067/task/32729/cwd (readlink: Permission denied) 2024-05-09T04:58:11.727 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32729 telemetry ceph rtd unknown /proc/30067/task/32729/root (readlink: Permission denied) 2024-05-09T04:58:11.727 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32729 telemetry ceph txt unknown /proc/30067/task/32729/exe (readlink: Permission denied) 2024-05-09T04:58:11.727 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32729 telemetry ceph NOFD /proc/30067/task/32729/fd (opendir: Permission denied) 2024-05-09T04:58:11.728 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32730 m-fin-tel ceph cwd unknown /proc/30067/task/32730/cwd (readlink: Permission denied) 2024-05-09T04:58:11.728 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32730 m-fin-tel ceph rtd unknown /proc/30067/task/32730/root (readlink: Permission denied) 2024-05-09T04:58:11.728 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32730 m-fin-tel ceph txt unknown /proc/30067/task/32730/exe (readlink: Permission denied) 2024-05-09T04:58:11.728 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32730 m-fin-tel ceph NOFD /proc/30067/task/32730/fd (opendir: Permission denied) 2024-05-09T04:58:11.728 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32731 mgr-fin ceph cwd unknown /proc/30067/task/32731/cwd (readlink: Permission denied) 2024-05-09T04:58:11.728 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32731 mgr-fin ceph rtd unknown /proc/30067/task/32731/root (readlink: Permission denied) 2024-05-09T04:58:11.728 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32731 mgr-fin ceph txt unknown /proc/30067/task/32731/exe (readlink: Permission denied) 2024-05-09T04:58:11.728 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32731 mgr-fin ceph NOFD /proc/30067/task/32731/fd (opendir: Permission denied) 2024-05-09T04:58:11.728 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32732 mgr-fin ceph cwd unknown /proc/30067/task/32732/cwd (readlink: Permission denied) 2024-05-09T04:58:11.728 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32732 mgr-fin ceph rtd unknown /proc/30067/task/32732/root (readlink: Permission denied) 2024-05-09T04:58:11.728 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32732 mgr-fin ceph txt unknown /proc/30067/task/32732/exe (readlink: Permission denied) 2024-05-09T04:58:11.728 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32732 mgr-fin ceph NOFD /proc/30067/task/32732/fd (opendir: Permission denied) 2024-05-09T04:58:11.728 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32733 mgr-fin ceph cwd unknown /proc/30067/task/32733/cwd (readlink: Permission denied) 2024-05-09T04:58:11.728 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32733 mgr-fin ceph rtd unknown /proc/30067/task/32733/root (readlink: Permission denied) 2024-05-09T04:58:11.729 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32733 mgr-fin ceph txt unknown /proc/30067/task/32733/exe (readlink: Permission denied) 2024-05-09T04:58:11.729 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32733 mgr-fin ceph NOFD /proc/30067/task/32733/fd (opendir: Permission denied) 2024-05-09T04:58:11.729 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32734 mgr-fin ceph cwd unknown /proc/30067/task/32734/cwd (readlink: Permission denied) 2024-05-09T04:58:11.729 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32734 mgr-fin ceph rtd unknown /proc/30067/task/32734/root (readlink: Permission denied) 2024-05-09T04:58:11.729 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32734 mgr-fin ceph txt unknown /proc/30067/task/32734/exe (readlink: Permission denied) 2024-05-09T04:58:11.729 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32734 mgr-fin ceph NOFD /proc/30067/task/32734/fd (opendir: Permission denied) 2024-05-09T04:58:11.729 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32738 mgr-fin ceph cwd unknown /proc/30067/task/32738/cwd (readlink: Permission denied) 2024-05-09T04:58:11.729 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32738 mgr-fin ceph rtd unknown /proc/30067/task/32738/root (readlink: Permission denied) 2024-05-09T04:58:11.729 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32738 mgr-fin ceph txt unknown /proc/30067/task/32738/exe (readlink: Permission denied) 2024-05-09T04:58:11.729 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32738 mgr-fin ceph NOFD /proc/30067/task/32738/fd (opendir: Permission denied) 2024-05-09T04:58:11.729 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32740 io_contex ceph cwd unknown /proc/30067/task/32740/cwd (readlink: Permission denied) 2024-05-09T04:58:11.729 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32740 io_contex ceph rtd unknown /proc/30067/task/32740/root (readlink: Permission denied) 2024-05-09T04:58:11.729 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32740 io_contex ceph txt unknown /proc/30067/task/32740/exe (readlink: Permission denied) 2024-05-09T04:58:11.729 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32740 io_contex ceph NOFD /proc/30067/task/32740/fd (opendir: Permission denied) 2024-05-09T04:58:11.729 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32741 io_contex ceph cwd unknown /proc/30067/task/32741/cwd (readlink: Permission denied) 2024-05-09T04:58:11.730 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32741 io_contex ceph rtd unknown /proc/30067/task/32741/root (readlink: Permission denied) 2024-05-09T04:58:11.730 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32741 io_contex ceph txt unknown /proc/30067/task/32741/exe (readlink: Permission denied) 2024-05-09T04:58:11.730 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32741 io_contex ceph NOFD /proc/30067/task/32741/fd (opendir: Permission denied) 2024-05-09T04:58:11.730 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32742 ceph_time ceph cwd unknown /proc/30067/task/32742/cwd (readlink: Permission denied) 2024-05-09T04:58:11.730 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32742 ceph_time ceph rtd unknown /proc/30067/task/32742/root (readlink: Permission denied) 2024-05-09T04:58:11.730 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32742 ceph_time ceph txt unknown /proc/30067/task/32742/exe (readlink: Permission denied) 2024-05-09T04:58:11.730 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32742 ceph_time ceph NOFD /proc/30067/task/32742/fd (opendir: Permission denied) 2024-05-09T04:58:11.730 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32743 ms_dispat ceph cwd unknown /proc/30067/task/32743/cwd (readlink: Permission denied) 2024-05-09T04:58:11.730 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32743 ms_dispat ceph rtd unknown /proc/30067/task/32743/root (readlink: Permission denied) 2024-05-09T04:58:11.730 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32743 ms_dispat ceph txt unknown /proc/30067/task/32743/exe (readlink: Permission denied) 2024-05-09T04:58:11.730 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32743 ms_dispat ceph NOFD /proc/30067/task/32743/fd (opendir: Permission denied) 2024-05-09T04:58:11.730 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32744 ms_local ceph cwd unknown /proc/30067/task/32744/cwd (readlink: Permission denied) 2024-05-09T04:58:11.730 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32744 ms_local ceph rtd unknown /proc/30067/task/32744/root (readlink: Permission denied) 2024-05-09T04:58:11.731 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32744 ms_local ceph txt unknown /proc/30067/task/32744/exe (readlink: Permission denied) 2024-05-09T04:58:11.731 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32744 ms_local ceph NOFD /proc/30067/task/32744/fd (opendir: Permission denied) 2024-05-09T04:58:11.731 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32745 safe_time ceph cwd unknown /proc/30067/task/32745/cwd (readlink: Permission denied) 2024-05-09T04:58:11.731 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32745 safe_time ceph rtd unknown /proc/30067/task/32745/root (readlink: Permission denied) 2024-05-09T04:58:11.731 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32745 safe_time ceph txt unknown /proc/30067/task/32745/exe (readlink: Permission denied) 2024-05-09T04:58:11.731 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32745 safe_time ceph NOFD /proc/30067/task/32745/fd (opendir: Permission denied) 2024-05-09T04:58:11.731 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32746 safe_time ceph cwd unknown /proc/30067/task/32746/cwd (readlink: Permission denied) 2024-05-09T04:58:11.731 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32746 safe_time ceph rtd unknown /proc/30067/task/32746/root (readlink: Permission denied) 2024-05-09T04:58:11.731 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32746 safe_time ceph txt unknown /proc/30067/task/32746/exe (readlink: Permission denied) 2024-05-09T04:58:11.731 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32746 safe_time ceph NOFD /proc/30067/task/32746/fd (opendir: Permission denied) 2024-05-09T04:58:11.731 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32747 mgr-fin ceph cwd unknown /proc/30067/task/32747/cwd (readlink: Permission denied) 2024-05-09T04:58:11.731 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32747 mgr-fin ceph rtd unknown /proc/30067/task/32747/root (readlink: Permission denied) 2024-05-09T04:58:11.731 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32747 mgr-fin ceph txt unknown /proc/30067/task/32747/exe (readlink: Permission denied) 2024-05-09T04:58:11.731 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32747 mgr-fin ceph NOFD /proc/30067/task/32747/fd (opendir: Permission denied) 2024-05-09T04:58:11.732 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32748 mgr-fin ceph cwd unknown /proc/30067/task/32748/cwd (readlink: Permission denied) 2024-05-09T04:58:11.732 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32748 mgr-fin ceph rtd unknown /proc/30067/task/32748/root (readlink: Permission denied) 2024-05-09T04:58:11.732 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32748 mgr-fin ceph txt unknown /proc/30067/task/32748/exe (readlink: Permission denied) 2024-05-09T04:58:11.732 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32748 mgr-fin ceph NOFD /proc/30067/task/32748/fd (opendir: Permission denied) 2024-05-09T04:58:11.732 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32749 mgr-fin ceph cwd unknown /proc/30067/task/32749/cwd (readlink: Permission denied) 2024-05-09T04:58:11.732 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32749 mgr-fin ceph rtd unknown /proc/30067/task/32749/root (readlink: Permission denied) 2024-05-09T04:58:11.732 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32749 mgr-fin ceph txt unknown /proc/30067/task/32749/exe (readlink: Permission denied) 2024-05-09T04:58:11.732 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32749 mgr-fin ceph NOFD /proc/30067/task/32749/fd (opendir: Permission denied) 2024-05-09T04:58:11.732 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32750 mgr-fin ceph cwd unknown /proc/30067/task/32750/cwd (readlink: Permission denied) 2024-05-09T04:58:11.732 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32750 mgr-fin ceph rtd unknown /proc/30067/task/32750/root (readlink: Permission denied) 2024-05-09T04:58:11.732 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32750 mgr-fin ceph txt unknown /proc/30067/task/32750/exe (readlink: Permission denied) 2024-05-09T04:58:11.732 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32750 mgr-fin ceph NOFD /proc/30067/task/32750/fd (opendir: Permission denied) 2024-05-09T04:58:11.732 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32751 mgr-fin ceph cwd unknown /proc/30067/task/32751/cwd (readlink: Permission denied) 2024-05-09T04:58:11.732 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32751 mgr-fin ceph rtd unknown /proc/30067/task/32751/root (readlink: Permission denied) 2024-05-09T04:58:11.733 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32751 mgr-fin ceph txt unknown /proc/30067/task/32751/exe (readlink: Permission denied) 2024-05-09T04:58:11.733 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32751 mgr-fin ceph NOFD /proc/30067/task/32751/fd (opendir: Permission denied) 2024-05-09T04:58:11.733 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32752 mgr-fin ceph cwd unknown /proc/30067/task/32752/cwd (readlink: Permission denied) 2024-05-09T04:58:11.733 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32752 mgr-fin ceph rtd unknown /proc/30067/task/32752/root (readlink: Permission denied) 2024-05-09T04:58:11.733 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32752 mgr-fin ceph txt unknown /proc/30067/task/32752/exe (readlink: Permission denied) 2024-05-09T04:58:11.733 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32752 mgr-fin ceph NOFD /proc/30067/task/32752/fd (opendir: Permission denied) 2024-05-09T04:58:11.733 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32753 mgr-fin ceph cwd unknown /proc/30067/task/32753/cwd (readlink: Permission denied) 2024-05-09T04:58:11.733 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32753 mgr-fin ceph rtd unknown /proc/30067/task/32753/root (readlink: Permission denied) 2024-05-09T04:58:11.733 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32753 mgr-fin ceph txt unknown /proc/30067/task/32753/exe (readlink: Permission denied) 2024-05-09T04:58:11.733 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32753 mgr-fin ceph NOFD /proc/30067/task/32753/fd (opendir: Permission denied) 2024-05-09T04:58:11.733 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32754 mgr-fin ceph cwd unknown /proc/30067/task/32754/cwd (readlink: Permission denied) 2024-05-09T04:58:11.733 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32754 mgr-fin ceph rtd unknown /proc/30067/task/32754/root (readlink: Permission denied) 2024-05-09T04:58:11.733 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32754 mgr-fin ceph txt unknown /proc/30067/task/32754/exe (readlink: Permission denied) 2024-05-09T04:58:11.733 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32754 mgr-fin ceph NOFD /proc/30067/task/32754/fd (opendir: Permission denied) 2024-05-09T04:58:11.733 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32755 mgr-fin ceph cwd unknown /proc/30067/task/32755/cwd (readlink: Permission denied) 2024-05-09T04:58:11.734 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32755 mgr-fin ceph rtd unknown /proc/30067/task/32755/root (readlink: Permission denied) 2024-05-09T04:58:11.734 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32755 mgr-fin ceph txt unknown /proc/30067/task/32755/exe (readlink: Permission denied) 2024-05-09T04:58:11.734 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32755 mgr-fin ceph NOFD /proc/30067/task/32755/fd (opendir: Permission denied) 2024-05-09T04:58:11.734 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32756 mgr-fin ceph cwd unknown /proc/30067/task/32756/cwd (readlink: Permission denied) 2024-05-09T04:58:11.734 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32756 mgr-fin ceph rtd unknown /proc/30067/task/32756/root (readlink: Permission denied) 2024-05-09T04:58:11.734 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32756 mgr-fin ceph txt unknown /proc/30067/task/32756/exe (readlink: Permission denied) 2024-05-09T04:58:11.734 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32756 mgr-fin ceph NOFD /proc/30067/task/32756/fd (opendir: Permission denied) 2024-05-09T04:58:11.734 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32757 mgr-fin ceph cwd unknown /proc/30067/task/32757/cwd (readlink: Permission denied) 2024-05-09T04:58:11.734 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32757 mgr-fin ceph rtd unknown /proc/30067/task/32757/root (readlink: Permission denied) 2024-05-09T04:58:11.734 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32757 mgr-fin ceph txt unknown /proc/30067/task/32757/exe (readlink: Permission denied) 2024-05-09T04:58:11.734 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32757 mgr-fin ceph NOFD /proc/30067/task/32757/fd (opendir: Permission denied) 2024-05-09T04:58:11.734 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32758 mgr-fin ceph cwd unknown /proc/30067/task/32758/cwd (readlink: Permission denied) 2024-05-09T04:58:11.734 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32758 mgr-fin ceph rtd unknown /proc/30067/task/32758/root (readlink: Permission denied) 2024-05-09T04:58:11.735 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32758 mgr-fin ceph txt unknown /proc/30067/task/32758/exe (readlink: Permission denied) 2024-05-09T04:58:11.735 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32758 mgr-fin ceph NOFD /proc/30067/task/32758/fd (opendir: Permission denied) 2024-05-09T04:58:11.735 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32760 m-fin-vol ceph cwd unknown /proc/30067/task/32760/cwd (readlink: Permission denied) 2024-05-09T04:58:11.735 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32760 m-fin-vol ceph rtd unknown /proc/30067/task/32760/root (readlink: Permission denied) 2024-05-09T04:58:11.735 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32760 m-fin-vol ceph txt unknown /proc/30067/task/32760/exe (readlink: Permission denied) 2024-05-09T04:58:11.735 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32760 m-fin-vol ceph NOFD /proc/30067/task/32760/fd (opendir: Permission denied) 2024-05-09T04:58:11.735 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32857 mgr-fin ceph cwd unknown /proc/30067/task/32857/cwd (readlink: Permission denied) 2024-05-09T04:58:11.735 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32857 mgr-fin ceph rtd unknown /proc/30067/task/32857/root (readlink: Permission denied) 2024-05-09T04:58:11.735 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32857 mgr-fin ceph txt unknown /proc/30067/task/32857/exe (readlink: Permission denied) 2024-05-09T04:58:11.735 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32857 mgr-fin ceph NOFD /proc/30067/task/32857/fd (opendir: Permission denied) 2024-05-09T04:58:11.735 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32858 mgr-fin ceph cwd unknown /proc/30067/task/32858/cwd (readlink: Permission denied) 2024-05-09T04:58:11.735 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32858 mgr-fin ceph rtd unknown /proc/30067/task/32858/root (readlink: Permission denied) 2024-05-09T04:58:11.735 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32858 mgr-fin ceph txt unknown /proc/30067/task/32858/exe (readlink: Permission denied) 2024-05-09T04:58:11.735 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32858 mgr-fin ceph NOFD /proc/30067/task/32858/fd (opendir: Permission denied) 2024-05-09T04:58:11.736 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32859 mgr-fin ceph cwd unknown /proc/30067/task/32859/cwd (readlink: Permission denied) 2024-05-09T04:58:11.736 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32859 mgr-fin ceph rtd unknown /proc/30067/task/32859/root (readlink: Permission denied) 2024-05-09T04:58:11.736 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32859 mgr-fin ceph txt unknown /proc/30067/task/32859/exe (readlink: Permission denied) 2024-05-09T04:58:11.736 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32859 mgr-fin ceph NOFD /proc/30067/task/32859/fd (opendir: Permission denied) 2024-05-09T04:58:11.736 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32860 mgr-fin ceph cwd unknown /proc/30067/task/32860/cwd (readlink: Permission denied) 2024-05-09T04:58:11.736 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32860 mgr-fin ceph rtd unknown /proc/30067/task/32860/root (readlink: Permission denied) 2024-05-09T04:58:11.736 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32860 mgr-fin ceph txt unknown /proc/30067/task/32860/exe (readlink: Permission denied) 2024-05-09T04:58:11.736 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32860 mgr-fin ceph NOFD /proc/30067/task/32860/fd (opendir: Permission denied) 2024-05-09T04:58:11.736 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32861 mgr-fin ceph cwd unknown /proc/30067/task/32861/cwd (readlink: Permission denied) 2024-05-09T04:58:11.736 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32861 mgr-fin ceph rtd unknown /proc/30067/task/32861/root (readlink: Permission denied) 2024-05-09T04:58:11.736 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32861 mgr-fin ceph txt unknown /proc/30067/task/32861/exe (readlink: Permission denied) 2024-05-09T04:58:11.736 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32861 mgr-fin ceph NOFD /proc/30067/task/32861/fd (opendir: Permission denied) 2024-05-09T04:58:11.736 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32862 mgr-fin ceph cwd unknown /proc/30067/task/32862/cwd (readlink: Permission denied) 2024-05-09T04:58:11.736 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32862 mgr-fin ceph rtd unknown /proc/30067/task/32862/root (readlink: Permission denied) 2024-05-09T04:58:11.736 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32862 mgr-fin ceph txt unknown /proc/30067/task/32862/exe (readlink: Permission denied) 2024-05-09T04:58:11.737 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32862 mgr-fin ceph NOFD /proc/30067/task/32862/fd (opendir: Permission denied) 2024-05-09T04:58:11.737 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32863 mgr-fin ceph cwd unknown /proc/30067/task/32863/cwd (readlink: Permission denied) 2024-05-09T04:58:11.737 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32863 mgr-fin ceph rtd unknown /proc/30067/task/32863/root (readlink: Permission denied) 2024-05-09T04:58:11.737 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32863 mgr-fin ceph txt unknown /proc/30067/task/32863/exe (readlink: Permission denied) 2024-05-09T04:58:11.737 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32863 mgr-fin ceph NOFD /proc/30067/task/32863/fd (opendir: Permission denied) 2024-05-09T04:58:11.737 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32864 mgr-fin ceph cwd unknown /proc/30067/task/32864/cwd (readlink: Permission denied) 2024-05-09T04:58:11.737 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32864 mgr-fin ceph rtd unknown /proc/30067/task/32864/root (readlink: Permission denied) 2024-05-09T04:58:11.737 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32864 mgr-fin ceph txt unknown /proc/30067/task/32864/exe (readlink: Permission denied) 2024-05-09T04:58:11.737 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32864 mgr-fin ceph NOFD /proc/30067/task/32864/fd (opendir: Permission denied) 2024-05-09T04:58:11.737 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32865 mgr-fin ceph cwd unknown /proc/30067/task/32865/cwd (readlink: Permission denied) 2024-05-09T04:58:11.737 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32865 mgr-fin ceph rtd unknown /proc/30067/task/32865/root (readlink: Permission denied) 2024-05-09T04:58:11.737 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32865 mgr-fin ceph txt unknown /proc/30067/task/32865/exe (readlink: Permission denied) 2024-05-09T04:58:11.737 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32865 mgr-fin ceph NOFD /proc/30067/task/32865/fd (opendir: Permission denied) 2024-05-09T04:58:11.738 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32866 mgr-fin ceph cwd unknown /proc/30067/task/32866/cwd (readlink: Permission denied) 2024-05-09T04:58:11.738 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32866 mgr-fin ceph rtd unknown /proc/30067/task/32866/root (readlink: Permission denied) 2024-05-09T04:58:11.738 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32866 mgr-fin ceph txt unknown /proc/30067/task/32866/exe (readlink: Permission denied) 2024-05-09T04:58:11.738 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32866 mgr-fin ceph NOFD /proc/30067/task/32866/fd (opendir: Permission denied) 2024-05-09T04:58:11.738 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32867 mgr-fin ceph cwd unknown /proc/30067/task/32867/cwd (readlink: Permission denied) 2024-05-09T04:58:11.738 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32867 mgr-fin ceph rtd unknown /proc/30067/task/32867/root (readlink: Permission denied) 2024-05-09T04:58:11.738 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32867 mgr-fin ceph txt unknown /proc/30067/task/32867/exe (readlink: Permission denied) 2024-05-09T04:58:11.738 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32867 mgr-fin ceph NOFD /proc/30067/task/32867/fd (opendir: Permission denied) 2024-05-09T04:58:11.738 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32894 mgr-fin ceph cwd unknown /proc/30067/task/32894/cwd (readlink: Permission denied) 2024-05-09T04:58:11.738 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32894 mgr-fin ceph rtd unknown /proc/30067/task/32894/root (readlink: Permission denied) 2024-05-09T04:58:11.738 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32894 mgr-fin ceph txt unknown /proc/30067/task/32894/exe (readlink: Permission denied) 2024-05-09T04:58:11.738 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32894 mgr-fin ceph NOFD /proc/30067/task/32894/fd (opendir: Permission denied) 2024-05-09T04:58:11.738 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32895 mgr-fin ceph cwd unknown /proc/30067/task/32895/cwd (readlink: Permission denied) 2024-05-09T04:58:11.738 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32895 mgr-fin ceph rtd unknown /proc/30067/task/32895/root (readlink: Permission denied) 2024-05-09T04:58:11.739 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32895 mgr-fin ceph txt unknown /proc/30067/task/32895/exe (readlink: Permission denied) 2024-05-09T04:58:11.739 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32895 mgr-fin ceph NOFD /proc/30067/task/32895/fd (opendir: Permission denied) 2024-05-09T04:58:11.739 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32896 mgr-fin ceph cwd unknown /proc/30067/task/32896/cwd (readlink: Permission denied) 2024-05-09T04:58:11.739 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32896 mgr-fin ceph rtd unknown /proc/30067/task/32896/root (readlink: Permission denied) 2024-05-09T04:58:11.739 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32896 mgr-fin ceph txt unknown /proc/30067/task/32896/exe (readlink: Permission denied) 2024-05-09T04:58:11.739 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32896 mgr-fin ceph NOFD /proc/30067/task/32896/fd (opendir: Permission denied) 2024-05-09T04:58:11.739 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32897 mgr-fin ceph cwd unknown /proc/30067/task/32897/cwd (readlink: Permission denied) 2024-05-09T04:58:11.739 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32897 mgr-fin ceph rtd unknown /proc/30067/task/32897/root (readlink: Permission denied) 2024-05-09T04:58:11.739 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32897 mgr-fin ceph txt unknown /proc/30067/task/32897/exe (readlink: Permission denied) 2024-05-09T04:58:11.739 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32897 mgr-fin ceph NOFD /proc/30067/task/32897/fd (opendir: Permission denied) 2024-05-09T04:58:11.739 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32898 mgr-fin ceph cwd unknown /proc/30067/task/32898/cwd (readlink: Permission denied) 2024-05-09T04:58:11.739 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32898 mgr-fin ceph rtd unknown /proc/30067/task/32898/root (readlink: Permission denied) 2024-05-09T04:58:11.739 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32898 mgr-fin ceph txt unknown /proc/30067/task/32898/exe (readlink: Permission denied) 2024-05-09T04:58:11.739 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32898 mgr-fin ceph NOFD /proc/30067/task/32898/fd (opendir: Permission denied) 2024-05-09T04:58:11.739 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32899 mgr-fin ceph cwd unknown /proc/30067/task/32899/cwd (readlink: Permission denied) 2024-05-09T04:58:11.740 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32899 mgr-fin ceph rtd unknown /proc/30067/task/32899/root (readlink: Permission denied) 2024-05-09T04:58:11.740 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32899 mgr-fin ceph txt unknown /proc/30067/task/32899/exe (readlink: Permission denied) 2024-05-09T04:58:11.740 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32899 mgr-fin ceph NOFD /proc/30067/task/32899/fd (opendir: Permission denied) 2024-05-09T04:58:11.740 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32900 mgr-fin ceph cwd unknown /proc/30067/task/32900/cwd (readlink: Permission denied) 2024-05-09T04:58:11.740 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32900 mgr-fin ceph rtd unknown /proc/30067/task/32900/root (readlink: Permission denied) 2024-05-09T04:58:11.740 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32900 mgr-fin ceph txt unknown /proc/30067/task/32900/exe (readlink: Permission denied) 2024-05-09T04:58:11.740 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32900 mgr-fin ceph NOFD /proc/30067/task/32900/fd (opendir: Permission denied) 2024-05-09T04:58:11.740 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32901 mgr-fin ceph cwd unknown /proc/30067/task/32901/cwd (readlink: Permission denied) 2024-05-09T04:58:11.740 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32901 mgr-fin ceph rtd unknown /proc/30067/task/32901/root (readlink: Permission denied) 2024-05-09T04:58:11.740 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32901 mgr-fin ceph txt unknown /proc/30067/task/32901/exe (readlink: Permission denied) 2024-05-09T04:58:11.740 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32901 mgr-fin ceph NOFD /proc/30067/task/32901/fd (opendir: Permission denied) 2024-05-09T04:58:11.740 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32902 mgr-fin ceph cwd unknown /proc/30067/task/32902/cwd (readlink: Permission denied) 2024-05-09T04:58:11.740 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32902 mgr-fin ceph rtd unknown /proc/30067/task/32902/root (readlink: Permission denied) 2024-05-09T04:58:11.741 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32902 mgr-fin ceph txt unknown /proc/30067/task/32902/exe (readlink: Permission denied) 2024-05-09T04:58:11.742 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32902 mgr-fin ceph NOFD /proc/30067/task/32902/fd (opendir: Permission denied) 2024-05-09T04:58:11.742 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32903 mgr-fin ceph cwd unknown /proc/30067/task/32903/cwd (readlink: Permission denied) 2024-05-09T04:58:11.742 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32903 mgr-fin ceph rtd unknown /proc/30067/task/32903/root (readlink: Permission denied) 2024-05-09T04:58:11.742 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32903 mgr-fin ceph txt unknown /proc/30067/task/32903/exe (readlink: Permission denied) 2024-05-09T04:58:11.742 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32903 mgr-fin ceph NOFD /proc/30067/task/32903/fd (opendir: Permission denied) 2024-05-09T04:58:11.742 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32904 mgr-fin ceph cwd unknown /proc/30067/task/32904/cwd (readlink: Permission denied) 2024-05-09T04:58:11.742 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32904 mgr-fin ceph rtd unknown /proc/30067/task/32904/root (readlink: Permission denied) 2024-05-09T04:58:11.742 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32904 mgr-fin ceph txt unknown /proc/30067/task/32904/exe (readlink: Permission denied) 2024-05-09T04:58:11.742 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 32904 mgr-fin ceph NOFD /proc/30067/task/32904/fd (opendir: Permission denied) 2024-05-09T04:58:11.742 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33281 dashboard ceph cwd unknown /proc/30067/task/33281/cwd (readlink: Permission denied) 2024-05-09T04:58:11.742 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33281 dashboard ceph rtd unknown /proc/30067/task/33281/root (readlink: Permission denied) 2024-05-09T04:58:11.743 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33281 dashboard ceph txt unknown /proc/30067/task/33281/exe (readlink: Permission denied) 2024-05-09T04:58:11.743 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33281 dashboard ceph NOFD /proc/30067/task/33281/fd (opendir: Permission denied) 2024-05-09T04:58:11.743 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33282 dashboard ceph cwd unknown /proc/30067/task/33282/cwd (readlink: Permission denied) 2024-05-09T04:58:11.743 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33282 dashboard ceph rtd unknown /proc/30067/task/33282/root (readlink: Permission denied) 2024-05-09T04:58:11.743 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33282 dashboard ceph txt unknown /proc/30067/task/33282/exe (readlink: Permission denied) 2024-05-09T04:58:11.743 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33282 dashboard ceph NOFD /proc/30067/task/33282/fd (opendir: Permission denied) 2024-05-09T04:58:11.743 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33283 dashboard ceph cwd unknown /proc/30067/task/33283/cwd (readlink: Permission denied) 2024-05-09T04:58:11.743 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33283 dashboard ceph rtd unknown /proc/30067/task/33283/root (readlink: Permission denied) 2024-05-09T04:58:11.743 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33283 dashboard ceph txt unknown /proc/30067/task/33283/exe (readlink: Permission denied) 2024-05-09T04:58:11.743 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33283 dashboard ceph NOFD /proc/30067/task/33283/fd (opendir: Permission denied) 2024-05-09T04:58:11.743 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33284 dashboard ceph cwd unknown /proc/30067/task/33284/cwd (readlink: Permission denied) 2024-05-09T04:58:11.743 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33284 dashboard ceph rtd unknown /proc/30067/task/33284/root (readlink: Permission denied) 2024-05-09T04:58:11.744 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33284 dashboard ceph txt unknown /proc/30067/task/33284/exe (readlink: Permission denied) 2024-05-09T04:58:11.744 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33284 dashboard ceph NOFD /proc/30067/task/33284/fd (opendir: Permission denied) 2024-05-09T04:58:11.744 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33285 dashboard ceph cwd unknown /proc/30067/task/33285/cwd (readlink: Permission denied) 2024-05-09T04:58:11.744 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33285 dashboard ceph rtd unknown /proc/30067/task/33285/root (readlink: Permission denied) 2024-05-09T04:58:11.744 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33285 dashboard ceph txt unknown /proc/30067/task/33285/exe (readlink: Permission denied) 2024-05-09T04:58:11.744 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33285 dashboard ceph NOFD /proc/30067/task/33285/fd (opendir: Permission denied) 2024-05-09T04:58:11.744 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33286 dashboard ceph cwd unknown /proc/30067/task/33286/cwd (readlink: Permission denied) 2024-05-09T04:58:11.744 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33286 dashboard ceph rtd unknown /proc/30067/task/33286/root (readlink: Permission denied) 2024-05-09T04:58:11.744 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33286 dashboard ceph txt unknown /proc/30067/task/33286/exe (readlink: Permission denied) 2024-05-09T04:58:11.744 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33286 dashboard ceph NOFD /proc/30067/task/33286/fd (opendir: Permission denied) 2024-05-09T04:58:11.744 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33287 dashboard ceph cwd unknown /proc/30067/task/33287/cwd (readlink: Permission denied) 2024-05-09T04:58:11.744 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33287 dashboard ceph rtd unknown /proc/30067/task/33287/root (readlink: Permission denied) 2024-05-09T04:58:11.744 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33287 dashboard ceph txt unknown /proc/30067/task/33287/exe (readlink: Permission denied) 2024-05-09T04:58:11.744 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33287 dashboard ceph NOFD /proc/30067/task/33287/fd (opendir: Permission denied) 2024-05-09T04:58:11.745 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33288 dashboard ceph cwd unknown /proc/30067/task/33288/cwd (readlink: Permission denied) 2024-05-09T04:58:11.745 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33288 dashboard ceph rtd unknown /proc/30067/task/33288/root (readlink: Permission denied) 2024-05-09T04:58:11.745 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33288 dashboard ceph txt unknown /proc/30067/task/33288/exe (readlink: Permission denied) 2024-05-09T04:58:11.745 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33288 dashboard ceph NOFD /proc/30067/task/33288/fd (opendir: Permission denied) 2024-05-09T04:58:11.745 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33289 dashboard ceph cwd unknown /proc/30067/task/33289/cwd (readlink: Permission denied) 2024-05-09T04:58:11.745 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33289 dashboard ceph rtd unknown /proc/30067/task/33289/root (readlink: Permission denied) 2024-05-09T04:58:11.745 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33289 dashboard ceph txt unknown /proc/30067/task/33289/exe (readlink: Permission denied) 2024-05-09T04:58:11.745 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33289 dashboard ceph NOFD /proc/30067/task/33289/fd (opendir: Permission denied) 2024-05-09T04:58:11.745 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33290 dashboard ceph cwd unknown /proc/30067/task/33290/cwd (readlink: Permission denied) 2024-05-09T04:58:11.745 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33290 dashboard ceph rtd unknown /proc/30067/task/33290/root (readlink: Permission denied) 2024-05-09T04:58:11.745 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33290 dashboard ceph txt unknown /proc/30067/task/33290/exe (readlink: Permission denied) 2024-05-09T04:58:11.745 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33290 dashboard ceph NOFD /proc/30067/task/33290/fd (opendir: Permission denied) 2024-05-09T04:58:11.745 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33291 dashboard ceph cwd unknown /proc/30067/task/33291/cwd (readlink: Permission denied) 2024-05-09T04:58:11.745 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33291 dashboard ceph rtd unknown /proc/30067/task/33291/root (readlink: Permission denied) 2024-05-09T04:58:11.745 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33291 dashboard ceph txt unknown /proc/30067/task/33291/exe (readlink: Permission denied) 2024-05-09T04:58:11.746 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33291 dashboard ceph NOFD /proc/30067/task/33291/fd (opendir: Permission denied) 2024-05-09T04:58:11.746 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33324 dashboard ceph cwd unknown /proc/30067/task/33324/cwd (readlink: Permission denied) 2024-05-09T04:58:11.746 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33324 dashboard ceph rtd unknown /proc/30067/task/33324/root (readlink: Permission denied) 2024-05-09T04:58:11.746 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33324 dashboard ceph txt unknown /proc/30067/task/33324/exe (readlink: Permission denied) 2024-05-09T04:58:11.746 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 33324 dashboard ceph NOFD /proc/30067/task/33324/fd (opendir: Permission denied) 2024-05-09T04:58:11.746 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37335 cephadm ceph cwd unknown /proc/30067/task/37335/cwd (readlink: Permission denied) 2024-05-09T04:58:11.746 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37335 cephadm ceph rtd unknown /proc/30067/task/37335/root (readlink: Permission denied) 2024-05-09T04:58:11.746 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37335 cephadm ceph txt unknown /proc/30067/task/37335/exe (readlink: Permission denied) 2024-05-09T04:58:11.746 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37335 cephadm ceph NOFD /proc/30067/task/37335/fd (opendir: Permission denied) 2024-05-09T04:58:11.746 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37336 cephadm ceph cwd unknown /proc/30067/task/37336/cwd (readlink: Permission denied) 2024-05-09T04:58:11.746 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37336 cephadm ceph rtd unknown /proc/30067/task/37336/root (readlink: Permission denied) 2024-05-09T04:58:11.746 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37336 cephadm ceph txt unknown /proc/30067/task/37336/exe (readlink: Permission denied) 2024-05-09T04:58:11.746 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37336 cephadm ceph NOFD /proc/30067/task/37336/fd (opendir: Permission denied) 2024-05-09T04:58:11.747 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37338 cephadm ceph cwd unknown /proc/30067/task/37338/cwd (readlink: Permission denied) 2024-05-09T04:58:11.747 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37338 cephadm ceph rtd unknown /proc/30067/task/37338/root (readlink: Permission denied) 2024-05-09T04:58:11.747 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37338 cephadm ceph txt unknown /proc/30067/task/37338/exe (readlink: Permission denied) 2024-05-09T04:58:11.747 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37338 cephadm ceph NOFD /proc/30067/task/37338/fd (opendir: Permission denied) 2024-05-09T04:58:11.747 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37339 cephadm ceph cwd unknown /proc/30067/task/37339/cwd (readlink: Permission denied) 2024-05-09T04:58:11.747 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37339 cephadm ceph rtd unknown /proc/30067/task/37339/root (readlink: Permission denied) 2024-05-09T04:58:11.747 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37339 cephadm ceph txt unknown /proc/30067/task/37339/exe (readlink: Permission denied) 2024-05-09T04:58:11.747 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37339 cephadm ceph NOFD /proc/30067/task/37339/fd (opendir: Permission denied) 2024-05-09T04:58:11.747 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37340 cephadm ceph cwd unknown /proc/30067/task/37340/cwd (readlink: Permission denied) 2024-05-09T04:58:11.747 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37340 cephadm ceph rtd unknown /proc/30067/task/37340/root (readlink: Permission denied) 2024-05-09T04:58:11.747 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37340 cephadm ceph txt unknown /proc/30067/task/37340/exe (readlink: Permission denied) 2024-05-09T04:58:11.748 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37340 cephadm ceph NOFD /proc/30067/task/37340/fd (opendir: Permission denied) 2024-05-09T04:58:11.748 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37344 cephadm ceph cwd unknown /proc/30067/task/37344/cwd (readlink: Permission denied) 2024-05-09T04:58:11.748 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37344 cephadm ceph rtd unknown /proc/30067/task/37344/root (readlink: Permission denied) 2024-05-09T04:58:11.748 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37344 cephadm ceph txt unknown /proc/30067/task/37344/exe (readlink: Permission denied) 2024-05-09T04:58:11.748 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37344 cephadm ceph NOFD /proc/30067/task/37344/fd (opendir: Permission denied) 2024-05-09T04:58:11.748 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37350 cephadm ceph cwd unknown /proc/30067/task/37350/cwd (readlink: Permission denied) 2024-05-09T04:58:11.748 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37350 cephadm ceph rtd unknown /proc/30067/task/37350/root (readlink: Permission denied) 2024-05-09T04:58:11.748 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37350 cephadm ceph txt unknown /proc/30067/task/37350/exe (readlink: Permission denied) 2024-05-09T04:58:11.748 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37350 cephadm ceph NOFD /proc/30067/task/37350/fd (opendir: Permission denied) 2024-05-09T04:58:11.748 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37351 cephadm ceph cwd unknown /proc/30067/task/37351/cwd (readlink: Permission denied) 2024-05-09T04:58:11.748 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37351 cephadm ceph rtd unknown /proc/30067/task/37351/root (readlink: Permission denied) 2024-05-09T04:58:11.748 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37351 cephadm ceph txt unknown /proc/30067/task/37351/exe (readlink: Permission denied) 2024-05-09T04:58:11.748 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37351 cephadm ceph NOFD /proc/30067/task/37351/fd (opendir: Permission denied) 2024-05-09T04:58:11.749 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37355 cephadm ceph cwd unknown /proc/30067/task/37355/cwd (readlink: Permission denied) 2024-05-09T04:58:11.749 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37355 cephadm ceph rtd unknown /proc/30067/task/37355/root (readlink: Permission denied) 2024-05-09T04:58:11.749 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37355 cephadm ceph txt unknown /proc/30067/task/37355/exe (readlink: Permission denied) 2024-05-09T04:58:11.749 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37355 cephadm ceph NOFD /proc/30067/task/37355/fd (opendir: Permission denied) 2024-05-09T04:58:11.749 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37356 cephadm ceph cwd unknown /proc/30067/task/37356/cwd (readlink: Permission denied) 2024-05-09T04:58:11.749 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37356 cephadm ceph rtd unknown /proc/30067/task/37356/root (readlink: Permission denied) 2024-05-09T04:58:11.749 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37356 cephadm ceph txt unknown /proc/30067/task/37356/exe (readlink: Permission denied) 2024-05-09T04:58:11.749 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37356 cephadm ceph NOFD /proc/30067/task/37356/fd (opendir: Permission denied) 2024-05-09T04:58:11.749 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37670 cephadm ceph cwd unknown /proc/30067/task/37670/cwd (readlink: Permission denied) 2024-05-09T04:58:11.749 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37670 cephadm ceph rtd unknown /proc/30067/task/37670/root (readlink: Permission denied) 2024-05-09T04:58:11.749 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37670 cephadm ceph txt unknown /proc/30067/task/37670/exe (readlink: Permission denied) 2024-05-09T04:58:11.749 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37670 cephadm ceph NOFD /proc/30067/task/37670/fd (opendir: Permission denied) 2024-05-09T04:58:11.749 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37681 cephadm ceph cwd unknown /proc/30067/task/37681/cwd (readlink: Permission denied) 2024-05-09T04:58:11.749 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37681 cephadm ceph rtd unknown /proc/30067/task/37681/root (readlink: Permission denied) 2024-05-09T04:58:11.750 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37681 cephadm ceph txt unknown /proc/30067/task/37681/exe (readlink: Permission denied) 2024-05-09T04:58:11.750 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37681 cephadm ceph NOFD /proc/30067/task/37681/fd (opendir: Permission denied) 2024-05-09T04:58:11.750 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37704 cephadm ceph cwd unknown /proc/30067/task/37704/cwd (readlink: Permission denied) 2024-05-09T04:58:11.750 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37704 cephadm ceph rtd unknown /proc/30067/task/37704/root (readlink: Permission denied) 2024-05-09T04:58:11.750 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37704 cephadm ceph txt unknown /proc/30067/task/37704/exe (readlink: Permission denied) 2024-05-09T04:58:11.750 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37704 cephadm ceph NOFD /proc/30067/task/37704/fd (opendir: Permission denied) 2024-05-09T04:58:11.750 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37705 cephadm ceph cwd unknown /proc/30067/task/37705/cwd (readlink: Permission denied) 2024-05-09T04:58:11.750 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37705 cephadm ceph rtd unknown /proc/30067/task/37705/root (readlink: Permission denied) 2024-05-09T04:58:11.750 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37705 cephadm ceph txt unknown /proc/30067/task/37705/exe (readlink: Permission denied) 2024-05-09T04:58:11.750 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37705 cephadm ceph NOFD /proc/30067/task/37705/fd (opendir: Permission denied) 2024-05-09T04:58:11.750 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37706 cephadm ceph cwd unknown /proc/30067/task/37706/cwd (readlink: Permission denied) 2024-05-09T04:58:11.750 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37706 cephadm ceph rtd unknown /proc/30067/task/37706/root (readlink: Permission denied) 2024-05-09T04:58:11.750 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37706 cephadm ceph txt unknown /proc/30067/task/37706/exe (readlink: Permission denied) 2024-05-09T04:58:11.751 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37706 cephadm ceph NOFD /proc/30067/task/37706/fd (opendir: Permission denied) 2024-05-09T04:58:11.751 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37707 cephadm ceph cwd unknown /proc/30067/task/37707/cwd (readlink: Permission denied) 2024-05-09T04:58:11.751 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37707 cephadm ceph rtd unknown /proc/30067/task/37707/root (readlink: Permission denied) 2024-05-09T04:58:11.751 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37707 cephadm ceph txt unknown /proc/30067/task/37707/exe (readlink: Permission denied) 2024-05-09T04:58:11.751 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37707 cephadm ceph NOFD /proc/30067/task/37707/fd (opendir: Permission denied) 2024-05-09T04:58:11.751 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37708 cephadm ceph cwd unknown /proc/30067/task/37708/cwd (readlink: Permission denied) 2024-05-09T04:58:11.751 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37708 cephadm ceph rtd unknown /proc/30067/task/37708/root (readlink: Permission denied) 2024-05-09T04:58:11.751 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37708 cephadm ceph txt unknown /proc/30067/task/37708/exe (readlink: Permission denied) 2024-05-09T04:58:11.751 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37708 cephadm ceph NOFD /proc/30067/task/37708/fd (opendir: Permission denied) 2024-05-09T04:58:11.751 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37709 cephadm ceph cwd unknown /proc/30067/task/37709/cwd (readlink: Permission denied) 2024-05-09T04:58:11.751 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37709 cephadm ceph rtd unknown /proc/30067/task/37709/root (readlink: Permission denied) 2024-05-09T04:58:11.751 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37709 cephadm ceph txt unknown /proc/30067/task/37709/exe (readlink: Permission denied) 2024-05-09T04:58:11.751 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37709 cephadm ceph NOFD /proc/30067/task/37709/fd (opendir: Permission denied) 2024-05-09T04:58:11.751 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37710 cephadm ceph cwd unknown /proc/30067/task/37710/cwd (readlink: Permission denied) 2024-05-09T04:58:11.752 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37710 cephadm ceph rtd unknown /proc/30067/task/37710/root (readlink: Permission denied) 2024-05-09T04:58:11.752 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37710 cephadm ceph txt unknown /proc/30067/task/37710/exe (readlink: Permission denied) 2024-05-09T04:58:11.752 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37710 cephadm ceph NOFD /proc/30067/task/37710/fd (opendir: Permission denied) 2024-05-09T04:58:11.752 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37711 cephadm ceph cwd unknown /proc/30067/task/37711/cwd (readlink: Permission denied) 2024-05-09T04:58:11.752 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37711 cephadm ceph rtd unknown /proc/30067/task/37711/root (readlink: Permission denied) 2024-05-09T04:58:11.752 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37711 cephadm ceph txt unknown /proc/30067/task/37711/exe (readlink: Permission denied) 2024-05-09T04:58:11.752 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 37711 cephadm ceph NOFD /proc/30067/task/37711/fd (opendir: Permission denied) 2024-05-09T04:58:11.752 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 46407 devicehea ceph cwd unknown /proc/30067/task/46407/cwd (readlink: Permission denied) 2024-05-09T04:58:11.752 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 46407 devicehea ceph rtd unknown /proc/30067/task/46407/root (readlink: Permission denied) 2024-05-09T04:58:11.752 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 46407 devicehea ceph txt unknown /proc/30067/task/46407/exe (readlink: Permission denied) 2024-05-09T04:58:11.752 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 46407 devicehea ceph NOFD /proc/30067/task/46407/fd (opendir: Permission denied) 2024-05-09T04:58:11.752 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 46411 io_contex ceph cwd unknown /proc/30067/task/46411/cwd (readlink: Permission denied) 2024-05-09T04:58:11.752 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 46411 io_contex ceph rtd unknown /proc/30067/task/46411/root (readlink: Permission denied) 2024-05-09T04:58:11.752 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 46411 io_contex ceph txt unknown /proc/30067/task/46411/exe (readlink: Permission denied) 2024-05-09T04:58:11.752 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 46411 io_contex ceph NOFD /proc/30067/task/46411/fd (opendir: Permission denied) 2024-05-09T04:58:11.753 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 46412 io_contex ceph cwd unknown /proc/30067/task/46412/cwd (readlink: Permission denied) 2024-05-09T04:58:11.753 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 46412 io_contex ceph rtd unknown /proc/30067/task/46412/root (readlink: Permission denied) 2024-05-09T04:58:11.753 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 46412 io_contex ceph txt unknown /proc/30067/task/46412/exe (readlink: Permission denied) 2024-05-09T04:58:11.753 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 46412 io_contex ceph NOFD /proc/30067/task/46412/fd (opendir: Permission denied) 2024-05-09T04:58:11.753 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 46413 ceph_time ceph cwd unknown /proc/30067/task/46413/cwd (readlink: Permission denied) 2024-05-09T04:58:11.753 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 46413 ceph_time ceph rtd unknown /proc/30067/task/46413/root (readlink: Permission denied) 2024-05-09T04:58:11.753 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 46413 ceph_time ceph txt unknown /proc/30067/task/46413/exe (readlink: Permission denied) 2024-05-09T04:58:11.753 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 46413 ceph_time ceph NOFD /proc/30067/task/46413/fd (opendir: Permission denied) 2024-05-09T04:58:11.753 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 46414 ms_dispat ceph cwd unknown /proc/30067/task/46414/cwd (readlink: Permission denied) 2024-05-09T04:58:11.753 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 46414 ms_dispat ceph rtd unknown /proc/30067/task/46414/root (readlink: Permission denied) 2024-05-09T04:58:11.753 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 46414 ms_dispat ceph txt unknown /proc/30067/task/46414/exe (readlink: Permission denied) 2024-05-09T04:58:11.753 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 46414 ms_dispat ceph NOFD /proc/30067/task/46414/fd (opendir: Permission denied) 2024-05-09T04:58:11.753 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 46415 ms_local ceph cwd unknown /proc/30067/task/46415/cwd (readlink: Permission denied) 2024-05-09T04:58:11.753 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 46415 ms_local ceph rtd unknown /proc/30067/task/46415/root (readlink: Permission denied) 2024-05-09T04:58:11.753 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 46415 ms_local ceph txt unknown /proc/30067/task/46415/exe (readlink: Permission denied) 2024-05-09T04:58:11.754 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 46415 ms_local ceph NOFD /proc/30067/task/46415/fd (opendir: Permission denied) 2024-05-09T04:58:11.754 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 46416 safe_time ceph cwd unknown /proc/30067/task/46416/cwd (readlink: Permission denied) 2024-05-09T04:58:11.754 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 46416 safe_time ceph rtd unknown /proc/30067/task/46416/root (readlink: Permission denied) 2024-05-09T04:58:11.754 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 46416 safe_time ceph txt unknown /proc/30067/task/46416/exe (readlink: Permission denied) 2024-05-09T04:58:11.754 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 46416 safe_time ceph NOFD /proc/30067/task/46416/fd (opendir: Permission denied) 2024-05-09T04:58:11.754 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 46417 safe_time ceph cwd unknown /proc/30067/task/46417/cwd (readlink: Permission denied) 2024-05-09T04:58:11.754 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 46417 safe_time ceph rtd unknown /proc/30067/task/46417/root (readlink: Permission denied) 2024-05-09T04:58:11.754 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 46417 safe_time ceph txt unknown /proc/30067/task/46417/exe (readlink: Permission denied) 2024-05-09T04:58:11.754 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mgr 30067 46417 safe_time ceph NOFD /proc/30067/task/46417/fd (opendir: Permission denied) 2024-05-09T04:58:11.754 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 31209 root cwd unknown /proc/31209/cwd (readlink: Permission denied) 2024-05-09T04:58:11.754 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 31209 root rtd unknown /proc/31209/root (readlink: Permission denied) 2024-05-09T04:58:11.754 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 31209 root txt unknown /proc/31209/exe (readlink: Permission denied) 2024-05-09T04:58:11.754 INFO:tasks.workunit.client.0.smithi006.stdout:systemd 31209 root NOFD /proc/31209/fd (opendir: Permission denied) 2024-05-09T04:58:11.754 INFO:tasks.workunit.client.0.smithi006.stdout:(sd-pam) 31212 root cwd unknown /proc/31212/cwd (readlink: Permission denied) 2024-05-09T04:58:11.755 INFO:tasks.workunit.client.0.smithi006.stdout:(sd-pam) 31212 root rtd unknown /proc/31212/root (readlink: Permission denied) 2024-05-09T04:58:11.755 INFO:tasks.workunit.client.0.smithi006.stdout:(sd-pam) 31212 root txt unknown /proc/31212/exe (readlink: Permission denied) 2024-05-09T04:58:11.755 INFO:tasks.workunit.client.0.smithi006.stdout:(sd-pam) 31212 root NOFD /proc/31212/fd (opendir: Permission denied) 2024-05-09T04:58:11.755 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/5 32558 root cwd unknown /proc/32558/cwd (readlink: Permission denied) 2024-05-09T04:58:11.755 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/5 32558 root rtd unknown /proc/32558/root (readlink: Permission denied) 2024-05-09T04:58:11.755 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/5 32558 root txt unknown /proc/32558/exe (readlink: Permission denied) 2024-05-09T04:58:11.755 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/5 32558 root NOFD /proc/32558/fd (opendir: Permission denied) 2024-05-09T04:58:11.755 INFO:tasks.workunit.client.0.smithi006.stdout:sshd 32739 root cwd unknown /proc/32739/cwd (readlink: Permission denied) 2024-05-09T04:58:11.755 INFO:tasks.workunit.client.0.smithi006.stdout:sshd 32739 root rtd unknown /proc/32739/root (readlink: Permission denied) 2024-05-09T04:58:11.755 INFO:tasks.workunit.client.0.smithi006.stdout:sshd 32739 root txt unknown /proc/32739/exe (readlink: Permission denied) 2024-05-09T04:58:11.755 INFO:tasks.workunit.client.0.smithi006.stdout:sshd 32739 root NOFD /proc/32739/fd (opendir: Permission denied) 2024-05-09T04:58:11.755 INFO:tasks.workunit.client.0.smithi006.stdout:sshd 32763 root cwd unknown /proc/32763/cwd (readlink: Permission denied) 2024-05-09T04:58:11.755 INFO:tasks.workunit.client.0.smithi006.stdout:sshd 32763 root rtd unknown /proc/32763/root (readlink: Permission denied) 2024-05-09T04:58:11.756 INFO:tasks.workunit.client.0.smithi006.stdout:sshd 32763 root txt unknown /proc/32763/exe (readlink: Permission denied) 2024-05-09T04:58:11.756 INFO:tasks.workunit.client.0.smithi006.stdout:sshd 32763 root NOFD /proc/32763/fd (opendir: Permission denied) 2024-05-09T04:58:11.756 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 42091 root cwd unknown /proc/42091/cwd (readlink: Permission denied) 2024-05-09T04:58:11.756 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 42091 root rtd unknown /proc/42091/root (readlink: Permission denied) 2024-05-09T04:58:11.756 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 42091 root txt unknown /proc/42091/exe (readlink: Permission denied) 2024-05-09T04:58:11.756 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 42091 root NOFD /proc/42091/fd (opendir: Permission denied) 2024-05-09T04:58:11.756 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 42093 root cwd unknown /proc/42093/cwd (readlink: Permission denied) 2024-05-09T04:58:11.756 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 42093 root rtd unknown /proc/42093/root (readlink: Permission denied) 2024-05-09T04:58:11.756 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 42093 root txt unknown /proc/42093/exe (readlink: Permission denied) 2024-05-09T04:58:11.756 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 42093 root NOFD /proc/42093/fd (opendir: Permission denied) 2024-05-09T04:58:11.756 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 ceph cwd unknown /proc/42096/cwd (readlink: Permission denied) 2024-05-09T04:58:11.756 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 ceph rtd unknown /proc/42096/root (readlink: Permission denied) 2024-05-09T04:58:11.756 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 ceph txt unknown /proc/42096/exe (readlink: Permission denied) 2024-05-09T04:58:11.756 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 ceph NOFD /proc/42096/fd (opendir: Permission denied) 2024-05-09T04:58:11.756 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42098 log ceph cwd unknown /proc/42096/task/42098/cwd (readlink: Permission denied) 2024-05-09T04:58:11.757 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42098 log ceph rtd unknown /proc/42096/task/42098/root (readlink: Permission denied) 2024-05-09T04:58:11.757 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42098 log ceph txt unknown /proc/42096/task/42098/exe (readlink: Permission denied) 2024-05-09T04:58:11.757 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42098 log ceph NOFD /proc/42096/task/42098/fd (opendir: Permission denied) 2024-05-09T04:58:11.757 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42100 msgr-work ceph cwd unknown /proc/42096/task/42100/cwd (readlink: Permission denied) 2024-05-09T04:58:11.757 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42100 msgr-work ceph rtd unknown /proc/42096/task/42100/root (readlink: Permission denied) 2024-05-09T04:58:11.757 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42100 msgr-work ceph txt unknown /proc/42096/task/42100/exe (readlink: Permission denied) 2024-05-09T04:58:11.757 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42100 msgr-work ceph NOFD /proc/42096/task/42100/fd (opendir: Permission denied) 2024-05-09T04:58:11.757 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42101 msgr-work ceph cwd unknown /proc/42096/task/42101/cwd (readlink: Permission denied) 2024-05-09T04:58:11.757 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42101 msgr-work ceph rtd unknown /proc/42096/task/42101/root (readlink: Permission denied) 2024-05-09T04:58:11.757 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42101 msgr-work ceph txt unknown /proc/42096/task/42101/exe (readlink: Permission denied) 2024-05-09T04:58:11.757 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42101 msgr-work ceph NOFD /proc/42096/task/42101/fd (opendir: Permission denied) 2024-05-09T04:58:11.757 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42102 msgr-work ceph cwd unknown /proc/42096/task/42102/cwd (readlink: Permission denied) 2024-05-09T04:58:11.757 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42102 msgr-work ceph rtd unknown /proc/42096/task/42102/root (readlink: Permission denied) 2024-05-09T04:58:11.757 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42102 msgr-work ceph txt unknown /proc/42096/task/42102/exe (readlink: Permission denied) 2024-05-09T04:58:11.758 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42102 msgr-work ceph NOFD /proc/42096/task/42102/fd (opendir: Permission denied) 2024-05-09T04:58:11.758 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42107 service ceph cwd unknown /proc/42096/task/42107/cwd (readlink: Permission denied) 2024-05-09T04:58:11.758 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42107 service ceph rtd unknown /proc/42096/task/42107/root (readlink: Permission denied) 2024-05-09T04:58:11.758 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42107 service ceph txt unknown /proc/42096/task/42107/exe (readlink: Permission denied) 2024-05-09T04:58:11.758 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42107 service ceph NOFD /proc/42096/task/42107/fd (opendir: Permission denied) 2024-05-09T04:58:11.758 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42108 admin_soc ceph cwd unknown /proc/42096/task/42108/cwd (readlink: Permission denied) 2024-05-09T04:58:11.758 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42108 admin_soc ceph rtd unknown /proc/42096/task/42108/root (readlink: Permission denied) 2024-05-09T04:58:11.758 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42108 admin_soc ceph txt unknown /proc/42096/task/42108/exe (readlink: Permission denied) 2024-05-09T04:58:11.758 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42108 admin_soc ceph NOFD /proc/42096/task/42108/fd (opendir: Permission denied) 2024-05-09T04:58:11.758 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42249 signal_ha ceph cwd unknown /proc/42096/task/42249/cwd (readlink: Permission denied) 2024-05-09T04:58:11.758 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42249 signal_ha ceph rtd unknown /proc/42096/task/42249/root (readlink: Permission denied) 2024-05-09T04:58:11.758 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42249 signal_ha ceph txt unknown /proc/42096/task/42249/exe (readlink: Permission denied) 2024-05-09T04:58:11.758 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42249 signal_ha ceph NOFD /proc/42096/task/42249/fd (opendir: Permission denied) 2024-05-09T04:58:11.758 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42250 io_contex ceph cwd unknown /proc/42096/task/42250/cwd (readlink: Permission denied) 2024-05-09T04:58:11.759 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42250 io_contex ceph rtd unknown /proc/42096/task/42250/root (readlink: Permission denied) 2024-05-09T04:58:11.759 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42250 io_contex ceph txt unknown /proc/42096/task/42250/exe (readlink: Permission denied) 2024-05-09T04:58:11.759 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42250 io_contex ceph NOFD /proc/42096/task/42250/fd (opendir: Permission denied) 2024-05-09T04:58:11.759 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42251 io_contex ceph cwd unknown /proc/42096/task/42251/cwd (readlink: Permission denied) 2024-05-09T04:58:11.759 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42251 io_contex ceph rtd unknown /proc/42096/task/42251/root (readlink: Permission denied) 2024-05-09T04:58:11.759 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42251 io_contex ceph txt unknown /proc/42096/task/42251/exe (readlink: Permission denied) 2024-05-09T04:58:11.759 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42251 io_contex ceph NOFD /proc/42096/task/42251/fd (opendir: Permission denied) 2024-05-09T04:58:11.759 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42301 OpHistory ceph cwd unknown /proc/42096/task/42301/cwd (readlink: Permission denied) 2024-05-09T04:58:11.759 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42301 OpHistory ceph rtd unknown /proc/42096/task/42301/root (readlink: Permission denied) 2024-05-09T04:58:11.759 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42301 OpHistory ceph txt unknown /proc/42096/task/42301/exe (readlink: Permission denied) 2024-05-09T04:58:11.759 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42301 OpHistory ceph NOFD /proc/42096/task/42301/fd (opendir: Permission denied) 2024-05-09T04:58:11.759 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42302 ceph_time ceph cwd unknown /proc/42096/task/42302/cwd (readlink: Permission denied) 2024-05-09T04:58:11.759 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42302 ceph_time ceph rtd unknown /proc/42096/task/42302/root (readlink: Permission denied) 2024-05-09T04:58:11.759 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42302 ceph_time ceph txt unknown /proc/42096/task/42302/exe (readlink: Permission denied) 2024-05-09T04:58:11.759 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42302 ceph_time ceph NOFD /proc/42096/task/42302/fd (opendir: Permission denied) 2024-05-09T04:58:11.760 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42486 ceph-osd ceph cwd unknown /proc/42096/task/42486/cwd (readlink: Permission denied) 2024-05-09T04:58:11.760 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42486 ceph-osd ceph rtd unknown /proc/42096/task/42486/root (readlink: Permission denied) 2024-05-09T04:58:11.760 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42486 ceph-osd ceph txt unknown /proc/42096/task/42486/exe (readlink: Permission denied) 2024-05-09T04:58:11.760 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42486 ceph-osd ceph NOFD /proc/42096/task/42486/fd (opendir: Permission denied) 2024-05-09T04:58:11.760 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42487 safe_time ceph cwd unknown /proc/42096/task/42487/cwd (readlink: Permission denied) 2024-05-09T04:58:11.760 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42487 safe_time ceph rtd unknown /proc/42096/task/42487/root (readlink: Permission denied) 2024-05-09T04:58:11.760 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42487 safe_time ceph txt unknown /proc/42096/task/42487/exe (readlink: Permission denied) 2024-05-09T04:58:11.760 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42487 safe_time ceph NOFD /proc/42096/task/42487/fd (opendir: Permission denied) 2024-05-09T04:58:11.760 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42488 safe_time ceph cwd unknown /proc/42096/task/42488/cwd (readlink: Permission denied) 2024-05-09T04:58:11.760 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42488 safe_time ceph rtd unknown /proc/42096/task/42488/root (readlink: Permission denied) 2024-05-09T04:58:11.760 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42488 safe_time ceph txt unknown /proc/42096/task/42488/exe (readlink: Permission denied) 2024-05-09T04:58:11.760 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42488 safe_time ceph NOFD /proc/42096/task/42488/fd (opendir: Permission denied) 2024-05-09T04:58:11.760 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42489 safe_time ceph cwd unknown /proc/42096/task/42489/cwd (readlink: Permission denied) 2024-05-09T04:58:11.760 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42489 safe_time ceph rtd unknown /proc/42096/task/42489/root (readlink: Permission denied) 2024-05-09T04:58:11.761 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42489 safe_time ceph txt unknown /proc/42096/task/42489/exe (readlink: Permission denied) 2024-05-09T04:58:11.761 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42489 safe_time ceph NOFD /proc/42096/task/42489/fd (opendir: Permission denied) 2024-05-09T04:58:11.761 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42490 safe_time ceph cwd unknown /proc/42096/task/42490/cwd (readlink: Permission denied) 2024-05-09T04:58:11.761 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42490 safe_time ceph rtd unknown /proc/42096/task/42490/root (readlink: Permission denied) 2024-05-09T04:58:11.761 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42490 safe_time ceph txt unknown /proc/42096/task/42490/exe (readlink: Permission denied) 2024-05-09T04:58:11.761 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42490 safe_time ceph NOFD /proc/42096/task/42490/fd (opendir: Permission denied) 2024-05-09T04:58:11.761 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42491 fn_anonym ceph cwd unknown /proc/42096/task/42491/cwd (readlink: Permission denied) 2024-05-09T04:58:11.761 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42491 fn_anonym ceph rtd unknown /proc/42096/task/42491/root (readlink: Permission denied) 2024-05-09T04:58:11.761 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42491 fn_anonym ceph txt unknown /proc/42096/task/42491/exe (readlink: Permission denied) 2024-05-09T04:58:11.761 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42491 fn_anonym ceph NOFD /proc/42096/task/42491/fd (opendir: Permission denied) 2024-05-09T04:58:11.761 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42498 rocksdb:l ceph cwd unknown /proc/42096/task/42498/cwd (readlink: Permission denied) 2024-05-09T04:58:11.761 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42498 rocksdb:l ceph rtd unknown /proc/42096/task/42498/root (readlink: Permission denied) 2024-05-09T04:58:11.761 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42498 rocksdb:l ceph txt unknown /proc/42096/task/42498/exe (readlink: Permission denied) 2024-05-09T04:58:11.761 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42498 rocksdb:l ceph NOFD /proc/42096/task/42498/fd (opendir: Permission denied) 2024-05-09T04:58:11.762 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42499 rocksdb:l ceph cwd unknown /proc/42096/task/42499/cwd (readlink: Permission denied) 2024-05-09T04:58:11.762 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42499 rocksdb:l ceph rtd unknown /proc/42096/task/42499/root (readlink: Permission denied) 2024-05-09T04:58:11.762 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42499 rocksdb:l ceph txt unknown /proc/42096/task/42499/exe (readlink: Permission denied) 2024-05-09T04:58:11.762 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42499 rocksdb:l ceph NOFD /proc/42096/task/42499/fd (opendir: Permission denied) 2024-05-09T04:58:11.762 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42500 rocksdb:l ceph cwd unknown /proc/42096/task/42500/cwd (readlink: Permission denied) 2024-05-09T04:58:11.762 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42500 rocksdb:l ceph rtd unknown /proc/42096/task/42500/root (readlink: Permission denied) 2024-05-09T04:58:11.762 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42500 rocksdb:l ceph txt unknown /proc/42096/task/42500/exe (readlink: Permission denied) 2024-05-09T04:58:11.762 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42500 rocksdb:l ceph NOFD /proc/42096/task/42500/fd (opendir: Permission denied) 2024-05-09T04:58:11.762 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42501 rocksdb:h ceph cwd unknown /proc/42096/task/42501/cwd (readlink: Permission denied) 2024-05-09T04:58:11.762 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42501 rocksdb:h ceph rtd unknown /proc/42096/task/42501/root (readlink: Permission denied) 2024-05-09T04:58:11.762 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42501 rocksdb:h ceph txt unknown /proc/42096/task/42501/exe (readlink: Permission denied) 2024-05-09T04:58:11.762 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42501 rocksdb:h ceph NOFD /proc/42096/task/42501/fd (opendir: Permission denied) 2024-05-09T04:58:11.762 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42947 bstore_ai ceph cwd unknown /proc/42096/task/42947/cwd (readlink: Permission denied) 2024-05-09T04:58:11.763 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42947 bstore_ai ceph rtd unknown /proc/42096/task/42947/root (readlink: Permission denied) 2024-05-09T04:58:11.763 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42947 bstore_ai ceph txt unknown /proc/42096/task/42947/exe (readlink: Permission denied) 2024-05-09T04:58:11.763 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 42947 bstore_ai ceph NOFD /proc/42096/task/42947/fd (opendir: Permission denied) 2024-05-09T04:58:11.763 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43138 bstore_ai ceph cwd unknown /proc/42096/task/43138/cwd (readlink: Permission denied) 2024-05-09T04:58:11.763 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43138 bstore_ai ceph rtd unknown /proc/42096/task/43138/root (readlink: Permission denied) 2024-05-09T04:58:11.763 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43138 bstore_ai ceph txt unknown /proc/42096/task/43138/exe (readlink: Permission denied) 2024-05-09T04:58:11.763 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43138 bstore_ai ceph NOFD /proc/42096/task/43138/fd (opendir: Permission denied) 2024-05-09T04:58:11.763 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43323 ceph-osd ceph cwd unknown /proc/42096/task/43323/cwd (readlink: Permission denied) 2024-05-09T04:58:11.763 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43323 ceph-osd ceph rtd unknown /proc/42096/task/43323/root (readlink: Permission denied) 2024-05-09T04:58:11.763 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43323 ceph-osd ceph txt unknown /proc/42096/task/43323/exe (readlink: Permission denied) 2024-05-09T04:58:11.763 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43323 ceph-osd ceph NOFD /proc/42096/task/43323/fd (opendir: Permission denied) 2024-05-09T04:58:11.763 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43325 cfin ceph cwd unknown /proc/42096/task/43325/cwd (readlink: Permission denied) 2024-05-09T04:58:11.763 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43325 cfin ceph rtd unknown /proc/42096/task/43325/root (readlink: Permission denied) 2024-05-09T04:58:11.763 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43325 cfin ceph txt unknown /proc/42096/task/43325/exe (readlink: Permission denied) 2024-05-09T04:58:11.763 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43325 cfin ceph NOFD /proc/42096/task/43325/fd (opendir: Permission denied) 2024-05-09T04:58:11.764 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43326 bstore_kv ceph cwd unknown /proc/42096/task/43326/cwd (readlink: Permission denied) 2024-05-09T04:58:11.764 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43326 bstore_kv ceph rtd unknown /proc/42096/task/43326/root (readlink: Permission denied) 2024-05-09T04:58:11.764 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43326 bstore_kv ceph txt unknown /proc/42096/task/43326/exe (readlink: Permission denied) 2024-05-09T04:58:11.764 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43326 bstore_kv ceph NOFD /proc/42096/task/43326/fd (opendir: Permission denied) 2024-05-09T04:58:11.764 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43327 bstore_kv ceph cwd unknown /proc/42096/task/43327/cwd (readlink: Permission denied) 2024-05-09T04:58:11.764 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43327 bstore_kv ceph rtd unknown /proc/42096/task/43327/root (readlink: Permission denied) 2024-05-09T04:58:11.764 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43327 bstore_kv ceph txt unknown /proc/42096/task/43327/exe (readlink: Permission denied) 2024-05-09T04:58:11.764 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43327 bstore_kv ceph NOFD /proc/42096/task/43327/fd (opendir: Permission denied) 2024-05-09T04:58:11.764 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43328 bstore_me ceph cwd unknown /proc/42096/task/43328/cwd (readlink: Permission denied) 2024-05-09T04:58:11.764 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43328 bstore_me ceph rtd unknown /proc/42096/task/43328/root (readlink: Permission denied) 2024-05-09T04:58:11.764 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43328 bstore_me ceph txt unknown /proc/42096/task/43328/exe (readlink: Permission denied) 2024-05-09T04:58:11.764 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43328 bstore_me ceph NOFD /proc/42096/task/43328/fd (opendir: Permission denied) 2024-05-09T04:58:11.764 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43329 ms_dispat ceph cwd unknown /proc/42096/task/43329/cwd (readlink: Permission denied) 2024-05-09T04:58:11.764 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43329 ms_dispat ceph rtd unknown /proc/42096/task/43329/root (readlink: Permission denied) 2024-05-09T04:58:11.764 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43329 ms_dispat ceph txt unknown /proc/42096/task/43329/exe (readlink: Permission denied) 2024-05-09T04:58:11.765 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43329 ms_dispat ceph NOFD /proc/42096/task/43329/fd (opendir: Permission denied) 2024-05-09T04:58:11.765 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43330 ms_local ceph cwd unknown /proc/42096/task/43330/cwd (readlink: Permission denied) 2024-05-09T04:58:11.765 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43330 ms_local ceph rtd unknown /proc/42096/task/43330/root (readlink: Permission denied) 2024-05-09T04:58:11.765 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43330 ms_local ceph txt unknown /proc/42096/task/43330/exe (readlink: Permission denied) 2024-05-09T04:58:11.765 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43330 ms_local ceph NOFD /proc/42096/task/43330/fd (opendir: Permission denied) 2024-05-09T04:58:11.765 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43331 safe_time ceph cwd unknown /proc/42096/task/43331/cwd (readlink: Permission denied) 2024-05-09T04:58:11.765 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43331 safe_time ceph rtd unknown /proc/42096/task/43331/root (readlink: Permission denied) 2024-05-09T04:58:11.765 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43331 safe_time ceph txt unknown /proc/42096/task/43331/exe (readlink: Permission denied) 2024-05-09T04:58:11.765 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43331 safe_time ceph NOFD /proc/42096/task/43331/fd (opendir: Permission denied) 2024-05-09T04:58:11.765 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43332 safe_time ceph cwd unknown /proc/42096/task/43332/cwd (readlink: Permission denied) 2024-05-09T04:58:11.765 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43332 safe_time ceph rtd unknown /proc/42096/task/43332/root (readlink: Permission denied) 2024-05-09T04:58:11.765 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43332 safe_time ceph txt unknown /proc/42096/task/43332/exe (readlink: Permission denied) 2024-05-09T04:58:11.765 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43332 safe_time ceph NOFD /proc/42096/task/43332/fd (opendir: Permission denied) 2024-05-09T04:58:11.766 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43333 ms_dispat ceph cwd unknown /proc/42096/task/43333/cwd (readlink: Permission denied) 2024-05-09T04:58:11.766 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43333 ms_dispat ceph rtd unknown /proc/42096/task/43333/root (readlink: Permission denied) 2024-05-09T04:58:11.766 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43333 ms_dispat ceph txt unknown /proc/42096/task/43333/exe (readlink: Permission denied) 2024-05-09T04:58:11.766 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43333 ms_dispat ceph NOFD /proc/42096/task/43333/fd (opendir: Permission denied) 2024-05-09T04:58:11.766 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43334 ms_local ceph cwd unknown /proc/42096/task/43334/cwd (readlink: Permission denied) 2024-05-09T04:58:11.766 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43334 ms_local ceph rtd unknown /proc/42096/task/43334/root (readlink: Permission denied) 2024-05-09T04:58:11.766 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43334 ms_local ceph txt unknown /proc/42096/task/43334/exe (readlink: Permission denied) 2024-05-09T04:58:11.766 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43334 ms_local ceph NOFD /proc/42096/task/43334/fd (opendir: Permission denied) 2024-05-09T04:58:11.766 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43335 ms_dispat ceph cwd unknown /proc/42096/task/43335/cwd (readlink: Permission denied) 2024-05-09T04:58:11.766 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43335 ms_dispat ceph rtd unknown /proc/42096/task/43335/root (readlink: Permission denied) 2024-05-09T04:58:11.766 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43335 ms_dispat ceph txt unknown /proc/42096/task/43335/exe (readlink: Permission denied) 2024-05-09T04:58:11.766 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43335 ms_dispat ceph NOFD /proc/42096/task/43335/fd (opendir: Permission denied) 2024-05-09T04:58:11.766 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43336 ms_local ceph cwd unknown /proc/42096/task/43336/cwd (readlink: Permission denied) 2024-05-09T04:58:11.766 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43336 ms_local ceph rtd unknown /proc/42096/task/43336/root (readlink: Permission denied) 2024-05-09T04:58:11.767 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43336 ms_local ceph txt unknown /proc/42096/task/43336/exe (readlink: Permission denied) 2024-05-09T04:58:11.767 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43336 ms_local ceph NOFD /proc/42096/task/43336/fd (opendir: Permission denied) 2024-05-09T04:58:11.767 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43337 ms_dispat ceph cwd unknown /proc/42096/task/43337/cwd (readlink: Permission denied) 2024-05-09T04:58:11.767 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43337 ms_dispat ceph rtd unknown /proc/42096/task/43337/root (readlink: Permission denied) 2024-05-09T04:58:11.767 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43337 ms_dispat ceph txt unknown /proc/42096/task/43337/exe (readlink: Permission denied) 2024-05-09T04:58:11.767 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43337 ms_dispat ceph NOFD /proc/42096/task/43337/fd (opendir: Permission denied) 2024-05-09T04:58:11.767 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43338 ms_local ceph cwd unknown /proc/42096/task/43338/cwd (readlink: Permission denied) 2024-05-09T04:58:11.767 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43338 ms_local ceph rtd unknown /proc/42096/task/43338/root (readlink: Permission denied) 2024-05-09T04:58:11.767 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43338 ms_local ceph txt unknown /proc/42096/task/43338/exe (readlink: Permission denied) 2024-05-09T04:58:11.767 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43338 ms_local ceph NOFD /proc/42096/task/43338/fd (opendir: Permission denied) 2024-05-09T04:58:11.767 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43339 ms_dispat ceph cwd unknown /proc/42096/task/43339/cwd (readlink: Permission denied) 2024-05-09T04:58:11.767 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43339 ms_dispat ceph rtd unknown /proc/42096/task/43339/root (readlink: Permission denied) 2024-05-09T04:58:11.767 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43339 ms_dispat ceph txt unknown /proc/42096/task/43339/exe (readlink: Permission denied) 2024-05-09T04:58:11.767 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43339 ms_dispat ceph NOFD /proc/42096/task/43339/fd (opendir: Permission denied) 2024-05-09T04:58:11.767 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43340 ms_local ceph cwd unknown /proc/42096/task/43340/cwd (readlink: Permission denied) 2024-05-09T04:58:11.768 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43340 ms_local ceph rtd unknown /proc/42096/task/43340/root (readlink: Permission denied) 2024-05-09T04:58:11.768 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43340 ms_local ceph txt unknown /proc/42096/task/43340/exe (readlink: Permission denied) 2024-05-09T04:58:11.768 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43340 ms_local ceph NOFD /proc/42096/task/43340/fd (opendir: Permission denied) 2024-05-09T04:58:11.768 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43341 ms_dispat ceph cwd unknown /proc/42096/task/43341/cwd (readlink: Permission denied) 2024-05-09T04:58:11.768 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43341 ms_dispat ceph rtd unknown /proc/42096/task/43341/root (readlink: Permission denied) 2024-05-09T04:58:11.768 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43341 ms_dispat ceph txt unknown /proc/42096/task/43341/exe (readlink: Permission denied) 2024-05-09T04:58:11.768 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43341 ms_dispat ceph NOFD /proc/42096/task/43341/fd (opendir: Permission denied) 2024-05-09T04:58:11.768 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43342 ms_local ceph cwd unknown /proc/42096/task/43342/cwd (readlink: Permission denied) 2024-05-09T04:58:11.768 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43342 ms_local ceph rtd unknown /proc/42096/task/43342/root (readlink: Permission denied) 2024-05-09T04:58:11.768 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43342 ms_local ceph txt unknown /proc/42096/task/43342/exe (readlink: Permission denied) 2024-05-09T04:58:11.768 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43342 ms_local ceph NOFD /proc/42096/task/43342/fd (opendir: Permission denied) 2024-05-09T04:58:11.768 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43343 ms_dispat ceph cwd unknown /proc/42096/task/43343/cwd (readlink: Permission denied) 2024-05-09T04:58:11.768 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43343 ms_dispat ceph rtd unknown /proc/42096/task/43343/root (readlink: Permission denied) 2024-05-09T04:58:11.768 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43343 ms_dispat ceph txt unknown /proc/42096/task/43343/exe (readlink: Permission denied) 2024-05-09T04:58:11.769 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43343 ms_dispat ceph NOFD /proc/42096/task/43343/fd (opendir: Permission denied) 2024-05-09T04:58:11.769 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43344 ms_local ceph cwd unknown /proc/42096/task/43344/cwd (readlink: Permission denied) 2024-05-09T04:58:11.769 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43344 ms_local ceph rtd unknown /proc/42096/task/43344/root (readlink: Permission denied) 2024-05-09T04:58:11.769 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43344 ms_local ceph txt unknown /proc/42096/task/43344/exe (readlink: Permission denied) 2024-05-09T04:58:11.769 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43344 ms_local ceph NOFD /proc/42096/task/43344/fd (opendir: Permission denied) 2024-05-09T04:58:11.769 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43345 fn_anonym ceph cwd unknown /proc/42096/task/43345/cwd (readlink: Permission denied) 2024-05-09T04:58:11.769 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43345 fn_anonym ceph rtd unknown /proc/42096/task/43345/root (readlink: Permission denied) 2024-05-09T04:58:11.769 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43345 fn_anonym ceph txt unknown /proc/42096/task/43345/exe (readlink: Permission denied) 2024-05-09T04:58:11.769 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43345 fn_anonym ceph NOFD /proc/42096/task/43345/fd (opendir: Permission denied) 2024-05-09T04:58:11.769 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43346 finisher ceph cwd unknown /proc/42096/task/43346/cwd (readlink: Permission denied) 2024-05-09T04:58:11.769 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43346 finisher ceph rtd unknown /proc/42096/task/43346/root (readlink: Permission denied) 2024-05-09T04:58:11.769 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43346 finisher ceph txt unknown /proc/42096/task/43346/exe (readlink: Permission denied) 2024-05-09T04:58:11.769 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43346 finisher ceph NOFD /proc/42096/task/43346/fd (opendir: Permission denied) 2024-05-09T04:58:11.769 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43347 safe_time ceph cwd unknown /proc/42096/task/43347/cwd (readlink: Permission denied) 2024-05-09T04:58:11.770 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43347 safe_time ceph rtd unknown /proc/42096/task/43347/root (readlink: Permission denied) 2024-05-09T04:58:11.770 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43347 safe_time ceph txt unknown /proc/42096/task/43347/exe (readlink: Permission denied) 2024-05-09T04:58:11.770 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43347 safe_time ceph NOFD /proc/42096/task/43347/fd (opendir: Permission denied) 2024-05-09T04:58:11.770 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43348 safe_time ceph cwd unknown /proc/42096/task/43348/cwd (readlink: Permission denied) 2024-05-09T04:58:11.770 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43348 safe_time ceph rtd unknown /proc/42096/task/43348/root (readlink: Permission denied) 2024-05-09T04:58:11.770 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43348 safe_time ceph txt unknown /proc/42096/task/43348/exe (readlink: Permission denied) 2024-05-09T04:58:11.770 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43348 safe_time ceph NOFD /proc/42096/task/43348/fd (opendir: Permission denied) 2024-05-09T04:58:11.770 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43349 ceph-osd ceph cwd unknown /proc/42096/task/43349/cwd (readlink: Permission denied) 2024-05-09T04:58:11.770 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43349 ceph-osd ceph rtd unknown /proc/42096/task/43349/root (readlink: Permission denied) 2024-05-09T04:58:11.770 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43349 ceph-osd ceph txt unknown /proc/42096/task/43349/exe (readlink: Permission denied) 2024-05-09T04:58:11.770 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43349 ceph-osd ceph NOFD /proc/42096/task/43349/fd (opendir: Permission denied) 2024-05-09T04:58:11.770 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43350 osd_srv_a ceph cwd unknown /proc/42096/task/43350/cwd (readlink: Permission denied) 2024-05-09T04:58:11.770 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43350 osd_srv_a ceph rtd unknown /proc/42096/task/43350/root (readlink: Permission denied) 2024-05-09T04:58:11.770 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43350 osd_srv_a ceph txt unknown /proc/42096/task/43350/exe (readlink: Permission denied) 2024-05-09T04:58:11.771 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43350 osd_srv_a ceph NOFD /proc/42096/task/43350/fd (opendir: Permission denied) 2024-05-09T04:58:11.771 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43351 tp_osd_tp ceph cwd unknown /proc/42096/task/43351/cwd (readlink: Permission denied) 2024-05-09T04:58:11.771 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43351 tp_osd_tp ceph rtd unknown /proc/42096/task/43351/root (readlink: Permission denied) 2024-05-09T04:58:11.771 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43351 tp_osd_tp ceph txt unknown /proc/42096/task/43351/exe (readlink: Permission denied) 2024-05-09T04:58:11.771 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43351 tp_osd_tp ceph NOFD /proc/42096/task/43351/fd (opendir: Permission denied) 2024-05-09T04:58:11.771 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43352 tp_osd_tp ceph cwd unknown /proc/42096/task/43352/cwd (readlink: Permission denied) 2024-05-09T04:58:11.771 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43352 tp_osd_tp ceph rtd unknown /proc/42096/task/43352/root (readlink: Permission denied) 2024-05-09T04:58:11.771 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43352 tp_osd_tp ceph txt unknown /proc/42096/task/43352/exe (readlink: Permission denied) 2024-05-09T04:58:11.771 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43352 tp_osd_tp ceph NOFD /proc/42096/task/43352/fd (opendir: Permission denied) 2024-05-09T04:58:11.771 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43353 tp_osd_tp ceph cwd unknown /proc/42096/task/43353/cwd (readlink: Permission denied) 2024-05-09T04:58:11.771 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43353 tp_osd_tp ceph rtd unknown /proc/42096/task/43353/root (readlink: Permission denied) 2024-05-09T04:58:11.771 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43353 tp_osd_tp ceph txt unknown /proc/42096/task/43353/exe (readlink: Permission denied) 2024-05-09T04:58:11.771 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43353 tp_osd_tp ceph NOFD /proc/42096/task/43353/fd (opendir: Permission denied) 2024-05-09T04:58:11.771 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43354 tp_osd_tp ceph cwd unknown /proc/42096/task/43354/cwd (readlink: Permission denied) 2024-05-09T04:58:11.772 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43354 tp_osd_tp ceph rtd unknown /proc/42096/task/43354/root (readlink: Permission denied) 2024-05-09T04:58:11.772 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43354 tp_osd_tp ceph txt unknown /proc/42096/task/43354/exe (readlink: Permission denied) 2024-05-09T04:58:11.772 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43354 tp_osd_tp ceph NOFD /proc/42096/task/43354/fd (opendir: Permission denied) 2024-05-09T04:58:11.772 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43355 tp_osd_tp ceph cwd unknown /proc/42096/task/43355/cwd (readlink: Permission denied) 2024-05-09T04:58:11.772 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43355 tp_osd_tp ceph rtd unknown /proc/42096/task/43355/root (readlink: Permission denied) 2024-05-09T04:58:11.772 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43355 tp_osd_tp ceph txt unknown /proc/42096/task/43355/exe (readlink: Permission denied) 2024-05-09T04:58:11.772 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43355 tp_osd_tp ceph NOFD /proc/42096/task/43355/fd (opendir: Permission denied) 2024-05-09T04:58:11.772 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43356 tp_osd_tp ceph cwd unknown /proc/42096/task/43356/cwd (readlink: Permission denied) 2024-05-09T04:58:11.772 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43356 tp_osd_tp ceph rtd unknown /proc/42096/task/43356/root (readlink: Permission denied) 2024-05-09T04:58:11.772 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43356 tp_osd_tp ceph txt unknown /proc/42096/task/43356/exe (readlink: Permission denied) 2024-05-09T04:58:11.772 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43356 tp_osd_tp ceph NOFD /proc/42096/task/43356/fd (opendir: Permission denied) 2024-05-09T04:58:11.772 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43357 tp_osd_tp ceph cwd unknown /proc/42096/task/43357/cwd (readlink: Permission denied) 2024-05-09T04:58:11.772 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43357 tp_osd_tp ceph rtd unknown /proc/42096/task/43357/root (readlink: Permission denied) 2024-05-09T04:58:11.773 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43357 tp_osd_tp ceph txt unknown /proc/42096/task/43357/exe (readlink: Permission denied) 2024-05-09T04:58:11.773 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43357 tp_osd_tp ceph NOFD /proc/42096/task/43357/fd (opendir: Permission denied) 2024-05-09T04:58:11.773 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43358 tp_osd_tp ceph cwd unknown /proc/42096/task/43358/cwd (readlink: Permission denied) 2024-05-09T04:58:11.773 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43358 tp_osd_tp ceph rtd unknown /proc/42096/task/43358/root (readlink: Permission denied) 2024-05-09T04:58:11.773 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43358 tp_osd_tp ceph txt unknown /proc/42096/task/43358/exe (readlink: Permission denied) 2024-05-09T04:58:11.773 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43358 tp_osd_tp ceph NOFD /proc/42096/task/43358/fd (opendir: Permission denied) 2024-05-09T04:58:11.773 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43359 tp_osd_tp ceph cwd unknown /proc/42096/task/43359/cwd (readlink: Permission denied) 2024-05-09T04:58:11.773 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43359 tp_osd_tp ceph rtd unknown /proc/42096/task/43359/root (readlink: Permission denied) 2024-05-09T04:58:11.773 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43359 tp_osd_tp ceph txt unknown /proc/42096/task/43359/exe (readlink: Permission denied) 2024-05-09T04:58:11.773 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43359 tp_osd_tp ceph NOFD /proc/42096/task/43359/fd (opendir: Permission denied) 2024-05-09T04:58:11.773 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43360 tp_osd_tp ceph cwd unknown /proc/42096/task/43360/cwd (readlink: Permission denied) 2024-05-09T04:58:11.773 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43360 tp_osd_tp ceph rtd unknown /proc/42096/task/43360/root (readlink: Permission denied) 2024-05-09T04:58:11.773 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43360 tp_osd_tp ceph txt unknown /proc/42096/task/43360/exe (readlink: Permission denied) 2024-05-09T04:58:11.773 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43360 tp_osd_tp ceph NOFD /proc/42096/task/43360/fd (opendir: Permission denied) 2024-05-09T04:58:11.773 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43361 tp_osd_tp ceph cwd unknown /proc/42096/task/43361/cwd (readlink: Permission denied) 2024-05-09T04:58:11.774 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43361 tp_osd_tp ceph rtd unknown /proc/42096/task/43361/root (readlink: Permission denied) 2024-05-09T04:58:11.774 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43361 tp_osd_tp ceph txt unknown /proc/42096/task/43361/exe (readlink: Permission denied) 2024-05-09T04:58:11.774 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43361 tp_osd_tp ceph NOFD /proc/42096/task/43361/fd (opendir: Permission denied) 2024-05-09T04:58:11.774 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43362 tp_osd_tp ceph cwd unknown /proc/42096/task/43362/cwd (readlink: Permission denied) 2024-05-09T04:58:11.774 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43362 tp_osd_tp ceph rtd unknown /proc/42096/task/43362/root (readlink: Permission denied) 2024-05-09T04:58:11.774 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43362 tp_osd_tp ceph txt unknown /proc/42096/task/43362/exe (readlink: Permission denied) 2024-05-09T04:58:11.774 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43362 tp_osd_tp ceph NOFD /proc/42096/task/43362/fd (opendir: Permission denied) 2024-05-09T04:58:11.774 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43363 tp_osd_tp ceph cwd unknown /proc/42096/task/43363/cwd (readlink: Permission denied) 2024-05-09T04:58:11.774 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43363 tp_osd_tp ceph rtd unknown /proc/42096/task/43363/root (readlink: Permission denied) 2024-05-09T04:58:11.774 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43363 tp_osd_tp ceph txt unknown /proc/42096/task/43363/exe (readlink: Permission denied) 2024-05-09T04:58:11.774 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43363 tp_osd_tp ceph NOFD /proc/42096/task/43363/fd (opendir: Permission denied) 2024-05-09T04:58:11.774 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43364 tp_osd_tp ceph cwd unknown /proc/42096/task/43364/cwd (readlink: Permission denied) 2024-05-09T04:58:11.774 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43364 tp_osd_tp ceph rtd unknown /proc/42096/task/43364/root (readlink: Permission denied) 2024-05-09T04:58:11.774 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43364 tp_osd_tp ceph txt unknown /proc/42096/task/43364/exe (readlink: Permission denied) 2024-05-09T04:58:11.775 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43364 tp_osd_tp ceph NOFD /proc/42096/task/43364/fd (opendir: Permission denied) 2024-05-09T04:58:11.775 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43365 tp_osd_tp ceph cwd unknown /proc/42096/task/43365/cwd (readlink: Permission denied) 2024-05-09T04:58:11.775 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43365 tp_osd_tp ceph rtd unknown /proc/42096/task/43365/root (readlink: Permission denied) 2024-05-09T04:58:11.775 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43365 tp_osd_tp ceph txt unknown /proc/42096/task/43365/exe (readlink: Permission denied) 2024-05-09T04:58:11.775 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43365 tp_osd_tp ceph NOFD /proc/42096/task/43365/fd (opendir: Permission denied) 2024-05-09T04:58:11.775 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43366 tp_osd_tp ceph cwd unknown /proc/42096/task/43366/cwd (readlink: Permission denied) 2024-05-09T04:58:11.775 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43366 tp_osd_tp ceph rtd unknown /proc/42096/task/43366/root (readlink: Permission denied) 2024-05-09T04:58:11.775 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43366 tp_osd_tp ceph txt unknown /proc/42096/task/43366/exe (readlink: Permission denied) 2024-05-09T04:58:11.775 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43366 tp_osd_tp ceph NOFD /proc/42096/task/43366/fd (opendir: Permission denied) 2024-05-09T04:58:11.775 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43367 osd_srv_h ceph cwd unknown /proc/42096/task/43367/cwd (readlink: Permission denied) 2024-05-09T04:58:11.775 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43367 osd_srv_h ceph rtd unknown /proc/42096/task/43367/root (readlink: Permission denied) 2024-05-09T04:58:11.775 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43367 osd_srv_h ceph txt unknown /proc/42096/task/43367/exe (readlink: Permission denied) 2024-05-09T04:58:11.776 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 42096 43367 osd_srv_h ceph NOFD /proc/42096/task/43367/fd (opendir: Permission denied) 2024-05-09T04:58:11.776 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 43501 root cwd unknown /proc/43501/cwd (readlink: Permission denied) 2024-05-09T04:58:11.776 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 43501 root rtd unknown /proc/43501/root (readlink: Permission denied) 2024-05-09T04:58:11.776 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 43501 root txt unknown /proc/43501/exe (readlink: Permission denied) 2024-05-09T04:58:11.776 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 43501 root NOFD /proc/43501/fd (opendir: Permission denied) 2024-05-09T04:58:11.776 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 43532 root cwd unknown /proc/43532/cwd (readlink: Permission denied) 2024-05-09T04:58:11.776 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 43532 root rtd unknown /proc/43532/root (readlink: Permission denied) 2024-05-09T04:58:11.776 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 43532 root txt unknown /proc/43532/exe (readlink: Permission denied) 2024-05-09T04:58:11.776 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 43532 root NOFD /proc/43532/fd (opendir: Permission denied) 2024-05-09T04:58:11.776 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/6 45994 root cwd unknown /proc/45994/cwd (readlink: Permission denied) 2024-05-09T04:58:11.776 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/6 45994 root rtd unknown /proc/45994/root (readlink: Permission denied) 2024-05-09T04:58:11.776 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/6 45994 root txt unknown /proc/45994/exe (readlink: Permission denied) 2024-05-09T04:58:11.776 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/6 45994 root NOFD /proc/45994/fd (opendir: Permission denied) 2024-05-09T04:58:11.776 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 48741 root cwd unknown /proc/48741/cwd (readlink: Permission denied) 2024-05-09T04:58:11.777 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 48741 root rtd unknown /proc/48741/root (readlink: Permission denied) 2024-05-09T04:58:11.777 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 48741 root txt unknown /proc/48741/exe (readlink: Permission denied) 2024-05-09T04:58:11.777 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 48741 root NOFD /proc/48741/fd (opendir: Permission denied) 2024-05-09T04:58:11.777 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 48743 root cwd unknown /proc/48743/cwd (readlink: Permission denied) 2024-05-09T04:58:11.777 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 48743 root rtd unknown /proc/48743/root (readlink: Permission denied) 2024-05-09T04:58:11.777 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 48743 root txt unknown /proc/48743/exe (readlink: Permission denied) 2024-05-09T04:58:11.777 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 48743 root NOFD /proc/48743/fd (opendir: Permission denied) 2024-05-09T04:58:11.777 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 ceph cwd unknown /proc/48745/cwd (readlink: Permission denied) 2024-05-09T04:58:11.777 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 ceph rtd unknown /proc/48745/root (readlink: Permission denied) 2024-05-09T04:58:11.777 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 ceph txt unknown /proc/48745/exe (readlink: Permission denied) 2024-05-09T04:58:11.777 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 ceph NOFD /proc/48745/fd (opendir: Permission denied) 2024-05-09T04:58:11.777 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48748 log ceph cwd unknown /proc/48745/task/48748/cwd (readlink: Permission denied) 2024-05-09T04:58:11.777 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48748 log ceph rtd unknown /proc/48745/task/48748/root (readlink: Permission denied) 2024-05-09T04:58:11.777 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48748 log ceph txt unknown /proc/48745/task/48748/exe (readlink: Permission denied) 2024-05-09T04:58:11.778 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48748 log ceph NOFD /proc/48745/task/48748/fd (opendir: Permission denied) 2024-05-09T04:58:11.778 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48750 msgr-work ceph cwd unknown /proc/48745/task/48750/cwd (readlink: Permission denied) 2024-05-09T04:58:11.778 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48750 msgr-work ceph rtd unknown /proc/48745/task/48750/root (readlink: Permission denied) 2024-05-09T04:58:11.778 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48750 msgr-work ceph txt unknown /proc/48745/task/48750/exe (readlink: Permission denied) 2024-05-09T04:58:11.778 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48750 msgr-work ceph NOFD /proc/48745/task/48750/fd (opendir: Permission denied) 2024-05-09T04:58:11.778 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48751 msgr-work ceph cwd unknown /proc/48745/task/48751/cwd (readlink: Permission denied) 2024-05-09T04:58:11.778 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48751 msgr-work ceph rtd unknown /proc/48745/task/48751/root (readlink: Permission denied) 2024-05-09T04:58:11.778 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48751 msgr-work ceph txt unknown /proc/48745/task/48751/exe (readlink: Permission denied) 2024-05-09T04:58:11.778 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48751 msgr-work ceph NOFD /proc/48745/task/48751/fd (opendir: Permission denied) 2024-05-09T04:58:11.778 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48752 msgr-work ceph cwd unknown /proc/48745/task/48752/cwd (readlink: Permission denied) 2024-05-09T04:58:11.778 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48752 msgr-work ceph rtd unknown /proc/48745/task/48752/root (readlink: Permission denied) 2024-05-09T04:58:11.778 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48752 msgr-work ceph txt unknown /proc/48745/task/48752/exe (readlink: Permission denied) 2024-05-09T04:58:11.778 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48752 msgr-work ceph NOFD /proc/48745/task/48752/fd (opendir: Permission denied) 2024-05-09T04:58:11.779 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48757 service ceph cwd unknown /proc/48745/task/48757/cwd (readlink: Permission denied) 2024-05-09T04:58:11.779 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48757 service ceph rtd unknown /proc/48745/task/48757/root (readlink: Permission denied) 2024-05-09T04:58:11.779 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48757 service ceph txt unknown /proc/48745/task/48757/exe (readlink: Permission denied) 2024-05-09T04:58:11.779 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48757 service ceph NOFD /proc/48745/task/48757/fd (opendir: Permission denied) 2024-05-09T04:58:11.779 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48758 admin_soc ceph cwd unknown /proc/48745/task/48758/cwd (readlink: Permission denied) 2024-05-09T04:58:11.779 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48758 admin_soc ceph rtd unknown /proc/48745/task/48758/root (readlink: Permission denied) 2024-05-09T04:58:11.779 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48758 admin_soc ceph txt unknown /proc/48745/task/48758/exe (readlink: Permission denied) 2024-05-09T04:58:11.779 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48758 admin_soc ceph NOFD /proc/48745/task/48758/fd (opendir: Permission denied) 2024-05-09T04:58:11.779 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48898 signal_ha ceph cwd unknown /proc/48745/task/48898/cwd (readlink: Permission denied) 2024-05-09T04:58:11.779 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48898 signal_ha ceph rtd unknown /proc/48745/task/48898/root (readlink: Permission denied) 2024-05-09T04:58:11.779 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48898 signal_ha ceph txt unknown /proc/48745/task/48898/exe (readlink: Permission denied) 2024-05-09T04:58:11.779 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48898 signal_ha ceph NOFD /proc/48745/task/48898/fd (opendir: Permission denied) 2024-05-09T04:58:11.779 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48899 io_contex ceph cwd unknown /proc/48745/task/48899/cwd (readlink: Permission denied) 2024-05-09T04:58:11.779 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48899 io_contex ceph rtd unknown /proc/48745/task/48899/root (readlink: Permission denied) 2024-05-09T04:58:11.780 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48899 io_contex ceph txt unknown /proc/48745/task/48899/exe (readlink: Permission denied) 2024-05-09T04:58:11.780 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48899 io_contex ceph NOFD /proc/48745/task/48899/fd (opendir: Permission denied) 2024-05-09T04:58:11.780 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48900 io_contex ceph cwd unknown /proc/48745/task/48900/cwd (readlink: Permission denied) 2024-05-09T04:58:11.780 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48900 io_contex ceph rtd unknown /proc/48745/task/48900/root (readlink: Permission denied) 2024-05-09T04:58:11.780 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48900 io_contex ceph txt unknown /proc/48745/task/48900/exe (readlink: Permission denied) 2024-05-09T04:58:11.780 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48900 io_contex ceph NOFD /proc/48745/task/48900/fd (opendir: Permission denied) 2024-05-09T04:58:11.780 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48957 OpHistory ceph cwd unknown /proc/48745/task/48957/cwd (readlink: Permission denied) 2024-05-09T04:58:11.780 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48957 OpHistory ceph rtd unknown /proc/48745/task/48957/root (readlink: Permission denied) 2024-05-09T04:58:11.780 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48957 OpHistory ceph txt unknown /proc/48745/task/48957/exe (readlink: Permission denied) 2024-05-09T04:58:11.780 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48957 OpHistory ceph NOFD /proc/48745/task/48957/fd (opendir: Permission denied) 2024-05-09T04:58:11.780 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48960 ceph_time ceph cwd unknown /proc/48745/task/48960/cwd (readlink: Permission denied) 2024-05-09T04:58:11.780 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48960 ceph_time ceph rtd unknown /proc/48745/task/48960/root (readlink: Permission denied) 2024-05-09T04:58:11.780 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48960 ceph_time ceph txt unknown /proc/48745/task/48960/exe (readlink: Permission denied) 2024-05-09T04:58:11.781 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 48960 ceph_time ceph NOFD /proc/48745/task/48960/fd (opendir: Permission denied) 2024-05-09T04:58:11.781 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49017 ceph-osd ceph cwd unknown /proc/48745/task/49017/cwd (readlink: Permission denied) 2024-05-09T04:58:11.781 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49017 ceph-osd ceph rtd unknown /proc/48745/task/49017/root (readlink: Permission denied) 2024-05-09T04:58:11.781 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49017 ceph-osd ceph txt unknown /proc/48745/task/49017/exe (readlink: Permission denied) 2024-05-09T04:58:11.781 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49017 ceph-osd ceph NOFD /proc/48745/task/49017/fd (opendir: Permission denied) 2024-05-09T04:58:11.781 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49040 ceph-osd ceph cwd unknown /proc/48745/task/49040/cwd (readlink: Permission denied) 2024-05-09T04:58:11.781 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49040 ceph-osd ceph rtd unknown /proc/48745/task/49040/root (readlink: Permission denied) 2024-05-09T04:58:11.781 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49040 ceph-osd ceph txt unknown /proc/48745/task/49040/exe (readlink: Permission denied) 2024-05-09T04:58:11.781 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49040 ceph-osd ceph NOFD /proc/48745/task/49040/fd (opendir: Permission denied) 2024-05-09T04:58:11.781 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49060 ceph-osd ceph cwd unknown /proc/48745/task/49060/cwd (readlink: Permission denied) 2024-05-09T04:58:11.781 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49060 ceph-osd ceph rtd unknown /proc/48745/task/49060/root (readlink: Permission denied) 2024-05-09T04:58:11.781 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49060 ceph-osd ceph txt unknown /proc/48745/task/49060/exe (readlink: Permission denied) 2024-05-09T04:58:11.781 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49060 ceph-osd ceph NOFD /proc/48745/task/49060/fd (opendir: Permission denied) 2024-05-09T04:58:11.782 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49081 ceph-osd ceph cwd unknown /proc/48745/task/49081/cwd (readlink: Permission denied) 2024-05-09T04:58:11.782 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49081 ceph-osd ceph rtd unknown /proc/48745/task/49081/root (readlink: Permission denied) 2024-05-09T04:58:11.782 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49081 ceph-osd ceph txt unknown /proc/48745/task/49081/exe (readlink: Permission denied) 2024-05-09T04:58:11.782 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49081 ceph-osd ceph NOFD /proc/48745/task/49081/fd (opendir: Permission denied) 2024-05-09T04:58:11.782 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49087 ceph-osd ceph cwd unknown /proc/48745/task/49087/cwd (readlink: Permission denied) 2024-05-09T04:58:11.782 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49087 ceph-osd ceph rtd unknown /proc/48745/task/49087/root (readlink: Permission denied) 2024-05-09T04:58:11.782 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49087 ceph-osd ceph txt unknown /proc/48745/task/49087/exe (readlink: Permission denied) 2024-05-09T04:58:11.782 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49087 ceph-osd ceph NOFD /proc/48745/task/49087/fd (opendir: Permission denied) 2024-05-09T04:58:11.782 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49097 ceph-osd ceph cwd unknown /proc/48745/task/49097/cwd (readlink: Permission denied) 2024-05-09T04:58:11.782 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49097 ceph-osd ceph rtd unknown /proc/48745/task/49097/root (readlink: Permission denied) 2024-05-09T04:58:11.782 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49097 ceph-osd ceph txt unknown /proc/48745/task/49097/exe (readlink: Permission denied) 2024-05-09T04:58:11.782 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49097 ceph-osd ceph NOFD /proc/48745/task/49097/fd (opendir: Permission denied) 2024-05-09T04:58:11.782 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49131 ceph-osd ceph cwd unknown /proc/48745/task/49131/cwd (readlink: Permission denied) 2024-05-09T04:58:11.782 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49131 ceph-osd ceph rtd unknown /proc/48745/task/49131/root (readlink: Permission denied) 2024-05-09T04:58:11.782 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49131 ceph-osd ceph txt unknown /proc/48745/task/49131/exe (readlink: Permission denied) 2024-05-09T04:58:11.783 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49131 ceph-osd ceph NOFD /proc/48745/task/49131/fd (opendir: Permission denied) 2024-05-09T04:58:11.783 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49139 ceph-osd ceph cwd unknown /proc/48745/task/49139/cwd (readlink: Permission denied) 2024-05-09T04:58:11.783 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49139 ceph-osd ceph rtd unknown /proc/48745/task/49139/root (readlink: Permission denied) 2024-05-09T04:58:11.783 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49139 ceph-osd ceph txt unknown /proc/48745/task/49139/exe (readlink: Permission denied) 2024-05-09T04:58:11.783 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49139 ceph-osd ceph NOFD /proc/48745/task/49139/fd (opendir: Permission denied) 2024-05-09T04:58:11.783 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49140 ceph-osd ceph cwd unknown /proc/48745/task/49140/cwd (readlink: Permission denied) 2024-05-09T04:58:11.783 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49140 ceph-osd ceph rtd unknown /proc/48745/task/49140/root (readlink: Permission denied) 2024-05-09T04:58:11.783 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49140 ceph-osd ceph txt unknown /proc/48745/task/49140/exe (readlink: Permission denied) 2024-05-09T04:58:11.783 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49140 ceph-osd ceph NOFD /proc/48745/task/49140/fd (opendir: Permission denied) 2024-05-09T04:58:11.783 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49142 safe_time ceph cwd unknown /proc/48745/task/49142/cwd (readlink: Permission denied) 2024-05-09T04:58:11.783 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49142 safe_time ceph rtd unknown /proc/48745/task/49142/root (readlink: Permission denied) 2024-05-09T04:58:11.783 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49142 safe_time ceph txt unknown /proc/48745/task/49142/exe (readlink: Permission denied) 2024-05-09T04:58:11.783 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49142 safe_time ceph NOFD /proc/48745/task/49142/fd (opendir: Permission denied) 2024-05-09T04:58:11.784 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49143 safe_time ceph cwd unknown /proc/48745/task/49143/cwd (readlink: Permission denied) 2024-05-09T04:58:11.784 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49143 safe_time ceph rtd unknown /proc/48745/task/49143/root (readlink: Permission denied) 2024-05-09T04:58:11.784 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49143 safe_time ceph txt unknown /proc/48745/task/49143/exe (readlink: Permission denied) 2024-05-09T04:58:11.784 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49143 safe_time ceph NOFD /proc/48745/task/49143/fd (opendir: Permission denied) 2024-05-09T04:58:11.784 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49144 safe_time ceph cwd unknown /proc/48745/task/49144/cwd (readlink: Permission denied) 2024-05-09T04:58:11.784 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49144 safe_time ceph rtd unknown /proc/48745/task/49144/root (readlink: Permission denied) 2024-05-09T04:58:11.784 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49144 safe_time ceph txt unknown /proc/48745/task/49144/exe (readlink: Permission denied) 2024-05-09T04:58:11.784 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49144 safe_time ceph NOFD /proc/48745/task/49144/fd (opendir: Permission denied) 2024-05-09T04:58:11.784 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49145 safe_time ceph cwd unknown /proc/48745/task/49145/cwd (readlink: Permission denied) 2024-05-09T04:58:11.784 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49145 safe_time ceph rtd unknown /proc/48745/task/49145/root (readlink: Permission denied) 2024-05-09T04:58:11.784 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49145 safe_time ceph txt unknown /proc/48745/task/49145/exe (readlink: Permission denied) 2024-05-09T04:58:11.784 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49145 safe_time ceph NOFD /proc/48745/task/49145/fd (opendir: Permission denied) 2024-05-09T04:58:11.785 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49146 fn_anonym ceph cwd unknown /proc/48745/task/49146/cwd (readlink: Permission denied) 2024-05-09T04:58:11.785 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49146 fn_anonym ceph rtd unknown /proc/48745/task/49146/root (readlink: Permission denied) 2024-05-09T04:58:11.785 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49146 fn_anonym ceph txt unknown /proc/48745/task/49146/exe (readlink: Permission denied) 2024-05-09T04:58:11.785 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49146 fn_anonym ceph NOFD /proc/48745/task/49146/fd (opendir: Permission denied) 2024-05-09T04:58:11.785 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49152 rocksdb:l ceph cwd unknown /proc/48745/task/49152/cwd (readlink: Permission denied) 2024-05-09T04:58:11.785 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49152 rocksdb:l ceph rtd unknown /proc/48745/task/49152/root (readlink: Permission denied) 2024-05-09T04:58:11.785 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49152 rocksdb:l ceph txt unknown /proc/48745/task/49152/exe (readlink: Permission denied) 2024-05-09T04:58:11.785 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49152 rocksdb:l ceph NOFD /proc/48745/task/49152/fd (opendir: Permission denied) 2024-05-09T04:58:11.785 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49153 rocksdb:l ceph cwd unknown /proc/48745/task/49153/cwd (readlink: Permission denied) 2024-05-09T04:58:11.785 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49153 rocksdb:l ceph rtd unknown /proc/48745/task/49153/root (readlink: Permission denied) 2024-05-09T04:58:11.785 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49153 rocksdb:l ceph txt unknown /proc/48745/task/49153/exe (readlink: Permission denied) 2024-05-09T04:58:11.786 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49153 rocksdb:l ceph NOFD /proc/48745/task/49153/fd (opendir: Permission denied) 2024-05-09T04:58:11.786 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49154 rocksdb:l ceph cwd unknown /proc/48745/task/49154/cwd (readlink: Permission denied) 2024-05-09T04:58:11.786 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49154 rocksdb:l ceph rtd unknown /proc/48745/task/49154/root (readlink: Permission denied) 2024-05-09T04:58:11.786 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49154 rocksdb:l ceph txt unknown /proc/48745/task/49154/exe (readlink: Permission denied) 2024-05-09T04:58:11.786 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49154 rocksdb:l ceph NOFD /proc/48745/task/49154/fd (opendir: Permission denied) 2024-05-09T04:58:11.786 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49155 rocksdb:h ceph cwd unknown /proc/48745/task/49155/cwd (readlink: Permission denied) 2024-05-09T04:58:11.786 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49155 rocksdb:h ceph rtd unknown /proc/48745/task/49155/root (readlink: Permission denied) 2024-05-09T04:58:11.786 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49155 rocksdb:h ceph txt unknown /proc/48745/task/49155/exe (readlink: Permission denied) 2024-05-09T04:58:11.786 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49155 rocksdb:h ceph NOFD /proc/48745/task/49155/fd (opendir: Permission denied) 2024-05-09T04:58:11.786 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49643 bstore_ai ceph cwd unknown /proc/48745/task/49643/cwd (readlink: Permission denied) 2024-05-09T04:58:11.786 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49643 bstore_ai ceph rtd unknown /proc/48745/task/49643/root (readlink: Permission denied) 2024-05-09T04:58:11.787 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49643 bstore_ai ceph txt unknown /proc/48745/task/49643/exe (readlink: Permission denied) 2024-05-09T04:58:11.787 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49643 bstore_ai ceph NOFD /proc/48745/task/49643/fd (opendir: Permission denied) 2024-05-09T04:58:11.787 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49848 bstore_ai ceph cwd unknown /proc/48745/task/49848/cwd (readlink: Permission denied) 2024-05-09T04:58:11.787 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49848 bstore_ai ceph rtd unknown /proc/48745/task/49848/root (readlink: Permission denied) 2024-05-09T04:58:11.787 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49848 bstore_ai ceph txt unknown /proc/48745/task/49848/exe (readlink: Permission denied) 2024-05-09T04:58:11.787 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 49848 bstore_ai ceph NOFD /proc/48745/task/49848/fd (opendir: Permission denied) 2024-05-09T04:58:11.787 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50031 ceph-osd ceph cwd unknown /proc/48745/task/50031/cwd (readlink: Permission denied) 2024-05-09T04:58:11.787 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50031 ceph-osd ceph rtd unknown /proc/48745/task/50031/root (readlink: Permission denied) 2024-05-09T04:58:11.787 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50031 ceph-osd ceph txt unknown /proc/48745/task/50031/exe (readlink: Permission denied) 2024-05-09T04:58:11.787 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50031 ceph-osd ceph NOFD /proc/48745/task/50031/fd (opendir: Permission denied) 2024-05-09T04:58:11.787 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50032 cfin ceph cwd unknown /proc/48745/task/50032/cwd (readlink: Permission denied) 2024-05-09T04:58:11.788 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50032 cfin ceph rtd unknown /proc/48745/task/50032/root (readlink: Permission denied) 2024-05-09T04:58:11.788 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50032 cfin ceph txt unknown /proc/48745/task/50032/exe (readlink: Permission denied) 2024-05-09T04:58:11.788 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50032 cfin ceph NOFD /proc/48745/task/50032/fd (opendir: Permission denied) 2024-05-09T04:58:11.788 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50033 bstore_kv ceph cwd unknown /proc/48745/task/50033/cwd (readlink: Permission denied) 2024-05-09T04:58:11.788 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50033 bstore_kv ceph rtd unknown /proc/48745/task/50033/root (readlink: Permission denied) 2024-05-09T04:58:11.788 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50033 bstore_kv ceph txt unknown /proc/48745/task/50033/exe (readlink: Permission denied) 2024-05-09T04:58:11.788 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50033 bstore_kv ceph NOFD /proc/48745/task/50033/fd (opendir: Permission denied) 2024-05-09T04:58:11.788 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50034 bstore_kv ceph cwd unknown /proc/48745/task/50034/cwd (readlink: Permission denied) 2024-05-09T04:58:11.788 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50034 bstore_kv ceph rtd unknown /proc/48745/task/50034/root (readlink: Permission denied) 2024-05-09T04:58:11.788 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50034 bstore_kv ceph txt unknown /proc/48745/task/50034/exe (readlink: Permission denied) 2024-05-09T04:58:11.788 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50034 bstore_kv ceph NOFD /proc/48745/task/50034/fd (opendir: Permission denied) 2024-05-09T04:58:11.788 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50035 bstore_me ceph cwd unknown /proc/48745/task/50035/cwd (readlink: Permission denied) 2024-05-09T04:58:11.788 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50035 bstore_me ceph rtd unknown /proc/48745/task/50035/root (readlink: Permission denied) 2024-05-09T04:58:11.789 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50035 bstore_me ceph txt unknown /proc/48745/task/50035/exe (readlink: Permission denied) 2024-05-09T04:58:11.789 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50035 bstore_me ceph NOFD /proc/48745/task/50035/fd (opendir: Permission denied) 2024-05-09T04:58:11.789 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50036 ms_dispat ceph cwd unknown /proc/48745/task/50036/cwd (readlink: Permission denied) 2024-05-09T04:58:11.789 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50036 ms_dispat ceph rtd unknown /proc/48745/task/50036/root (readlink: Permission denied) 2024-05-09T04:58:11.789 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50036 ms_dispat ceph txt unknown /proc/48745/task/50036/exe (readlink: Permission denied) 2024-05-09T04:58:11.789 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50036 ms_dispat ceph NOFD /proc/48745/task/50036/fd (opendir: Permission denied) 2024-05-09T04:58:11.789 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50037 ms_local ceph cwd unknown /proc/48745/task/50037/cwd (readlink: Permission denied) 2024-05-09T04:58:11.789 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50037 ms_local ceph rtd unknown /proc/48745/task/50037/root (readlink: Permission denied) 2024-05-09T04:58:11.789 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50037 ms_local ceph txt unknown /proc/48745/task/50037/exe (readlink: Permission denied) 2024-05-09T04:58:11.789 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50037 ms_local ceph NOFD /proc/48745/task/50037/fd (opendir: Permission denied) 2024-05-09T04:58:11.789 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50038 safe_time ceph cwd unknown /proc/48745/task/50038/cwd (readlink: Permission denied) 2024-05-09T04:58:11.790 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50038 safe_time ceph rtd unknown /proc/48745/task/50038/root (readlink: Permission denied) 2024-05-09T04:58:11.790 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50038 safe_time ceph txt unknown /proc/48745/task/50038/exe (readlink: Permission denied) 2024-05-09T04:58:11.790 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50038 safe_time ceph NOFD /proc/48745/task/50038/fd (opendir: Permission denied) 2024-05-09T04:58:11.790 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50039 safe_time ceph cwd unknown /proc/48745/task/50039/cwd (readlink: Permission denied) 2024-05-09T04:58:11.790 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50039 safe_time ceph rtd unknown /proc/48745/task/50039/root (readlink: Permission denied) 2024-05-09T04:58:11.790 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50039 safe_time ceph txt unknown /proc/48745/task/50039/exe (readlink: Permission denied) 2024-05-09T04:58:11.790 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50039 safe_time ceph NOFD /proc/48745/task/50039/fd (opendir: Permission denied) 2024-05-09T04:58:11.790 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50040 ms_dispat ceph cwd unknown /proc/48745/task/50040/cwd (readlink: Permission denied) 2024-05-09T04:58:11.790 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50040 ms_dispat ceph rtd unknown /proc/48745/task/50040/root (readlink: Permission denied) 2024-05-09T04:58:11.790 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50040 ms_dispat ceph txt unknown /proc/48745/task/50040/exe (readlink: Permission denied) 2024-05-09T04:58:11.791 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50040 ms_dispat ceph NOFD /proc/48745/task/50040/fd (opendir: Permission denied) 2024-05-09T04:58:11.791 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50041 ms_local ceph cwd unknown /proc/48745/task/50041/cwd (readlink: Permission denied) 2024-05-09T04:58:11.791 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50041 ms_local ceph rtd unknown /proc/48745/task/50041/root (readlink: Permission denied) 2024-05-09T04:58:11.791 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50041 ms_local ceph txt unknown /proc/48745/task/50041/exe (readlink: Permission denied) 2024-05-09T04:58:11.791 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50041 ms_local ceph NOFD /proc/48745/task/50041/fd (opendir: Permission denied) 2024-05-09T04:58:11.791 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50042 ms_dispat ceph cwd unknown /proc/48745/task/50042/cwd (readlink: Permission denied) 2024-05-09T04:58:11.791 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50042 ms_dispat ceph rtd unknown /proc/48745/task/50042/root (readlink: Permission denied) 2024-05-09T04:58:11.791 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50042 ms_dispat ceph txt unknown /proc/48745/task/50042/exe (readlink: Permission denied) 2024-05-09T04:58:11.791 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50042 ms_dispat ceph NOFD /proc/48745/task/50042/fd (opendir: Permission denied) 2024-05-09T04:58:11.791 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50043 ms_local ceph cwd unknown /proc/48745/task/50043/cwd (readlink: Permission denied) 2024-05-09T04:58:11.791 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50043 ms_local ceph rtd unknown /proc/48745/task/50043/root (readlink: Permission denied) 2024-05-09T04:58:11.792 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50043 ms_local ceph txt unknown /proc/48745/task/50043/exe (readlink: Permission denied) 2024-05-09T04:58:11.792 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50043 ms_local ceph NOFD /proc/48745/task/50043/fd (opendir: Permission denied) 2024-05-09T04:58:11.792 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50044 ms_dispat ceph cwd unknown /proc/48745/task/50044/cwd (readlink: Permission denied) 2024-05-09T04:58:11.792 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50044 ms_dispat ceph rtd unknown /proc/48745/task/50044/root (readlink: Permission denied) 2024-05-09T04:58:11.792 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50044 ms_dispat ceph txt unknown /proc/48745/task/50044/exe (readlink: Permission denied) 2024-05-09T04:58:11.792 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50044 ms_dispat ceph NOFD /proc/48745/task/50044/fd (opendir: Permission denied) 2024-05-09T04:58:11.792 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50045 ms_local ceph cwd unknown /proc/48745/task/50045/cwd (readlink: Permission denied) 2024-05-09T04:58:11.792 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50045 ms_local ceph rtd unknown /proc/48745/task/50045/root (readlink: Permission denied) 2024-05-09T04:58:11.792 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50045 ms_local ceph txt unknown /proc/48745/task/50045/exe (readlink: Permission denied) 2024-05-09T04:58:11.792 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50045 ms_local ceph NOFD /proc/48745/task/50045/fd (opendir: Permission denied) 2024-05-09T04:58:11.793 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50046 ms_dispat ceph cwd unknown /proc/48745/task/50046/cwd (readlink: Permission denied) 2024-05-09T04:58:11.793 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50046 ms_dispat ceph rtd unknown /proc/48745/task/50046/root (readlink: Permission denied) 2024-05-09T04:58:11.793 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50046 ms_dispat ceph txt unknown /proc/48745/task/50046/exe (readlink: Permission denied) 2024-05-09T04:58:11.793 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50046 ms_dispat ceph NOFD /proc/48745/task/50046/fd (opendir: Permission denied) 2024-05-09T04:58:11.793 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50047 ms_local ceph cwd unknown /proc/48745/task/50047/cwd (readlink: Permission denied) 2024-05-09T04:58:11.793 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50047 ms_local ceph rtd unknown /proc/48745/task/50047/root (readlink: Permission denied) 2024-05-09T04:58:11.793 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50047 ms_local ceph txt unknown /proc/48745/task/50047/exe (readlink: Permission denied) 2024-05-09T04:58:11.793 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50047 ms_local ceph NOFD /proc/48745/task/50047/fd (opendir: Permission denied) 2024-05-09T04:58:11.793 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50048 ms_dispat ceph cwd unknown /proc/48745/task/50048/cwd (readlink: Permission denied) 2024-05-09T04:58:11.793 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50048 ms_dispat ceph rtd unknown /proc/48745/task/50048/root (readlink: Permission denied) 2024-05-09T04:58:11.794 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50048 ms_dispat ceph txt unknown /proc/48745/task/50048/exe (readlink: Permission denied) 2024-05-09T04:58:11.794 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50048 ms_dispat ceph NOFD /proc/48745/task/50048/fd (opendir: Permission denied) 2024-05-09T04:58:11.794 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50049 ms_local ceph cwd unknown /proc/48745/task/50049/cwd (readlink: Permission denied) 2024-05-09T04:58:11.794 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50049 ms_local ceph rtd unknown /proc/48745/task/50049/root (readlink: Permission denied) 2024-05-09T04:58:11.794 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50049 ms_local ceph txt unknown /proc/48745/task/50049/exe (readlink: Permission denied) 2024-05-09T04:58:11.794 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50049 ms_local ceph NOFD /proc/48745/task/50049/fd (opendir: Permission denied) 2024-05-09T04:58:11.794 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50050 ms_dispat ceph cwd unknown /proc/48745/task/50050/cwd (readlink: Permission denied) 2024-05-09T04:58:11.794 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50050 ms_dispat ceph rtd unknown /proc/48745/task/50050/root (readlink: Permission denied) 2024-05-09T04:58:11.794 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50050 ms_dispat ceph txt unknown /proc/48745/task/50050/exe (readlink: Permission denied) 2024-05-09T04:58:11.794 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50050 ms_dispat ceph NOFD /proc/48745/task/50050/fd (opendir: Permission denied) 2024-05-09T04:58:11.794 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50051 ms_local ceph cwd unknown /proc/48745/task/50051/cwd (readlink: Permission denied) 2024-05-09T04:58:11.794 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50051 ms_local ceph rtd unknown /proc/48745/task/50051/root (readlink: Permission denied) 2024-05-09T04:58:11.794 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50051 ms_local ceph txt unknown /proc/48745/task/50051/exe (readlink: Permission denied) 2024-05-09T04:58:11.794 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50051 ms_local ceph NOFD /proc/48745/task/50051/fd (opendir: Permission denied) 2024-05-09T04:58:11.795 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50052 fn_anonym ceph cwd unknown /proc/48745/task/50052/cwd (readlink: Permission denied) 2024-05-09T04:58:11.795 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50052 fn_anonym ceph rtd unknown /proc/48745/task/50052/root (readlink: Permission denied) 2024-05-09T04:58:11.795 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50052 fn_anonym ceph txt unknown /proc/48745/task/50052/exe (readlink: Permission denied) 2024-05-09T04:58:11.795 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50052 fn_anonym ceph NOFD /proc/48745/task/50052/fd (opendir: Permission denied) 2024-05-09T04:58:11.795 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50053 finisher ceph cwd unknown /proc/48745/task/50053/cwd (readlink: Permission denied) 2024-05-09T04:58:11.795 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50053 finisher ceph rtd unknown /proc/48745/task/50053/root (readlink: Permission denied) 2024-05-09T04:58:11.795 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50053 finisher ceph txt unknown /proc/48745/task/50053/exe (readlink: Permission denied) 2024-05-09T04:58:11.795 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50053 finisher ceph NOFD /proc/48745/task/50053/fd (opendir: Permission denied) 2024-05-09T04:58:11.795 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50054 safe_time ceph cwd unknown /proc/48745/task/50054/cwd (readlink: Permission denied) 2024-05-09T04:58:11.795 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50054 safe_time ceph rtd unknown /proc/48745/task/50054/root (readlink: Permission denied) 2024-05-09T04:58:11.795 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50054 safe_time ceph txt unknown /proc/48745/task/50054/exe (readlink: Permission denied) 2024-05-09T04:58:11.795 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50054 safe_time ceph NOFD /proc/48745/task/50054/fd (opendir: Permission denied) 2024-05-09T04:58:11.795 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50055 safe_time ceph cwd unknown /proc/48745/task/50055/cwd (readlink: Permission denied) 2024-05-09T04:58:11.795 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50055 safe_time ceph rtd unknown /proc/48745/task/50055/root (readlink: Permission denied) 2024-05-09T04:58:11.795 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50055 safe_time ceph txt unknown /proc/48745/task/50055/exe (readlink: Permission denied) 2024-05-09T04:58:11.796 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50055 safe_time ceph NOFD /proc/48745/task/50055/fd (opendir: Permission denied) 2024-05-09T04:58:11.796 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50056 ceph-osd ceph cwd unknown /proc/48745/task/50056/cwd (readlink: Permission denied) 2024-05-09T04:58:11.796 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50056 ceph-osd ceph rtd unknown /proc/48745/task/50056/root (readlink: Permission denied) 2024-05-09T04:58:11.796 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50056 ceph-osd ceph txt unknown /proc/48745/task/50056/exe (readlink: Permission denied) 2024-05-09T04:58:11.796 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50056 ceph-osd ceph NOFD /proc/48745/task/50056/fd (opendir: Permission denied) 2024-05-09T04:58:11.796 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50057 osd_srv_a ceph cwd unknown /proc/48745/task/50057/cwd (readlink: Permission denied) 2024-05-09T04:58:11.796 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50057 osd_srv_a ceph rtd unknown /proc/48745/task/50057/root (readlink: Permission denied) 2024-05-09T04:58:11.796 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50057 osd_srv_a ceph txt unknown /proc/48745/task/50057/exe (readlink: Permission denied) 2024-05-09T04:58:11.796 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50057 osd_srv_a ceph NOFD /proc/48745/task/50057/fd (opendir: Permission denied) 2024-05-09T04:58:11.796 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50058 tp_osd_tp ceph cwd unknown /proc/48745/task/50058/cwd (readlink: Permission denied) 2024-05-09T04:58:11.796 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50058 tp_osd_tp ceph rtd unknown /proc/48745/task/50058/root (readlink: Permission denied) 2024-05-09T04:58:11.796 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50058 tp_osd_tp ceph txt unknown /proc/48745/task/50058/exe (readlink: Permission denied) 2024-05-09T04:58:11.796 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50058 tp_osd_tp ceph NOFD /proc/48745/task/50058/fd (opendir: Permission denied) 2024-05-09T04:58:11.796 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50059 tp_osd_tp ceph cwd unknown /proc/48745/task/50059/cwd (readlink: Permission denied) 2024-05-09T04:58:11.797 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50059 tp_osd_tp ceph rtd unknown /proc/48745/task/50059/root (readlink: Permission denied) 2024-05-09T04:58:11.797 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50059 tp_osd_tp ceph txt unknown /proc/48745/task/50059/exe (readlink: Permission denied) 2024-05-09T04:58:11.797 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50059 tp_osd_tp ceph NOFD /proc/48745/task/50059/fd (opendir: Permission denied) 2024-05-09T04:58:11.797 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50060 tp_osd_tp ceph cwd unknown /proc/48745/task/50060/cwd (readlink: Permission denied) 2024-05-09T04:58:11.797 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50060 tp_osd_tp ceph rtd unknown /proc/48745/task/50060/root (readlink: Permission denied) 2024-05-09T04:58:11.797 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50060 tp_osd_tp ceph txt unknown /proc/48745/task/50060/exe (readlink: Permission denied) 2024-05-09T04:58:11.797 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50060 tp_osd_tp ceph NOFD /proc/48745/task/50060/fd (opendir: Permission denied) 2024-05-09T04:58:11.797 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50061 tp_osd_tp ceph cwd unknown /proc/48745/task/50061/cwd (readlink: Permission denied) 2024-05-09T04:58:11.797 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50061 tp_osd_tp ceph rtd unknown /proc/48745/task/50061/root (readlink: Permission denied) 2024-05-09T04:58:11.797 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50061 tp_osd_tp ceph txt unknown /proc/48745/task/50061/exe (readlink: Permission denied) 2024-05-09T04:58:11.797 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50061 tp_osd_tp ceph NOFD /proc/48745/task/50061/fd (opendir: Permission denied) 2024-05-09T04:58:11.797 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50062 tp_osd_tp ceph cwd unknown /proc/48745/task/50062/cwd (readlink: Permission denied) 2024-05-09T04:58:11.797 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50062 tp_osd_tp ceph rtd unknown /proc/48745/task/50062/root (readlink: Permission denied) 2024-05-09T04:58:11.797 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50062 tp_osd_tp ceph txt unknown /proc/48745/task/50062/exe (readlink: Permission denied) 2024-05-09T04:58:11.798 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50062 tp_osd_tp ceph NOFD /proc/48745/task/50062/fd (opendir: Permission denied) 2024-05-09T04:58:11.798 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50063 tp_osd_tp ceph cwd unknown /proc/48745/task/50063/cwd (readlink: Permission denied) 2024-05-09T04:58:11.798 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50063 tp_osd_tp ceph rtd unknown /proc/48745/task/50063/root (readlink: Permission denied) 2024-05-09T04:58:11.798 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50063 tp_osd_tp ceph txt unknown /proc/48745/task/50063/exe (readlink: Permission denied) 2024-05-09T04:58:11.798 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50063 tp_osd_tp ceph NOFD /proc/48745/task/50063/fd (opendir: Permission denied) 2024-05-09T04:58:11.798 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50064 tp_osd_tp ceph cwd unknown /proc/48745/task/50064/cwd (readlink: Permission denied) 2024-05-09T04:58:11.798 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50064 tp_osd_tp ceph rtd unknown /proc/48745/task/50064/root (readlink: Permission denied) 2024-05-09T04:58:11.798 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50064 tp_osd_tp ceph txt unknown /proc/48745/task/50064/exe (readlink: Permission denied) 2024-05-09T04:58:11.798 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50064 tp_osd_tp ceph NOFD /proc/48745/task/50064/fd (opendir: Permission denied) 2024-05-09T04:58:11.798 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50065 tp_osd_tp ceph cwd unknown /proc/48745/task/50065/cwd (readlink: Permission denied) 2024-05-09T04:58:11.798 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50065 tp_osd_tp ceph rtd unknown /proc/48745/task/50065/root (readlink: Permission denied) 2024-05-09T04:58:11.798 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50065 tp_osd_tp ceph txt unknown /proc/48745/task/50065/exe (readlink: Permission denied) 2024-05-09T04:58:11.798 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50065 tp_osd_tp ceph NOFD /proc/48745/task/50065/fd (opendir: Permission denied) 2024-05-09T04:58:11.798 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50066 tp_osd_tp ceph cwd unknown /proc/48745/task/50066/cwd (readlink: Permission denied) 2024-05-09T04:58:11.798 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50066 tp_osd_tp ceph rtd unknown /proc/48745/task/50066/root (readlink: Permission denied) 2024-05-09T04:58:11.798 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50066 tp_osd_tp ceph txt unknown /proc/48745/task/50066/exe (readlink: Permission denied) 2024-05-09T04:58:11.799 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50066 tp_osd_tp ceph NOFD /proc/48745/task/50066/fd (opendir: Permission denied) 2024-05-09T04:58:11.799 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50067 tp_osd_tp ceph cwd unknown /proc/48745/task/50067/cwd (readlink: Permission denied) 2024-05-09T04:58:11.799 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50067 tp_osd_tp ceph rtd unknown /proc/48745/task/50067/root (readlink: Permission denied) 2024-05-09T04:58:11.799 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50067 tp_osd_tp ceph txt unknown /proc/48745/task/50067/exe (readlink: Permission denied) 2024-05-09T04:58:11.799 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50067 tp_osd_tp ceph NOFD /proc/48745/task/50067/fd (opendir: Permission denied) 2024-05-09T04:58:11.799 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50068 tp_osd_tp ceph cwd unknown /proc/48745/task/50068/cwd (readlink: Permission denied) 2024-05-09T04:58:11.799 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50068 tp_osd_tp ceph rtd unknown /proc/48745/task/50068/root (readlink: Permission denied) 2024-05-09T04:58:11.799 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50068 tp_osd_tp ceph txt unknown /proc/48745/task/50068/exe (readlink: Permission denied) 2024-05-09T04:58:11.799 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50068 tp_osd_tp ceph NOFD /proc/48745/task/50068/fd (opendir: Permission denied) 2024-05-09T04:58:11.799 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50069 tp_osd_tp ceph cwd unknown /proc/48745/task/50069/cwd (readlink: Permission denied) 2024-05-09T04:58:11.799 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50069 tp_osd_tp ceph rtd unknown /proc/48745/task/50069/root (readlink: Permission denied) 2024-05-09T04:58:11.799 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50069 tp_osd_tp ceph txt unknown /proc/48745/task/50069/exe (readlink: Permission denied) 2024-05-09T04:58:11.799 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50069 tp_osd_tp ceph NOFD /proc/48745/task/50069/fd (opendir: Permission denied) 2024-05-09T04:58:11.799 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50070 tp_osd_tp ceph cwd unknown /proc/48745/task/50070/cwd (readlink: Permission denied) 2024-05-09T04:58:11.800 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50070 tp_osd_tp ceph rtd unknown /proc/48745/task/50070/root (readlink: Permission denied) 2024-05-09T04:58:11.800 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50070 tp_osd_tp ceph txt unknown /proc/48745/task/50070/exe (readlink: Permission denied) 2024-05-09T04:58:11.800 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50070 tp_osd_tp ceph NOFD /proc/48745/task/50070/fd (opendir: Permission denied) 2024-05-09T04:58:11.800 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50071 tp_osd_tp ceph cwd unknown /proc/48745/task/50071/cwd (readlink: Permission denied) 2024-05-09T04:58:11.800 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50071 tp_osd_tp ceph rtd unknown /proc/48745/task/50071/root (readlink: Permission denied) 2024-05-09T04:58:11.800 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50071 tp_osd_tp ceph txt unknown /proc/48745/task/50071/exe (readlink: Permission denied) 2024-05-09T04:58:11.800 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50071 tp_osd_tp ceph NOFD /proc/48745/task/50071/fd (opendir: Permission denied) 2024-05-09T04:58:11.800 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50072 tp_osd_tp ceph cwd unknown /proc/48745/task/50072/cwd (readlink: Permission denied) 2024-05-09T04:58:11.800 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50072 tp_osd_tp ceph rtd unknown /proc/48745/task/50072/root (readlink: Permission denied) 2024-05-09T04:58:11.800 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50072 tp_osd_tp ceph txt unknown /proc/48745/task/50072/exe (readlink: Permission denied) 2024-05-09T04:58:11.800 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50072 tp_osd_tp ceph NOFD /proc/48745/task/50072/fd (opendir: Permission denied) 2024-05-09T04:58:11.800 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50073 tp_osd_tp ceph cwd unknown /proc/48745/task/50073/cwd (readlink: Permission denied) 2024-05-09T04:58:11.800 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50073 tp_osd_tp ceph rtd unknown /proc/48745/task/50073/root (readlink: Permission denied) 2024-05-09T04:58:11.801 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50073 tp_osd_tp ceph txt unknown /proc/48745/task/50073/exe (readlink: Permission denied) 2024-05-09T04:58:11.801 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50073 tp_osd_tp ceph NOFD /proc/48745/task/50073/fd (opendir: Permission denied) 2024-05-09T04:58:11.801 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50074 osd_srv_h ceph cwd unknown /proc/48745/task/50074/cwd (readlink: Permission denied) 2024-05-09T04:58:11.801 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50074 osd_srv_h ceph rtd unknown /proc/48745/task/50074/root (readlink: Permission denied) 2024-05-09T04:58:11.801 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50074 osd_srv_h ceph txt unknown /proc/48745/task/50074/exe (readlink: Permission denied) 2024-05-09T04:58:11.801 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 48745 50074 osd_srv_h ceph NOFD /proc/48745/task/50074/fd (opendir: Permission denied) 2024-05-09T04:58:11.801 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 50275 root cwd unknown /proc/50275/cwd (readlink: Permission denied) 2024-05-09T04:58:11.801 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 50275 root rtd unknown /proc/50275/root (readlink: Permission denied) 2024-05-09T04:58:11.801 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 50275 root txt unknown /proc/50275/exe (readlink: Permission denied) 2024-05-09T04:58:11.801 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 50275 root NOFD /proc/50275/fd (opendir: Permission denied) 2024-05-09T04:58:11.801 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 50306 root cwd unknown /proc/50306/cwd (readlink: Permission denied) 2024-05-09T04:58:11.801 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 50306 root rtd unknown /proc/50306/root (readlink: Permission denied) 2024-05-09T04:58:11.801 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 50306 root txt unknown /proc/50306/exe (readlink: Permission denied) 2024-05-09T04:58:11.801 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 50306 root NOFD /proc/50306/fd (opendir: Permission denied) 2024-05-09T04:58:11.801 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/4 52842 root cwd unknown /proc/52842/cwd (readlink: Permission denied) 2024-05-09T04:58:11.802 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/4 52842 root rtd unknown /proc/52842/root (readlink: Permission denied) 2024-05-09T04:58:11.802 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/4 52842 root txt unknown /proc/52842/exe (readlink: Permission denied) 2024-05-09T04:58:11.802 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/4 52842 root NOFD /proc/52842/fd (opendir: Permission denied) 2024-05-09T04:58:11.802 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/7 53140 root cwd unknown /proc/53140/cwd (readlink: Permission denied) 2024-05-09T04:58:11.802 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/7 53140 root rtd unknown /proc/53140/root (readlink: Permission denied) 2024-05-09T04:58:11.802 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/7 53140 root txt unknown /proc/53140/exe (readlink: Permission denied) 2024-05-09T04:58:11.802 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/7 53140 root NOFD /proc/53140/fd (opendir: Permission denied) 2024-05-09T04:58:11.802 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 55562 root cwd unknown /proc/55562/cwd (readlink: Permission denied) 2024-05-09T04:58:11.802 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 55562 root rtd unknown /proc/55562/root (readlink: Permission denied) 2024-05-09T04:58:11.802 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 55562 root txt unknown /proc/55562/exe (readlink: Permission denied) 2024-05-09T04:58:11.802 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 55562 root NOFD /proc/55562/fd (opendir: Permission denied) 2024-05-09T04:58:11.802 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 55564 root cwd unknown /proc/55564/cwd (readlink: Permission denied) 2024-05-09T04:58:11.802 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 55564 root rtd unknown /proc/55564/root (readlink: Permission denied) 2024-05-09T04:58:11.803 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 55564 root txt unknown /proc/55564/exe (readlink: Permission denied) 2024-05-09T04:58:11.803 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 55564 root NOFD /proc/55564/fd (opendir: Permission denied) 2024-05-09T04:58:11.803 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 ceph cwd unknown /proc/55566/cwd (readlink: Permission denied) 2024-05-09T04:58:11.803 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 ceph rtd unknown /proc/55566/root (readlink: Permission denied) 2024-05-09T04:58:11.803 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 ceph txt unknown /proc/55566/exe (readlink: Permission denied) 2024-05-09T04:58:11.803 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 ceph NOFD /proc/55566/fd (opendir: Permission denied) 2024-05-09T04:58:11.803 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55567 log ceph cwd unknown /proc/55566/task/55567/cwd (readlink: Permission denied) 2024-05-09T04:58:11.803 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55567 log ceph rtd unknown /proc/55566/task/55567/root (readlink: Permission denied) 2024-05-09T04:58:11.803 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55567 log ceph txt unknown /proc/55566/task/55567/exe (readlink: Permission denied) 2024-05-09T04:58:11.803 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55567 log ceph NOFD /proc/55566/task/55567/fd (opendir: Permission denied) 2024-05-09T04:58:11.803 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55569 msgr-work ceph cwd unknown /proc/55566/task/55569/cwd (readlink: Permission denied) 2024-05-09T04:58:11.803 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55569 msgr-work ceph rtd unknown /proc/55566/task/55569/root (readlink: Permission denied) 2024-05-09T04:58:11.803 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55569 msgr-work ceph txt unknown /proc/55566/task/55569/exe (readlink: Permission denied) 2024-05-09T04:58:11.803 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55569 msgr-work ceph NOFD /proc/55566/task/55569/fd (opendir: Permission denied) 2024-05-09T04:58:11.804 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55570 msgr-work ceph cwd unknown /proc/55566/task/55570/cwd (readlink: Permission denied) 2024-05-09T04:58:11.804 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55570 msgr-work ceph rtd unknown /proc/55566/task/55570/root (readlink: Permission denied) 2024-05-09T04:58:11.804 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55570 msgr-work ceph txt unknown /proc/55566/task/55570/exe (readlink: Permission denied) 2024-05-09T04:58:11.804 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55570 msgr-work ceph NOFD /proc/55566/task/55570/fd (opendir: Permission denied) 2024-05-09T04:58:11.804 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55571 msgr-work ceph cwd unknown /proc/55566/task/55571/cwd (readlink: Permission denied) 2024-05-09T04:58:11.804 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55571 msgr-work ceph rtd unknown /proc/55566/task/55571/root (readlink: Permission denied) 2024-05-09T04:58:11.804 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55571 msgr-work ceph txt unknown /proc/55566/task/55571/exe (readlink: Permission denied) 2024-05-09T04:58:11.804 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55571 msgr-work ceph NOFD /proc/55566/task/55571/fd (opendir: Permission denied) 2024-05-09T04:58:11.804 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55576 service ceph cwd unknown /proc/55566/task/55576/cwd (readlink: Permission denied) 2024-05-09T04:58:11.804 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55576 service ceph rtd unknown /proc/55566/task/55576/root (readlink: Permission denied) 2024-05-09T04:58:11.804 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55576 service ceph txt unknown /proc/55566/task/55576/exe (readlink: Permission denied) 2024-05-09T04:58:11.804 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55576 service ceph NOFD /proc/55566/task/55576/fd (opendir: Permission denied) 2024-05-09T04:58:11.804 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55577 admin_soc ceph cwd unknown /proc/55566/task/55577/cwd (readlink: Permission denied) 2024-05-09T04:58:11.805 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55577 admin_soc ceph rtd unknown /proc/55566/task/55577/root (readlink: Permission denied) 2024-05-09T04:58:11.805 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55577 admin_soc ceph txt unknown /proc/55566/task/55577/exe (readlink: Permission denied) 2024-05-09T04:58:11.805 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55577 admin_soc ceph NOFD /proc/55566/task/55577/fd (opendir: Permission denied) 2024-05-09T04:58:11.805 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55719 signal_ha ceph cwd unknown /proc/55566/task/55719/cwd (readlink: Permission denied) 2024-05-09T04:58:11.805 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55719 signal_ha ceph rtd unknown /proc/55566/task/55719/root (readlink: Permission denied) 2024-05-09T04:58:11.805 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55719 signal_ha ceph txt unknown /proc/55566/task/55719/exe (readlink: Permission denied) 2024-05-09T04:58:11.805 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55719 signal_ha ceph NOFD /proc/55566/task/55719/fd (opendir: Permission denied) 2024-05-09T04:58:11.805 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55720 io_contex ceph cwd unknown /proc/55566/task/55720/cwd (readlink: Permission denied) 2024-05-09T04:58:11.805 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55720 io_contex ceph rtd unknown /proc/55566/task/55720/root (readlink: Permission denied) 2024-05-09T04:58:11.805 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55720 io_contex ceph txt unknown /proc/55566/task/55720/exe (readlink: Permission denied) 2024-05-09T04:58:11.805 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55720 io_contex ceph NOFD /proc/55566/task/55720/fd (opendir: Permission denied) 2024-05-09T04:58:11.805 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55721 io_contex ceph cwd unknown /proc/55566/task/55721/cwd (readlink: Permission denied) 2024-05-09T04:58:11.805 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55721 io_contex ceph rtd unknown /proc/55566/task/55721/root (readlink: Permission denied) 2024-05-09T04:58:11.805 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55721 io_contex ceph txt unknown /proc/55566/task/55721/exe (readlink: Permission denied) 2024-05-09T04:58:11.806 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55721 io_contex ceph NOFD /proc/55566/task/55721/fd (opendir: Permission denied) 2024-05-09T04:58:11.806 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55768 OpHistory ceph cwd unknown /proc/55566/task/55768/cwd (readlink: Permission denied) 2024-05-09T04:58:11.806 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55768 OpHistory ceph rtd unknown /proc/55566/task/55768/root (readlink: Permission denied) 2024-05-09T04:58:11.806 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55768 OpHistory ceph txt unknown /proc/55566/task/55768/exe (readlink: Permission denied) 2024-05-09T04:58:11.806 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55768 OpHistory ceph NOFD /proc/55566/task/55768/fd (opendir: Permission denied) 2024-05-09T04:58:11.806 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55769 ceph_time ceph cwd unknown /proc/55566/task/55769/cwd (readlink: Permission denied) 2024-05-09T04:58:11.806 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55769 ceph_time ceph rtd unknown /proc/55566/task/55769/root (readlink: Permission denied) 2024-05-09T04:58:11.806 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55769 ceph_time ceph txt unknown /proc/55566/task/55769/exe (readlink: Permission denied) 2024-05-09T04:58:11.806 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55769 ceph_time ceph NOFD /proc/55566/task/55769/fd (opendir: Permission denied) 2024-05-09T04:58:11.806 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55953 ceph-osd ceph cwd unknown /proc/55566/task/55953/cwd (readlink: Permission denied) 2024-05-09T04:58:11.806 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55953 ceph-osd ceph rtd unknown /proc/55566/task/55953/root (readlink: Permission denied) 2024-05-09T04:58:11.806 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55953 ceph-osd ceph txt unknown /proc/55566/task/55953/exe (readlink: Permission denied) 2024-05-09T04:58:11.806 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55953 ceph-osd ceph NOFD /proc/55566/task/55953/fd (opendir: Permission denied) 2024-05-09T04:58:11.806 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55954 safe_time ceph cwd unknown /proc/55566/task/55954/cwd (readlink: Permission denied) 2024-05-09T04:58:11.807 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55954 safe_time ceph rtd unknown /proc/55566/task/55954/root (readlink: Permission denied) 2024-05-09T04:58:11.807 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55954 safe_time ceph txt unknown /proc/55566/task/55954/exe (readlink: Permission denied) 2024-05-09T04:58:11.807 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55954 safe_time ceph NOFD /proc/55566/task/55954/fd (opendir: Permission denied) 2024-05-09T04:58:11.807 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55955 safe_time ceph cwd unknown /proc/55566/task/55955/cwd (readlink: Permission denied) 2024-05-09T04:58:11.807 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55955 safe_time ceph rtd unknown /proc/55566/task/55955/root (readlink: Permission denied) 2024-05-09T04:58:11.807 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55955 safe_time ceph txt unknown /proc/55566/task/55955/exe (readlink: Permission denied) 2024-05-09T04:58:11.807 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55955 safe_time ceph NOFD /proc/55566/task/55955/fd (opendir: Permission denied) 2024-05-09T04:58:11.807 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55956 safe_time ceph cwd unknown /proc/55566/task/55956/cwd (readlink: Permission denied) 2024-05-09T04:58:11.807 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55956 safe_time ceph rtd unknown /proc/55566/task/55956/root (readlink: Permission denied) 2024-05-09T04:58:11.807 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55956 safe_time ceph txt unknown /proc/55566/task/55956/exe (readlink: Permission denied) 2024-05-09T04:58:11.807 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55956 safe_time ceph NOFD /proc/55566/task/55956/fd (opendir: Permission denied) 2024-05-09T04:58:11.807 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55957 safe_time ceph cwd unknown /proc/55566/task/55957/cwd (readlink: Permission denied) 2024-05-09T04:58:11.807 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55957 safe_time ceph rtd unknown /proc/55566/task/55957/root (readlink: Permission denied) 2024-05-09T04:58:11.807 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55957 safe_time ceph txt unknown /proc/55566/task/55957/exe (readlink: Permission denied) 2024-05-09T04:58:11.808 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55957 safe_time ceph NOFD /proc/55566/task/55957/fd (opendir: Permission denied) 2024-05-09T04:58:11.808 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55958 fn_anonym ceph cwd unknown /proc/55566/task/55958/cwd (readlink: Permission denied) 2024-05-09T04:58:11.808 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55958 fn_anonym ceph rtd unknown /proc/55566/task/55958/root (readlink: Permission denied) 2024-05-09T04:58:11.808 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55958 fn_anonym ceph txt unknown /proc/55566/task/55958/exe (readlink: Permission denied) 2024-05-09T04:58:11.808 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55958 fn_anonym ceph NOFD /proc/55566/task/55958/fd (opendir: Permission denied) 2024-05-09T04:58:11.808 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55963 rocksdb:l ceph cwd unknown /proc/55566/task/55963/cwd (readlink: Permission denied) 2024-05-09T04:58:11.808 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55963 rocksdb:l ceph rtd unknown /proc/55566/task/55963/root (readlink: Permission denied) 2024-05-09T04:58:11.808 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55963 rocksdb:l ceph txt unknown /proc/55566/task/55963/exe (readlink: Permission denied) 2024-05-09T04:58:11.808 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55963 rocksdb:l ceph NOFD /proc/55566/task/55963/fd (opendir: Permission denied) 2024-05-09T04:58:11.808 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55964 rocksdb:l ceph cwd unknown /proc/55566/task/55964/cwd (readlink: Permission denied) 2024-05-09T04:58:11.808 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55964 rocksdb:l ceph rtd unknown /proc/55566/task/55964/root (readlink: Permission denied) 2024-05-09T04:58:11.808 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55964 rocksdb:l ceph txt unknown /proc/55566/task/55964/exe (readlink: Permission denied) 2024-05-09T04:58:11.808 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55964 rocksdb:l ceph NOFD /proc/55566/task/55964/fd (opendir: Permission denied) 2024-05-09T04:58:11.808 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55965 rocksdb:l ceph cwd unknown /proc/55566/task/55965/cwd (readlink: Permission denied) 2024-05-09T04:58:11.808 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55965 rocksdb:l ceph rtd unknown /proc/55566/task/55965/root (readlink: Permission denied) 2024-05-09T04:58:11.809 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55965 rocksdb:l ceph txt unknown /proc/55566/task/55965/exe (readlink: Permission denied) 2024-05-09T04:58:11.809 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55965 rocksdb:l ceph NOFD /proc/55566/task/55965/fd (opendir: Permission denied) 2024-05-09T04:58:11.809 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55967 rocksdb:h ceph cwd unknown /proc/55566/task/55967/cwd (readlink: Permission denied) 2024-05-09T04:58:11.809 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55967 rocksdb:h ceph rtd unknown /proc/55566/task/55967/root (readlink: Permission denied) 2024-05-09T04:58:11.809 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55967 rocksdb:h ceph txt unknown /proc/55566/task/55967/exe (readlink: Permission denied) 2024-05-09T04:58:11.809 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 55967 rocksdb:h ceph NOFD /proc/55566/task/55967/fd (opendir: Permission denied) 2024-05-09T04:58:11.809 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56472 bstore_ai ceph cwd unknown /proc/55566/task/56472/cwd (readlink: Permission denied) 2024-05-09T04:58:11.809 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56472 bstore_ai ceph rtd unknown /proc/55566/task/56472/root (readlink: Permission denied) 2024-05-09T04:58:11.809 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56472 bstore_ai ceph txt unknown /proc/55566/task/56472/exe (readlink: Permission denied) 2024-05-09T04:58:11.809 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56472 bstore_ai ceph NOFD /proc/55566/task/56472/fd (opendir: Permission denied) 2024-05-09T04:58:11.809 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56685 bstore_ai ceph cwd unknown /proc/55566/task/56685/cwd (readlink: Permission denied) 2024-05-09T04:58:11.809 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56685 bstore_ai ceph rtd unknown /proc/55566/task/56685/root (readlink: Permission denied) 2024-05-09T04:58:11.809 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56685 bstore_ai ceph txt unknown /proc/55566/task/56685/exe (readlink: Permission denied) 2024-05-09T04:58:11.809 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56685 bstore_ai ceph NOFD /proc/55566/task/56685/fd (opendir: Permission denied) 2024-05-09T04:58:11.809 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56869 ceph-osd ceph cwd unknown /proc/55566/task/56869/cwd (readlink: Permission denied) 2024-05-09T04:58:11.810 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56869 ceph-osd ceph rtd unknown /proc/55566/task/56869/root (readlink: Permission denied) 2024-05-09T04:58:11.810 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56869 ceph-osd ceph txt unknown /proc/55566/task/56869/exe (readlink: Permission denied) 2024-05-09T04:58:11.810 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56869 ceph-osd ceph NOFD /proc/55566/task/56869/fd (opendir: Permission denied) 2024-05-09T04:58:11.810 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56870 cfin ceph cwd unknown /proc/55566/task/56870/cwd (readlink: Permission denied) 2024-05-09T04:58:11.810 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56870 cfin ceph rtd unknown /proc/55566/task/56870/root (readlink: Permission denied) 2024-05-09T04:58:11.810 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56870 cfin ceph txt unknown /proc/55566/task/56870/exe (readlink: Permission denied) 2024-05-09T04:58:11.810 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56870 cfin ceph NOFD /proc/55566/task/56870/fd (opendir: Permission denied) 2024-05-09T04:58:11.810 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56871 bstore_kv ceph cwd unknown /proc/55566/task/56871/cwd (readlink: Permission denied) 2024-05-09T04:58:11.810 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56871 bstore_kv ceph rtd unknown /proc/55566/task/56871/root (readlink: Permission denied) 2024-05-09T04:58:11.810 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56871 bstore_kv ceph txt unknown /proc/55566/task/56871/exe (readlink: Permission denied) 2024-05-09T04:58:11.810 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56871 bstore_kv ceph NOFD /proc/55566/task/56871/fd (opendir: Permission denied) 2024-05-09T04:58:11.810 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56872 bstore_kv ceph cwd unknown /proc/55566/task/56872/cwd (readlink: Permission denied) 2024-05-09T04:58:11.810 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56872 bstore_kv ceph rtd unknown /proc/55566/task/56872/root (readlink: Permission denied) 2024-05-09T04:58:11.810 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56872 bstore_kv ceph txt unknown /proc/55566/task/56872/exe (readlink: Permission denied) 2024-05-09T04:58:11.810 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56872 bstore_kv ceph NOFD /proc/55566/task/56872/fd (opendir: Permission denied) 2024-05-09T04:58:11.811 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56873 bstore_me ceph cwd unknown /proc/55566/task/56873/cwd (readlink: Permission denied) 2024-05-09T04:58:11.811 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56873 bstore_me ceph rtd unknown /proc/55566/task/56873/root (readlink: Permission denied) 2024-05-09T04:58:11.811 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56873 bstore_me ceph txt unknown /proc/55566/task/56873/exe (readlink: Permission denied) 2024-05-09T04:58:11.811 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56873 bstore_me ceph NOFD /proc/55566/task/56873/fd (opendir: Permission denied) 2024-05-09T04:58:11.811 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56874 ms_dispat ceph cwd unknown /proc/55566/task/56874/cwd (readlink: Permission denied) 2024-05-09T04:58:11.811 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56874 ms_dispat ceph rtd unknown /proc/55566/task/56874/root (readlink: Permission denied) 2024-05-09T04:58:11.811 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56874 ms_dispat ceph txt unknown /proc/55566/task/56874/exe (readlink: Permission denied) 2024-05-09T04:58:11.811 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56874 ms_dispat ceph NOFD /proc/55566/task/56874/fd (opendir: Permission denied) 2024-05-09T04:58:11.811 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56875 ms_local ceph cwd unknown /proc/55566/task/56875/cwd (readlink: Permission denied) 2024-05-09T04:58:11.811 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56875 ms_local ceph rtd unknown /proc/55566/task/56875/root (readlink: Permission denied) 2024-05-09T04:58:11.811 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56875 ms_local ceph txt unknown /proc/55566/task/56875/exe (readlink: Permission denied) 2024-05-09T04:58:11.811 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56875 ms_local ceph NOFD /proc/55566/task/56875/fd (opendir: Permission denied) 2024-05-09T04:58:11.811 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56876 safe_time ceph cwd unknown /proc/55566/task/56876/cwd (readlink: Permission denied) 2024-05-09T04:58:11.811 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56876 safe_time ceph rtd unknown /proc/55566/task/56876/root (readlink: Permission denied) 2024-05-09T04:58:11.811 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56876 safe_time ceph txt unknown /proc/55566/task/56876/exe (readlink: Permission denied) 2024-05-09T04:58:11.812 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56876 safe_time ceph NOFD /proc/55566/task/56876/fd (opendir: Permission denied) 2024-05-09T04:58:11.812 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56877 safe_time ceph cwd unknown /proc/55566/task/56877/cwd (readlink: Permission denied) 2024-05-09T04:58:11.812 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56877 safe_time ceph rtd unknown /proc/55566/task/56877/root (readlink: Permission denied) 2024-05-09T04:58:11.812 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56877 safe_time ceph txt unknown /proc/55566/task/56877/exe (readlink: Permission denied) 2024-05-09T04:58:11.812 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56877 safe_time ceph NOFD /proc/55566/task/56877/fd (opendir: Permission denied) 2024-05-09T04:58:11.812 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56878 ms_dispat ceph cwd unknown /proc/55566/task/56878/cwd (readlink: Permission denied) 2024-05-09T04:58:11.812 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56878 ms_dispat ceph rtd unknown /proc/55566/task/56878/root (readlink: Permission denied) 2024-05-09T04:58:11.812 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56878 ms_dispat ceph txt unknown /proc/55566/task/56878/exe (readlink: Permission denied) 2024-05-09T04:58:11.812 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56878 ms_dispat ceph NOFD /proc/55566/task/56878/fd (opendir: Permission denied) 2024-05-09T04:58:11.812 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56879 ms_local ceph cwd unknown /proc/55566/task/56879/cwd (readlink: Permission denied) 2024-05-09T04:58:11.812 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56879 ms_local ceph rtd unknown /proc/55566/task/56879/root (readlink: Permission denied) 2024-05-09T04:58:11.812 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56879 ms_local ceph txt unknown /proc/55566/task/56879/exe (readlink: Permission denied) 2024-05-09T04:58:11.812 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56879 ms_local ceph NOFD /proc/55566/task/56879/fd (opendir: Permission denied) 2024-05-09T04:58:11.812 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56880 ms_dispat ceph cwd unknown /proc/55566/task/56880/cwd (readlink: Permission denied) 2024-05-09T04:58:11.812 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56880 ms_dispat ceph rtd unknown /proc/55566/task/56880/root (readlink: Permission denied) 2024-05-09T04:58:11.812 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56880 ms_dispat ceph txt unknown /proc/55566/task/56880/exe (readlink: Permission denied) 2024-05-09T04:58:11.813 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56880 ms_dispat ceph NOFD /proc/55566/task/56880/fd (opendir: Permission denied) 2024-05-09T04:58:11.813 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56881 ms_local ceph cwd unknown /proc/55566/task/56881/cwd (readlink: Permission denied) 2024-05-09T04:58:11.813 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56881 ms_local ceph rtd unknown /proc/55566/task/56881/root (readlink: Permission denied) 2024-05-09T04:58:11.813 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56881 ms_local ceph txt unknown /proc/55566/task/56881/exe (readlink: Permission denied) 2024-05-09T04:58:11.813 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56881 ms_local ceph NOFD /proc/55566/task/56881/fd (opendir: Permission denied) 2024-05-09T04:58:11.813 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56882 ms_dispat ceph cwd unknown /proc/55566/task/56882/cwd (readlink: Permission denied) 2024-05-09T04:58:11.813 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56882 ms_dispat ceph rtd unknown /proc/55566/task/56882/root (readlink: Permission denied) 2024-05-09T04:58:11.813 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56882 ms_dispat ceph txt unknown /proc/55566/task/56882/exe (readlink: Permission denied) 2024-05-09T04:58:11.813 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56882 ms_dispat ceph NOFD /proc/55566/task/56882/fd (opendir: Permission denied) 2024-05-09T04:58:11.813 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56883 ms_local ceph cwd unknown /proc/55566/task/56883/cwd (readlink: Permission denied) 2024-05-09T04:58:11.813 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56883 ms_local ceph rtd unknown /proc/55566/task/56883/root (readlink: Permission denied) 2024-05-09T04:58:11.813 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56883 ms_local ceph txt unknown /proc/55566/task/56883/exe (readlink: Permission denied) 2024-05-09T04:58:11.813 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56883 ms_local ceph NOFD /proc/55566/task/56883/fd (opendir: Permission denied) 2024-05-09T04:58:11.813 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56884 ms_dispat ceph cwd unknown /proc/55566/task/56884/cwd (readlink: Permission denied) 2024-05-09T04:58:11.813 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56884 ms_dispat ceph rtd unknown /proc/55566/task/56884/root (readlink: Permission denied) 2024-05-09T04:58:11.814 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56884 ms_dispat ceph txt unknown /proc/55566/task/56884/exe (readlink: Permission denied) 2024-05-09T04:58:11.814 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56884 ms_dispat ceph NOFD /proc/55566/task/56884/fd (opendir: Permission denied) 2024-05-09T04:58:11.814 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56885 ms_local ceph cwd unknown /proc/55566/task/56885/cwd (readlink: Permission denied) 2024-05-09T04:58:11.814 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56885 ms_local ceph rtd unknown /proc/55566/task/56885/root (readlink: Permission denied) 2024-05-09T04:58:11.814 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56885 ms_local ceph txt unknown /proc/55566/task/56885/exe (readlink: Permission denied) 2024-05-09T04:58:11.814 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56885 ms_local ceph NOFD /proc/55566/task/56885/fd (opendir: Permission denied) 2024-05-09T04:58:11.814 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56886 ms_dispat ceph cwd unknown /proc/55566/task/56886/cwd (readlink: Permission denied) 2024-05-09T04:58:11.814 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56886 ms_dispat ceph rtd unknown /proc/55566/task/56886/root (readlink: Permission denied) 2024-05-09T04:58:11.814 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56886 ms_dispat ceph txt unknown /proc/55566/task/56886/exe (readlink: Permission denied) 2024-05-09T04:58:11.814 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56886 ms_dispat ceph NOFD /proc/55566/task/56886/fd (opendir: Permission denied) 2024-05-09T04:58:11.814 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56887 ms_local ceph cwd unknown /proc/55566/task/56887/cwd (readlink: Permission denied) 2024-05-09T04:58:11.815 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56887 ms_local ceph rtd unknown /proc/55566/task/56887/root (readlink: Permission denied) 2024-05-09T04:58:11.815 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56887 ms_local ceph txt unknown /proc/55566/task/56887/exe (readlink: Permission denied) 2024-05-09T04:58:11.815 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56887 ms_local ceph NOFD /proc/55566/task/56887/fd (opendir: Permission denied) 2024-05-09T04:58:11.815 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56888 ms_dispat ceph cwd unknown /proc/55566/task/56888/cwd (readlink: Permission denied) 2024-05-09T04:58:11.815 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56888 ms_dispat ceph rtd unknown /proc/55566/task/56888/root (readlink: Permission denied) 2024-05-09T04:58:11.815 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56888 ms_dispat ceph txt unknown /proc/55566/task/56888/exe (readlink: Permission denied) 2024-05-09T04:58:11.815 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56888 ms_dispat ceph NOFD /proc/55566/task/56888/fd (opendir: Permission denied) 2024-05-09T04:58:11.815 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56889 ms_local ceph cwd unknown /proc/55566/task/56889/cwd (readlink: Permission denied) 2024-05-09T04:58:11.815 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56889 ms_local ceph rtd unknown /proc/55566/task/56889/root (readlink: Permission denied) 2024-05-09T04:58:11.815 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56889 ms_local ceph txt unknown /proc/55566/task/56889/exe (readlink: Permission denied) 2024-05-09T04:58:11.815 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56889 ms_local ceph NOFD /proc/55566/task/56889/fd (opendir: Permission denied) 2024-05-09T04:58:11.815 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56890 fn_anonym ceph cwd unknown /proc/55566/task/56890/cwd (readlink: Permission denied) 2024-05-09T04:58:11.815 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56890 fn_anonym ceph rtd unknown /proc/55566/task/56890/root (readlink: Permission denied) 2024-05-09T04:58:11.815 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56890 fn_anonym ceph txt unknown /proc/55566/task/56890/exe (readlink: Permission denied) 2024-05-09T04:58:11.816 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56890 fn_anonym ceph NOFD /proc/55566/task/56890/fd (opendir: Permission denied) 2024-05-09T04:58:11.816 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56891 finisher ceph cwd unknown /proc/55566/task/56891/cwd (readlink: Permission denied) 2024-05-09T04:58:11.816 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56891 finisher ceph rtd unknown /proc/55566/task/56891/root (readlink: Permission denied) 2024-05-09T04:58:11.816 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56891 finisher ceph txt unknown /proc/55566/task/56891/exe (readlink: Permission denied) 2024-05-09T04:58:11.816 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56891 finisher ceph NOFD /proc/55566/task/56891/fd (opendir: Permission denied) 2024-05-09T04:58:11.816 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56892 safe_time ceph cwd unknown /proc/55566/task/56892/cwd (readlink: Permission denied) 2024-05-09T04:58:11.816 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56892 safe_time ceph rtd unknown /proc/55566/task/56892/root (readlink: Permission denied) 2024-05-09T04:58:11.816 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56892 safe_time ceph txt unknown /proc/55566/task/56892/exe (readlink: Permission denied) 2024-05-09T04:58:11.816 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56892 safe_time ceph NOFD /proc/55566/task/56892/fd (opendir: Permission denied) 2024-05-09T04:58:11.816 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56893 safe_time ceph cwd unknown /proc/55566/task/56893/cwd (readlink: Permission denied) 2024-05-09T04:58:11.816 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56893 safe_time ceph rtd unknown /proc/55566/task/56893/root (readlink: Permission denied) 2024-05-09T04:58:11.817 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56893 safe_time ceph txt unknown /proc/55566/task/56893/exe (readlink: Permission denied) 2024-05-09T04:58:11.817 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56893 safe_time ceph NOFD /proc/55566/task/56893/fd (opendir: Permission denied) 2024-05-09T04:58:11.817 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56894 ceph-osd ceph cwd unknown /proc/55566/task/56894/cwd (readlink: Permission denied) 2024-05-09T04:58:11.817 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56894 ceph-osd ceph rtd unknown /proc/55566/task/56894/root (readlink: Permission denied) 2024-05-09T04:58:11.817 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56894 ceph-osd ceph txt unknown /proc/55566/task/56894/exe (readlink: Permission denied) 2024-05-09T04:58:11.817 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56894 ceph-osd ceph NOFD /proc/55566/task/56894/fd (opendir: Permission denied) 2024-05-09T04:58:11.817 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56895 osd_srv_a ceph cwd unknown /proc/55566/task/56895/cwd (readlink: Permission denied) 2024-05-09T04:58:11.817 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56895 osd_srv_a ceph rtd unknown /proc/55566/task/56895/root (readlink: Permission denied) 2024-05-09T04:58:11.817 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56895 osd_srv_a ceph txt unknown /proc/55566/task/56895/exe (readlink: Permission denied) 2024-05-09T04:58:11.817 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56895 osd_srv_a ceph NOFD /proc/55566/task/56895/fd (opendir: Permission denied) 2024-05-09T04:58:11.817 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56896 tp_osd_tp ceph cwd unknown /proc/55566/task/56896/cwd (readlink: Permission denied) 2024-05-09T04:58:11.817 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56896 tp_osd_tp ceph rtd unknown /proc/55566/task/56896/root (readlink: Permission denied) 2024-05-09T04:58:11.817 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56896 tp_osd_tp ceph txt unknown /proc/55566/task/56896/exe (readlink: Permission denied) 2024-05-09T04:58:11.817 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56896 tp_osd_tp ceph NOFD /proc/55566/task/56896/fd (opendir: Permission denied) 2024-05-09T04:58:11.817 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56897 tp_osd_tp ceph cwd unknown /proc/55566/task/56897/cwd (readlink: Permission denied) 2024-05-09T04:58:11.818 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56897 tp_osd_tp ceph rtd unknown /proc/55566/task/56897/root (readlink: Permission denied) 2024-05-09T04:58:11.818 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56897 tp_osd_tp ceph txt unknown /proc/55566/task/56897/exe (readlink: Permission denied) 2024-05-09T04:58:11.818 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56897 tp_osd_tp ceph NOFD /proc/55566/task/56897/fd (opendir: Permission denied) 2024-05-09T04:58:11.818 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56898 tp_osd_tp ceph cwd unknown /proc/55566/task/56898/cwd (readlink: Permission denied) 2024-05-09T04:58:11.818 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56898 tp_osd_tp ceph rtd unknown /proc/55566/task/56898/root (readlink: Permission denied) 2024-05-09T04:58:11.818 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56898 tp_osd_tp ceph txt unknown /proc/55566/task/56898/exe (readlink: Permission denied) 2024-05-09T04:58:11.818 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56898 tp_osd_tp ceph NOFD /proc/55566/task/56898/fd (opendir: Permission denied) 2024-05-09T04:58:11.818 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56899 tp_osd_tp ceph cwd unknown /proc/55566/task/56899/cwd (readlink: Permission denied) 2024-05-09T04:58:11.818 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56899 tp_osd_tp ceph rtd unknown /proc/55566/task/56899/root (readlink: Permission denied) 2024-05-09T04:58:11.818 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56899 tp_osd_tp ceph txt unknown /proc/55566/task/56899/exe (readlink: Permission denied) 2024-05-09T04:58:11.819 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56899 tp_osd_tp ceph NOFD /proc/55566/task/56899/fd (opendir: Permission denied) 2024-05-09T04:58:11.819 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56900 tp_osd_tp ceph cwd unknown /proc/55566/task/56900/cwd (readlink: Permission denied) 2024-05-09T04:58:11.819 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56900 tp_osd_tp ceph rtd unknown /proc/55566/task/56900/root (readlink: Permission denied) 2024-05-09T04:58:11.819 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56900 tp_osd_tp ceph txt unknown /proc/55566/task/56900/exe (readlink: Permission denied) 2024-05-09T04:58:11.819 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56900 tp_osd_tp ceph NOFD /proc/55566/task/56900/fd (opendir: Permission denied) 2024-05-09T04:58:11.819 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56901 tp_osd_tp ceph cwd unknown /proc/55566/task/56901/cwd (readlink: Permission denied) 2024-05-09T04:58:11.819 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56901 tp_osd_tp ceph rtd unknown /proc/55566/task/56901/root (readlink: Permission denied) 2024-05-09T04:58:11.819 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56901 tp_osd_tp ceph txt unknown /proc/55566/task/56901/exe (readlink: Permission denied) 2024-05-09T04:58:11.819 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56901 tp_osd_tp ceph NOFD /proc/55566/task/56901/fd (opendir: Permission denied) 2024-05-09T04:58:11.819 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56902 tp_osd_tp ceph cwd unknown /proc/55566/task/56902/cwd (readlink: Permission denied) 2024-05-09T04:58:11.819 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56902 tp_osd_tp ceph rtd unknown /proc/55566/task/56902/root (readlink: Permission denied) 2024-05-09T04:58:11.820 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56902 tp_osd_tp ceph txt unknown /proc/55566/task/56902/exe (readlink: Permission denied) 2024-05-09T04:58:11.820 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56902 tp_osd_tp ceph NOFD /proc/55566/task/56902/fd (opendir: Permission denied) 2024-05-09T04:58:11.820 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56903 tp_osd_tp ceph cwd unknown /proc/55566/task/56903/cwd (readlink: Permission denied) 2024-05-09T04:58:11.820 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56903 tp_osd_tp ceph rtd unknown /proc/55566/task/56903/root (readlink: Permission denied) 2024-05-09T04:58:11.820 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56903 tp_osd_tp ceph txt unknown /proc/55566/task/56903/exe (readlink: Permission denied) 2024-05-09T04:58:11.820 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56903 tp_osd_tp ceph NOFD /proc/55566/task/56903/fd (opendir: Permission denied) 2024-05-09T04:58:11.820 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56904 tp_osd_tp ceph cwd unknown /proc/55566/task/56904/cwd (readlink: Permission denied) 2024-05-09T04:58:11.820 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56904 tp_osd_tp ceph rtd unknown /proc/55566/task/56904/root (readlink: Permission denied) 2024-05-09T04:58:11.820 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56904 tp_osd_tp ceph txt unknown /proc/55566/task/56904/exe (readlink: Permission denied) 2024-05-09T04:58:11.821 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56904 tp_osd_tp ceph NOFD /proc/55566/task/56904/fd (opendir: Permission denied) 2024-05-09T04:58:11.821 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56905 tp_osd_tp ceph cwd unknown /proc/55566/task/56905/cwd (readlink: Permission denied) 2024-05-09T04:58:11.821 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56905 tp_osd_tp ceph rtd unknown /proc/55566/task/56905/root (readlink: Permission denied) 2024-05-09T04:58:11.821 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56905 tp_osd_tp ceph txt unknown /proc/55566/task/56905/exe (readlink: Permission denied) 2024-05-09T04:58:11.821 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56905 tp_osd_tp ceph NOFD /proc/55566/task/56905/fd (opendir: Permission denied) 2024-05-09T04:58:11.821 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56906 tp_osd_tp ceph cwd unknown /proc/55566/task/56906/cwd (readlink: Permission denied) 2024-05-09T04:58:11.821 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56906 tp_osd_tp ceph rtd unknown /proc/55566/task/56906/root (readlink: Permission denied) 2024-05-09T04:58:11.821 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56906 tp_osd_tp ceph txt unknown /proc/55566/task/56906/exe (readlink: Permission denied) 2024-05-09T04:58:11.821 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56906 tp_osd_tp ceph NOFD /proc/55566/task/56906/fd (opendir: Permission denied) 2024-05-09T04:58:11.821 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56907 tp_osd_tp ceph cwd unknown /proc/55566/task/56907/cwd (readlink: Permission denied) 2024-05-09T04:58:11.821 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56907 tp_osd_tp ceph rtd unknown /proc/55566/task/56907/root (readlink: Permission denied) 2024-05-09T04:58:11.821 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56907 tp_osd_tp ceph txt unknown /proc/55566/task/56907/exe (readlink: Permission denied) 2024-05-09T04:58:11.822 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56907 tp_osd_tp ceph NOFD /proc/55566/task/56907/fd (opendir: Permission denied) 2024-05-09T04:58:11.822 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56908 tp_osd_tp ceph cwd unknown /proc/55566/task/56908/cwd (readlink: Permission denied) 2024-05-09T04:58:11.822 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56908 tp_osd_tp ceph rtd unknown /proc/55566/task/56908/root (readlink: Permission denied) 2024-05-09T04:58:11.822 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56908 tp_osd_tp ceph txt unknown /proc/55566/task/56908/exe (readlink: Permission denied) 2024-05-09T04:58:11.822 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56908 tp_osd_tp ceph NOFD /proc/55566/task/56908/fd (opendir: Permission denied) 2024-05-09T04:58:11.822 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56909 tp_osd_tp ceph cwd unknown /proc/55566/task/56909/cwd (readlink: Permission denied) 2024-05-09T04:58:11.822 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56909 tp_osd_tp ceph rtd unknown /proc/55566/task/56909/root (readlink: Permission denied) 2024-05-09T04:58:11.822 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56909 tp_osd_tp ceph txt unknown /proc/55566/task/56909/exe (readlink: Permission denied) 2024-05-09T04:58:11.822 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56909 tp_osd_tp ceph NOFD /proc/55566/task/56909/fd (opendir: Permission denied) 2024-05-09T04:58:11.822 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56910 tp_osd_tp ceph cwd unknown /proc/55566/task/56910/cwd (readlink: Permission denied) 2024-05-09T04:58:11.823 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56910 tp_osd_tp ceph rtd unknown /proc/55566/task/56910/root (readlink: Permission denied) 2024-05-09T04:58:11.823 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56910 tp_osd_tp ceph txt unknown /proc/55566/task/56910/exe (readlink: Permission denied) 2024-05-09T04:58:11.823 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56910 tp_osd_tp ceph NOFD /proc/55566/task/56910/fd (opendir: Permission denied) 2024-05-09T04:58:11.823 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56911 tp_osd_tp ceph cwd unknown /proc/55566/task/56911/cwd (readlink: Permission denied) 2024-05-09T04:58:11.823 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56911 tp_osd_tp ceph rtd unknown /proc/55566/task/56911/root (readlink: Permission denied) 2024-05-09T04:58:11.823 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56911 tp_osd_tp ceph txt unknown /proc/55566/task/56911/exe (readlink: Permission denied) 2024-05-09T04:58:11.823 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56911 tp_osd_tp ceph NOFD /proc/55566/task/56911/fd (opendir: Permission denied) 2024-05-09T04:58:11.823 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56912 osd_srv_h ceph cwd unknown /proc/55566/task/56912/cwd (readlink: Permission denied) 2024-05-09T04:58:11.823 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56912 osd_srv_h ceph rtd unknown /proc/55566/task/56912/root (readlink: Permission denied) 2024-05-09T04:58:11.823 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56912 osd_srv_h ceph txt unknown /proc/55566/task/56912/exe (readlink: Permission denied) 2024-05-09T04:58:11.823 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 55566 56912 osd_srv_h ceph NOFD /proc/55566/task/56912/fd (opendir: Permission denied) 2024-05-09T04:58:11.824 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 57114 root cwd unknown /proc/57114/cwd (readlink: Permission denied) 2024-05-09T04:58:11.824 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 57114 root rtd unknown /proc/57114/root (readlink: Permission denied) 2024-05-09T04:58:11.824 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 57114 root txt unknown /proc/57114/exe (readlink: Permission denied) 2024-05-09T04:58:11.824 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 57114 root NOFD /proc/57114/fd (opendir: Permission denied) 2024-05-09T04:58:11.824 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 57145 root cwd unknown /proc/57145/cwd (readlink: Permission denied) 2024-05-09T04:58:11.824 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 57145 root rtd unknown /proc/57145/root (readlink: Permission denied) 2024-05-09T04:58:11.824 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 57145 root txt unknown /proc/57145/exe (readlink: Permission denied) 2024-05-09T04:58:11.824 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 57145 root NOFD /proc/57145/fd (opendir: Permission denied) 2024-05-09T04:58:11.824 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/1 59514 root cwd unknown /proc/59514/cwd (readlink: Permission denied) 2024-05-09T04:58:11.825 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/1 59514 root rtd unknown /proc/59514/root (readlink: Permission denied) 2024-05-09T04:58:11.825 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/1 59514 root txt unknown /proc/59514/exe (readlink: Permission denied) 2024-05-09T04:58:11.825 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/1 59514 root NOFD /proc/59514/fd (opendir: Permission denied) 2024-05-09T04:58:11.825 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/u 61465 root cwd unknown /proc/61465/cwd (readlink: Permission denied) 2024-05-09T04:58:11.825 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/u 61465 root rtd unknown /proc/61465/root (readlink: Permission denied) 2024-05-09T04:58:11.825 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/u 61465 root txt unknown /proc/61465/exe (readlink: Permission denied) 2024-05-09T04:58:11.825 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/u 61465 root NOFD /proc/61465/fd (opendir: Permission denied) 2024-05-09T04:58:11.825 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/5 62251 root cwd unknown /proc/62251/cwd (readlink: Permission denied) 2024-05-09T04:58:11.825 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/5 62251 root rtd unknown /proc/62251/root (readlink: Permission denied) 2024-05-09T04:58:11.825 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/5 62251 root txt unknown /proc/62251/exe (readlink: Permission denied) 2024-05-09T04:58:11.825 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/5 62251 root NOFD /proc/62251/fd (opendir: Permission denied) 2024-05-09T04:58:11.826 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 62320 root cwd unknown /proc/62320/cwd (readlink: Permission denied) 2024-05-09T04:58:11.826 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 62320 root rtd unknown /proc/62320/root (readlink: Permission denied) 2024-05-09T04:58:11.826 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 62320 root txt unknown /proc/62320/exe (readlink: Permission denied) 2024-05-09T04:58:11.826 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 62320 root NOFD /proc/62320/fd (opendir: Permission denied) 2024-05-09T04:58:11.826 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 62322 root cwd unknown /proc/62322/cwd (readlink: Permission denied) 2024-05-09T04:58:11.826 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 62322 root rtd unknown /proc/62322/root (readlink: Permission denied) 2024-05-09T04:58:11.826 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 62322 root txt unknown /proc/62322/exe (readlink: Permission denied) 2024-05-09T04:58:11.826 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 62322 root NOFD /proc/62322/fd (opendir: Permission denied) 2024-05-09T04:58:11.826 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 ceph cwd unknown /proc/62324/cwd (readlink: Permission denied) 2024-05-09T04:58:11.827 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 ceph rtd unknown /proc/62324/root (readlink: Permission denied) 2024-05-09T04:58:11.827 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 ceph txt unknown /proc/62324/exe (readlink: Permission denied) 2024-05-09T04:58:11.827 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 ceph NOFD /proc/62324/fd (opendir: Permission denied) 2024-05-09T04:58:11.827 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62327 log ceph cwd unknown /proc/62324/task/62327/cwd (readlink: Permission denied) 2024-05-09T04:58:11.827 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62327 log ceph rtd unknown /proc/62324/task/62327/root (readlink: Permission denied) 2024-05-09T04:58:11.827 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62327 log ceph txt unknown /proc/62324/task/62327/exe (readlink: Permission denied) 2024-05-09T04:58:11.827 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62327 log ceph NOFD /proc/62324/task/62327/fd (opendir: Permission denied) 2024-05-09T04:58:11.827 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62329 msgr-work ceph cwd unknown /proc/62324/task/62329/cwd (readlink: Permission denied) 2024-05-09T04:58:11.827 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62329 msgr-work ceph rtd unknown /proc/62324/task/62329/root (readlink: Permission denied) 2024-05-09T04:58:11.827 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62329 msgr-work ceph txt unknown /proc/62324/task/62329/exe (readlink: Permission denied) 2024-05-09T04:58:11.827 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62329 msgr-work ceph NOFD /proc/62324/task/62329/fd (opendir: Permission denied) 2024-05-09T04:58:11.827 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62330 msgr-work ceph cwd unknown /proc/62324/task/62330/cwd (readlink: Permission denied) 2024-05-09T04:58:11.827 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62330 msgr-work ceph rtd unknown /proc/62324/task/62330/root (readlink: Permission denied) 2024-05-09T04:58:11.828 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62330 msgr-work ceph txt unknown /proc/62324/task/62330/exe (readlink: Permission denied) 2024-05-09T04:58:11.828 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62330 msgr-work ceph NOFD /proc/62324/task/62330/fd (opendir: Permission denied) 2024-05-09T04:58:11.828 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62331 msgr-work ceph cwd unknown /proc/62324/task/62331/cwd (readlink: Permission denied) 2024-05-09T04:58:11.828 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62331 msgr-work ceph rtd unknown /proc/62324/task/62331/root (readlink: Permission denied) 2024-05-09T04:58:11.828 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62331 msgr-work ceph txt unknown /proc/62324/task/62331/exe (readlink: Permission denied) 2024-05-09T04:58:11.828 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62331 msgr-work ceph NOFD /proc/62324/task/62331/fd (opendir: Permission denied) 2024-05-09T04:58:11.828 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62335 service ceph cwd unknown /proc/62324/task/62335/cwd (readlink: Permission denied) 2024-05-09T04:58:11.828 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62335 service ceph rtd unknown /proc/62324/task/62335/root (readlink: Permission denied) 2024-05-09T04:58:11.828 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62335 service ceph txt unknown /proc/62324/task/62335/exe (readlink: Permission denied) 2024-05-09T04:58:11.828 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62335 service ceph NOFD /proc/62324/task/62335/fd (opendir: Permission denied) 2024-05-09T04:58:11.828 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62336 admin_soc ceph cwd unknown /proc/62324/task/62336/cwd (readlink: Permission denied) 2024-05-09T04:58:11.828 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62336 admin_soc ceph rtd unknown /proc/62324/task/62336/root (readlink: Permission denied) 2024-05-09T04:58:11.829 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62336 admin_soc ceph txt unknown /proc/62324/task/62336/exe (readlink: Permission denied) 2024-05-09T04:58:11.829 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62336 admin_soc ceph NOFD /proc/62324/task/62336/fd (opendir: Permission denied) 2024-05-09T04:58:11.829 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62479 signal_ha ceph cwd unknown /proc/62324/task/62479/cwd (readlink: Permission denied) 2024-05-09T04:58:11.829 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62479 signal_ha ceph rtd unknown /proc/62324/task/62479/root (readlink: Permission denied) 2024-05-09T04:58:11.829 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62479 signal_ha ceph txt unknown /proc/62324/task/62479/exe (readlink: Permission denied) 2024-05-09T04:58:11.829 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62479 signal_ha ceph NOFD /proc/62324/task/62479/fd (opendir: Permission denied) 2024-05-09T04:58:11.829 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62480 io_contex ceph cwd unknown /proc/62324/task/62480/cwd (readlink: Permission denied) 2024-05-09T04:58:11.829 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62480 io_contex ceph rtd unknown /proc/62324/task/62480/root (readlink: Permission denied) 2024-05-09T04:58:11.829 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62480 io_contex ceph txt unknown /proc/62324/task/62480/exe (readlink: Permission denied) 2024-05-09T04:58:11.829 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62480 io_contex ceph NOFD /proc/62324/task/62480/fd (opendir: Permission denied) 2024-05-09T04:58:11.829 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62481 io_contex ceph cwd unknown /proc/62324/task/62481/cwd (readlink: Permission denied) 2024-05-09T04:58:11.829 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62481 io_contex ceph rtd unknown /proc/62324/task/62481/root (readlink: Permission denied) 2024-05-09T04:58:11.829 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62481 io_contex ceph txt unknown /proc/62324/task/62481/exe (readlink: Permission denied) 2024-05-09T04:58:11.830 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62481 io_contex ceph NOFD /proc/62324/task/62481/fd (opendir: Permission denied) 2024-05-09T04:58:11.830 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62549 OpHistory ceph cwd unknown /proc/62324/task/62549/cwd (readlink: Permission denied) 2024-05-09T04:58:11.830 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62549 OpHistory ceph rtd unknown /proc/62324/task/62549/root (readlink: Permission denied) 2024-05-09T04:58:11.830 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62549 OpHistory ceph txt unknown /proc/62324/task/62549/exe (readlink: Permission denied) 2024-05-09T04:58:11.830 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62549 OpHistory ceph NOFD /proc/62324/task/62549/fd (opendir: Permission denied) 2024-05-09T04:58:11.830 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62550 ceph_time ceph cwd unknown /proc/62324/task/62550/cwd (readlink: Permission denied) 2024-05-09T04:58:11.830 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62550 ceph_time ceph rtd unknown /proc/62324/task/62550/root (readlink: Permission denied) 2024-05-09T04:58:11.830 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62550 ceph_time ceph txt unknown /proc/62324/task/62550/exe (readlink: Permission denied) 2024-05-09T04:58:11.830 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62550 ceph_time ceph NOFD /proc/62324/task/62550/fd (opendir: Permission denied) 2024-05-09T04:58:11.830 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62710 ceph-osd ceph cwd unknown /proc/62324/task/62710/cwd (readlink: Permission denied) 2024-05-09T04:58:11.830 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62710 ceph-osd ceph rtd unknown /proc/62324/task/62710/root (readlink: Permission denied) 2024-05-09T04:58:11.830 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62710 ceph-osd ceph txt unknown /proc/62324/task/62710/exe (readlink: Permission denied) 2024-05-09T04:58:11.830 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62710 ceph-osd ceph NOFD /proc/62324/task/62710/fd (opendir: Permission denied) 2024-05-09T04:58:11.831 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62711 safe_time ceph cwd unknown /proc/62324/task/62711/cwd (readlink: Permission denied) 2024-05-09T04:58:11.831 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62711 safe_time ceph rtd unknown /proc/62324/task/62711/root (readlink: Permission denied) 2024-05-09T04:58:11.831 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62711 safe_time ceph txt unknown /proc/62324/task/62711/exe (readlink: Permission denied) 2024-05-09T04:58:11.831 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62711 safe_time ceph NOFD /proc/62324/task/62711/fd (opendir: Permission denied) 2024-05-09T04:58:11.831 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62712 safe_time ceph cwd unknown /proc/62324/task/62712/cwd (readlink: Permission denied) 2024-05-09T04:58:11.831 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62712 safe_time ceph rtd unknown /proc/62324/task/62712/root (readlink: Permission denied) 2024-05-09T04:58:11.831 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62712 safe_time ceph txt unknown /proc/62324/task/62712/exe (readlink: Permission denied) 2024-05-09T04:58:11.831 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62712 safe_time ceph NOFD /proc/62324/task/62712/fd (opendir: Permission denied) 2024-05-09T04:58:11.831 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62713 safe_time ceph cwd unknown /proc/62324/task/62713/cwd (readlink: Permission denied) 2024-05-09T04:58:11.831 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62713 safe_time ceph rtd unknown /proc/62324/task/62713/root (readlink: Permission denied) 2024-05-09T04:58:11.831 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62713 safe_time ceph txt unknown /proc/62324/task/62713/exe (readlink: Permission denied) 2024-05-09T04:58:11.831 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62713 safe_time ceph NOFD /proc/62324/task/62713/fd (opendir: Permission denied) 2024-05-09T04:58:11.831 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62714 safe_time ceph cwd unknown /proc/62324/task/62714/cwd (readlink: Permission denied) 2024-05-09T04:58:11.832 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62714 safe_time ceph rtd unknown /proc/62324/task/62714/root (readlink: Permission denied) 2024-05-09T04:58:11.832 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62714 safe_time ceph txt unknown /proc/62324/task/62714/exe (readlink: Permission denied) 2024-05-09T04:58:11.832 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62714 safe_time ceph NOFD /proc/62324/task/62714/fd (opendir: Permission denied) 2024-05-09T04:58:11.832 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62716 fn_anonym ceph cwd unknown /proc/62324/task/62716/cwd (readlink: Permission denied) 2024-05-09T04:58:11.832 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62716 fn_anonym ceph rtd unknown /proc/62324/task/62716/root (readlink: Permission denied) 2024-05-09T04:58:11.832 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62716 fn_anonym ceph txt unknown /proc/62324/task/62716/exe (readlink: Permission denied) 2024-05-09T04:58:11.832 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62716 fn_anonym ceph NOFD /proc/62324/task/62716/fd (opendir: Permission denied) 2024-05-09T04:58:11.832 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62721 rocksdb:l ceph cwd unknown /proc/62324/task/62721/cwd (readlink: Permission denied) 2024-05-09T04:58:11.832 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62721 rocksdb:l ceph rtd unknown /proc/62324/task/62721/root (readlink: Permission denied) 2024-05-09T04:58:11.832 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62721 rocksdb:l ceph txt unknown /proc/62324/task/62721/exe (readlink: Permission denied) 2024-05-09T04:58:11.832 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62721 rocksdb:l ceph NOFD /proc/62324/task/62721/fd (opendir: Permission denied) 2024-05-09T04:58:11.832 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62722 rocksdb:l ceph cwd unknown /proc/62324/task/62722/cwd (readlink: Permission denied) 2024-05-09T04:58:11.832 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62722 rocksdb:l ceph rtd unknown /proc/62324/task/62722/root (readlink: Permission denied) 2024-05-09T04:58:11.833 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62722 rocksdb:l ceph txt unknown /proc/62324/task/62722/exe (readlink: Permission denied) 2024-05-09T04:58:11.833 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62722 rocksdb:l ceph NOFD /proc/62324/task/62722/fd (opendir: Permission denied) 2024-05-09T04:58:11.833 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62723 rocksdb:l ceph cwd unknown /proc/62324/task/62723/cwd (readlink: Permission denied) 2024-05-09T04:58:11.833 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62723 rocksdb:l ceph rtd unknown /proc/62324/task/62723/root (readlink: Permission denied) 2024-05-09T04:58:11.833 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62723 rocksdb:l ceph txt unknown /proc/62324/task/62723/exe (readlink: Permission denied) 2024-05-09T04:58:11.833 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62723 rocksdb:l ceph NOFD /proc/62324/task/62723/fd (opendir: Permission denied) 2024-05-09T04:58:11.833 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62724 rocksdb:h ceph cwd unknown /proc/62324/task/62724/cwd (readlink: Permission denied) 2024-05-09T04:58:11.833 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62724 rocksdb:h ceph rtd unknown /proc/62324/task/62724/root (readlink: Permission denied) 2024-05-09T04:58:11.833 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62724 rocksdb:h ceph txt unknown /proc/62324/task/62724/exe (readlink: Permission denied) 2024-05-09T04:58:11.833 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 62724 rocksdb:h ceph NOFD /proc/62324/task/62724/fd (opendir: Permission denied) 2024-05-09T04:58:11.833 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63227 bstore_ai ceph cwd unknown /proc/62324/task/63227/cwd (readlink: Permission denied) 2024-05-09T04:58:11.833 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63227 bstore_ai ceph rtd unknown /proc/62324/task/63227/root (readlink: Permission denied) 2024-05-09T04:58:11.833 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63227 bstore_ai ceph txt unknown /proc/62324/task/63227/exe (readlink: Permission denied) 2024-05-09T04:58:11.834 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63227 bstore_ai ceph NOFD /proc/62324/task/63227/fd (opendir: Permission denied) 2024-05-09T04:58:11.834 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63456 bstore_ai ceph cwd unknown /proc/62324/task/63456/cwd (readlink: Permission denied) 2024-05-09T04:58:11.834 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63456 bstore_ai ceph rtd unknown /proc/62324/task/63456/root (readlink: Permission denied) 2024-05-09T04:58:11.834 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63456 bstore_ai ceph txt unknown /proc/62324/task/63456/exe (readlink: Permission denied) 2024-05-09T04:58:11.834 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63456 bstore_ai ceph NOFD /proc/62324/task/63456/fd (opendir: Permission denied) 2024-05-09T04:58:11.834 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63639 ceph-osd ceph cwd unknown /proc/62324/task/63639/cwd (readlink: Permission denied) 2024-05-09T04:58:11.834 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63639 ceph-osd ceph rtd unknown /proc/62324/task/63639/root (readlink: Permission denied) 2024-05-09T04:58:11.834 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63639 ceph-osd ceph txt unknown /proc/62324/task/63639/exe (readlink: Permission denied) 2024-05-09T04:58:11.834 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63639 ceph-osd ceph NOFD /proc/62324/task/63639/fd (opendir: Permission denied) 2024-05-09T04:58:11.834 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63640 cfin ceph cwd unknown /proc/62324/task/63640/cwd (readlink: Permission denied) 2024-05-09T04:58:11.834 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63640 cfin ceph rtd unknown /proc/62324/task/63640/root (readlink: Permission denied) 2024-05-09T04:58:11.834 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63640 cfin ceph txt unknown /proc/62324/task/63640/exe (readlink: Permission denied) 2024-05-09T04:58:11.835 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63640 cfin ceph NOFD /proc/62324/task/63640/fd (opendir: Permission denied) 2024-05-09T04:58:11.835 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63641 bstore_kv ceph cwd unknown /proc/62324/task/63641/cwd (readlink: Permission denied) 2024-05-09T04:58:11.835 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63641 bstore_kv ceph rtd unknown /proc/62324/task/63641/root (readlink: Permission denied) 2024-05-09T04:58:11.835 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63641 bstore_kv ceph txt unknown /proc/62324/task/63641/exe (readlink: Permission denied) 2024-05-09T04:58:11.835 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63641 bstore_kv ceph NOFD /proc/62324/task/63641/fd (opendir: Permission denied) 2024-05-09T04:58:11.835 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63642 bstore_kv ceph cwd unknown /proc/62324/task/63642/cwd (readlink: Permission denied) 2024-05-09T04:58:11.835 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63642 bstore_kv ceph rtd unknown /proc/62324/task/63642/root (readlink: Permission denied) 2024-05-09T04:58:11.835 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63642 bstore_kv ceph txt unknown /proc/62324/task/63642/exe (readlink: Permission denied) 2024-05-09T04:58:11.835 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63642 bstore_kv ceph NOFD /proc/62324/task/63642/fd (opendir: Permission denied) 2024-05-09T04:58:11.835 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63643 bstore_me ceph cwd unknown /proc/62324/task/63643/cwd (readlink: Permission denied) 2024-05-09T04:58:11.835 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63643 bstore_me ceph rtd unknown /proc/62324/task/63643/root (readlink: Permission denied) 2024-05-09T04:58:11.835 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63643 bstore_me ceph txt unknown /proc/62324/task/63643/exe (readlink: Permission denied) 2024-05-09T04:58:11.835 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63643 bstore_me ceph NOFD /proc/62324/task/63643/fd (opendir: Permission denied) 2024-05-09T04:58:11.835 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63646 ms_dispat ceph cwd unknown /proc/62324/task/63646/cwd (readlink: Permission denied) 2024-05-09T04:58:11.835 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63646 ms_dispat ceph rtd unknown /proc/62324/task/63646/root (readlink: Permission denied) 2024-05-09T04:58:11.836 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63646 ms_dispat ceph txt unknown /proc/62324/task/63646/exe (readlink: Permission denied) 2024-05-09T04:58:11.836 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63646 ms_dispat ceph NOFD /proc/62324/task/63646/fd (opendir: Permission denied) 2024-05-09T04:58:11.836 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63648 ms_local ceph cwd unknown /proc/62324/task/63648/cwd (readlink: Permission denied) 2024-05-09T04:58:11.836 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63648 ms_local ceph rtd unknown /proc/62324/task/63648/root (readlink: Permission denied) 2024-05-09T04:58:11.836 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63648 ms_local ceph txt unknown /proc/62324/task/63648/exe (readlink: Permission denied) 2024-05-09T04:58:11.836 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63648 ms_local ceph NOFD /proc/62324/task/63648/fd (opendir: Permission denied) 2024-05-09T04:58:11.836 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63649 safe_time ceph cwd unknown /proc/62324/task/63649/cwd (readlink: Permission denied) 2024-05-09T04:58:11.836 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63649 safe_time ceph rtd unknown /proc/62324/task/63649/root (readlink: Permission denied) 2024-05-09T04:58:11.836 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63649 safe_time ceph txt unknown /proc/62324/task/63649/exe (readlink: Permission denied) 2024-05-09T04:58:11.836 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63649 safe_time ceph NOFD /proc/62324/task/63649/fd (opendir: Permission denied) 2024-05-09T04:58:11.836 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63650 safe_time ceph cwd unknown /proc/62324/task/63650/cwd (readlink: Permission denied) 2024-05-09T04:58:11.836 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63650 safe_time ceph rtd unknown /proc/62324/task/63650/root (readlink: Permission denied) 2024-05-09T04:58:11.836 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63650 safe_time ceph txt unknown /proc/62324/task/63650/exe (readlink: Permission denied) 2024-05-09T04:58:11.836 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63650 safe_time ceph NOFD /proc/62324/task/63650/fd (opendir: Permission denied) 2024-05-09T04:58:11.836 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63651 ms_dispat ceph cwd unknown /proc/62324/task/63651/cwd (readlink: Permission denied) 2024-05-09T04:58:11.837 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63651 ms_dispat ceph rtd unknown /proc/62324/task/63651/root (readlink: Permission denied) 2024-05-09T04:58:11.837 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63651 ms_dispat ceph txt unknown /proc/62324/task/63651/exe (readlink: Permission denied) 2024-05-09T04:58:11.837 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63651 ms_dispat ceph NOFD /proc/62324/task/63651/fd (opendir: Permission denied) 2024-05-09T04:58:11.837 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63652 ms_local ceph cwd unknown /proc/62324/task/63652/cwd (readlink: Permission denied) 2024-05-09T04:58:11.837 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63652 ms_local ceph rtd unknown /proc/62324/task/63652/root (readlink: Permission denied) 2024-05-09T04:58:11.837 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63652 ms_local ceph txt unknown /proc/62324/task/63652/exe (readlink: Permission denied) 2024-05-09T04:58:11.837 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63652 ms_local ceph NOFD /proc/62324/task/63652/fd (opendir: Permission denied) 2024-05-09T04:58:11.837 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63653 ms_dispat ceph cwd unknown /proc/62324/task/63653/cwd (readlink: Permission denied) 2024-05-09T04:58:11.837 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63653 ms_dispat ceph rtd unknown /proc/62324/task/63653/root (readlink: Permission denied) 2024-05-09T04:58:11.837 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63653 ms_dispat ceph txt unknown /proc/62324/task/63653/exe (readlink: Permission denied) 2024-05-09T04:58:11.837 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63653 ms_dispat ceph NOFD /proc/62324/task/63653/fd (opendir: Permission denied) 2024-05-09T04:58:11.837 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63654 ms_local ceph cwd unknown /proc/62324/task/63654/cwd (readlink: Permission denied) 2024-05-09T04:58:11.837 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63654 ms_local ceph rtd unknown /proc/62324/task/63654/root (readlink: Permission denied) 2024-05-09T04:58:11.837 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63654 ms_local ceph txt unknown /proc/62324/task/63654/exe (readlink: Permission denied) 2024-05-09T04:58:11.837 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63654 ms_local ceph NOFD /proc/62324/task/63654/fd (opendir: Permission denied) 2024-05-09T04:58:11.837 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63655 ms_dispat ceph cwd unknown /proc/62324/task/63655/cwd (readlink: Permission denied) 2024-05-09T04:58:11.838 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63655 ms_dispat ceph rtd unknown /proc/62324/task/63655/root (readlink: Permission denied) 2024-05-09T04:58:11.838 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63655 ms_dispat ceph txt unknown /proc/62324/task/63655/exe (readlink: Permission denied) 2024-05-09T04:58:11.838 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63655 ms_dispat ceph NOFD /proc/62324/task/63655/fd (opendir: Permission denied) 2024-05-09T04:58:11.838 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63656 ms_local ceph cwd unknown /proc/62324/task/63656/cwd (readlink: Permission denied) 2024-05-09T04:58:11.838 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63656 ms_local ceph rtd unknown /proc/62324/task/63656/root (readlink: Permission denied) 2024-05-09T04:58:11.838 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63656 ms_local ceph txt unknown /proc/62324/task/63656/exe (readlink: Permission denied) 2024-05-09T04:58:11.838 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63656 ms_local ceph NOFD /proc/62324/task/63656/fd (opendir: Permission denied) 2024-05-09T04:58:11.838 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63657 ms_dispat ceph cwd unknown /proc/62324/task/63657/cwd (readlink: Permission denied) 2024-05-09T04:58:11.838 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63657 ms_dispat ceph rtd unknown /proc/62324/task/63657/root (readlink: Permission denied) 2024-05-09T04:58:11.838 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63657 ms_dispat ceph txt unknown /proc/62324/task/63657/exe (readlink: Permission denied) 2024-05-09T04:58:11.838 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63657 ms_dispat ceph NOFD /proc/62324/task/63657/fd (opendir: Permission denied) 2024-05-09T04:58:11.838 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63658 ms_local ceph cwd unknown /proc/62324/task/63658/cwd (readlink: Permission denied) 2024-05-09T04:58:11.838 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63658 ms_local ceph rtd unknown /proc/62324/task/63658/root (readlink: Permission denied) 2024-05-09T04:58:11.838 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63658 ms_local ceph txt unknown /proc/62324/task/63658/exe (readlink: Permission denied) 2024-05-09T04:58:11.839 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63658 ms_local ceph NOFD /proc/62324/task/63658/fd (opendir: Permission denied) 2024-05-09T04:58:11.839 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63659 ms_dispat ceph cwd unknown /proc/62324/task/63659/cwd (readlink: Permission denied) 2024-05-09T04:58:11.839 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63659 ms_dispat ceph rtd unknown /proc/62324/task/63659/root (readlink: Permission denied) 2024-05-09T04:58:11.839 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63659 ms_dispat ceph txt unknown /proc/62324/task/63659/exe (readlink: Permission denied) 2024-05-09T04:58:11.839 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63659 ms_dispat ceph NOFD /proc/62324/task/63659/fd (opendir: Permission denied) 2024-05-09T04:58:11.839 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63660 ms_local ceph cwd unknown /proc/62324/task/63660/cwd (readlink: Permission denied) 2024-05-09T04:58:11.839 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63660 ms_local ceph rtd unknown /proc/62324/task/63660/root (readlink: Permission denied) 2024-05-09T04:58:11.839 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63660 ms_local ceph txt unknown /proc/62324/task/63660/exe (readlink: Permission denied) 2024-05-09T04:58:11.839 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63660 ms_local ceph NOFD /proc/62324/task/63660/fd (opendir: Permission denied) 2024-05-09T04:58:11.839 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63661 ms_dispat ceph cwd unknown /proc/62324/task/63661/cwd (readlink: Permission denied) 2024-05-09T04:58:11.839 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63661 ms_dispat ceph rtd unknown /proc/62324/task/63661/root (readlink: Permission denied) 2024-05-09T04:58:11.839 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63661 ms_dispat ceph txt unknown /proc/62324/task/63661/exe (readlink: Permission denied) 2024-05-09T04:58:11.839 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63661 ms_dispat ceph NOFD /proc/62324/task/63661/fd (opendir: Permission denied) 2024-05-09T04:58:11.839 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63662 ms_local ceph cwd unknown /proc/62324/task/63662/cwd (readlink: Permission denied) 2024-05-09T04:58:11.839 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63662 ms_local ceph rtd unknown /proc/62324/task/63662/root (readlink: Permission denied) 2024-05-09T04:58:11.839 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63662 ms_local ceph txt unknown /proc/62324/task/63662/exe (readlink: Permission denied) 2024-05-09T04:58:11.840 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63662 ms_local ceph NOFD /proc/62324/task/63662/fd (opendir: Permission denied) 2024-05-09T04:58:11.840 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63663 fn_anonym ceph cwd unknown /proc/62324/task/63663/cwd (readlink: Permission denied) 2024-05-09T04:58:11.840 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63663 fn_anonym ceph rtd unknown /proc/62324/task/63663/root (readlink: Permission denied) 2024-05-09T04:58:11.840 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63663 fn_anonym ceph txt unknown /proc/62324/task/63663/exe (readlink: Permission denied) 2024-05-09T04:58:11.840 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63663 fn_anonym ceph NOFD /proc/62324/task/63663/fd (opendir: Permission denied) 2024-05-09T04:58:11.840 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63664 finisher ceph cwd unknown /proc/62324/task/63664/cwd (readlink: Permission denied) 2024-05-09T04:58:11.840 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63664 finisher ceph rtd unknown /proc/62324/task/63664/root (readlink: Permission denied) 2024-05-09T04:58:11.840 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63664 finisher ceph txt unknown /proc/62324/task/63664/exe (readlink: Permission denied) 2024-05-09T04:58:11.840 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63664 finisher ceph NOFD /proc/62324/task/63664/fd (opendir: Permission denied) 2024-05-09T04:58:11.840 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63665 safe_time ceph cwd unknown /proc/62324/task/63665/cwd (readlink: Permission denied) 2024-05-09T04:58:11.840 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63665 safe_time ceph rtd unknown /proc/62324/task/63665/root (readlink: Permission denied) 2024-05-09T04:58:11.840 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63665 safe_time ceph txt unknown /proc/62324/task/63665/exe (readlink: Permission denied) 2024-05-09T04:58:11.840 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63665 safe_time ceph NOFD /proc/62324/task/63665/fd (opendir: Permission denied) 2024-05-09T04:58:11.840 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63666 safe_time ceph cwd unknown /proc/62324/task/63666/cwd (readlink: Permission denied) 2024-05-09T04:58:11.840 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63666 safe_time ceph rtd unknown /proc/62324/task/63666/root (readlink: Permission denied) 2024-05-09T04:58:11.841 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63666 safe_time ceph txt unknown /proc/62324/task/63666/exe (readlink: Permission denied) 2024-05-09T04:58:11.841 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63666 safe_time ceph NOFD /proc/62324/task/63666/fd (opendir: Permission denied) 2024-05-09T04:58:11.841 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63667 ceph-osd ceph cwd unknown /proc/62324/task/63667/cwd (readlink: Permission denied) 2024-05-09T04:58:11.841 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63667 ceph-osd ceph rtd unknown /proc/62324/task/63667/root (readlink: Permission denied) 2024-05-09T04:58:11.841 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63667 ceph-osd ceph txt unknown /proc/62324/task/63667/exe (readlink: Permission denied) 2024-05-09T04:58:11.841 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63667 ceph-osd ceph NOFD /proc/62324/task/63667/fd (opendir: Permission denied) 2024-05-09T04:58:11.841 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63668 osd_srv_a ceph cwd unknown /proc/62324/task/63668/cwd (readlink: Permission denied) 2024-05-09T04:58:11.841 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63668 osd_srv_a ceph rtd unknown /proc/62324/task/63668/root (readlink: Permission denied) 2024-05-09T04:58:11.841 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63668 osd_srv_a ceph txt unknown /proc/62324/task/63668/exe (readlink: Permission denied) 2024-05-09T04:58:11.841 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63668 osd_srv_a ceph NOFD /proc/62324/task/63668/fd (opendir: Permission denied) 2024-05-09T04:58:11.841 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63669 tp_osd_tp ceph cwd unknown /proc/62324/task/63669/cwd (readlink: Permission denied) 2024-05-09T04:58:11.841 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63669 tp_osd_tp ceph rtd unknown /proc/62324/task/63669/root (readlink: Permission denied) 2024-05-09T04:58:11.841 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63669 tp_osd_tp ceph txt unknown /proc/62324/task/63669/exe (readlink: Permission denied) 2024-05-09T04:58:11.842 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63669 tp_osd_tp ceph NOFD /proc/62324/task/63669/fd (opendir: Permission denied) 2024-05-09T04:58:11.842 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63670 tp_osd_tp ceph cwd unknown /proc/62324/task/63670/cwd (readlink: Permission denied) 2024-05-09T04:58:11.842 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63670 tp_osd_tp ceph rtd unknown /proc/62324/task/63670/root (readlink: Permission denied) 2024-05-09T04:58:11.842 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63670 tp_osd_tp ceph txt unknown /proc/62324/task/63670/exe (readlink: Permission denied) 2024-05-09T04:58:11.842 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63670 tp_osd_tp ceph NOFD /proc/62324/task/63670/fd (opendir: Permission denied) 2024-05-09T04:58:11.842 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63671 tp_osd_tp ceph cwd unknown /proc/62324/task/63671/cwd (readlink: Permission denied) 2024-05-09T04:58:11.842 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63671 tp_osd_tp ceph rtd unknown /proc/62324/task/63671/root (readlink: Permission denied) 2024-05-09T04:58:11.842 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63671 tp_osd_tp ceph txt unknown /proc/62324/task/63671/exe (readlink: Permission denied) 2024-05-09T04:58:11.842 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63671 tp_osd_tp ceph NOFD /proc/62324/task/63671/fd (opendir: Permission denied) 2024-05-09T04:58:11.842 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63672 tp_osd_tp ceph cwd unknown /proc/62324/task/63672/cwd (readlink: Permission denied) 2024-05-09T04:58:11.842 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63672 tp_osd_tp ceph rtd unknown /proc/62324/task/63672/root (readlink: Permission denied) 2024-05-09T04:58:11.842 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63672 tp_osd_tp ceph txt unknown /proc/62324/task/63672/exe (readlink: Permission denied) 2024-05-09T04:58:11.842 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63672 tp_osd_tp ceph NOFD /proc/62324/task/63672/fd (opendir: Permission denied) 2024-05-09T04:58:11.842 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63673 tp_osd_tp ceph cwd unknown /proc/62324/task/63673/cwd (readlink: Permission denied) 2024-05-09T04:58:11.843 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63673 tp_osd_tp ceph rtd unknown /proc/62324/task/63673/root (readlink: Permission denied) 2024-05-09T04:58:11.843 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63673 tp_osd_tp ceph txt unknown /proc/62324/task/63673/exe (readlink: Permission denied) 2024-05-09T04:58:11.843 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63673 tp_osd_tp ceph NOFD /proc/62324/task/63673/fd (opendir: Permission denied) 2024-05-09T04:58:11.843 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63674 tp_osd_tp ceph cwd unknown /proc/62324/task/63674/cwd (readlink: Permission denied) 2024-05-09T04:58:11.843 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63674 tp_osd_tp ceph rtd unknown /proc/62324/task/63674/root (readlink: Permission denied) 2024-05-09T04:58:11.843 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63674 tp_osd_tp ceph txt unknown /proc/62324/task/63674/exe (readlink: Permission denied) 2024-05-09T04:58:11.843 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63674 tp_osd_tp ceph NOFD /proc/62324/task/63674/fd (opendir: Permission denied) 2024-05-09T04:58:11.843 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63675 tp_osd_tp ceph cwd unknown /proc/62324/task/63675/cwd (readlink: Permission denied) 2024-05-09T04:58:11.843 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63675 tp_osd_tp ceph rtd unknown /proc/62324/task/63675/root (readlink: Permission denied) 2024-05-09T04:58:11.843 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63675 tp_osd_tp ceph txt unknown /proc/62324/task/63675/exe (readlink: Permission denied) 2024-05-09T04:58:11.843 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63675 tp_osd_tp ceph NOFD /proc/62324/task/63675/fd (opendir: Permission denied) 2024-05-09T04:58:11.843 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63676 tp_osd_tp ceph cwd unknown /proc/62324/task/63676/cwd (readlink: Permission denied) 2024-05-09T04:58:11.843 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63676 tp_osd_tp ceph rtd unknown /proc/62324/task/63676/root (readlink: Permission denied) 2024-05-09T04:58:11.843 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63676 tp_osd_tp ceph txt unknown /proc/62324/task/63676/exe (readlink: Permission denied) 2024-05-09T04:58:11.843 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63676 tp_osd_tp ceph NOFD /proc/62324/task/63676/fd (opendir: Permission denied) 2024-05-09T04:58:11.844 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63677 tp_osd_tp ceph cwd unknown /proc/62324/task/63677/cwd (readlink: Permission denied) 2024-05-09T04:58:11.844 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63677 tp_osd_tp ceph rtd unknown /proc/62324/task/63677/root (readlink: Permission denied) 2024-05-09T04:58:11.844 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63677 tp_osd_tp ceph txt unknown /proc/62324/task/63677/exe (readlink: Permission denied) 2024-05-09T04:58:11.844 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63677 tp_osd_tp ceph NOFD /proc/62324/task/63677/fd (opendir: Permission denied) 2024-05-09T04:58:11.844 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63678 tp_osd_tp ceph cwd unknown /proc/62324/task/63678/cwd (readlink: Permission denied) 2024-05-09T04:58:11.844 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63678 tp_osd_tp ceph rtd unknown /proc/62324/task/63678/root (readlink: Permission denied) 2024-05-09T04:58:11.844 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63678 tp_osd_tp ceph txt unknown /proc/62324/task/63678/exe (readlink: Permission denied) 2024-05-09T04:58:11.844 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63678 tp_osd_tp ceph NOFD /proc/62324/task/63678/fd (opendir: Permission denied) 2024-05-09T04:58:11.844 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63679 tp_osd_tp ceph cwd unknown /proc/62324/task/63679/cwd (readlink: Permission denied) 2024-05-09T04:58:11.844 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63679 tp_osd_tp ceph rtd unknown /proc/62324/task/63679/root (readlink: Permission denied) 2024-05-09T04:58:11.844 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63679 tp_osd_tp ceph txt unknown /proc/62324/task/63679/exe (readlink: Permission denied) 2024-05-09T04:58:11.844 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63679 tp_osd_tp ceph NOFD /proc/62324/task/63679/fd (opendir: Permission denied) 2024-05-09T04:58:11.844 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63680 tp_osd_tp ceph cwd unknown /proc/62324/task/63680/cwd (readlink: Permission denied) 2024-05-09T04:58:11.844 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63680 tp_osd_tp ceph rtd unknown /proc/62324/task/63680/root (readlink: Permission denied) 2024-05-09T04:58:11.845 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63680 tp_osd_tp ceph txt unknown /proc/62324/task/63680/exe (readlink: Permission denied) 2024-05-09T04:58:11.845 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63680 tp_osd_tp ceph NOFD /proc/62324/task/63680/fd (opendir: Permission denied) 2024-05-09T04:58:11.845 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63681 tp_osd_tp ceph cwd unknown /proc/62324/task/63681/cwd (readlink: Permission denied) 2024-05-09T04:58:11.845 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63681 tp_osd_tp ceph rtd unknown /proc/62324/task/63681/root (readlink: Permission denied) 2024-05-09T04:58:11.845 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63681 tp_osd_tp ceph txt unknown /proc/62324/task/63681/exe (readlink: Permission denied) 2024-05-09T04:58:11.845 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63681 tp_osd_tp ceph NOFD /proc/62324/task/63681/fd (opendir: Permission denied) 2024-05-09T04:58:11.845 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63682 tp_osd_tp ceph cwd unknown /proc/62324/task/63682/cwd (readlink: Permission denied) 2024-05-09T04:58:11.845 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63682 tp_osd_tp ceph rtd unknown /proc/62324/task/63682/root (readlink: Permission denied) 2024-05-09T04:58:11.845 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63682 tp_osd_tp ceph txt unknown /proc/62324/task/63682/exe (readlink: Permission denied) 2024-05-09T04:58:11.845 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63682 tp_osd_tp ceph NOFD /proc/62324/task/63682/fd (opendir: Permission denied) 2024-05-09T04:58:11.845 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63683 tp_osd_tp ceph cwd unknown /proc/62324/task/63683/cwd (readlink: Permission denied) 2024-05-09T04:58:11.845 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63683 tp_osd_tp ceph rtd unknown /proc/62324/task/63683/root (readlink: Permission denied) 2024-05-09T04:58:11.845 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63683 tp_osd_tp ceph txt unknown /proc/62324/task/63683/exe (readlink: Permission denied) 2024-05-09T04:58:11.846 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63683 tp_osd_tp ceph NOFD /proc/62324/task/63683/fd (opendir: Permission denied) 2024-05-09T04:58:11.846 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63684 tp_osd_tp ceph cwd unknown /proc/62324/task/63684/cwd (readlink: Permission denied) 2024-05-09T04:58:11.846 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63684 tp_osd_tp ceph rtd unknown /proc/62324/task/63684/root (readlink: Permission denied) 2024-05-09T04:58:11.846 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63684 tp_osd_tp ceph txt unknown /proc/62324/task/63684/exe (readlink: Permission denied) 2024-05-09T04:58:11.846 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63684 tp_osd_tp ceph NOFD /proc/62324/task/63684/fd (opendir: Permission denied) 2024-05-09T04:58:11.846 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63685 osd_srv_h ceph cwd unknown /proc/62324/task/63685/cwd (readlink: Permission denied) 2024-05-09T04:58:11.846 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63685 osd_srv_h ceph rtd unknown /proc/62324/task/63685/root (readlink: Permission denied) 2024-05-09T04:58:11.846 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63685 osd_srv_h ceph txt unknown /proc/62324/task/63685/exe (readlink: Permission denied) 2024-05-09T04:58:11.846 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-osd 62324 63685 osd_srv_h ceph NOFD /proc/62324/task/63685/fd (opendir: Permission denied) 2024-05-09T04:58:11.846 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 63888 root cwd unknown /proc/63888/cwd (readlink: Permission denied) 2024-05-09T04:58:11.846 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 63888 root rtd unknown /proc/63888/root (readlink: Permission denied) 2024-05-09T04:58:11.846 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 63888 root txt unknown /proc/63888/exe (readlink: Permission denied) 2024-05-09T04:58:11.846 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 63888 root NOFD /proc/63888/fd (opendir: Permission denied) 2024-05-09T04:58:11.847 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 63919 root cwd unknown /proc/63919/cwd (readlink: Permission denied) 2024-05-09T04:58:11.847 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 63919 root rtd unknown /proc/63919/root (readlink: Permission denied) 2024-05-09T04:58:11.847 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 63919 root txt unknown /proc/63919/exe (readlink: Permission denied) 2024-05-09T04:58:11.847 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 63919 root NOFD /proc/63919/fd (opendir: Permission denied) 2024-05-09T04:58:11.847 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 66869 root cwd unknown /proc/66869/cwd (readlink: Permission denied) 2024-05-09T04:58:11.847 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 66869 root rtd unknown /proc/66869/root (readlink: Permission denied) 2024-05-09T04:58:11.847 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 66869 root txt unknown /proc/66869/exe (readlink: Permission denied) 2024-05-09T04:58:11.847 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 66869 root NOFD /proc/66869/fd (opendir: Permission denied) 2024-05-09T04:58:11.847 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 66892 root cwd unknown /proc/66892/cwd (readlink: Permission denied) 2024-05-09T04:58:11.847 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 66892 root rtd unknown /proc/66892/root (readlink: Permission denied) 2024-05-09T04:58:11.848 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 66892 root txt unknown /proc/66892/exe (readlink: Permission denied) 2024-05-09T04:58:11.848 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 66892 root NOFD /proc/66892/fd (opendir: Permission denied) 2024-05-09T04:58:11.848 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 66904 root cwd unknown /proc/66904/cwd (readlink: Permission denied) 2024-05-09T04:58:11.848 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 66904 root rtd unknown /proc/66904/root (readlink: Permission denied) 2024-05-09T04:58:11.848 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 66904 root txt unknown /proc/66904/exe (readlink: Permission denied) 2024-05-09T04:58:11.848 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 66904 root NOFD /proc/66904/fd (opendir: Permission denied) 2024-05-09T04:58:11.848 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 66917 root cwd unknown /proc/66917/cwd (readlink: Permission denied) 2024-05-09T04:58:11.848 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 66917 root rtd unknown /proc/66917/root (readlink: Permission denied) 2024-05-09T04:58:11.848 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 66917 root txt unknown /proc/66917/exe (readlink: Permission denied) 2024-05-09T04:58:11.848 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 66917 root NOFD /proc/66917/fd (opendir: Permission denied) 2024-05-09T04:58:11.848 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 66923 root cwd unknown /proc/66923/cwd (readlink: Permission denied) 2024-05-09T04:58:11.848 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 66923 root rtd unknown /proc/66923/root (readlink: Permission denied) 2024-05-09T04:58:11.848 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 66923 root txt unknown /proc/66923/exe (readlink: Permission denied) 2024-05-09T04:58:11.849 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 66923 root NOFD /proc/66923/fd (opendir: Permission denied) 2024-05-09T04:58:11.849 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 66935 root cwd unknown /proc/66935/cwd (readlink: Permission denied) 2024-05-09T04:58:11.849 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 66935 root rtd unknown /proc/66935/root (readlink: Permission denied) 2024-05-09T04:58:11.849 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 66935 root txt unknown /proc/66935/exe (readlink: Permission denied) 2024-05-09T04:58:11.849 INFO:tasks.workunit.client.0.smithi006.stdout:sudo 66935 root NOFD /proc/66935/fd (opendir: Permission denied) 2024-05-09T04:58:11.849 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 66940 root cwd unknown /proc/66940/cwd (readlink: Permission denied) 2024-05-09T04:58:11.849 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 66940 root rtd unknown /proc/66940/root (readlink: Permission denied) 2024-05-09T04:58:11.849 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 66940 root txt unknown /proc/66940/exe (readlink: Permission denied) 2024-05-09T04:58:11.849 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 66940 root NOFD /proc/66940/fd (opendir: Permission denied) 2024-05-09T04:58:11.849 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 66957 root cwd unknown /proc/66957/cwd (readlink: Permission denied) 2024-05-09T04:58:11.849 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 66957 root rtd unknown /proc/66957/root (readlink: Permission denied) 2024-05-09T04:58:11.849 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 66957 root txt unknown /proc/66957/exe (readlink: Permission denied) 2024-05-09T04:58:11.849 INFO:tasks.workunit.client.0.smithi006.stdout:journalct 66957 root NOFD /proc/66957/fd (opendir: Permission denied) 2024-05-09T04:58:11.850 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 67555 root cwd unknown /proc/67555/cwd (readlink: Permission denied) 2024-05-09T04:58:11.850 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 67555 root rtd unknown /proc/67555/root (readlink: Permission denied) 2024-05-09T04:58:11.850 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 67555 root txt unknown /proc/67555/exe (readlink: Permission denied) 2024-05-09T04:58:11.850 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 67555 root NOFD /proc/67555/fd (opendir: Permission denied) 2024-05-09T04:58:11.850 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 67557 root cwd unknown /proc/67557/cwd (readlink: Permission denied) 2024-05-09T04:58:11.850 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 67557 root rtd unknown /proc/67557/root (readlink: Permission denied) 2024-05-09T04:58:11.850 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 67557 root txt unknown /proc/67557/exe (readlink: Permission denied) 2024-05-09T04:58:11.850 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 67557 root NOFD /proc/67557/fd (opendir: Permission denied) 2024-05-09T04:58:11.850 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 ceph cwd unknown /proc/67559/cwd (readlink: Permission denied) 2024-05-09T04:58:11.850 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 ceph rtd unknown /proc/67559/root (readlink: Permission denied) 2024-05-09T04:58:11.850 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 ceph txt unknown /proc/67559/exe (readlink: Permission denied) 2024-05-09T04:58:11.851 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 ceph NOFD /proc/67559/fd (opendir: Permission denied) 2024-05-09T04:58:11.851 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67562 log ceph cwd unknown /proc/67559/task/67562/cwd (readlink: Permission denied) 2024-05-09T04:58:11.851 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67562 log ceph rtd unknown /proc/67559/task/67562/root (readlink: Permission denied) 2024-05-09T04:58:11.851 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67562 log ceph txt unknown /proc/67559/task/67562/exe (readlink: Permission denied) 2024-05-09T04:58:11.851 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67562 log ceph NOFD /proc/67559/task/67562/fd (opendir: Permission denied) 2024-05-09T04:58:11.851 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67564 msgr-work ceph cwd unknown /proc/67559/task/67564/cwd (readlink: Permission denied) 2024-05-09T04:58:11.851 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67564 msgr-work ceph rtd unknown /proc/67559/task/67564/root (readlink: Permission denied) 2024-05-09T04:58:11.851 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67564 msgr-work ceph txt unknown /proc/67559/task/67564/exe (readlink: Permission denied) 2024-05-09T04:58:11.851 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67564 msgr-work ceph NOFD /proc/67559/task/67564/fd (opendir: Permission denied) 2024-05-09T04:58:11.851 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67565 msgr-work ceph cwd unknown /proc/67559/task/67565/cwd (readlink: Permission denied) 2024-05-09T04:58:11.851 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67565 msgr-work ceph rtd unknown /proc/67559/task/67565/root (readlink: Permission denied) 2024-05-09T04:58:11.851 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67565 msgr-work ceph txt unknown /proc/67559/task/67565/exe (readlink: Permission denied) 2024-05-09T04:58:11.852 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67565 msgr-work ceph NOFD /proc/67559/task/67565/fd (opendir: Permission denied) 2024-05-09T04:58:11.852 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67566 msgr-work ceph cwd unknown /proc/67559/task/67566/cwd (readlink: Permission denied) 2024-05-09T04:58:11.852 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67566 msgr-work ceph rtd unknown /proc/67559/task/67566/root (readlink: Permission denied) 2024-05-09T04:58:11.852 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67566 msgr-work ceph txt unknown /proc/67559/task/67566/exe (readlink: Permission denied) 2024-05-09T04:58:11.852 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67566 msgr-work ceph NOFD /proc/67559/task/67566/fd (opendir: Permission denied) 2024-05-09T04:58:11.852 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67570 service ceph cwd unknown /proc/67559/task/67570/cwd (readlink: Permission denied) 2024-05-09T04:58:11.852 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67570 service ceph rtd unknown /proc/67559/task/67570/root (readlink: Permission denied) 2024-05-09T04:58:11.852 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67570 service ceph txt unknown /proc/67559/task/67570/exe (readlink: Permission denied) 2024-05-09T04:58:11.852 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67570 service ceph NOFD /proc/67559/task/67570/fd (opendir: Permission denied) 2024-05-09T04:58:11.852 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67571 admin_soc ceph cwd unknown /proc/67559/task/67571/cwd (readlink: Permission denied) 2024-05-09T04:58:11.852 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67571 admin_soc ceph rtd unknown /proc/67559/task/67571/root (readlink: Permission denied) 2024-05-09T04:58:11.853 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67571 admin_soc ceph txt unknown /proc/67559/task/67571/exe (readlink: Permission denied) 2024-05-09T04:58:11.853 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67571 admin_soc ceph NOFD /proc/67559/task/67571/fd (opendir: Permission denied) 2024-05-09T04:58:11.853 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67572 signal_ha ceph cwd unknown /proc/67559/task/67572/cwd (readlink: Permission denied) 2024-05-09T04:58:11.853 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67572 signal_ha ceph rtd unknown /proc/67559/task/67572/root (readlink: Permission denied) 2024-05-09T04:58:11.853 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67572 signal_ha ceph txt unknown /proc/67559/task/67572/exe (readlink: Permission denied) 2024-05-09T04:58:11.853 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67572 signal_ha ceph NOFD /proc/67559/task/67572/fd (opendir: Permission denied) 2024-05-09T04:58:11.853 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67573 io_contex ceph cwd unknown /proc/67559/task/67573/cwd (readlink: Permission denied) 2024-05-09T04:58:11.853 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67573 io_contex ceph rtd unknown /proc/67559/task/67573/root (readlink: Permission denied) 2024-05-09T04:58:11.853 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67573 io_contex ceph txt unknown /proc/67559/task/67573/exe (readlink: Permission denied) 2024-05-09T04:58:11.853 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67573 io_contex ceph NOFD /proc/67559/task/67573/fd (opendir: Permission denied) 2024-05-09T04:58:11.853 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67574 io_contex ceph cwd unknown /proc/67559/task/67574/cwd (readlink: Permission denied) 2024-05-09T04:58:11.853 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67574 io_contex ceph rtd unknown /proc/67559/task/67574/root (readlink: Permission denied) 2024-05-09T04:58:11.853 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67574 io_contex ceph txt unknown /proc/67559/task/67574/exe (readlink: Permission denied) 2024-05-09T04:58:11.853 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67574 io_contex ceph NOFD /proc/67559/task/67574/fd (opendir: Permission denied) 2024-05-09T04:58:11.854 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67575 ms_dispat ceph cwd unknown /proc/67559/task/67575/cwd (readlink: Permission denied) 2024-05-09T04:58:11.854 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67575 ms_dispat ceph rtd unknown /proc/67559/task/67575/root (readlink: Permission denied) 2024-05-09T04:58:11.854 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67575 ms_dispat ceph txt unknown /proc/67559/task/67575/exe (readlink: Permission denied) 2024-05-09T04:58:11.854 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67575 ms_dispat ceph NOFD /proc/67559/task/67575/fd (opendir: Permission denied) 2024-05-09T04:58:11.854 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67576 ms_local ceph cwd unknown /proc/67559/task/67576/cwd (readlink: Permission denied) 2024-05-09T04:58:11.854 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67576 ms_local ceph rtd unknown /proc/67559/task/67576/root (readlink: Permission denied) 2024-05-09T04:58:11.854 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67576 ms_local ceph txt unknown /proc/67559/task/67576/exe (readlink: Permission denied) 2024-05-09T04:58:11.854 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67576 ms_local ceph NOFD /proc/67559/task/67576/fd (opendir: Permission denied) 2024-05-09T04:58:11.854 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67577 safe_time ceph cwd unknown /proc/67559/task/67577/cwd (readlink: Permission denied) 2024-05-09T04:58:11.854 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67577 safe_time ceph rtd unknown /proc/67559/task/67577/root (readlink: Permission denied) 2024-05-09T04:58:11.854 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67577 safe_time ceph txt unknown /proc/67559/task/67577/exe (readlink: Permission denied) 2024-05-09T04:58:11.854 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67577 safe_time ceph NOFD /proc/67559/task/67577/fd (opendir: Permission denied) 2024-05-09T04:58:11.854 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67578 safe_time ceph cwd unknown /proc/67559/task/67578/cwd (readlink: Permission denied) 2024-05-09T04:58:11.854 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67578 safe_time ceph rtd unknown /proc/67559/task/67578/root (readlink: Permission denied) 2024-05-09T04:58:11.855 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67578 safe_time ceph txt unknown /proc/67559/task/67578/exe (readlink: Permission denied) 2024-05-09T04:58:11.855 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67578 safe_time ceph NOFD /proc/67559/task/67578/fd (opendir: Permission denied) 2024-05-09T04:58:11.855 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67579 ceph-mds ceph cwd unknown /proc/67559/task/67579/cwd (readlink: Permission denied) 2024-05-09T04:58:11.855 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67579 ceph-mds ceph rtd unknown /proc/67559/task/67579/root (readlink: Permission denied) 2024-05-09T04:58:11.855 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67579 ceph-mds ceph txt unknown /proc/67559/task/67579/exe (readlink: Permission denied) 2024-05-09T04:58:11.855 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67579 ceph-mds ceph NOFD /proc/67559/task/67579/fd (opendir: Permission denied) 2024-05-09T04:58:11.855 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67682 safe_time ceph cwd unknown /proc/67559/task/67682/cwd (readlink: Permission denied) 2024-05-09T04:58:11.855 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67682 safe_time ceph rtd unknown /proc/67559/task/67682/root (readlink: Permission denied) 2024-05-09T04:58:11.855 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67682 safe_time ceph txt unknown /proc/67559/task/67682/exe (readlink: Permission denied) 2024-05-09T04:58:11.855 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67559 67682 safe_time ceph NOFD /proc/67559/task/67682/fd (opendir: Permission denied) 2024-05-09T04:58:11.855 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 67968 root cwd unknown /proc/67968/cwd (readlink: Permission denied) 2024-05-09T04:58:11.855 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 67968 root rtd unknown /proc/67968/root (readlink: Permission denied) 2024-05-09T04:58:11.855 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 67968 root txt unknown /proc/67968/exe (readlink: Permission denied) 2024-05-09T04:58:11.855 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 67968 root NOFD /proc/67968/fd (opendir: Permission denied) 2024-05-09T04:58:11.856 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 67970 root cwd unknown /proc/67970/cwd (readlink: Permission denied) 2024-05-09T04:58:11.856 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 67970 root rtd unknown /proc/67970/root (readlink: Permission denied) 2024-05-09T04:58:11.856 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 67970 root txt unknown /proc/67970/exe (readlink: Permission denied) 2024-05-09T04:58:11.856 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 67970 root NOFD /proc/67970/fd (opendir: Permission denied) 2024-05-09T04:58:11.856 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 ceph cwd unknown /proc/67972/cwd (readlink: Permission denied) 2024-05-09T04:58:11.856 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 ceph rtd unknown /proc/67972/root (readlink: Permission denied) 2024-05-09T04:58:11.856 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 ceph txt unknown /proc/67972/exe (readlink: Permission denied) 2024-05-09T04:58:11.856 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 ceph NOFD /proc/67972/fd (opendir: Permission denied) 2024-05-09T04:58:11.856 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67974 log ceph cwd unknown /proc/67972/task/67974/cwd (readlink: Permission denied) 2024-05-09T04:58:11.856 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67974 log ceph rtd unknown /proc/67972/task/67974/root (readlink: Permission denied) 2024-05-09T04:58:11.856 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67974 log ceph txt unknown /proc/67972/task/67974/exe (readlink: Permission denied) 2024-05-09T04:58:11.856 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67974 log ceph NOFD /proc/67972/task/67974/fd (opendir: Permission denied) 2024-05-09T04:58:11.856 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67977 msgr-work ceph cwd unknown /proc/67972/task/67977/cwd (readlink: Permission denied) 2024-05-09T04:58:11.856 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67977 msgr-work ceph rtd unknown /proc/67972/task/67977/root (readlink: Permission denied) 2024-05-09T04:58:11.857 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67977 msgr-work ceph txt unknown /proc/67972/task/67977/exe (readlink: Permission denied) 2024-05-09T04:58:11.857 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67977 msgr-work ceph NOFD /proc/67972/task/67977/fd (opendir: Permission denied) 2024-05-09T04:58:11.857 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67978 msgr-work ceph cwd unknown /proc/67972/task/67978/cwd (readlink: Permission denied) 2024-05-09T04:58:11.857 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67978 msgr-work ceph rtd unknown /proc/67972/task/67978/root (readlink: Permission denied) 2024-05-09T04:58:11.857 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67978 msgr-work ceph txt unknown /proc/67972/task/67978/exe (readlink: Permission denied) 2024-05-09T04:58:11.857 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67978 msgr-work ceph NOFD /proc/67972/task/67978/fd (opendir: Permission denied) 2024-05-09T04:58:11.857 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67979 msgr-work ceph cwd unknown /proc/67972/task/67979/cwd (readlink: Permission denied) 2024-05-09T04:58:11.857 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67979 msgr-work ceph rtd unknown /proc/67972/task/67979/root (readlink: Permission denied) 2024-05-09T04:58:11.857 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67979 msgr-work ceph txt unknown /proc/67972/task/67979/exe (readlink: Permission denied) 2024-05-09T04:58:11.857 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67979 msgr-work ceph NOFD /proc/67972/task/67979/fd (opendir: Permission denied) 2024-05-09T04:58:11.857 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67983 service ceph cwd unknown /proc/67972/task/67983/cwd (readlink: Permission denied) 2024-05-09T04:58:11.858 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67983 service ceph rtd unknown /proc/67972/task/67983/root (readlink: Permission denied) 2024-05-09T04:58:11.858 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67983 service ceph txt unknown /proc/67972/task/67983/exe (readlink: Permission denied) 2024-05-09T04:58:11.858 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67983 service ceph NOFD /proc/67972/task/67983/fd (opendir: Permission denied) 2024-05-09T04:58:11.858 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67984 admin_soc ceph cwd unknown /proc/67972/task/67984/cwd (readlink: Permission denied) 2024-05-09T04:58:11.858 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67984 admin_soc ceph rtd unknown /proc/67972/task/67984/root (readlink: Permission denied) 2024-05-09T04:58:11.858 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67984 admin_soc ceph txt unknown /proc/67972/task/67984/exe (readlink: Permission denied) 2024-05-09T04:58:11.858 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67984 admin_soc ceph NOFD /proc/67972/task/67984/fd (opendir: Permission denied) 2024-05-09T04:58:11.858 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67985 signal_ha ceph cwd unknown /proc/67972/task/67985/cwd (readlink: Permission denied) 2024-05-09T04:58:11.859 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67985 signal_ha ceph rtd unknown /proc/67972/task/67985/root (readlink: Permission denied) 2024-05-09T04:58:11.859 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67985 signal_ha ceph txt unknown /proc/67972/task/67985/exe (readlink: Permission denied) 2024-05-09T04:58:11.859 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67985 signal_ha ceph NOFD /proc/67972/task/67985/fd (opendir: Permission denied) 2024-05-09T04:58:11.859 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67986 io_contex ceph cwd unknown /proc/67972/task/67986/cwd (readlink: Permission denied) 2024-05-09T04:58:11.859 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67986 io_contex ceph rtd unknown /proc/67972/task/67986/root (readlink: Permission denied) 2024-05-09T04:58:11.859 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67986 io_contex ceph txt unknown /proc/67972/task/67986/exe (readlink: Permission denied) 2024-05-09T04:58:11.859 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67986 io_contex ceph NOFD /proc/67972/task/67986/fd (opendir: Permission denied) 2024-05-09T04:58:11.859 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67987 io_contex ceph cwd unknown /proc/67972/task/67987/cwd (readlink: Permission denied) 2024-05-09T04:58:11.859 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67987 io_contex ceph rtd unknown /proc/67972/task/67987/root (readlink: Permission denied) 2024-05-09T04:58:11.859 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67987 io_contex ceph txt unknown /proc/67972/task/67987/exe (readlink: Permission denied) 2024-05-09T04:58:11.860 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67987 io_contex ceph NOFD /proc/67972/task/67987/fd (opendir: Permission denied) 2024-05-09T04:58:11.860 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67988 ms_dispat ceph cwd unknown /proc/67972/task/67988/cwd (readlink: Permission denied) 2024-05-09T04:58:11.860 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67988 ms_dispat ceph rtd unknown /proc/67972/task/67988/root (readlink: Permission denied) 2024-05-09T04:58:11.860 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67988 ms_dispat ceph txt unknown /proc/67972/task/67988/exe (readlink: Permission denied) 2024-05-09T04:58:11.860 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67988 ms_dispat ceph NOFD /proc/67972/task/67988/fd (opendir: Permission denied) 2024-05-09T04:58:11.860 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67989 ms_local ceph cwd unknown /proc/67972/task/67989/cwd (readlink: Permission denied) 2024-05-09T04:58:11.860 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67989 ms_local ceph rtd unknown /proc/67972/task/67989/root (readlink: Permission denied) 2024-05-09T04:58:11.860 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67989 ms_local ceph txt unknown /proc/67972/task/67989/exe (readlink: Permission denied) 2024-05-09T04:58:11.860 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67989 ms_local ceph NOFD /proc/67972/task/67989/fd (opendir: Permission denied) 2024-05-09T04:58:11.860 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67990 safe_time ceph cwd unknown /proc/67972/task/67990/cwd (readlink: Permission denied) 2024-05-09T04:58:11.860 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67990 safe_time ceph rtd unknown /proc/67972/task/67990/root (readlink: Permission denied) 2024-05-09T04:58:11.861 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67990 safe_time ceph txt unknown /proc/67972/task/67990/exe (readlink: Permission denied) 2024-05-09T04:58:11.861 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67990 safe_time ceph NOFD /proc/67972/task/67990/fd (opendir: Permission denied) 2024-05-09T04:58:11.861 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67991 safe_time ceph cwd unknown /proc/67972/task/67991/cwd (readlink: Permission denied) 2024-05-09T04:58:11.861 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67991 safe_time ceph rtd unknown /proc/67972/task/67991/root (readlink: Permission denied) 2024-05-09T04:58:11.861 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67991 safe_time ceph txt unknown /proc/67972/task/67991/exe (readlink: Permission denied) 2024-05-09T04:58:11.861 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67991 safe_time ceph NOFD /proc/67972/task/67991/fd (opendir: Permission denied) 2024-05-09T04:58:11.861 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67992 ceph-mds ceph cwd unknown /proc/67972/task/67992/cwd (readlink: Permission denied) 2024-05-09T04:58:11.861 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67992 ceph-mds ceph rtd unknown /proc/67972/task/67992/root (readlink: Permission denied) 2024-05-09T04:58:11.861 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67992 ceph-mds ceph txt unknown /proc/67972/task/67992/exe (readlink: Permission denied) 2024-05-09T04:58:11.861 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 67992 ceph-mds ceph NOFD /proc/67972/task/67992/fd (opendir: Permission denied) 2024-05-09T04:58:11.862 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 68024 safe_time ceph cwd unknown /proc/67972/task/68024/cwd (readlink: Permission denied) 2024-05-09T04:58:11.862 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 68024 safe_time ceph rtd unknown /proc/67972/task/68024/root (readlink: Permission denied) 2024-05-09T04:58:11.862 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 68024 safe_time ceph txt unknown /proc/67972/task/68024/exe (readlink: Permission denied) 2024-05-09T04:58:11.862 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 68024 safe_time ceph NOFD /proc/67972/task/68024/fd (opendir: Permission denied) 2024-05-09T04:58:11.862 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74806 ceph_time ceph cwd unknown /proc/67972/task/74806/cwd (readlink: Permission denied) 2024-05-09T04:58:11.862 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74806 ceph_time ceph rtd unknown /proc/67972/task/74806/root (readlink: Permission denied) 2024-05-09T04:58:11.862 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74806 ceph_time ceph txt unknown /proc/67972/task/74806/exe (readlink: Permission denied) 2024-05-09T04:58:11.862 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74806 ceph_time ceph NOFD /proc/67972/task/74806/fd (opendir: Permission denied) 2024-05-09T04:58:11.862 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74807 OpHistory ceph cwd unknown /proc/67972/task/74807/cwd (readlink: Permission denied) 2024-05-09T04:58:11.862 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74807 OpHistory ceph rtd unknown /proc/67972/task/74807/root (readlink: Permission denied) 2024-05-09T04:58:11.863 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74807 OpHistory ceph txt unknown /proc/67972/task/74807/exe (readlink: Permission denied) 2024-05-09T04:58:11.863 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74807 OpHistory ceph NOFD /proc/67972/task/74807/fd (opendir: Permission denied) 2024-05-09T04:58:11.863 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74808 ms_dispat ceph cwd unknown /proc/67972/task/74808/cwd (readlink: Permission denied) 2024-05-09T04:58:11.863 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74808 ms_dispat ceph rtd unknown /proc/67972/task/74808/root (readlink: Permission denied) 2024-05-09T04:58:11.863 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74808 ms_dispat ceph txt unknown /proc/67972/task/74808/exe (readlink: Permission denied) 2024-05-09T04:58:11.863 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74808 ms_dispat ceph NOFD /proc/67972/task/74808/fd (opendir: Permission denied) 2024-05-09T04:58:11.863 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74809 ms_dispat ceph cwd unknown /proc/67972/task/74809/cwd (readlink: Permission denied) 2024-05-09T04:58:11.863 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74809 ms_dispat ceph rtd unknown /proc/67972/task/74809/root (readlink: Permission denied) 2024-05-09T04:58:11.863 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74809 ms_dispat ceph txt unknown /proc/67972/task/74809/exe (readlink: Permission denied) 2024-05-09T04:58:11.863 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74809 ms_dispat ceph NOFD /proc/67972/task/74809/fd (opendir: Permission denied) 2024-05-09T04:58:11.863 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74810 mds_rank_ ceph cwd unknown /proc/67972/task/74810/cwd (readlink: Permission denied) 2024-05-09T04:58:11.864 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74810 mds_rank_ ceph rtd unknown /proc/67972/task/74810/root (readlink: Permission denied) 2024-05-09T04:58:11.864 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74810 mds_rank_ ceph txt unknown /proc/67972/task/74810/exe (readlink: Permission denied) 2024-05-09T04:58:11.864 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74810 mds_rank_ ceph NOFD /proc/67972/task/74810/fd (opendir: Permission denied) 2024-05-09T04:58:11.864 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74811 PQ_Finish ceph cwd unknown /proc/67972/task/74811/cwd (readlink: Permission denied) 2024-05-09T04:58:11.864 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74811 PQ_Finish ceph rtd unknown /proc/67972/task/74811/root (readlink: Permission denied) 2024-05-09T04:58:11.864 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74811 PQ_Finish ceph txt unknown /proc/67972/task/74811/exe (readlink: Permission denied) 2024-05-09T04:58:11.864 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74811 PQ_Finish ceph NOFD /proc/67972/task/74811/fd (opendir: Permission denied) 2024-05-09T04:58:11.864 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74812 safe_time ceph cwd unknown /proc/67972/task/74812/cwd (readlink: Permission denied) 2024-05-09T04:58:11.864 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74812 safe_time ceph rtd unknown /proc/67972/task/74812/root (readlink: Permission denied) 2024-05-09T04:58:11.864 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74812 safe_time ceph txt unknown /proc/67972/task/74812/exe (readlink: Permission denied) 2024-05-09T04:58:11.864 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74812 safe_time ceph NOFD /proc/67972/task/74812/fd (opendir: Permission denied) 2024-05-09T04:58:11.865 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74813 MR_Finish ceph cwd unknown /proc/67972/task/74813/cwd (readlink: Permission denied) 2024-05-09T04:58:11.865 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74813 MR_Finish ceph rtd unknown /proc/67972/task/74813/root (readlink: Permission denied) 2024-05-09T04:58:11.865 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74813 MR_Finish ceph txt unknown /proc/67972/task/74813/exe (readlink: Permission denied) 2024-05-09T04:58:11.865 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74813 MR_Finish ceph NOFD /proc/67972/task/74813/fd (opendir: Permission denied) 2024-05-09T04:58:11.865 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74815 md_submit ceph cwd unknown /proc/67972/task/74815/cwd (readlink: Permission denied) 2024-05-09T04:58:11.865 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74815 md_submit ceph rtd unknown /proc/67972/task/74815/root (readlink: Permission denied) 2024-05-09T04:58:11.865 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74815 md_submit ceph txt unknown /proc/67972/task/74815/exe (readlink: Permission denied) 2024-05-09T04:58:11.865 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 67972 74815 md_submit ceph NOFD /proc/67972/task/74815/fd (opendir: Permission denied) 2024-05-09T04:58:11.865 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/0 68366 root cwd unknown /proc/68366/cwd (readlink: Permission denied) 2024-05-09T04:58:11.865 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/0 68366 root rtd unknown /proc/68366/root (readlink: Permission denied) 2024-05-09T04:58:11.865 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/0 68366 root txt unknown /proc/68366/exe (readlink: Permission denied) 2024-05-09T04:58:11.865 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/0 68366 root NOFD /proc/68366/fd (opendir: Permission denied) 2024-05-09T04:58:11.865 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 68748 root cwd unknown /proc/68748/cwd (readlink: Permission denied) 2024-05-09T04:58:11.865 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 68748 root rtd unknown /proc/68748/root (readlink: Permission denied) 2024-05-09T04:58:11.865 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 68748 root txt unknown /proc/68748/exe (readlink: Permission denied) 2024-05-09T04:58:11.866 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 68748 root NOFD /proc/68748/fd (opendir: Permission denied) 2024-05-09T04:58:11.866 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 68750 root cwd unknown /proc/68750/cwd (readlink: Permission denied) 2024-05-09T04:58:11.866 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 68750 root rtd unknown /proc/68750/root (readlink: Permission denied) 2024-05-09T04:58:11.866 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 68750 root txt unknown /proc/68750/exe (readlink: Permission denied) 2024-05-09T04:58:11.866 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 68750 root NOFD /proc/68750/fd (opendir: Permission denied) 2024-05-09T04:58:11.866 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 ceph cwd unknown /proc/68756/cwd (readlink: Permission denied) 2024-05-09T04:58:11.866 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 ceph rtd unknown /proc/68756/root (readlink: Permission denied) 2024-05-09T04:58:11.866 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 ceph txt unknown /proc/68756/exe (readlink: Permission denied) 2024-05-09T04:58:11.866 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 ceph NOFD /proc/68756/fd (opendir: Permission denied) 2024-05-09T04:58:11.866 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68757 log ceph cwd unknown /proc/68756/task/68757/cwd (readlink: Permission denied) 2024-05-09T04:58:11.866 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68757 log ceph rtd unknown /proc/68756/task/68757/root (readlink: Permission denied) 2024-05-09T04:58:11.866 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68757 log ceph txt unknown /proc/68756/task/68757/exe (readlink: Permission denied) 2024-05-09T04:58:11.866 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68757 log ceph NOFD /proc/68756/task/68757/fd (opendir: Permission denied) 2024-05-09T04:58:11.866 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68759 msgr-work ceph cwd unknown /proc/68756/task/68759/cwd (readlink: Permission denied) 2024-05-09T04:58:11.866 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68759 msgr-work ceph rtd unknown /proc/68756/task/68759/root (readlink: Permission denied) 2024-05-09T04:58:11.867 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68759 msgr-work ceph txt unknown /proc/68756/task/68759/exe (readlink: Permission denied) 2024-05-09T04:58:11.867 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68759 msgr-work ceph NOFD /proc/68756/task/68759/fd (opendir: Permission denied) 2024-05-09T04:58:11.867 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68760 msgr-work ceph cwd unknown /proc/68756/task/68760/cwd (readlink: Permission denied) 2024-05-09T04:58:11.867 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68760 msgr-work ceph rtd unknown /proc/68756/task/68760/root (readlink: Permission denied) 2024-05-09T04:58:11.867 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68760 msgr-work ceph txt unknown /proc/68756/task/68760/exe (readlink: Permission denied) 2024-05-09T04:58:11.867 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68760 msgr-work ceph NOFD /proc/68756/task/68760/fd (opendir: Permission denied) 2024-05-09T04:58:11.867 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68761 msgr-work ceph cwd unknown /proc/68756/task/68761/cwd (readlink: Permission denied) 2024-05-09T04:58:11.867 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68761 msgr-work ceph rtd unknown /proc/68756/task/68761/root (readlink: Permission denied) 2024-05-09T04:58:11.867 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68761 msgr-work ceph txt unknown /proc/68756/task/68761/exe (readlink: Permission denied) 2024-05-09T04:58:11.867 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68761 msgr-work ceph NOFD /proc/68756/task/68761/fd (opendir: Permission denied) 2024-05-09T04:58:11.867 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68767 service ceph cwd unknown /proc/68756/task/68767/cwd (readlink: Permission denied) 2024-05-09T04:58:11.867 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68767 service ceph rtd unknown /proc/68756/task/68767/root (readlink: Permission denied) 2024-05-09T04:58:11.867 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68767 service ceph txt unknown /proc/68756/task/68767/exe (readlink: Permission denied) 2024-05-09T04:58:11.867 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68767 service ceph NOFD /proc/68756/task/68767/fd (opendir: Permission denied) 2024-05-09T04:58:11.867 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68768 admin_soc ceph cwd unknown /proc/68756/task/68768/cwd (readlink: Permission denied) 2024-05-09T04:58:11.867 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68768 admin_soc ceph rtd unknown /proc/68756/task/68768/root (readlink: Permission denied) 2024-05-09T04:58:11.868 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68768 admin_soc ceph txt unknown /proc/68756/task/68768/exe (readlink: Permission denied) 2024-05-09T04:58:11.868 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68768 admin_soc ceph NOFD /proc/68756/task/68768/fd (opendir: Permission denied) 2024-05-09T04:58:11.868 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68769 signal_ha ceph cwd unknown /proc/68756/task/68769/cwd (readlink: Permission denied) 2024-05-09T04:58:11.868 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68769 signal_ha ceph rtd unknown /proc/68756/task/68769/root (readlink: Permission denied) 2024-05-09T04:58:11.868 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68769 signal_ha ceph txt unknown /proc/68756/task/68769/exe (readlink: Permission denied) 2024-05-09T04:58:11.868 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68769 signal_ha ceph NOFD /proc/68756/task/68769/fd (opendir: Permission denied) 2024-05-09T04:58:11.868 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68770 io_contex ceph cwd unknown /proc/68756/task/68770/cwd (readlink: Permission denied) 2024-05-09T04:58:11.868 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68770 io_contex ceph rtd unknown /proc/68756/task/68770/root (readlink: Permission denied) 2024-05-09T04:58:11.868 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68770 io_contex ceph txt unknown /proc/68756/task/68770/exe (readlink: Permission denied) 2024-05-09T04:58:11.868 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68770 io_contex ceph NOFD /proc/68756/task/68770/fd (opendir: Permission denied) 2024-05-09T04:58:11.868 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68771 io_contex ceph cwd unknown /proc/68756/task/68771/cwd (readlink: Permission denied) 2024-05-09T04:58:11.868 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68771 io_contex ceph rtd unknown /proc/68756/task/68771/root (readlink: Permission denied) 2024-05-09T04:58:11.868 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68771 io_contex ceph txt unknown /proc/68756/task/68771/exe (readlink: Permission denied) 2024-05-09T04:58:11.868 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68771 io_contex ceph NOFD /proc/68756/task/68771/fd (opendir: Permission denied) 2024-05-09T04:58:11.868 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68772 ms_dispat ceph cwd unknown /proc/68756/task/68772/cwd (readlink: Permission denied) 2024-05-09T04:58:11.869 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68772 ms_dispat ceph rtd unknown /proc/68756/task/68772/root (readlink: Permission denied) 2024-05-09T04:58:11.869 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68772 ms_dispat ceph txt unknown /proc/68756/task/68772/exe (readlink: Permission denied) 2024-05-09T04:58:11.869 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68772 ms_dispat ceph NOFD /proc/68756/task/68772/fd (opendir: Permission denied) 2024-05-09T04:58:11.869 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68773 ms_local ceph cwd unknown /proc/68756/task/68773/cwd (readlink: Permission denied) 2024-05-09T04:58:11.869 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68773 ms_local ceph rtd unknown /proc/68756/task/68773/root (readlink: Permission denied) 2024-05-09T04:58:11.869 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68773 ms_local ceph txt unknown /proc/68756/task/68773/exe (readlink: Permission denied) 2024-05-09T04:58:11.869 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68773 ms_local ceph NOFD /proc/68756/task/68773/fd (opendir: Permission denied) 2024-05-09T04:58:11.869 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68774 safe_time ceph cwd unknown /proc/68756/task/68774/cwd (readlink: Permission denied) 2024-05-09T04:58:11.869 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68774 safe_time ceph rtd unknown /proc/68756/task/68774/root (readlink: Permission denied) 2024-05-09T04:58:11.869 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68774 safe_time ceph txt unknown /proc/68756/task/68774/exe (readlink: Permission denied) 2024-05-09T04:58:11.869 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68774 safe_time ceph NOFD /proc/68756/task/68774/fd (opendir: Permission denied) 2024-05-09T04:58:11.869 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68775 safe_time ceph cwd unknown /proc/68756/task/68775/cwd (readlink: Permission denied) 2024-05-09T04:58:11.869 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68775 safe_time ceph rtd unknown /proc/68756/task/68775/root (readlink: Permission denied) 2024-05-09T04:58:11.869 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68775 safe_time ceph txt unknown /proc/68756/task/68775/exe (readlink: Permission denied) 2024-05-09T04:58:11.869 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68775 safe_time ceph NOFD /proc/68756/task/68775/fd (opendir: Permission denied) 2024-05-09T04:58:11.869 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68776 ceph-mds ceph cwd unknown /proc/68756/task/68776/cwd (readlink: Permission denied) 2024-05-09T04:58:11.870 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68776 ceph-mds ceph rtd unknown /proc/68756/task/68776/root (readlink: Permission denied) 2024-05-09T04:58:11.870 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68776 ceph-mds ceph txt unknown /proc/68756/task/68776/exe (readlink: Permission denied) 2024-05-09T04:58:11.870 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68776 ceph-mds ceph NOFD /proc/68756/task/68776/fd (opendir: Permission denied) 2024-05-09T04:58:11.870 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68891 safe_time ceph cwd unknown /proc/68756/task/68891/cwd (readlink: Permission denied) 2024-05-09T04:58:11.870 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68891 safe_time ceph rtd unknown /proc/68756/task/68891/root (readlink: Permission denied) 2024-05-09T04:58:11.870 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68891 safe_time ceph txt unknown /proc/68756/task/68891/exe (readlink: Permission denied) 2024-05-09T04:58:11.870 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 68891 safe_time ceph NOFD /proc/68756/task/68891/fd (opendir: Permission denied) 2024-05-09T04:58:11.870 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74638 ceph_time ceph cwd unknown /proc/68756/task/74638/cwd (readlink: Permission denied) 2024-05-09T04:58:11.870 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74638 ceph_time ceph rtd unknown /proc/68756/task/74638/root (readlink: Permission denied) 2024-05-09T04:58:11.870 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74638 ceph_time ceph txt unknown /proc/68756/task/74638/exe (readlink: Permission denied) 2024-05-09T04:58:11.870 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74638 ceph_time ceph NOFD /proc/68756/task/74638/fd (opendir: Permission denied) 2024-05-09T04:58:11.870 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74639 OpHistory ceph cwd unknown /proc/68756/task/74639/cwd (readlink: Permission denied) 2024-05-09T04:58:11.870 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74639 OpHistory ceph rtd unknown /proc/68756/task/74639/root (readlink: Permission denied) 2024-05-09T04:58:11.871 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74639 OpHistory ceph txt unknown /proc/68756/task/74639/exe (readlink: Permission denied) 2024-05-09T04:58:11.871 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74639 OpHistory ceph NOFD /proc/68756/task/74639/fd (opendir: Permission denied) 2024-05-09T04:58:11.871 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74640 ms_dispat ceph cwd unknown /proc/68756/task/74640/cwd (readlink: Permission denied) 2024-05-09T04:58:11.871 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74640 ms_dispat ceph rtd unknown /proc/68756/task/74640/root (readlink: Permission denied) 2024-05-09T04:58:11.871 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74640 ms_dispat ceph txt unknown /proc/68756/task/74640/exe (readlink: Permission denied) 2024-05-09T04:58:11.871 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74640 ms_dispat ceph NOFD /proc/68756/task/74640/fd (opendir: Permission denied) 2024-05-09T04:58:11.871 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74641 ms_dispat ceph cwd unknown /proc/68756/task/74641/cwd (readlink: Permission denied) 2024-05-09T04:58:11.871 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74641 ms_dispat ceph rtd unknown /proc/68756/task/74641/root (readlink: Permission denied) 2024-05-09T04:58:11.871 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74641 ms_dispat ceph txt unknown /proc/68756/task/74641/exe (readlink: Permission denied) 2024-05-09T04:58:11.871 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74641 ms_dispat ceph NOFD /proc/68756/task/74641/fd (opendir: Permission denied) 2024-05-09T04:58:11.871 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74642 mds_rank_ ceph cwd unknown /proc/68756/task/74642/cwd (readlink: Permission denied) 2024-05-09T04:58:11.871 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74642 mds_rank_ ceph rtd unknown /proc/68756/task/74642/root (readlink: Permission denied) 2024-05-09T04:58:11.871 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74642 mds_rank_ ceph txt unknown /proc/68756/task/74642/exe (readlink: Permission denied) 2024-05-09T04:58:11.871 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74642 mds_rank_ ceph NOFD /proc/68756/task/74642/fd (opendir: Permission denied) 2024-05-09T04:58:11.872 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74643 PQ_Finish ceph cwd unknown /proc/68756/task/74643/cwd (readlink: Permission denied) 2024-05-09T04:58:11.872 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74643 PQ_Finish ceph rtd unknown /proc/68756/task/74643/root (readlink: Permission denied) 2024-05-09T04:58:11.872 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74643 PQ_Finish ceph txt unknown /proc/68756/task/74643/exe (readlink: Permission denied) 2024-05-09T04:58:11.872 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74643 PQ_Finish ceph NOFD /proc/68756/task/74643/fd (opendir: Permission denied) 2024-05-09T04:58:11.872 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74644 safe_time ceph cwd unknown /proc/68756/task/74644/cwd (readlink: Permission denied) 2024-05-09T04:58:11.872 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74644 safe_time ceph rtd unknown /proc/68756/task/74644/root (readlink: Permission denied) 2024-05-09T04:58:11.872 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74644 safe_time ceph txt unknown /proc/68756/task/74644/exe (readlink: Permission denied) 2024-05-09T04:58:11.872 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74644 safe_time ceph NOFD /proc/68756/task/74644/fd (opendir: Permission denied) 2024-05-09T04:58:11.872 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74645 MR_Finish ceph cwd unknown /proc/68756/task/74645/cwd (readlink: Permission denied) 2024-05-09T04:58:11.872 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74645 MR_Finish ceph rtd unknown /proc/68756/task/74645/root (readlink: Permission denied) 2024-05-09T04:58:11.872 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74645 MR_Finish ceph txt unknown /proc/68756/task/74645/exe (readlink: Permission denied) 2024-05-09T04:58:11.872 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74645 MR_Finish ceph NOFD /proc/68756/task/74645/fd (opendir: Permission denied) 2024-05-09T04:58:11.872 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74646 md_submit ceph cwd unknown /proc/68756/task/74646/cwd (readlink: Permission denied) 2024-05-09T04:58:11.872 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74646 md_submit ceph rtd unknown /proc/68756/task/74646/root (readlink: Permission denied) 2024-05-09T04:58:11.873 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74646 md_submit ceph txt unknown /proc/68756/task/74646/exe (readlink: Permission denied) 2024-05-09T04:58:11.873 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74646 md_submit ceph NOFD /proc/68756/task/74646/fd (opendir: Permission denied) 2024-05-09T04:58:11.873 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74680 ms_dispat ceph cwd unknown /proc/68756/task/74680/cwd (readlink: Permission denied) 2024-05-09T04:58:11.873 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74680 ms_dispat ceph rtd unknown /proc/68756/task/74680/root (readlink: Permission denied) 2024-05-09T04:58:11.873 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74680 ms_dispat ceph txt unknown /proc/68756/task/74680/exe (readlink: Permission denied) 2024-05-09T04:58:11.873 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74680 ms_dispat ceph NOFD /proc/68756/task/74680/fd (opendir: Permission denied) 2024-05-09T04:58:11.873 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74681 quiesce.a ceph cwd unknown /proc/68756/task/74681/cwd (readlink: Permission denied) 2024-05-09T04:58:11.873 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74681 quiesce.a ceph rtd unknown /proc/68756/task/74681/root (readlink: Permission denied) 2024-05-09T04:58:11.873 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74681 quiesce.a ceph txt unknown /proc/68756/task/74681/exe (readlink: Permission denied) 2024-05-09T04:58:11.873 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74681 quiesce.a ceph NOFD /proc/68756/task/74681/fd (opendir: Permission denied) 2024-05-09T04:58:11.873 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74682 quiesce_d ceph cwd unknown /proc/68756/task/74682/cwd (readlink: Permission denied) 2024-05-09T04:58:11.873 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74682 quiesce_d ceph rtd unknown /proc/68756/task/74682/root (readlink: Permission denied) 2024-05-09T04:58:11.873 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74682 quiesce_d ceph txt unknown /proc/68756/task/74682/exe (readlink: Permission denied) 2024-05-09T04:58:11.873 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 68756 74682 quiesce_d ceph NOFD /proc/68756/task/74682/fd (opendir: Permission denied) 2024-05-09T04:58:11.873 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 69196 root cwd unknown /proc/69196/cwd (readlink: Permission denied) 2024-05-09T04:58:11.874 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 69196 root rtd unknown /proc/69196/root (readlink: Permission denied) 2024-05-09T04:58:11.874 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 69196 root txt unknown /proc/69196/exe (readlink: Permission denied) 2024-05-09T04:58:11.874 INFO:tasks.workunit.client.0.smithi006.stdout:conmon 69196 root NOFD /proc/69196/fd (opendir: Permission denied) 2024-05-09T04:58:11.874 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 69198 root cwd unknown /proc/69198/cwd (readlink: Permission denied) 2024-05-09T04:58:11.874 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 69198 root rtd unknown /proc/69198/root (readlink: Permission denied) 2024-05-09T04:58:11.874 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 69198 root txt unknown /proc/69198/exe (readlink: Permission denied) 2024-05-09T04:58:11.874 INFO:tasks.workunit.client.0.smithi006.stdout:podman-in 69198 root NOFD /proc/69198/fd (opendir: Permission denied) 2024-05-09T04:58:11.874 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 ceph cwd unknown /proc/69201/cwd (readlink: Permission denied) 2024-05-09T04:58:11.874 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 ceph rtd unknown /proc/69201/root (readlink: Permission denied) 2024-05-09T04:58:11.874 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 ceph txt unknown /proc/69201/exe (readlink: Permission denied) 2024-05-09T04:58:11.874 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 ceph NOFD /proc/69201/fd (opendir: Permission denied) 2024-05-09T04:58:11.874 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69204 log ceph cwd unknown /proc/69201/task/69204/cwd (readlink: Permission denied) 2024-05-09T04:58:11.874 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69204 log ceph rtd unknown /proc/69201/task/69204/root (readlink: Permission denied) 2024-05-09T04:58:11.874 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69204 log ceph txt unknown /proc/69201/task/69204/exe (readlink: Permission denied) 2024-05-09T04:58:11.874 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69204 log ceph NOFD /proc/69201/task/69204/fd (opendir: Permission denied) 2024-05-09T04:58:11.875 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69206 msgr-work ceph cwd unknown /proc/69201/task/69206/cwd (readlink: Permission denied) 2024-05-09T04:58:11.875 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69206 msgr-work ceph rtd unknown /proc/69201/task/69206/root (readlink: Permission denied) 2024-05-09T04:58:11.875 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69206 msgr-work ceph txt unknown /proc/69201/task/69206/exe (readlink: Permission denied) 2024-05-09T04:58:11.875 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69206 msgr-work ceph NOFD /proc/69201/task/69206/fd (opendir: Permission denied) 2024-05-09T04:58:11.875 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69207 msgr-work ceph cwd unknown /proc/69201/task/69207/cwd (readlink: Permission denied) 2024-05-09T04:58:11.875 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69207 msgr-work ceph rtd unknown /proc/69201/task/69207/root (readlink: Permission denied) 2024-05-09T04:58:11.875 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69207 msgr-work ceph txt unknown /proc/69201/task/69207/exe (readlink: Permission denied) 2024-05-09T04:58:11.875 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69207 msgr-work ceph NOFD /proc/69201/task/69207/fd (opendir: Permission denied) 2024-05-09T04:58:11.875 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69208 msgr-work ceph cwd unknown /proc/69201/task/69208/cwd (readlink: Permission denied) 2024-05-09T04:58:11.875 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69208 msgr-work ceph rtd unknown /proc/69201/task/69208/root (readlink: Permission denied) 2024-05-09T04:58:11.875 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69208 msgr-work ceph txt unknown /proc/69201/task/69208/exe (readlink: Permission denied) 2024-05-09T04:58:11.875 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69208 msgr-work ceph NOFD /proc/69201/task/69208/fd (opendir: Permission denied) 2024-05-09T04:58:11.875 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69212 service ceph cwd unknown /proc/69201/task/69212/cwd (readlink: Permission denied) 2024-05-09T04:58:11.875 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69212 service ceph rtd unknown /proc/69201/task/69212/root (readlink: Permission denied) 2024-05-09T04:58:11.876 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69212 service ceph txt unknown /proc/69201/task/69212/exe (readlink: Permission denied) 2024-05-09T04:58:11.876 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69212 service ceph NOFD /proc/69201/task/69212/fd (opendir: Permission denied) 2024-05-09T04:58:11.876 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69213 admin_soc ceph cwd unknown /proc/69201/task/69213/cwd (readlink: Permission denied) 2024-05-09T04:58:11.876 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69213 admin_soc ceph rtd unknown /proc/69201/task/69213/root (readlink: Permission denied) 2024-05-09T04:58:11.876 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69213 admin_soc ceph txt unknown /proc/69201/task/69213/exe (readlink: Permission denied) 2024-05-09T04:58:11.876 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69213 admin_soc ceph NOFD /proc/69201/task/69213/fd (opendir: Permission denied) 2024-05-09T04:58:11.876 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69214 signal_ha ceph cwd unknown /proc/69201/task/69214/cwd (readlink: Permission denied) 2024-05-09T04:58:11.876 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69214 signal_ha ceph rtd unknown /proc/69201/task/69214/root (readlink: Permission denied) 2024-05-09T04:58:11.876 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69214 signal_ha ceph txt unknown /proc/69201/task/69214/exe (readlink: Permission denied) 2024-05-09T04:58:11.876 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69214 signal_ha ceph NOFD /proc/69201/task/69214/fd (opendir: Permission denied) 2024-05-09T04:58:11.876 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69215 io_contex ceph cwd unknown /proc/69201/task/69215/cwd (readlink: Permission denied) 2024-05-09T04:58:11.876 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69215 io_contex ceph rtd unknown /proc/69201/task/69215/root (readlink: Permission denied) 2024-05-09T04:58:11.876 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69215 io_contex ceph txt unknown /proc/69201/task/69215/exe (readlink: Permission denied) 2024-05-09T04:58:11.876 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69215 io_contex ceph NOFD /proc/69201/task/69215/fd (opendir: Permission denied) 2024-05-09T04:58:11.876 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69216 io_contex ceph cwd unknown /proc/69201/task/69216/cwd (readlink: Permission denied) 2024-05-09T04:58:11.876 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69216 io_contex ceph rtd unknown /proc/69201/task/69216/root (readlink: Permission denied) 2024-05-09T04:58:11.877 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69216 io_contex ceph txt unknown /proc/69201/task/69216/exe (readlink: Permission denied) 2024-05-09T04:58:11.877 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69216 io_contex ceph NOFD /proc/69201/task/69216/fd (opendir: Permission denied) 2024-05-09T04:58:11.877 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69217 ms_dispat ceph cwd unknown /proc/69201/task/69217/cwd (readlink: Permission denied) 2024-05-09T04:58:11.877 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69217 ms_dispat ceph rtd unknown /proc/69201/task/69217/root (readlink: Permission denied) 2024-05-09T04:58:11.877 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69217 ms_dispat ceph txt unknown /proc/69201/task/69217/exe (readlink: Permission denied) 2024-05-09T04:58:11.877 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69217 ms_dispat ceph NOFD /proc/69201/task/69217/fd (opendir: Permission denied) 2024-05-09T04:58:11.877 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69218 ms_local ceph cwd unknown /proc/69201/task/69218/cwd (readlink: Permission denied) 2024-05-09T04:58:11.877 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69218 ms_local ceph rtd unknown /proc/69201/task/69218/root (readlink: Permission denied) 2024-05-09T04:58:11.877 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69218 ms_local ceph txt unknown /proc/69201/task/69218/exe (readlink: Permission denied) 2024-05-09T04:58:11.877 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69218 ms_local ceph NOFD /proc/69201/task/69218/fd (opendir: Permission denied) 2024-05-09T04:58:11.877 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69219 safe_time ceph cwd unknown /proc/69201/task/69219/cwd (readlink: Permission denied) 2024-05-09T04:58:11.877 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69219 safe_time ceph rtd unknown /proc/69201/task/69219/root (readlink: Permission denied) 2024-05-09T04:58:11.877 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69219 safe_time ceph txt unknown /proc/69201/task/69219/exe (readlink: Permission denied) 2024-05-09T04:58:11.877 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69219 safe_time ceph NOFD /proc/69201/task/69219/fd (opendir: Permission denied) 2024-05-09T04:58:11.878 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69220 safe_time ceph cwd unknown /proc/69201/task/69220/cwd (readlink: Permission denied) 2024-05-09T04:58:11.878 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69220 safe_time ceph rtd unknown /proc/69201/task/69220/root (readlink: Permission denied) 2024-05-09T04:58:11.878 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69220 safe_time ceph txt unknown /proc/69201/task/69220/exe (readlink: Permission denied) 2024-05-09T04:58:11.878 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69220 safe_time ceph NOFD /proc/69201/task/69220/fd (opendir: Permission denied) 2024-05-09T04:58:11.878 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69221 ceph-mds ceph cwd unknown /proc/69201/task/69221/cwd (readlink: Permission denied) 2024-05-09T04:58:11.878 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69221 ceph-mds ceph rtd unknown /proc/69201/task/69221/root (readlink: Permission denied) 2024-05-09T04:58:11.878 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69221 ceph-mds ceph txt unknown /proc/69201/task/69221/exe (readlink: Permission denied) 2024-05-09T04:58:11.878 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69221 ceph-mds ceph NOFD /proc/69201/task/69221/fd (opendir: Permission denied) 2024-05-09T04:58:11.878 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69253 safe_time ceph cwd unknown /proc/69201/task/69253/cwd (readlink: Permission denied) 2024-05-09T04:58:11.878 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69253 safe_time ceph rtd unknown /proc/69201/task/69253/root (readlink: Permission denied) 2024-05-09T04:58:11.878 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69253 safe_time ceph txt unknown /proc/69201/task/69253/exe (readlink: Permission denied) 2024-05-09T04:58:11.878 INFO:tasks.workunit.client.0.smithi006.stdout:ceph-mds 69201 69253 safe_time ceph NOFD /proc/69201/task/69253/fd (opendir: Permission denied) 2024-05-09T04:58:11.878 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/2 70222 root cwd unknown /proc/70222/cwd (readlink: Permission denied) 2024-05-09T04:58:11.878 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/2 70222 root rtd unknown /proc/70222/root (readlink: Permission denied) 2024-05-09T04:58:11.879 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/2 70222 root txt unknown /proc/70222/exe (readlink: Permission denied) 2024-05-09T04:58:11.879 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/2 70222 root NOFD /proc/70222/fd (opendir: Permission denied) 2024-05-09T04:58:11.879 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/7 73537 root cwd unknown /proc/73537/cwd (readlink: Permission denied) 2024-05-09T04:58:11.879 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/7 73537 root rtd unknown /proc/73537/root (readlink: Permission denied) 2024-05-09T04:58:11.879 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/7 73537 root txt unknown /proc/73537/exe (readlink: Permission denied) 2024-05-09T04:58:11.879 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/7 73537 root NOFD /proc/73537/fd (opendir: Permission denied) 2024-05-09T04:58:11.879 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/3 74580 root cwd unknown /proc/74580/cwd (readlink: Permission denied) 2024-05-09T04:58:11.879 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/3 74580 root rtd unknown /proc/74580/root (readlink: Permission denied) 2024-05-09T04:58:11.879 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/3 74580 root txt unknown /proc/74580/exe (readlink: Permission denied) 2024-05-09T04:58:11.879 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/3 74580 root NOFD /proc/74580/fd (opendir: Permission denied) 2024-05-09T04:58:11.879 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/6 80102 root cwd unknown /proc/80102/cwd (readlink: Permission denied) 2024-05-09T04:58:11.879 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/6 80102 root rtd unknown /proc/80102/root (readlink: Permission denied) 2024-05-09T04:58:11.879 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/6 80102 root txt unknown /proc/80102/exe (readlink: Permission denied) 2024-05-09T04:58:11.879 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/6 80102 root NOFD /proc/80102/fd (opendir: Permission denied) 2024-05-09T04:58:11.880 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/5 81183 root cwd unknown /proc/81183/cwd (readlink: Permission denied) 2024-05-09T04:58:11.880 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/5 81183 root rtd unknown /proc/81183/root (readlink: Permission denied) 2024-05-09T04:58:11.880 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/5 81183 root txt unknown /proc/81183/exe (readlink: Permission denied) 2024-05-09T04:58:11.880 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/5 81183 root NOFD /proc/81183/fd (opendir: Permission denied) 2024-05-09T04:58:11.880 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/2 81510 root cwd unknown /proc/81510/cwd (readlink: Permission denied) 2024-05-09T04:58:11.880 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/2 81510 root rtd unknown /proc/81510/root (readlink: Permission denied) 2024-05-09T04:58:11.880 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/2 81510 root txt unknown /proc/81510/exe (readlink: Permission denied) 2024-05-09T04:58:11.880 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/2 81510 root NOFD /proc/81510/fd (opendir: Permission denied) 2024-05-09T04:58:11.880 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/5 84251 root cwd unknown /proc/84251/cwd (readlink: Permission denied) 2024-05-09T04:58:11.880 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/5 84251 root rtd unknown /proc/84251/root (readlink: Permission denied) 2024-05-09T04:58:11.880 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/5 84251 root txt unknown /proc/84251/exe (readlink: Permission denied) 2024-05-09T04:58:11.880 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/5 84251 root NOFD /proc/84251/fd (opendir: Permission denied) 2024-05-09T04:58:11.880 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/1 85044 root cwd unknown /proc/85044/cwd (readlink: Permission denied) 2024-05-09T04:58:11.880 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/1 85044 root rtd unknown /proc/85044/root (readlink: Permission denied) 2024-05-09T04:58:11.880 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/1 85044 root txt unknown /proc/85044/exe (readlink: Permission denied) 2024-05-09T04:58:11.881 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/1 85044 root NOFD /proc/85044/fd (opendir: Permission denied) 2024-05-09T04:58:11.881 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/0 85464 root cwd unknown /proc/85464/cwd (readlink: Permission denied) 2024-05-09T04:58:11.881 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/0 85464 root rtd unknown /proc/85464/root (readlink: Permission denied) 2024-05-09T04:58:11.881 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/0 85464 root txt unknown /proc/85464/exe (readlink: Permission denied) 2024-05-09T04:58:11.881 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/0 85464 root NOFD /proc/85464/fd (opendir: Permission denied) 2024-05-09T04:58:11.881 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/0 85472 root cwd unknown /proc/85472/cwd (readlink: Permission denied) 2024-05-09T04:58:11.881 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/0 85472 root rtd unknown /proc/85472/root (readlink: Permission denied) 2024-05-09T04:58:11.881 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/0 85472 root txt unknown /proc/85472/exe (readlink: Permission denied) 2024-05-09T04:58:11.881 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/0 85472 root NOFD /proc/85472/fd (opendir: Permission denied) 2024-05-09T04:58:11.881 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 86674 root cwd unknown /proc/86674/cwd (readlink: Permission denied) 2024-05-09T04:58:11.881 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 86674 root rtd unknown /proc/86674/root (readlink: Permission denied) 2024-05-09T04:58:11.881 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 86674 root txt unknown /proc/86674/exe (readlink: Permission denied) 2024-05-09T04:58:11.881 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 86674 root NOFD /proc/86674/fd (opendir: Permission denied) 2024-05-09T04:58:11.881 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 86689 root cwd unknown /proc/86689/cwd (readlink: Permission denied) 2024-05-09T04:58:11.882 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 86689 root rtd unknown /proc/86689/root (readlink: Permission denied) 2024-05-09T04:58:11.882 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 86689 root txt unknown /proc/86689/exe (readlink: Permission denied) 2024-05-09T04:58:11.882 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 86689 root NOFD /proc/86689/fd (opendir: Permission denied) 2024-05-09T04:58:11.882 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 86690 root cwd unknown /proc/86690/cwd (readlink: Permission denied) 2024-05-09T04:58:11.882 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 86690 root rtd unknown /proc/86690/root (readlink: Permission denied) 2024-05-09T04:58:11.882 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 86690 root txt unknown /proc/86690/exe (readlink: Permission denied) 2024-05-09T04:58:11.882 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/R 86690 root NOFD /proc/86690/fd (opendir: Permission denied) 2024-05-09T04:58:11.882 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/4 88458 root cwd unknown /proc/88458/cwd (readlink: Permission denied) 2024-05-09T04:58:11.882 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/4 88458 root rtd unknown /proc/88458/root (readlink: Permission denied) 2024-05-09T04:58:11.882 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/4 88458 root txt unknown /proc/88458/exe (readlink: Permission denied) 2024-05-09T04:58:11.882 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/4 88458 root NOFD /proc/88458/fd (opendir: Permission denied) 2024-05-09T04:58:11.882 INFO:tasks.workunit.client.0.smithi006.stdout:timeout 90462 ubuntu cwd DIR 0,114 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-09T04:58:11.883 INFO:tasks.workunit.client.0.smithi006.stdout:timeout 90462 ubuntu rtd DIR 8,1 4096 2 / 2024-05-09T04:58:11.883 INFO:tasks.workunit.client.0.smithi006.stdout:timeout 90462 ubuntu txt REG 8,1 41144 7580 /usr/bin/timeout 2024-05-09T04:58:11.883 INFO:tasks.workunit.client.0.smithi006.stdout:timeout 90462 ubuntu mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-05-09T04:58:11.883 INFO:tasks.workunit.client.0.smithi006.stdout:timeout 90462 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-05-09T04:58:11.883 INFO:tasks.workunit.client.0.smithi006.stdout:timeout 90462 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-05-09T04:58:11.883 INFO:tasks.workunit.client.0.smithi006.stdout:timeout 90462 ubuntu mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-05-09T04:58:11.883 INFO:tasks.workunit.client.0.smithi006.stdout:timeout 90462 ubuntu mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-05-09T04:58:11.883 INFO:tasks.workunit.client.0.smithi006.stdout:timeout 90462 ubuntu mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-05-09T04:58:11.883 INFO:tasks.workunit.client.0.smithi006.stdout:timeout 90462 ubuntu mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-05-09T04:58:11.883 INFO:tasks.workunit.client.0.smithi006.stdout:timeout 90462 ubuntu mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-05-09T04:58:11.883 INFO:tasks.workunit.client.0.smithi006.stdout:timeout 90462 ubuntu mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-05-09T04:58:11.883 INFO:tasks.workunit.client.0.smithi006.stdout:timeout 90462 ubuntu mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-05-09T04:58:11.883 INFO:tasks.workunit.client.0.smithi006.stdout:timeout 90462 ubuntu mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-05-09T04:58:11.883 INFO:tasks.workunit.client.0.smithi006.stdout:timeout 90462 ubuntu mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-05-09T04:58:11.884 INFO:tasks.workunit.client.0.smithi006.stdout:timeout 90462 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-05-09T04:58:11.884 INFO:tasks.workunit.client.0.smithi006.stdout:timeout 90462 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-05-09T04:58:11.884 INFO:tasks.workunit.client.0.smithi006.stdout:timeout 90462 ubuntu mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-05-09T04:58:11.884 INFO:tasks.workunit.client.0.smithi006.stdout:timeout 90462 ubuntu 0r FIFO 0,14 0t0 253091 pipe 2024-05-09T04:58:11.884 INFO:tasks.workunit.client.0.smithi006.stdout:timeout 90462 ubuntu 1w FIFO 0,14 0t0 253092 pipe 2024-05-09T04:58:11.884 INFO:tasks.workunit.client.0.smithi006.stdout:timeout 90462 ubuntu 2w FIFO 0,14 0t0 253093 pipe 2024-05-09T04:58:11.884 INFO:tasks.workunit.client.0.smithi006.stdout:fsync-tes 90795 ubuntu cwd DIR 0,114 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-09T04:58:11.884 INFO:tasks.workunit.client.0.smithi006.stdout:fsync-tes 90795 ubuntu rtd DIR 8,1 4096 2 / 2024-05-09T04:58:11.884 INFO:tasks.workunit.client.0.smithi006.stdout:fsync-tes 90795 ubuntu txt REG 8,1 1389072 7149 /usr/bin/bash 2024-05-09T04:58:11.884 INFO:tasks.workunit.client.0.smithi006.stdout:fsync-tes 90795 ubuntu mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-05-09T04:58:11.884 INFO:tasks.workunit.client.0.smithi006.stdout:fsync-tes 90795 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-05-09T04:58:11.885 INFO:tasks.workunit.client.0.smithi006.stdout:fsync-tes 90795 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-05-09T04:58:11.885 INFO:tasks.workunit.client.0.smithi006.stdout:fsync-tes 90795 ubuntu mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-05-09T04:58:11.885 INFO:tasks.workunit.client.0.smithi006.stdout:fsync-tes 90795 ubuntu mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-05-09T04:58:11.885 INFO:tasks.workunit.client.0.smithi006.stdout:fsync-tes 90795 ubuntu mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-05-09T04:58:11.885 INFO:tasks.workunit.client.0.smithi006.stdout:fsync-tes 90795 ubuntu mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-05-09T04:58:11.885 INFO:tasks.workunit.client.0.smithi006.stdout:fsync-tes 90795 ubuntu mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-05-09T04:58:11.885 INFO:tasks.workunit.client.0.smithi006.stdout:fsync-tes 90795 ubuntu mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-05-09T04:58:11.885 INFO:tasks.workunit.client.0.smithi006.stdout:fsync-tes 90795 ubuntu mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-05-09T04:58:11.885 INFO:tasks.workunit.client.0.smithi006.stdout:fsync-tes 90795 ubuntu mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-05-09T04:58:11.885 INFO:tasks.workunit.client.0.smithi006.stdout:fsync-tes 90795 ubuntu mem REG 8,1 195080 7147 /usr/lib64/libtinfo.so.6.2 2024-05-09T04:58:11.885 INFO:tasks.workunit.client.0.smithi006.stdout:fsync-tes 90795 ubuntu mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-05-09T04:58:11.885 INFO:tasks.workunit.client.0.smithi006.stdout:fsync-tes 90795 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-05-09T04:58:11.885 INFO:tasks.workunit.client.0.smithi006.stdout:fsync-tes 90795 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-05-09T04:58:11.886 INFO:tasks.workunit.client.0.smithi006.stdout:fsync-tes 90795 ubuntu mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-05-09T04:58:11.886 INFO:tasks.workunit.client.0.smithi006.stdout:fsync-tes 90795 ubuntu 0r FIFO 0,14 0t0 253091 pipe 2024-05-09T04:58:11.886 INFO:tasks.workunit.client.0.smithi006.stdout:fsync-tes 90795 ubuntu 1w FIFO 0,14 0t0 253092 pipe 2024-05-09T04:58:11.886 INFO:tasks.workunit.client.0.smithi006.stdout:fsync-tes 90795 ubuntu 2w FIFO 0,14 0t0 253093 pipe 2024-05-09T04:58:11.886 INFO:tasks.workunit.client.0.smithi006.stdout:fsync-tes 90795 ubuntu 255r REG 8,1 207 661742 /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-05-09T04:58:11.886 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/3 90820 root cwd unknown /proc/90820/cwd (readlink: Permission denied) 2024-05-09T04:58:11.886 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/3 90820 root rtd unknown /proc/90820/root (readlink: Permission denied) 2024-05-09T04:58:11.886 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/3 90820 root txt unknown /proc/90820/exe (readlink: Permission denied) 2024-05-09T04:58:11.886 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/3 90820 root NOFD /proc/90820/fd (opendir: Permission denied) 2024-05-09T04:58:11.886 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/3 90821 root cwd unknown /proc/90821/cwd (readlink: Permission denied) 2024-05-09T04:58:11.886 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/3 90821 root rtd unknown /proc/90821/root (readlink: Permission denied) 2024-05-09T04:58:11.886 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/3 90821 root txt unknown /proc/90821/exe (readlink: Permission denied) 2024-05-09T04:58:11.886 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/3 90821 root NOFD /proc/90821/fd (opendir: Permission denied) 2024-05-09T04:58:11.887 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/u 90833 root cwd unknown /proc/90833/cwd (readlink: Permission denied) 2024-05-09T04:58:11.887 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/u 90833 root rtd unknown /proc/90833/root (readlink: Permission denied) 2024-05-09T04:58:11.887 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/u 90833 root txt unknown /proc/90833/exe (readlink: Permission denied) 2024-05-09T04:58:11.887 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/u 90833 root NOFD /proc/90833/fd (opendir: Permission denied) 2024-05-09T04:58:11.887 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/7 90834 root cwd unknown /proc/90834/cwd (readlink: Permission denied) 2024-05-09T04:58:11.887 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/7 90834 root rtd unknown /proc/90834/root (readlink: Permission denied) 2024-05-09T04:58:11.887 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/7 90834 root txt unknown /proc/90834/exe (readlink: Permission denied) 2024-05-09T04:58:11.887 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/7 90834 root NOFD /proc/90834/fd (opendir: Permission denied) 2024-05-09T04:58:11.887 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/5 90835 root cwd unknown /proc/90835/cwd (readlink: Permission denied) 2024-05-09T04:58:11.887 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/5 90835 root rtd unknown /proc/90835/root (readlink: Permission denied) 2024-05-09T04:58:11.888 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/5 90835 root txt unknown /proc/90835/exe (readlink: Permission denied) 2024-05-09T04:58:11.888 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/5 90835 root NOFD /proc/90835/fd (opendir: Permission denied) 2024-05-09T04:58:11.888 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/4 90836 root cwd unknown /proc/90836/cwd (readlink: Permission denied) 2024-05-09T04:58:11.888 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/4 90836 root rtd unknown /proc/90836/root (readlink: Permission denied) 2024-05-09T04:58:11.888 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/4 90836 root txt unknown /proc/90836/exe (readlink: Permission denied) 2024-05-09T04:58:11.888 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/4 90836 root NOFD /proc/90836/fd (opendir: Permission denied) 2024-05-09T04:58:11.888 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/1 90965 root cwd unknown /proc/90965/cwd (readlink: Permission denied) 2024-05-09T04:58:11.888 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/1 90965 root rtd unknown /proc/90965/root (readlink: Permission denied) 2024-05-09T04:58:11.888 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/1 90965 root txt unknown /proc/90965/exe (readlink: Permission denied) 2024-05-09T04:58:11.888 INFO:tasks.workunit.client.0.smithi006.stdout:kworker/1 90965 root NOFD /proc/90965/fd (opendir: Permission denied) 2024-05-09T04:58:11.888 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90967 ubuntu cwd DIR 0,114 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-09T04:58:11.888 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90967 ubuntu rtd DIR 8,1 4096 2 / 2024-05-09T04:58:11.888 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90967 ubuntu txt REG 8,1 179232 9797 /usr/bin/lsof 2024-05-09T04:58:11.889 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90967 ubuntu mem REG 8,1 5431656 62287 /usr/lib64/libcrypto.so.3.2.1 2024-05-09T04:58:11.889 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90967 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-05-09T04:58:11.889 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90967 ubuntu mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-05-09T04:58:11.889 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90967 ubuntu mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-05-09T04:58:11.889 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90967 ubuntu mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-05-09T04:58:11.889 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90967 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-05-09T04:58:11.889 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90967 ubuntu mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-05-09T04:58:11.889 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90967 ubuntu mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-05-09T04:58:11.889 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90967 ubuntu mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-05-09T04:58:11.889 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90967 ubuntu mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-05-09T04:58:11.890 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90967 ubuntu mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-05-09T04:58:11.890 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90967 ubuntu mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-05-09T04:58:11.890 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90967 ubuntu mem REG 8,1 196224 8914 /usr/lib64/libtirpc.so.3.0.0 2024-05-09T04:58:11.890 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90967 ubuntu mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-05-09T04:58:11.890 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90967 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-05-09T04:58:11.890 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90967 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-05-09T04:58:11.890 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90967 ubuntu 0r FIFO 0,14 0t0 253091 pipe 2024-05-09T04:58:11.890 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90967 ubuntu 1w FIFO 0,14 0t0 253092 pipe 2024-05-09T04:58:11.890 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90967 ubuntu 2w FIFO 0,14 0t0 253093 pipe 2024-05-09T04:58:11.890 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90967 ubuntu 3r DIR 0,21 0 1 /proc 2024-05-09T04:58:11.890 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90967 ubuntu 4r DIR 0,21 7 248700 /proc/90967/fd 2024-05-09T04:58:11.890 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90967 ubuntu 5w FIFO 0,14 0t0 248705 pipe 2024-05-09T04:58:11.890 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90967 ubuntu 6r FIFO 0,14 0t0 248706 pipe 2024-05-09T04:58:11.891 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90968 ubuntu cwd DIR 0,114 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-09T04:58:11.891 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90968 ubuntu rtd DIR 8,1 4096 2 / 2024-05-09T04:58:11.891 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90968 ubuntu txt REG 8,1 179232 9797 /usr/bin/lsof 2024-05-09T04:58:11.891 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90968 ubuntu mem REG 8,1 5431656 62287 /usr/lib64/libcrypto.so.3.2.1 2024-05-09T04:58:11.891 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90968 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-05-09T04:58:11.891 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90968 ubuntu mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-05-09T04:58:11.891 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90968 ubuntu mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-05-09T04:58:11.891 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90968 ubuntu mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-05-09T04:58:11.891 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90968 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-05-09T04:58:11.891 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90968 ubuntu mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-05-09T04:58:11.891 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90968 ubuntu mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-05-09T04:58:11.892 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90968 ubuntu mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-05-09T04:58:11.892 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90968 ubuntu mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-05-09T04:58:11.892 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90968 ubuntu mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-05-09T04:58:11.892 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90968 ubuntu mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-05-09T04:58:11.892 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90968 ubuntu mem REG 8,1 196224 8914 /usr/lib64/libtirpc.so.3.0.0 2024-05-09T04:58:11.892 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90968 ubuntu mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-05-09T04:58:11.892 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90968 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-05-09T04:58:11.892 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90968 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-05-09T04:58:11.892 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90968 ubuntu 4r FIFO 0,14 0t0 248705 pipe 2024-05-09T04:58:11.892 INFO:tasks.workunit.client.0.smithi006.stdout:lsof 90968 ubuntu 7w FIFO 0,14 0t0 248706 pipe 2024-05-09T04:58:11.893 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-09T04:58:11.893 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-09T04:58:12.019 INFO:tasks.workunit:Stopping ['suites/fsync-tester.sh'] on client.0... 2024-05-09T04:58:12.019 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-09T04:58:13.129 DEBUG:teuthology.parallel:result is None 2024-05-09T04:58:13.130 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T04:58:13.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:12 smithi006 ceph-mon[29835]: pgmap v597: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 21 op/s 2024-05-09T04:58:13.236 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T04:58:13.236 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-05-09T04:58:13.247 INFO:tasks.quiescer:joining Quiescers 2024-05-09T04:58:13.247 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-05-09T04:58:13.247 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:58:13.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:12 smithi186 ceph-mon[34668]: pgmap v597: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 21 op/s 2024-05-09T04:58:13.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:12 smithi169 ceph-mon[34075]: pgmap v597: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 21 op/s 2024-05-09T04:58:13.722 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:58:15.152 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:14 smithi186 ceph-mon[34668]: pgmap v598: 129 pgs: 129 active+clean; 258 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 MiB/s wr, 26 op/s 2024-05-09T04:58:15.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:14 smithi006 ceph-mon[29835]: pgmap v598: 129 pgs: 129 active+clean; 258 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 MiB/s wr, 26 op/s 2024-05-09T04:58:15.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:14 smithi169 ceph-mon[34075]: pgmap v598: 129 pgs: 129 active+clean; 258 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 MiB/s wr, 26 op/s 2024-05-09T04:58:16.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.338+0000 7f4b93800700 1 -- 172.21.15.6:0/3087059959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b8c075b60 msgr2=0x7f4b8c075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:16.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.338+0000 7f4b93800700 1 --2- 172.21.15.6:0/3087059959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b8c075b60 0x7f4b8c075f70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009990 tx=0x7f4b80009ca0 comp rx=0 tx=0).stop 2024-05-09T04:58:16.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.339+0000 7f4b93800700 1 -- 172.21.15.6:0/3087059959 shutdown_connections 2024-05-09T04:58:16.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.339+0000 7f4b93800700 1 --2- 172.21.15.6:0/3087059959 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b8c115370 0x7f4b8c117790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:16.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.339+0000 7f4b93800700 1 --2- 172.21.15.6:0/3087059959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b8c075b60 0x7f4b8c075f70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:16.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.339+0000 7f4b93800700 1 --2- 172.21.15.6:0/3087059959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b8c0764b0 0x7f4b8c114e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:16.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.339+0000 7f4b93800700 1 -- 172.21.15.6:0/3087059959 >> 172.21.15.6:0/3087059959 conn(0x7f4b8c100040 msgr2=0x7f4b8c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:16.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.339+0000 7f4b93800700 1 -- 172.21.15.6:0/3087059959 shutdown_connections 2024-05-09T04:58:16.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.339+0000 7f4b93800700 1 -- 172.21.15.6:0/3087059959 wait complete. 2024-05-09T04:58:16.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.341+0000 7f4b93800700 1 Processor -- start 2024-05-09T04:58:16.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.341+0000 7f4b93800700 1 -- start start 2024-05-09T04:58:16.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.341+0000 7f4b93800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b8c075b60 0x7f4b8c1a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:16.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.341+0000 7f4b93800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b8c0764b0 0x7f4b8c1a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:16.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.341+0000 7f4b93800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b8c115370 0x7f4b8c1a5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:16.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.342+0000 7f4b93800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8c11aad0 con 0x7f4b8c115370 2024-05-09T04:58:16.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.342+0000 7f4b93800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8c11a950 con 0x7f4b8c075b60 2024-05-09T04:58:16.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.342+0000 7f4b93800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8c11ac50 con 0x7f4b8c0764b0 2024-05-09T04:58:16.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.342+0000 7f4b90e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b8c075b60 0x7f4b8c1a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:16.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.342+0000 7f4b90e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b8c075b60 0x7f4b8c1a1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:60992/0 (socket says 172.21.15.6:60992) 2024-05-09T04:58:16.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.342+0000 7f4b90e00700 1 -- 172.21.15.6:0/4151429008 learned_addr learned my addr 172.21.15.6:0/4151429008 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:58:16.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.342+0000 7f4b8be00700 1 --2- 172.21.15.6:0/4151429008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b8c0764b0 0x7f4b8c1a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:16.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.342+0000 7f4b91800700 1 --2- 172.21.15.6:0/4151429008 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b8c115370 0x7f4b8c1a5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:16.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.342+0000 7f4b90e00700 1 -- 172.21.15.6:0/4151429008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b8c0764b0 msgr2=0x7f4b8c1a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:16.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.342+0000 7f4b90e00700 1 --2- 172.21.15.6:0/4151429008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b8c0764b0 0x7f4b8c1a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:16.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.342+0000 7f4b90e00700 1 -- 172.21.15.6:0/4151429008 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b8c115370 msgr2=0x7f4b8c1a5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:16.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.342+0000 7f4b90e00700 1 --2- 172.21.15.6:0/4151429008 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b8c115370 0x7f4b8c1a5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:16.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.342+0000 7f4b90e00700 1 -- 172.21.15.6:0/4151429008 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b8c1a6350 con 0x7f4b8c075b60 2024-05-09T04:58:16.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.343+0000 7f4b90e00700 1 --2- 172.21.15.6:0/4151429008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b8c075b60 0x7f4b8c1a1480 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4b80000c00 tx=0x7f4b800057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:16.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.343+0000 7f4b89600700 1 -- 172.21.15.6:0/4151429008 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b80027070 con 0x7f4b8c075b60 2024-05-09T04:58:16.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.343+0000 7f4b93800700 1 -- 172.21.15.6:0/4151429008 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b8c1a65e0 con 0x7f4b8c075b60 2024-05-09T04:58:16.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.343+0000 7f4b89600700 1 -- 172.21.15.6:0/4151429008 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b80022470 con 0x7f4b8c075b60 2024-05-09T04:58:16.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.343+0000 7f4b89600700 1 -- 172.21.15.6:0/4151429008 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b80017650 con 0x7f4b8c075b60 2024-05-09T04:58:16.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.344+0000 7f4b93800700 1 -- 172.21.15.6:0/4151429008 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b8c1adf60 con 0x7f4b8c075b60 2024-05-09T04:58:16.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.345+0000 7f4b93800700 1 -- 172.21.15.6:0/4151429008 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b8c0772a0 con 0x7f4b8c075b60 2024-05-09T04:58:16.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.347+0000 7f4b89600700 1 -- 172.21.15.6:0/4151429008 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f4b8002a070 con 0x7f4b8c075b60 2024-05-09T04:58:16.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.349+0000 7f4b89600700 1 --2- 172.21.15.6:0/4151429008 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4b740b2880 0x7f4b740b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:16.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.349+0000 7f4b89600700 1 -- 172.21.15.6:0/4151429008 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f4b800e32d0 con 0x7f4b8c075b60 2024-05-09T04:58:16.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.349+0000 7f4b8be00700 1 --2- 172.21.15.6:0/4151429008 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4b740b2880 0x7f4b740b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:16.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.350+0000 7f4b8be00700 1 --2- 172.21.15.6:0/4151429008 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4b740b2880 0x7f4b740b4d90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4b78009680 tx=0x7f4b78009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:16.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.353+0000 7f4b89600700 1 -- 172.21.15.6:0/4151429008 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4b80013070 con 0x7f4b8c075b60 2024-05-09T04:58:16.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.641+0000 7f4b93800700 1 -- 172.21.15.6:0/4151429008 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b8c1a27e0 con 0x7f4b8c075b60 2024-05-09T04:58:16.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.643+0000 7f4b89600700 1 -- 172.21.15.6:0/4151429008 <== mon.2 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11927 (secure 0 0 0) 0x7f4b800aaca0 con 0x7f4b8c075b60 2024-05-09T04:58:16.643 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:58:16.643 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/162094599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":162094599},{"type":"v1","addr":"172.21.15.6:6835","nonce":162094599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/589294363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":589294363},{"type":"v1","addr":"172.21.15.6:6841","nonce":589294363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:49:36.920376+0000","modified":"2024-05-09T04:54:56.247431+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24554,"mds_1":24530,"mds_2":24512,"mds_3":24511,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24442":{"gid":24442,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6833/2977145206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2977145206},{"type":"v1","addr":"172.21.15.186:6833","nonce":2977145206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6833/133345948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":133345948},{"type":"v1","addr":"172.21.15.169:6833","nonce":133345948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.6:6837/1281521088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":1281521088},{"type":"v1","addr":"172.21.15.6:6837","nonce":1281521088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2268322007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2268322007},{"type":"v1","addr":"172.21.15.186:6835","nonce":2268322007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/3811005052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3811005052},{"type":"v1","addr":"172.21.15.169:6835","nonce":3811005052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.186:6837/1647607118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1647607118},{"type":"v1","addr":"172.21.15.186:6837","nonce":1647607118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":32,"addr":"172.21.15.186:6839/3954883105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":3954883105},{"type":"v1","addr":"172.21.15.186:6839","nonce":3954883105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.169:6837/1845715294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1845715294},{"type":"v1","addr":"172.21.15.169:6837","nonce":1845715294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.6:6839/2780306492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":2780306492},{"type":"v1","addr":"172.21.15.6:6839","nonce":2780306492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.169:6839/614388771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":614388771},{"type":"v1","addr":"172.21.15.169:6839","nonce":614388771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24512,24554,24530,24493,24511]},"id":1}]} 2024-05-09T04:58:16.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.647+0000 7f4b93800700 1 -- 172.21.15.6:0/4151429008 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4b740b2880 msgr2=0x7f4b740b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:16.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.647+0000 7f4b93800700 1 --2- 172.21.15.6:0/4151429008 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4b740b2880 0x7f4b740b4d90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4b78009680 tx=0x7f4b78009260 comp rx=0 tx=0).stop 2024-05-09T04:58:16.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.647+0000 7f4b93800700 1 -- 172.21.15.6:0/4151429008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b8c075b60 msgr2=0x7f4b8c1a1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:16.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.647+0000 7f4b93800700 1 --2- 172.21.15.6:0/4151429008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b8c075b60 0x7f4b8c1a1480 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4b80000c00 tx=0x7f4b800057c0 comp rx=0 tx=0).stop 2024-05-09T04:58:16.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.647+0000 7f4b93800700 1 -- 172.21.15.6:0/4151429008 shutdown_connections 2024-05-09T04:58:16.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.647+0000 7f4b93800700 1 --2- 172.21.15.6:0/4151429008 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b8c115370 0x7f4b8c1a5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:16.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.647+0000 7f4b93800700 1 --2- 172.21.15.6:0/4151429008 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4b740b2880 0x7f4b740b4d90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:16.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.647+0000 7f4b93800700 1 --2- 172.21.15.6:0/4151429008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b8c075b60 0x7f4b8c1a1480 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:16.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.647+0000 7f4b93800700 1 --2- 172.21.15.6:0/4151429008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b8c0764b0 0x7f4b8c1a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:16.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.647+0000 7f4b93800700 1 -- 172.21.15.6:0/4151429008 >> 172.21.15.6:0/4151429008 conn(0x7f4b8c100040 msgr2=0x7f4b8c100650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:16.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.648+0000 7f4b93800700 1 -- 172.21.15.6:0/4151429008 shutdown_connections 2024-05-09T04:58:16.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:16.648+0000 7f4b93800700 1 -- 172.21.15.6:0/4151429008 wait complete. 2024-05-09T04:58:16.649 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 30 2024-05-09T04:58:16.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:16 smithi006 ceph-mon[29835]: pgmap v599: 129 pgs: 129 active+clean; 258 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 28 op/s 2024-05-09T04:58:16.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:16 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/4151429008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:58:17.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:16 smithi186 ceph-mon[34668]: pgmap v599: 129 pgs: 129 active+clean; 258 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 28 op/s 2024-05-09T04:58:17.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:16 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/4151429008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:58:17.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:16 smithi169 ceph-mon[34075]: pgmap v599: 129 pgs: 129 active+clean; 258 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 28 op/s 2024-05-09T04:58:17.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:16 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/4151429008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:58:17.594 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24554 quiesce db --cancel --all' 2024-05-09T04:58:17.594 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24554 quiesce db --cancel --all 2024-05-09T04:58:18.054 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:58:19.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:18 smithi006 ceph-mon[29835]: pgmap v600: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 26 op/s 2024-05-09T04:58:19.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:18 smithi186 ceph-mon[34668]: pgmap v600: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 26 op/s 2024-05-09T04:58:19.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:18 smithi169 ceph-mon[34075]: pgmap v600: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 26 op/s 2024-05-09T04:58:20.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.682+0000 7f9ca6e00700 1 -- 172.21.15.6:0/1664177787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ca0076e40 msgr2=0x7f9ca0077250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:20.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.682+0000 7f9ca6e00700 1 --2- 172.21.15.6:0/1664177787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ca0076e40 0x7f9ca0077250 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9c90009990 tx=0x7f9c90009ca0 comp rx=0 tx=0).stop 2024-05-09T04:58:20.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.682+0000 7f9ca6e00700 1 -- 172.21.15.6:0/1664177787 shutdown_connections 2024-05-09T04:58:20.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.682+0000 7f9ca6e00700 1 --2- 172.21.15.6:0/1664177787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ca0110ba0 0x7f9ca0113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:20.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.682+0000 7f9ca6e00700 1 --2- 172.21.15.6:0/1664177787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ca0076e40 0x7f9ca0077250 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:20.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.682+0000 7f9ca6e00700 1 --2- 172.21.15.6:0/1664177787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ca0077790 0x7f9ca0110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:20.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.682+0000 7f9ca6e00700 1 -- 172.21.15.6:0/1664177787 >> 172.21.15.6:0/1664177787 conn(0x7f9ca00ffec0 msgr2=0x7f9ca0102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:20.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.682+0000 7f9ca6e00700 1 -- 172.21.15.6:0/1664177787 shutdown_connections 2024-05-09T04:58:20.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.682+0000 7f9ca6e00700 1 -- 172.21.15.6:0/1664177787 wait complete. 2024-05-09T04:58:20.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.684+0000 7f9ca6e00700 1 Processor -- start 2024-05-09T04:58:20.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.684+0000 7f9ca6e00700 1 -- start start 2024-05-09T04:58:20.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.684+0000 7f9ca6e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ca0076e40 0x7f9ca019d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:20.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.684+0000 7f9ca6e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ca0077790 0x7f9ca019d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:20.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.684+0000 7f9ca6e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ca0110ba0 0x7f9ca01a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:20.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.684+0000 7f9ca6e00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca01166e0 con 0x7f9ca0110ba0 2024-05-09T04:58:20.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.684+0000 7f9ca6e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca0116560 con 0x7f9ca0077790 2024-05-09T04:58:20.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.684+0000 7f9ca6e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca0116860 con 0x7f9ca0076e40 2024-05-09T04:58:20.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.685+0000 7f9ca4e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ca0110ba0 0x7f9ca01a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:20.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.685+0000 7f9c9f400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ca0077790 0x7f9ca019d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:20.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.685+0000 7f9c9fe00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ca0076e40 0x7f9ca019d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:20.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.685+0000 7f9ca4e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ca0110ba0 0x7f9ca01a1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39202/0 (socket says 172.21.15.6:39202) 2024-05-09T04:58:20.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.685+0000 7f9c9f400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ca0077790 0x7f9ca019d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:44934/0 (socket says 172.21.15.6:44934) 2024-05-09T04:58:20.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.685+0000 7f9c9fe00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ca0076e40 0x7f9ca019d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:55972/0 (socket says 172.21.15.6:55972) 2024-05-09T04:58:20.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.685+0000 7f9ca4e00700 1 -- 172.21.15.6:0/290993658 learned_addr learned my addr 172.21.15.6:0/290993658 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:58:20.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.685+0000 7f9ca4e00700 1 -- 172.21.15.6:0/290993658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ca0076e40 msgr2=0x7f9ca019d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:20.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.685+0000 7f9ca4e00700 1 --2- 172.21.15.6:0/290993658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ca0076e40 0x7f9ca019d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:20.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.685+0000 7f9ca4e00700 1 -- 172.21.15.6:0/290993658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ca0077790 msgr2=0x7f9ca019d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:20.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.685+0000 7f9ca4e00700 1 --2- 172.21.15.6:0/290993658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ca0077790 0x7f9ca019d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:20.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.685+0000 7f9ca4e00700 1 -- 172.21.15.6:0/290993658 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca01a1f10 con 0x7f9ca0110ba0 2024-05-09T04:58:20.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.686+0000 7f9ca4e00700 1 --2- 172.21.15.6:0/290993658 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ca0110ba0 0x7f9ca01a1870 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800c110 tx=0x7f9c9800c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:20.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.686+0000 7f9c9cc00700 1 -- 172.21.15.6:0/290993658 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c980092e0 con 0x7f9ca0110ba0 2024-05-09T04:58:20.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.686+0000 7f9c9cc00700 1 -- 172.21.15.6:0/290993658 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c98003a00 con 0x7f9ca0110ba0 2024-05-09T04:58:20.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.686+0000 7f9ca6e00700 1 -- 172.21.15.6:0/290993658 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ca01a21a0 con 0x7f9ca0110ba0 2024-05-09T04:58:20.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.686+0000 7f9c9cc00700 1 -- 172.21.15.6:0/290993658 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c98004650 con 0x7f9ca0110ba0 2024-05-09T04:58:20.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.686+0000 7f9ca6e00700 1 -- 172.21.15.6:0/290993658 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ca01a2690 con 0x7f9ca0110ba0 2024-05-09T04:58:20.693 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.692+0000 7f9c9cc00700 1 -- 172.21.15.6:0/290993658 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f9c9801a020 con 0x7f9ca0110ba0 2024-05-09T04:58:20.693 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.693+0000 7f9cb5a3db80 1 Processor -- start 2024-05-09T04:58:20.693 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.693+0000 7f9cb5a3db80 1 -- start start 2024-05-09T04:58:20.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.693+0000 7f9cb5a3db80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b3a52e1510 0x55b3a54644b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:20.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.693+0000 7f9cb5a3db80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3a54649f0 0x55b3a545e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:20.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.693+0000 7f9cb5a3db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a545ec20 0x55b3a545f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:20.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.693+0000 7f9cb5a3db80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55b3a5381320 con 0x55b3a52e1510 2024-05-09T04:58:20.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.693+0000 7f9cb5a3db80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b3a531f150 con 0x55b3a54649f0 2024-05-09T04:58:20.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.693+0000 7f9cb5a3db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55b3a537d2e0 con 0x55b3a545ec20 2024-05-09T04:58:20.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.694+0000 7f9c9f400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b3a52e1510 0x55b3a54644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:20.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.694+0000 7f9c9fe00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3a54649f0 0x55b3a545e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:20.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.694+0000 7f9c9f400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b3a52e1510 0x55b3a54644b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39216/0 (socket says 172.21.15.6:39216) 2024-05-09T04:58:20.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.694+0000 7f9c9f400700 1 -- 172.21.15.6:0/1248606309 learned_addr learned my addr 172.21.15.6:0/1248606309 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:58:20.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.694+0000 7f9c9f400700 1 --2- 172.21.15.6:0/1248606309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a545ec20 0x55b3a545f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:20.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.694+0000 7f9c9cc00700 1 --2- 172.21.15.6:0/290993658 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9c840b2880 0x7f9c840b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:20.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.694+0000 7f9c9fe00700 1 -- 172.21.15.6:0/1248606309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a545ec20 msgr2=0x55b3a545f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:20.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.694+0000 7f9c9fe00700 1 --2- 172.21.15.6:0/1248606309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a545ec20 0x55b3a545f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:20.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.694+0000 7f9c9fe00700 1 -- 172.21.15.6:0/1248606309 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b3a52e1510 msgr2=0x55b3a54644b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:20.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.694+0000 7f9c9fe00700 1 --2- 172.21.15.6:0/1248606309 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b3a52e1510 0x55b3a54644b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:20.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.694+0000 7f9c9fe00700 1 -- 172.21.15.6:0/1248606309 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3a51fc500 con 0x55b3a54649f0 2024-05-09T04:58:20.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.694+0000 7f9c9f400700 1 --2- 172.21.15.6:0/1248606309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a545ec20 0x55b3a545f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:58:20.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.694+0000 7f9c9cc00700 1 -- 172.21.15.6:0/290993658 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f9c980e1db0 con 0x7f9ca0110ba0 2024-05-09T04:58:20.697 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.695+0000 7f9c9fe00700 1 --2- 172.21.15.6:0/1248606309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3a54649f0 0x55b3a545e6e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9c900098e0 tx=0x7f9c900096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0bb6b1dddffc116 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:20.697 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.695+0000 7f9ca4e00700 1 --2- 172.21.15.6:0/290993658 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9c840b2880 0x7f9c840b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:20.697 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.695+0000 7f9c95600700 1 -- 172.21.15.6:0/1248606309 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c90027070 con 0x55b3a54649f0 2024-05-09T04:58:20.697 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.695+0000 7f9c95600700 1 -- 172.21.15.6:0/1248606309 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c900056b0 con 0x55b3a54649f0 2024-05-09T04:58:20.697 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.695+0000 7f9c95600700 1 -- 172.21.15.6:0/1248606309 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c90003bc0 con 0x55b3a54649f0 2024-05-09T04:58:20.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.696+0000 7f9cb5a3db80 1 -- 172.21.15.6:0/1248606309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3a54649f0 msgr2=0x55b3a545e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:20.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.696+0000 7f9ca4e00700 1 --2- 172.21.15.6:0/290993658 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9c840b2880 0x7f9c840b4d90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800e1d0 tx=0x7f9c98004020 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:20.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.696+0000 7f9cb5a3db80 1 --2- 172.21.15.6:0/1248606309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3a54649f0 0x55b3a545e6e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9c900098e0 tx=0x7f9c900096a0 comp rx=0 tx=0).stop 2024-05-09T04:58:20.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.696+0000 7f9cb5a3db80 1 -- 172.21.15.6:0/1248606309 shutdown_connections 2024-05-09T04:58:20.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.696+0000 7f9cb5a3db80 1 --2- 172.21.15.6:0/1248606309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a545ec20 0x55b3a545f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:20.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.696+0000 7f9cb5a3db80 1 --2- 172.21.15.6:0/1248606309 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b3a52e1510 0x55b3a54644b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:20.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.696+0000 7f9cb5a3db80 1 --2- 172.21.15.6:0/1248606309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3a54649f0 0x55b3a545e6e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:20.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.696+0000 7f9cb5a3db80 1 -- 172.21.15.6:0/1248606309 >> 172.21.15.6:0/1248606309 conn(0x55b3a5325ee0 msgr2=0x55b3a54592e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:20.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.697+0000 7f9cb5a3db80 1 -- 172.21.15.6:0/1248606309 shutdown_connections 2024-05-09T04:58:20.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.697+0000 7f9cb5a3db80 1 -- 172.21.15.6:0/1248606309 wait complete. 2024-05-09T04:58:20.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.697+0000 7f9cb5a3db80 1 -- start start 2024-05-09T04:58:20.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.697+0000 7f9cb5a3db80 1 Processor -- start 2024-05-09T04:58:20.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.698+0000 7f9cb5a3db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a52e1510 0x55b3a5468180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:20.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.698+0000 7f9cb5a3db80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b3a545ec20 0x55b3a54686c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:20.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.698+0000 7f9cb5a3db80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3a54649f0 0x55b3a5468c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:20.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.698+0000 7f9cb5a3db80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x55b3a546b430 con 0x55b3a545ec20 2024-05-09T04:58:20.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.698+0000 7f9cb5a3db80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b3a52e6730 con 0x55b3a54649f0 2024-05-09T04:58:20.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.698+0000 7f9cb5a3db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55b3a546b5b0 con 0x55b3a52e1510 2024-05-09T04:58:20.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.698+0000 7f9cb5a3db80 10 client.0 _ll_register_callbacks cb 0x55b3a5318600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:58:20.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.699+0000 7f9c9f400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a52e1510 0x55b3a5468180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:20.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.699+0000 7f9c9f400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a52e1510 0x55b3a5468180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:55980/0 (socket says 172.21.15.6:55980) 2024-05-09T04:58:20.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.699+0000 7f9c9f400700 1 -- 172.21.15.6:0/1427500883 learned_addr learned my addr 172.21.15.6:0/1427500883 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:58:20.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.699+0000 7f9c9fe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b3a545ec20 0x55b3a54686c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:20.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.699+0000 7f9c9f400700 1 --2- 172.21.15.6:0/1427500883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3a54649f0 0x55b3a5468c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:20.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.700+0000 7f9c9f400700 1 -- 172.21.15.6:0/1427500883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3a54649f0 msgr2=0x55b3a5468c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:20.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.700+0000 7f9c9f400700 1 --2- 172.21.15.6:0/1427500883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3a54649f0 0x55b3a5468c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:20.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.700+0000 7f9c9f400700 1 -- 172.21.15.6:0/1427500883 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b3a545ec20 msgr2=0x55b3a54686c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:20.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.700+0000 7f9c9f400700 1 --2- 172.21.15.6:0/1427500883 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b3a545ec20 0x55b3a54686c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:20.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.700+0000 7f9c9f400700 1 -- 172.21.15.6:0/1427500883 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3a54748e0 con 0x55b3a52e1510 2024-05-09T04:58:20.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.700+0000 7f9c9f400700 1 --2- 172.21.15.6:0/1427500883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a52e1510 0x55b3a5468180 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c013cd0 tx=0x7f9c8c015040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1efcd1e4547cfafd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:20.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.700+0000 7f9c8a000700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-05-09T04:58:20.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.700+0000 7f9c8a000700 1 -- 172.21.15.6:0/1427500883 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c027060 con 0x55b3a52e1510 2024-05-09T04:58:20.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.700+0000 7f9cb5a3db80 1 -- 172.21.15.6:0/1427500883 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c900056b0 con 0x55b3a52e1510 2024-05-09T04:58:20.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.700+0000 7f9c8a000700 1 -- 172.21.15.6:0/1427500883 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f9c8c011c70 con 0x55b3a52e1510 2024-05-09T04:58:20.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.701+0000 7f9c8a000700 1 -- 172.21.15.6:0/1427500883 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9c8c011e70 con 0x55b3a52e1510 2024-05-09T04:58:20.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.701+0000 7f9cb5a3db80 10 client.25471 fetch_fsmap learned FSMap version 30 2024-05-09T04:58:20.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.701+0000 7f9cb5a3db80 1 -- 172.21.15.6:0/1427500883 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b3a5476300 con 0x55b3a52e1510 2024-05-09T04:58:20.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.701+0000 7f9c8a000700 1 -- 172.21.15.6:0/1427500883 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c900056b0 con 0x55b3a52e1510 2024-05-09T04:58:20.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.701+0000 7f9c8a000700 1 -- 172.21.15.6:0/1427500883 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9c8c00f450 con 0x55b3a52e1510 2024-05-09T04:58:20.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.701+0000 7f9cb5a3db80 10 client.25471 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T04:58:20.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.701+0000 7f9cb5a3db80 10 client.25471 resolve_mds: validated gid 24554 aka daemon mds.b 2024-05-09T04:58:20.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.701+0000 7f9cb5a3db80 20 client.25471 populate_metadata read hostname 'smithi006' 2024-05-09T04:58:20.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.701+0000 7f9cb5a3db80 1 --2- 172.21.15.6:0/1427500883 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x55b3a54787d0 0x55b3a547abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:20.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.701+0000 7f9cb5a3db80 4 client.25471 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:58:20.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.701+0000 7f9cb5a3db80 1 -- 172.21.15.6:0/1427500883 --> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3a542a8e0 con 0x55b3a54787d0 2024-05-09T04:58:20.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.702+0000 7f9c9fe00700 1 --2- 172.21.15.6:0/1427500883 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x55b3a54787d0 0x55b3a547abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:20.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.703+0000 7f9c9fe00700 1 --2- 172.21.15.6:0/1427500883 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x55b3a54787d0 0x55b3a547abf0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93e06be6961b9b3 server_cookie=2ceb36b558fd6257 in_seq=0 out_seq=0 2024-05-09T04:58:20.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.703+0000 7f9c8a000700 10 client.25471 ms_handle_connect on v2:172.21.15.169:6838/614388771 2024-05-09T04:58:20.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.704+0000 7f9c8a000700 1 -- 172.21.15.6:0/1427500883 <== mds.0 v2:172.21.15.169:6838/614388771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b3a542a8e0 con 0x55b3a54787d0 2024-05-09T04:58:20.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.704+0000 7f9c8a000700 10 client.25471 handle_command_reply: tid=0 2024-05-09T04:58:20.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.704+0000 7f9cb5a3db80 1 client.25471 shutdown 2024-05-09T04:58:20.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.704+0000 7f9cb5a3db80 1 -- 172.21.15.6:0/1427500883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a52e1510 msgr2=0x55b3a5468180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:20.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.704+0000 7f9cb5a3db80 1 --2- 172.21.15.6:0/1427500883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a52e1510 0x55b3a5468180 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c013cd0 tx=0x7f9c8c015040 comp rx=0 tx=0).stop 2024-05-09T04:58:20.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.704+0000 7f9cb5a3db80 1 -- 172.21.15.6:0/1427500883 shutdown_connections 2024-05-09T04:58:20.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.704+0000 7f9cb5a3db80 1 --2- 172.21.15.6:0/1427500883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3a54649f0 0x55b3a5468c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:20.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.704+0000 7f9cb5a3db80 1 --2- 172.21.15.6:0/1427500883 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x55b3a54787d0 0x55b3a547abf0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:20.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.704+0000 7f9cb5a3db80 1 --2- 172.21.15.6:0/1427500883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a52e1510 0x55b3a5468180 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:20.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.704+0000 7f9cb5a3db80 1 --2- 172.21.15.6:0/1427500883 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b3a545ec20 0x55b3a54686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:20.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.704+0000 7f9cb5a3db80 1 -- 172.21.15.6:0/1427500883 >> 172.21.15.6:0/1427500883 conn(0x55b3a5325ee0 msgr2=0x55b3a5458710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:20.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.704+0000 7f9c8a000700 0 client.25471 ms_handle_reset on v2:172.21.15.169:6838/614388771 2024-05-09T04:58:20.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.704+0000 7f9cb5a3db80 1 -- 172.21.15.6:0/1427500883 shutdown_connections 2024-05-09T04:58:20.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.704+0000 7f9cb5a3db80 1 -- 172.21.15.6:0/1427500883 wait complete. 2024-05-09T04:58:20.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.705+0000 7f9cb5a3db80 20 client.25471 trim_cache size 0 max 16384 2024-05-09T04:58:20.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.727+0000 7f9cb5a3db80 1 Processor -- start 2024-05-09T04:58:20.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.727+0000 7f9cb5a3db80 1 -- start start 2024-05-09T04:58:20.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.727+0000 7f9cb5a3db80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b3a52e1510 0x55b3a545e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:20.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.727+0000 7f9cb5a3db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a545ec20 0x55b3a5468b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:20.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.727+0000 7f9cb5a3db80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3a54649f0 0x55b3a546a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:20.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.727+0000 7f9cb5a3db80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca01b24a0 con 0x55b3a52e1510 2024-05-09T04:58:20.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.727+0000 7f9cb5a3db80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca01b3fe0 con 0x55b3a54649f0 2024-05-09T04:58:20.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.727+0000 7f9cb5a3db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca01b2b70 con 0x55b3a545ec20 2024-05-09T04:58:20.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.727+0000 7f9c9f400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b3a52e1510 0x55b3a545e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:20.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.727+0000 7f9c9f400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b3a52e1510 0x55b3a545e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39244/0 (socket says 172.21.15.6:39244) 2024-05-09T04:58:20.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.727+0000 7f9c9f400700 1 -- 172.21.15.6:0/2933663381 learned_addr learned my addr 172.21.15.6:0/2933663381 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:58:20.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.728+0000 7f9c9fe00700 1 --2- 172.21.15.6:0/2933663381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a545ec20 0x55b3a5468b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:20.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.728+0000 7f9c9f400700 1 --2- 172.21.15.6:0/2933663381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3a54649f0 0x55b3a546a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:20.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.728+0000 7f9c9f400700 1 -- 172.21.15.6:0/2933663381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a545ec20 msgr2=0x55b3a5468b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:20.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.728+0000 7f9c9f400700 1 --2- 172.21.15.6:0/2933663381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a545ec20 0x55b3a5468b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:20.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.728+0000 7f9c9f400700 1 -- 172.21.15.6:0/2933663381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3a54649f0 msgr2=0x55b3a546a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:20.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.728+0000 7f9c9f400700 1 --2- 172.21.15.6:0/2933663381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3a54649f0 0x55b3a546a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:20.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.728+0000 7f9c9f400700 1 -- 172.21.15.6:0/2933663381 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3a54666f0 con 0x55b3a52e1510 2024-05-09T04:58:20.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.729+0000 7f9c9f400700 1 --2- 172.21.15.6:0/2933663381 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b3a52e1510 0x55b3a545e6e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55b3a542aa60 tx=0x7f9c8c0058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a71b9df16fd0f4e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:20.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.729+0000 7f9c94c00700 1 -- 172.21.15.6:0/2933663381 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c00fc90 con 0x55b3a52e1510 2024-05-09T04:58:20.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.729+0000 7f9c94c00700 1 -- 172.21.15.6:0/2933663381 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c8c005560 con 0x55b3a52e1510 2024-05-09T04:58:20.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.729+0000 7f9c94c00700 1 -- 172.21.15.6:0/2933663381 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c003680 con 0x55b3a52e1510 2024-05-09T04:58:20.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.730+0000 7f9cb5a3db80 1 -- 172.21.15.6:0/2933663381 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b3a52e1510 msgr2=0x55b3a545e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:20.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.730+0000 7f9cb5a3db80 1 --2- 172.21.15.6:0/2933663381 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b3a52e1510 0x55b3a545e6e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55b3a542aa60 tx=0x7f9c8c0058e0 comp rx=0 tx=0).stop 2024-05-09T04:58:20.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.730+0000 7f9cb5a3db80 1 -- 172.21.15.6:0/2933663381 shutdown_connections 2024-05-09T04:58:20.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.730+0000 7f9cb5a3db80 1 --2- 172.21.15.6:0/2933663381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3a54649f0 0x55b3a546a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:20.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.730+0000 7f9cb5a3db80 1 --2- 172.21.15.6:0/2933663381 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b3a52e1510 0x55b3a545e6e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:20.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.730+0000 7f9cb5a3db80 1 --2- 172.21.15.6:0/2933663381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a545ec20 0x55b3a5468b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:20.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.730+0000 7f9cb5a3db80 1 -- 172.21.15.6:0/2933663381 >> 172.21.15.6:0/2933663381 conn(0x55b3a5325ee0 msgr2=0x55b3a5461dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:20.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.730+0000 7f9cb5a3db80 1 -- 172.21.15.6:0/2933663381 shutdown_connections 2024-05-09T04:58:20.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.730+0000 7f9cb5a3db80 1 -- 172.21.15.6:0/2933663381 wait complete. 2024-05-09T04:58:20.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.731+0000 7f9cb5a3db80 1 -- start start 2024-05-09T04:58:20.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.731+0000 7f9cb5a3db80 1 Processor -- start 2024-05-09T04:58:20.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.731+0000 7f9cb5a3db80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b3a52e1510 0x55b3a546d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:20.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.731+0000 7f9cb5a3db80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3a545ec20 0x55b3a546dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:20.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.732+0000 7f9cb5a3db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a54649f0 0x55b3a546e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:20.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.732+0000 7f9cb5a3db80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca01b3910 con 0x55b3a52e1510 2024-05-09T04:58:20.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.732+0000 7f9cb5a3db80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca01b3240 con 0x55b3a545ec20 2024-05-09T04:58:20.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.732+0000 7f9cb5a3db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca01b0440 con 0x55b3a54649f0 2024-05-09T04:58:20.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.732+0000 7f9cb5a3db80 10 client.0 _ll_register_callbacks cb 0x55b3a5318600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:58:20.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.732+0000 7f9c9f400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b3a52e1510 0x55b3a546d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:20.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.732+0000 7f9c9f400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b3a52e1510 0x55b3a546d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39254/0 (socket says 172.21.15.6:39254) 2024-05-09T04:58:20.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.732+0000 7f9c9f400700 1 -- 172.21.15.6:0/359559505 learned_addr learned my addr 172.21.15.6:0/359559505 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:58:20.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.732+0000 7f9c9fe00700 1 --2- 172.21.15.6:0/359559505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3a545ec20 0x55b3a546dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:20.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.732+0000 7f9c9f400700 1 -- 172.21.15.6:0/359559505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a54649f0 msgr2=0x55b3a546e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:20.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.732+0000 7f9c9f400700 1 --2- 172.21.15.6:0/359559505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a54649f0 0x55b3a546e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:20.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.733+0000 7f9c9f400700 1 -- 172.21.15.6:0/359559505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3a545ec20 msgr2=0x55b3a546dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:20.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.733+0000 7f9c9f400700 1 --2- 172.21.15.6:0/359559505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3a545ec20 0x55b3a546dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:20.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.733+0000 7f9c9f400700 1 -- 172.21.15.6:0/359559505 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3a54e2f70 con 0x55b3a52e1510 2024-05-09T04:58:20.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.733+0000 7f9c9fe00700 1 --2- 172.21.15.6:0/359559505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3a545ec20 0x55b3a546dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:58:20.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.733+0000 7f9c9f400700 1 --2- 172.21.15.6:0/359559505 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b3a52e1510 0x55b3a546d810 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c00f8a0 tx=0x7f9c8c009980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=846e74414d6d312c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:20.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.733+0000 7f9c8a000700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2024-05-09T04:58:20.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.733+0000 7f9c8a000700 1 -- 172.21.15.6:0/359559505 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c044060 con 0x55b3a52e1510 2024-05-09T04:58:20.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.733+0000 7f9c8a000700 1 -- 172.21.15.6:0/359559505 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f9c8c00fc20 con 0x55b3a52e1510 2024-05-09T04:58:20.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.733+0000 7f9cb5a3db80 1 -- 172.21.15.6:0/359559505 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c8c005560 con 0x55b3a52e1510 2024-05-09T04:58:20.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.734+0000 7f9c8a000700 1 -- 172.21.15.6:0/359559505 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9c8c009260 con 0x55b3a52e1510 2024-05-09T04:58:20.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.734+0000 7f9cb5a3db80 10 client.15339 fetch_fsmap learned FSMap version 30 2024-05-09T04:58:20.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.734+0000 7f9cb5a3db80 1 -- 172.21.15.6:0/359559505 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b3a54e41b0 con 0x55b3a52e1510 2024-05-09T04:58:20.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.734+0000 7f9c8a000700 1 -- 172.21.15.6:0/359559505 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c8c005560 con 0x55b3a52e1510 2024-05-09T04:58:20.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.734+0000 7f9c8a000700 1 -- 172.21.15.6:0/359559505 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9c8c003b30 con 0x55b3a52e1510 2024-05-09T04:58:20.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.734+0000 7f9cb5a3db80 10 client.15339 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T04:58:20.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.734+0000 7f9cb5a3db80 10 client.15339 resolve_mds: validated gid 24554 aka daemon mds.b 2024-05-09T04:58:20.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.734+0000 7f9cb5a3db80 20 client.15339 populate_metadata read hostname 'smithi006' 2024-05-09T04:58:20.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.734+0000 7f9cb5a3db80 1 --2- 172.21.15.6:0/359559505 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x55b3a54e4810 0x55b3a54e6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:20.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.735+0000 7f9cb5a3db80 4 client.15339 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-05-09T04:58:20.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.735+0000 7f9cb5a3db80 1 -- 172.21.15.6:0/359559505 --> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55b3a538ed00 con 0x55b3a54e4810 2024-05-09T04:58:20.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.735+0000 7f9c9fe00700 1 --2- 172.21.15.6:0/359559505 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x55b3a54e4810 0x55b3a54e6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:20.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.736+0000 7f9c9fe00700 1 --2- 172.21.15.6:0/359559505 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x55b3a54e4810 0x55b3a54e6c30 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=660d10acaf50c849 server_cookie=3e6b25ad5f416ffa in_seq=0 out_seq=0 2024-05-09T04:58:20.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.736+0000 7f9c8a000700 10 client.15339 ms_handle_connect on v2:172.21.15.169:6838/614388771 2024-05-09T04:58:20.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.736+0000 7f9c8a000700 1 -- 172.21.15.6:0/359559505 <== mds.0 v2:172.21.15.169:6838/614388771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+79 (crc 0 0 0) 0x55b3a538ed00 con 0x55b3a54e4810 2024-05-09T04:58:20.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.736+0000 7f9c8a000700 10 client.15339 handle_command_reply: tid=0 2024-05-09T04:58:20.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.736+0000 7f9cb5a3db80 1 client.15339 shutdown 2024-05-09T04:58:20.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.736+0000 7f9cb5a3db80 1 -- 172.21.15.6:0/359559505 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b3a52e1510 msgr2=0x55b3a546d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:20.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.736+0000 7f9cb5a3db80 1 --2- 172.21.15.6:0/359559505 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b3a52e1510 0x55b3a546d810 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c00f8a0 tx=0x7f9c8c009980 comp rx=0 tx=0).stop 2024-05-09T04:58:20.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.736+0000 7f9cb5a3db80 1 -- 172.21.15.6:0/359559505 shutdown_connections 2024-05-09T04:58:20.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.736+0000 7f9cb5a3db80 1 --2- 172.21.15.6:0/359559505 >> [v2:172.21.15.169:6838/614388771,v1:172.21.15.169:6839/614388771] conn(0x55b3a54e4810 0x55b3a54e6c30 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:20.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.736+0000 7f9cb5a3db80 1 --2- 172.21.15.6:0/359559505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b3a54649f0 0x55b3a546e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:20.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.736+0000 7f9cb5a3db80 1 --2- 172.21.15.6:0/359559505 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b3a52e1510 0x55b3a546d810 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:20.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.736+0000 7f9cb5a3db80 1 --2- 172.21.15.6:0/359559505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3a545ec20 0x55b3a546dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:20.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.736+0000 7f9cb5a3db80 1 -- 172.21.15.6:0/359559505 >> 172.21.15.6:0/359559505 conn(0x55b3a5325ee0 msgr2=0x55b3a54627f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:20.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.736+0000 7f9c8a000700 0 client.15339 ms_handle_reset on v2:172.21.15.169:6838/614388771 2024-05-09T04:58:20.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.737+0000 7f9cb5a3db80 1 -- 172.21.15.6:0/359559505 shutdown_connections 2024-05-09T04:58:20.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.737+0000 7f9cb5a3db80 1 -- 172.21.15.6:0/359559505 wait complete. 2024-05-09T04:58:20.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.737+0000 7f9cb5a3db80 20 client.15339 trim_cache size 0 max 16384 2024-05-09T04:58:20.740 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-09T04:58:20.740 INFO:teuthology.orchestra.run.smithi006.stdout: "epoch": 30, 2024-05-09T04:58:20.740 INFO:teuthology.orchestra.run.smithi006.stdout: "leader": 24554, 2024-05-09T04:58:20.740 INFO:teuthology.orchestra.run.smithi006.stdout: "set_version": 8, 2024-05-09T04:58:20.740 INFO:teuthology.orchestra.run.smithi006.stdout: "sets": {} 2024-05-09T04:58:20.740 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-09T04:58:20.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.737+0000 7f9ca6e00700 1 -- 172.21.15.6:0/290993658 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9c840b2880 msgr2=0x7f9c840b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:20.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.737+0000 7f9ca6e00700 1 --2- 172.21.15.6:0/290993658 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9c840b2880 0x7f9c840b4d90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800e1d0 tx=0x7f9c98004020 comp rx=0 tx=0).stop 2024-05-09T04:58:20.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.737+0000 7f9ca6e00700 1 -- 172.21.15.6:0/290993658 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ca0110ba0 msgr2=0x7f9ca01a1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:20.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.737+0000 7f9ca6e00700 1 --2- 172.21.15.6:0/290993658 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ca0110ba0 0x7f9ca01a1870 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800c110 tx=0x7f9c9800c420 comp rx=0 tx=0).stop 2024-05-09T04:58:20.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.738+0000 7f9ca6e00700 1 -- 172.21.15.6:0/290993658 shutdown_connections 2024-05-09T04:58:20.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.738+0000 7f9ca6e00700 1 --2- 172.21.15.6:0/290993658 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ca0110ba0 0x7f9ca01a1870 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:20.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.738+0000 7f9ca6e00700 1 --2- 172.21.15.6:0/290993658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ca0076e40 0x7f9ca019d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:20.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.738+0000 7f9ca6e00700 1 --2- 172.21.15.6:0/290993658 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9c840b2880 0x7f9c840b4d90 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:20.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.738+0000 7f9ca6e00700 1 --2- 172.21.15.6:0/290993658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ca0077790 0x7f9ca019d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:20.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.738+0000 7f9ca6e00700 1 -- 172.21.15.6:0/290993658 >> 172.21.15.6:0/290993658 conn(0x7f9ca00ffec0 msgr2=0x7f9ca0101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:20.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.738+0000 7f9ca6e00700 1 -- 172.21.15.6:0/290993658 shutdown_connections 2024-05-09T04:58:20.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:20.738+0000 7f9ca6e00700 1 -- 172.21.15.6:0/290993658 wait complete. 2024-05-09T04:58:20.947 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:20 smithi006 ceph-mon[29835]: pgmap v601: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 26 op/s 2024-05-09T04:58:21.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:20 smithi186 ceph-mon[34668]: pgmap v601: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 26 op/s 2024-05-09T04:58:21.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:20 smithi169 ceph-mon[34075]: pgmap v601: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 26 op/s 2024-05-09T04:58:21.807 INFO:tasks.quiescer:done joining Quiescers 2024-05-09T04:58:21.807 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-09T04:58:21.817 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T04:58:22.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.106+0000 7f988e600640 1 -- 172.21.15.6:0/265268595 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9888105c00 msgr2=0x7f9888106900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:22.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.106+0000 7f988e600640 1 --2- 172.21.15.6:0/265268595 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9888105c00 0x7f9888106900 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9878009900 tx=0x7f987802a400 comp rx=0 tx=0).stop 2024-05-09T04:58:22.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.106+0000 7f988e600640 1 -- 172.21.15.6:0/265268595 shutdown_connections 2024-05-09T04:58:22.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.106+0000 7f988e600640 1 --2- 172.21.15.6:0/265268595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f988810bb40 0x7f988810df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:22.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.106+0000 7f988e600640 1 --2- 172.21.15.6:0/265268595 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9888106e40 0x7f9888107250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:22.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.106+0000 7f988e600640 1 --2- 172.21.15.6:0/265268595 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9888105c00 0x7f9888106900 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:22.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.106+0000 7f988e600640 1 -- 172.21.15.6:0/265268595 >> 172.21.15.6:0/265268595 conn(0x7f9888100df0 msgr2=0x7f98881032a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:22.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.106+0000 7f988e600640 1 -- 172.21.15.6:0/265268595 shutdown_connections 2024-05-09T04:58:22.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.106+0000 7f988e600640 1 -- 172.21.15.6:0/265268595 wait complete. 2024-05-09T04:58:22.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.108+0000 7f988e600640 1 Processor -- start 2024-05-09T04:58:22.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.108+0000 7f988e600640 1 -- start start 2024-05-09T04:58:22.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.108+0000 7f988e600640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9888105c00 0x7f988819c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:22.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.108+0000 7f988e600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9888106e40 0x7f988819c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:22.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.108+0000 7f988e600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f988810bb40 0x7f98881a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:22.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.108+0000 7f988e600640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9888111730 con 0x7f9888105c00 2024-05-09T04:58:22.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.108+0000 7f988e600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f98881115b0 con 0x7f9888106e40 2024-05-09T04:58:22.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.108+0000 7f988e600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f98881118b0 con 0x7f988810bb40 2024-05-09T04:58:22.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.109+0000 7f988d200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9888105c00 0x7f988819c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:22.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.109+0000 7f988d200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9888105c00 0x7f988819c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39270/0 (socket says 172.21.15.6:39270) 2024-05-09T04:58:22.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.109+0000 7f988d200640 1 -- 172.21.15.6:0/891575450 learned_addr learned my addr 172.21.15.6:0/891575450 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:58:22.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.109+0000 7f988dc00640 1 --2- 172.21.15.6:0/891575450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f988810bb40 0x7f98881a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:22.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.109+0000 7f987fe00640 1 --2- 172.21.15.6:0/891575450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9888106e40 0x7f988819c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:22.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.109+0000 7f988d200640 1 -- 172.21.15.6:0/891575450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f988810bb40 msgr2=0x7f98881a3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:22.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.109+0000 7f988d200640 1 --2- 172.21.15.6:0/891575450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f988810bb40 0x7f98881a3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:22.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.109+0000 7f988d200640 1 -- 172.21.15.6:0/891575450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9888106e40 msgr2=0x7f988819c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:22.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.109+0000 7f988d200640 1 --2- 172.21.15.6:0/891575450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9888106e40 0x7f988819c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:22.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.109+0000 7f988d200640 1 -- 172.21.15.6:0/891575450 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98881a3e70 con 0x7f9888105c00 2024-05-09T04:58:22.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.109+0000 7f988dc00640 1 --2- 172.21.15.6:0/891575450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f988810bb40 0x7f98881a3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:58:22.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.110+0000 7f988d200640 1 --2- 172.21.15.6:0/891575450 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9888105c00 0x7f988819c330 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9878031440 tx=0x7f9878003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:22.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.110+0000 7f9886000640 1 -- 172.21.15.6:0/891575450 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987803f070 con 0x7f9888105c00 2024-05-09T04:58:22.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.110+0000 7f9886000640 1 -- 172.21.15.6:0/891575450 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9878031c50 con 0x7f9888105c00 2024-05-09T04:58:22.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.110+0000 7f9886000640 1 -- 172.21.15.6:0/891575450 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9878035030 con 0x7f9888105c00 2024-05-09T04:58:22.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.110+0000 7f988e600640 1 -- 172.21.15.6:0/891575450 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98881a4100 con 0x7f9888105c00 2024-05-09T04:58:22.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.110+0000 7f988e600640 1 -- 172.21.15.6:0/891575450 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98881a45a0 con 0x7f9888105c00 2024-05-09T04:58:22.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.112+0000 7f988e600640 1 -- 172.21.15.6:0/891575450 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f988810a950 con 0x7f9888105c00 2024-05-09T04:58:22.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.115+0000 7f9886000640 1 -- 172.21.15.6:0/891575450 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f9878042070 con 0x7f9888105c00 2024-05-09T04:58:22.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.116+0000 7f9886000640 1 --2- 172.21.15.6:0/891575450 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f98680b2870 0x7f98680b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:22.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.116+0000 7f9886000640 1 -- 172.21.15.6:0/891575450 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f98781034d0 con 0x7f9888105c00 2024-05-09T04:58:22.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.117+0000 7f987fe00640 1 --2- 172.21.15.6:0/891575450 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f98680b2870 0x7f98680b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:22.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.117+0000 7f987fe00640 1 --2- 172.21.15.6:0/891575450 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f98680b2870 0x7f98680b4d80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f988819d3d0 tx=0x7f9870005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:22.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.118+0000 7f9886000640 1 -- 172.21.15.6:0/891575450 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f98780cae10 con 0x7f9888105c00 2024-05-09T04:58:22.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.345+0000 7f988e600640 1 -- 172.21.15.6:0/891575450 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f988802de00 con 0x7f9888105c00 2024-05-09T04:58:22.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.347+0000 7f9886000640 1 -- 172.21.15.6:0/891575450 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24659 (secure 0 0 0) 0x7f98780ca710 con 0x7f9888105c00 2024-05-09T04:58:22.349 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:58:22.349 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":89,"fsid":"f74d813c-0dbd-11ef-bc97-c7b262605968","created":"2024-05-09T04:39:35.287470+0000","modified":"2024-05-09T04:50:57.686719+0000","last_up_change":"2024-05-09T04:48:16.919842+0000","last_in_change":"2024-05-09T04:47:56.452645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:43:47.538306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:49:28.805447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T04:49:32.970287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T04:49:45.637000+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1b732fd3-e648-4b0d-aece-083bdaaa5d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6803","nonce":1738750914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6805","nonce":1738750914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6809","nonce":1738750914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6807","nonce":1738750914}]},"public_addr":"172.21.15.6:6803/1738750914","cluster_addr":"172.21.15.6:6805/1738750914","heartbeat_back_addr":"172.21.15.6:6809/1738750914","heartbeat_front_addr":"172.21.15.6:6807/1738750914","state":["exists","up"]},{"osd":1,"uuid":"9770c07d-4243-43bf-a0cd-9d2ac558556c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6801","nonce":2552207737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6803","nonce":2552207737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6807","nonce":2552207737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6805","nonce":2552207737}]},"public_addr":"172.21.15.169:6801/2552207737","cluster_addr":"172.21.15.169:6803/2552207737","heartbeat_back_addr":"172.21.15.169:6807/2552207737","heartbeat_front_addr":"172.21.15.169:6805/2552207737","state":["exists","up"]},{"osd":2,"uuid":"3f50ed89-b2a8-415e-b78d-8510d98d9368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6801","nonce":231435105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6803","nonce":231435105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6807","nonce":231435105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6805","nonce":231435105}]},"public_addr":"172.21.15.186:6801/231435105","cluster_addr":"172.21.15.186:6803/231435105","heartbeat_back_addr":"172.21.15.186:6807/231435105","heartbeat_front_addr":"172.21.15.186:6805/231435105","state":["exists","up"]},{"osd":3,"uuid":"afcbc36b-5669-4044-8dc5-2ebfe7970774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6811","nonce":2846613703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6813","nonce":2846613703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6817","nonce":2846613703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6815","nonce":2846613703}]},"public_addr":"172.21.15.6:6811/2846613703","cluster_addr":"172.21.15.6:6813/2846613703","heartbeat_back_addr":"172.21.15.6:6817/2846613703","heartbeat_front_addr":"172.21.15.6:6815/2846613703","state":["exists","up"]},{"osd":4,"uuid":"bb84cc5e-0297-4e23-be50-7ee2f7e59665","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6809","nonce":1928722453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6811","nonce":1928722453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6815","nonce":1928722453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6813","nonce":1928722453}]},"public_addr":"172.21.15.169:6809/1928722453","cluster_addr":"172.21.15.169:6811/1928722453","heartbeat_back_addr":"172.21.15.169:6815/1928722453","heartbeat_front_addr":"172.21.15.169:6813/1928722453","state":["exists","up"]},{"osd":5,"uuid":"830dc7f3-352e-4b01-8d4c-a9b68dcb40d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6809","nonce":3644445439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6811","nonce":3644445439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6815","nonce":3644445439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6813","nonce":3644445439}]},"public_addr":"172.21.15.186:6809/3644445439","cluster_addr":"172.21.15.186:6811/3644445439","heartbeat_back_addr":"172.21.15.186:6815/3644445439","heartbeat_front_addr":"172.21.15.186:6813/3644445439","state":["exists","up"]},{"osd":6,"uuid":"2e8478cb-c8c9-47ce-a3a5-48ffc959ab24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6819","nonce":3068252891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6821","nonce":3068252891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6825","nonce":3068252891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6823","nonce":3068252891}]},"public_addr":"172.21.15.6:6819/3068252891","cluster_addr":"172.21.15.6:6821/3068252891","heartbeat_back_addr":"172.21.15.6:6825/3068252891","heartbeat_front_addr":"172.21.15.6:6823/3068252891","state":["exists","up"]},{"osd":7,"uuid":"0f432ee2-42a4-4efa-9233-8e5a2fd65d10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6817","nonce":1303584711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6819","nonce":1303584711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6823","nonce":1303584711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6821","nonce":1303584711}]},"public_addr":"172.21.15.169:6817/1303584711","cluster_addr":"172.21.15.169:6819/1303584711","heartbeat_back_addr":"172.21.15.169:6823/1303584711","heartbeat_front_addr":"172.21.15.169:6821/1303584711","state":["exists","up"]},{"osd":8,"uuid":"cf5dca81-131b-48e7-9417-d01d4be47316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6817","nonce":3475163771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6819","nonce":3475163771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6823","nonce":3475163771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6821","nonce":3475163771}]},"public_addr":"172.21.15.186:6817/3475163771","cluster_addr":"172.21.15.186:6819/3475163771","heartbeat_back_addr":"172.21.15.186:6823/3475163771","heartbeat_front_addr":"172.21.15.186:6821/3475163771","state":["exists","up"]},{"osd":9,"uuid":"bca7bedd-1a57-4465-9033-064ca468add2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6827","nonce":3590990973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6829","nonce":3590990973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6833","nonce":3590990973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6831","nonce":3590990973}]},"public_addr":"172.21.15.6:6827/3590990973","cluster_addr":"172.21.15.6:6829/3590990973","heartbeat_back_addr":"172.21.15.6:6833/3590990973","heartbeat_front_addr":"172.21.15.6:6831/3590990973","state":["exists","up"]},{"osd":10,"uuid":"97b0e058-64f8-42ee-9b99-3ef8e39c8a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6825","nonce":61007341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6827","nonce":61007341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6831","nonce":61007341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6829","nonce":61007341}]},"public_addr":"172.21.15.169:6825/61007341","cluster_addr":"172.21.15.169:6827/61007341","heartbeat_back_addr":"172.21.15.169:6831/61007341","heartbeat_front_addr":"172.21.15.169:6829/61007341","state":["exists","up"]},{"osd":11,"uuid":"7f2220b0-24cf-48b2-95ae-1d66e00e57fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6825","nonce":1138940441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6827","nonce":1138940441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6831","nonce":1138940441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6829","nonce":1138940441}]},"public_addr":"172.21.15.186:6825/1138940441","cluster_addr":"172.21.15.186:6827/1138940441","heartbeat_back_addr":"172.21.15.186:6831/1138940441","heartbeat_front_addr":"172.21.15.186:6829/1138940441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:53.766295+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:20.195243+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:44.153812+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:20.472086+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:48.415371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:13.795415+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:50.236398+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:18.066623+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:43.781009+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:47:20.084719+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:47:48.219901+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:48:14.965536+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/3742411767":"2024-05-10T04:40:55.357057+0000","172.21.15.6:0/2410771702":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6801/937713898":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6800/937713898":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6801/3021503778":"2024-05-10T04:40:19.639151+0000","172.21.15.6:6800/3021503778":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/2559510850":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/3810474743":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/4133970555":"2024-05-10T04:40:55.357057+0000","172.21.15.6:0/664817743":"2024-05-10T04:40:19.639151+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:58:22.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.350+0000 7f988e600640 1 -- 172.21.15.6:0/891575450 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f98680b2870 msgr2=0x7f98680b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:22.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.350+0000 7f988e600640 1 --2- 172.21.15.6:0/891575450 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f98680b2870 0x7f98680b4d80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f988819d3d0 tx=0x7f9870005f90 comp rx=0 tx=0).stop 2024-05-09T04:58:22.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.350+0000 7f988e600640 1 -- 172.21.15.6:0/891575450 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9888105c00 msgr2=0x7f988819c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:22.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.350+0000 7f988e600640 1 --2- 172.21.15.6:0/891575450 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9888105c00 0x7f988819c330 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9878031440 tx=0x7f9878003c10 comp rx=0 tx=0).stop 2024-05-09T04:58:22.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.350+0000 7f988e600640 1 -- 172.21.15.6:0/891575450 shutdown_connections 2024-05-09T04:58:22.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.350+0000 7f988e600640 1 --2- 172.21.15.6:0/891575450 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f98680b2870 0x7f98680b4d80 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:22.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.350+0000 7f988e600640 1 --2- 172.21.15.6:0/891575450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f988810bb40 0x7f98881a3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:22.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.350+0000 7f988e600640 1 --2- 172.21.15.6:0/891575450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9888106e40 0x7f988819c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:22.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.350+0000 7f988e600640 1 --2- 172.21.15.6:0/891575450 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9888105c00 0x7f988819c330 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:22.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.350+0000 7f988e600640 1 -- 172.21.15.6:0/891575450 >> 172.21.15.6:0/891575450 conn(0x7f9888100df0 msgr2=0x7f9888107b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:22.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.351+0000 7f988e600640 1 -- 172.21.15.6:0/891575450 shutdown_connections 2024-05-09T04:58:22.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.351+0000 7f988e600640 1 -- 172.21.15.6:0/891575450 wait complete. 2024-05-09T04:58:22.381 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T04:43:47.538306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T04:49:28.805447+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T04:49:32.970287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T04:49:45.637000+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T04:58:22.381 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T04:58:22.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.676+0000 7fec5ca00640 1 -- 172.21.15.6:0/3679365534 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec580799d0 msgr2=0x7fec58079de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:22.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.676+0000 7fec5ca00640 1 --2- 172.21.15.6:0/3679365534 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec580799d0 0x7fec58079de0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fec50009be0 tx=0x7fec5002a6d0 comp rx=0 tx=0).stop 2024-05-09T04:58:22.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.677+0000 7fec5ca00640 1 -- 172.21.15.6:0/3679365534 shutdown_connections 2024-05-09T04:58:22.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.677+0000 7fec5ca00640 1 --2- 172.21.15.6:0/3679365534 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec580799d0 0x7fec58079de0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:22.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.677+0000 7fec5ca00640 1 --2- 172.21.15.6:0/3679365534 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec58078cd0 0x7fec580790e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:22.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.677+0000 7fec5ca00640 1 --2- 172.21.15.6:0/3679365534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec58077a40 0x7fec58078790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:22.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.677+0000 7fec5ca00640 1 -- 172.21.15.6:0/3679365534 >> 172.21.15.6:0/3679365534 conn(0x7fec58100e50 msgr2=0x7fec58103300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:22.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.677+0000 7fec5ca00640 1 -- 172.21.15.6:0/3679365534 shutdown_connections 2024-05-09T04:58:22.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.677+0000 7fec5ca00640 1 -- 172.21.15.6:0/3679365534 wait complete. 2024-05-09T04:58:22.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.678+0000 7fec5ca00640 1 Processor -- start 2024-05-09T04:58:22.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.678+0000 7fec5ca00640 1 -- start start 2024-05-09T04:58:22.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.678+0000 7fec5ca00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec58077a40 0x7fec581a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:22.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.678+0000 7fec5ca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec58078cd0 0x7fec581a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:22.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.678+0000 7fec5ca00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec580799d0 0x7fec581a7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:22.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.678+0000 7fec5ca00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fec58115b70 con 0x7fec58077a40 2024-05-09T04:58:22.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.678+0000 7fec5ca00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fec581159f0 con 0x7fec58078cd0 2024-05-09T04:58:22.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.678+0000 7fec5ca00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fec58115cf0 con 0x7fec580799d0 2024-05-09T04:58:22.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.679+0000 7fec57400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec58077a40 0x7fec581a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:22.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.679+0000 7fec57400640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec58077a40 0x7fec581a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39294/0 (socket says 172.21.15.6:39294) 2024-05-09T04:58:22.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.679+0000 7fec57400640 1 -- 172.21.15.6:0/2964448869 learned_addr learned my addr 172.21.15.6:0/2964448869 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:58:22.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.679+0000 7fec56a00640 1 --2- 172.21.15.6:0/2964448869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec58078cd0 0x7fec581a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:22.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.679+0000 7fec57e00640 1 --2- 172.21.15.6:0/2964448869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec580799d0 0x7fec581a7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:22.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.679+0000 7fec57400640 1 -- 172.21.15.6:0/2964448869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec580799d0 msgr2=0x7fec581a7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:22.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.679+0000 7fec57400640 1 --2- 172.21.15.6:0/2964448869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec580799d0 0x7fec581a7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:22.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.679+0000 7fec57400640 1 -- 172.21.15.6:0/2964448869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec58078cd0 msgr2=0x7fec581a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:22.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.679+0000 7fec57400640 1 --2- 172.21.15.6:0/2964448869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec58078cd0 0x7fec581a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:22.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.679+0000 7fec57400640 1 -- 172.21.15.6:0/2964448869 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec581a8210 con 0x7fec58077a40 2024-05-09T04:58:22.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.680+0000 7fec57400640 1 --2- 172.21.15.6:0/2964448869 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec58077a40 0x7fec581a06d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fec4800e9a0 tx=0x7fec4800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:22.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.680+0000 7fec4fe00640 1 -- 172.21.15.6:0/2964448869 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4800cd80 con 0x7fec58077a40 2024-05-09T04:58:22.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.680+0000 7fec4fe00640 1 -- 172.21.15.6:0/2964448869 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec480035f0 con 0x7fec58077a40 2024-05-09T04:58:22.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.680+0000 7fec5ca00640 1 -- 172.21.15.6:0/2964448869 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec581a8500 con 0x7fec58077a40 2024-05-09T04:58:22.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.680+0000 7fec4fe00640 1 -- 172.21.15.6:0/2964448869 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec48010690 con 0x7fec58077a40 2024-05-09T04:58:22.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.680+0000 7fec5ca00640 1 -- 172.21.15.6:0/2964448869 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec581a8a50 con 0x7fec58077a40 2024-05-09T04:58:22.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.681+0000 7fec5ca00640 1 -- 172.21.15.6:0/2964448869 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec58112220 con 0x7fec58077a40 2024-05-09T04:58:22.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.684+0000 7fec4fe00640 1 -- 172.21.15.6:0/2964448869 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7fec48004000 con 0x7fec58077a40 2024-05-09T04:58:22.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.687+0000 7fec4fe00640 1 --2- 172.21.15.6:0/2964448869 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fec3c0b27e0 0x7fec3c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:22.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.687+0000 7fec4fe00640 1 -- 172.21.15.6:0/2964448869 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fec480e2920 con 0x7fec58077a40 2024-05-09T04:58:22.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.687+0000 7fec56a00640 1 --2- 172.21.15.6:0/2964448869 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fec3c0b27e0 0x7fec3c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:22.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.688+0000 7fec56a00640 1 --2- 172.21.15.6:0/2964448869 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fec3c0b27e0 0x7fec3c0b4cf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fec400096c0 tx=0x7fec40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:22.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.690+0000 7fec4fe00640 1 -- 172.21.15.6:0/2964448869 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fec480042f0 con 0x7fec58077a40 2024-05-09T04:58:22.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.906+0000 7fec5ca00640 1 -- 172.21.15.6:0/2964448869 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fec580638d0 con 0x7fec58077a40 2024-05-09T04:58:22.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.909+0000 7fec4fe00640 1 -- 172.21.15.6:0/2964448869 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7fec480aa2f0 con 0x7fec58077a40 2024-05-09T04:58:22.910 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2024-05-09T04:58:22.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.912+0000 7fec5ca00640 1 -- 172.21.15.6:0/2964448869 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fec3c0b27e0 msgr2=0x7fec3c0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:22.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.912+0000 7fec5ca00640 1 --2- 172.21.15.6:0/2964448869 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fec3c0b27e0 0x7fec3c0b4cf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fec400096c0 tx=0x7fec40009260 comp rx=0 tx=0).stop 2024-05-09T04:58:22.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.912+0000 7fec5ca00640 1 -- 172.21.15.6:0/2964448869 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec58077a40 msgr2=0x7fec581a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:22.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.912+0000 7fec5ca00640 1 --2- 172.21.15.6:0/2964448869 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec58077a40 0x7fec581a06d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fec4800e9a0 tx=0x7fec4800ee60 comp rx=0 tx=0).stop 2024-05-09T04:58:22.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.912+0000 7fec5ca00640 1 -- 172.21.15.6:0/2964448869 shutdown_connections 2024-05-09T04:58:22.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.912+0000 7fec5ca00640 1 --2- 172.21.15.6:0/2964448869 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fec3c0b27e0 0x7fec3c0b4cf0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:22.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.912+0000 7fec5ca00640 1 --2- 172.21.15.6:0/2964448869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec580799d0 0x7fec581a7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:22.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.912+0000 7fec5ca00640 1 --2- 172.21.15.6:0/2964448869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec58078cd0 0x7fec581a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:22.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.912+0000 7fec5ca00640 1 --2- 172.21.15.6:0/2964448869 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec58077a40 0x7fec581a06d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:22.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.912+0000 7fec5ca00640 1 -- 172.21.15.6:0/2964448869 >> 172.21.15.6:0/2964448869 conn(0x7fec58100e50 msgr2=0x7fec581014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:22.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.913+0000 7fec5ca00640 1 -- 172.21.15.6:0/2964448869 shutdown_connections 2024-05-09T04:58:22.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:22.913+0000 7fec5ca00640 1 -- 172.21.15.6:0/2964448869 wait complete. 2024-05-09T04:58:22.943 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T04:58:23.169 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:22 smithi169 ceph-mon[34075]: pgmap v602: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 27 op/s 2024-05-09T04:58:23.169 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:22 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/891575450' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:58:23.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:22 smithi006 ceph-mon[29835]: pgmap v602: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 27 op/s 2024-05-09T04:58:23.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:22 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/891575450' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:58:23.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.235+0000 7f5b04e00640 1 -- 172.21.15.6:0/1904024762 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b00103a40 msgr2=0x7f5b00104740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:23.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.235+0000 7f5b04e00640 1 --2- 172.21.15.6:0/1904024762 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b00103a40 0x7f5b00104740 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5af0009ca0 tx=0x7f5af002a8c0 comp rx=0 tx=0).stop 2024-05-09T04:58:23.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.235+0000 7f5b04e00640 1 -- 172.21.15.6:0/1904024762 shutdown_connections 2024-05-09T04:58:23.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.235+0000 7f5b04e00640 1 --2- 172.21.15.6:0/1904024762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b00109980 0x7f5b0010bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:23.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.235+0000 7f5b04e00640 1 --2- 172.21.15.6:0/1904024762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b00104c80 0x7f5b00105090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:23.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.235+0000 7f5b04e00640 1 --2- 172.21.15.6:0/1904024762 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b00103a40 0x7f5b00104740 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:23.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.235+0000 7f5b04e00640 1 -- 172.21.15.6:0/1904024762 >> 172.21.15.6:0/1904024762 conn(0x7f5b000fec50 msgr2=0x7f5b001010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:23.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.236+0000 7f5b04e00640 1 -- 172.21.15.6:0/1904024762 shutdown_connections 2024-05-09T04:58:23.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.236+0000 7f5b04e00640 1 -- 172.21.15.6:0/1904024762 wait complete. 2024-05-09T04:58:23.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.237+0000 7f5b04e00640 1 Processor -- start 2024-05-09T04:58:23.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.237+0000 7f5b04e00640 1 -- start start 2024-05-09T04:58:23.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.237+0000 7f5b04e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b00103a40 0x7f5b0019c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:23.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.237+0000 7f5b04e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b00104c80 0x7f5b0019c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:23.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.237+0000 7f5b04e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b00109980 0x7f5b001a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:23.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.237+0000 7f5b04e00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0010f570 con 0x7f5b00104c80 2024-05-09T04:58:23.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.237+0000 7f5b04e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0010f3f0 con 0x7f5b00109980 2024-05-09T04:58:23.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.237+0000 7f5b04e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0010f6f0 con 0x7f5b00103a40 2024-05-09T04:58:23.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.238+0000 7f5afea00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b00104c80 0x7f5b0019c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:23.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.238+0000 7f5aff400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b00103a40 0x7f5b0019c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:23.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.238+0000 7f5afea00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b00104c80 0x7f5b0019c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39304/0 (socket says 172.21.15.6:39304) 2024-05-09T04:58:23.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.238+0000 7f5affe00640 1 --2- 172.21.15.6:0/3058557078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b00109980 0x7f5b001a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:23.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.238+0000 7f5afea00640 1 -- 172.21.15.6:0/3058557078 learned_addr learned my addr 172.21.15.6:0/3058557078 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:58:23.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.238+0000 7f5affe00640 1 -- 172.21.15.6:0/3058557078 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b00103a40 msgr2=0x7f5b0019c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:23.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.238+0000 7f5affe00640 1 --2- 172.21.15.6:0/3058557078 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b00103a40 0x7f5b0019c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:23.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.238+0000 7f5affe00640 1 -- 172.21.15.6:0/3058557078 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b00104c80 msgr2=0x7f5b0019c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:23.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.238+0000 7f5affe00640 1 --2- 172.21.15.6:0/3058557078 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b00104c80 0x7f5b0019c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:23.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.238+0000 7f5affe00640 1 -- 172.21.15.6:0/3058557078 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b001a3f20 con 0x7f5b00109980 2024-05-09T04:58:23.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.238+0000 7f5afea00640 1 --2- 172.21.15.6:0/3058557078 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b00104c80 0x7f5b0019c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:58:23.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.238+0000 7f5affe00640 1 --2- 172.21.15.6:0/3058557078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b00109980 0x7f5b001a38b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5af800e970 tx=0x7f5af800ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:23.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.239+0000 7f5af7e00640 1 -- 172.21.15.6:0/3058557078 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af800ccc0 con 0x7f5b00109980 2024-05-09T04:58:23.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.239+0000 7f5b04e00640 1 -- 172.21.15.6:0/3058557078 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b001a4210 con 0x7f5b00109980 2024-05-09T04:58:23.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.239+0000 7f5af7e00640 1 -- 172.21.15.6:0/3058557078 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5af8005020 con 0x7f5b00109980 2024-05-09T04:58:23.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.239+0000 7f5af7e00640 1 -- 172.21.15.6:0/3058557078 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af8004a00 con 0x7f5b00109980 2024-05-09T04:58:23.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.239+0000 7f5b04e00640 1 -- 172.21.15.6:0/3058557078 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b001a4710 con 0x7f5b00109980 2024-05-09T04:58:23.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.240+0000 7f5b04e00640 1 -- 172.21.15.6:0/3058557078 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b001a49d0 con 0x7f5b00109980 2024-05-09T04:58:23.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.243+0000 7f5af7e00640 1 -- 172.21.15.6:0/3058557078 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f5af8004ba0 con 0x7f5b00109980 2024-05-09T04:58:23.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.244+0000 7f5af7e00640 1 --2- 172.21.15.6:0/3058557078 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f5ae40b27e0 0x7f5ae40b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:23.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.245+0000 7f5af7e00640 1 -- 172.21.15.6:0/3058557078 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f5af80e2650 con 0x7f5b00109980 2024-05-09T04:58:23.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.245+0000 7f5aff400640 1 --2- 172.21.15.6:0/3058557078 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f5ae40b27e0 0x7f5ae40b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:23.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.245+0000 7f5aff400640 1 --2- 172.21.15.6:0/3058557078 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f5ae40b27e0 0x7f5ae40b4cf0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5af0000f80 tx=0x7f5af0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:23.247 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.246+0000 7f5af7e00640 1 -- 172.21.15.6:0/3058557078 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5af80aa020 con 0x7f5b00109980 2024-05-09T04:58:23.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:22 smithi186 ceph-mon[34668]: pgmap v602: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 27 op/s 2024-05-09T04:58:23.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:22 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/891575450' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:58:23.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.462+0000 7f5b04e00640 1 -- 172.21.15.6:0/3058557078 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5b001a4ba0 con 0x7f5b00109980 2024-05-09T04:58:23.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.464+0000 7f5af7e00640 1 -- 172.21.15.6:0/3058557078 <== mon.2 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f5af80a9920 con 0x7f5b00109980 2024-05-09T04:58:23.465 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2024-05-09T04:58:23.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.467+0000 7f5b04e00640 1 -- 172.21.15.6:0/3058557078 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f5ae40b27e0 msgr2=0x7f5ae40b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:23.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.467+0000 7f5b04e00640 1 --2- 172.21.15.6:0/3058557078 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f5ae40b27e0 0x7f5ae40b4cf0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5af0000f80 tx=0x7f5af0005fd0 comp rx=0 tx=0).stop 2024-05-09T04:58:23.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.467+0000 7f5b04e00640 1 -- 172.21.15.6:0/3058557078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b00109980 msgr2=0x7f5b001a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:23.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.467+0000 7f5b04e00640 1 --2- 172.21.15.6:0/3058557078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b00109980 0x7f5b001a38b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5af800e970 tx=0x7f5af800ee30 comp rx=0 tx=0).stop 2024-05-09T04:58:23.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.467+0000 7f5b04e00640 1 -- 172.21.15.6:0/3058557078 shutdown_connections 2024-05-09T04:58:23.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.467+0000 7f5b04e00640 1 --2- 172.21.15.6:0/3058557078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b00109980 0x7f5b001a38b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:23.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.467+0000 7f5b04e00640 1 --2- 172.21.15.6:0/3058557078 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b00104c80 0x7f5b0019c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:23.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.467+0000 7f5b04e00640 1 --2- 172.21.15.6:0/3058557078 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f5ae40b27e0 0x7f5ae40b4cf0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:23.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.467+0000 7f5b04e00640 1 --2- 172.21.15.6:0/3058557078 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b00103a40 0x7f5b0019c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:23.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.467+0000 7f5b04e00640 1 -- 172.21.15.6:0/3058557078 >> 172.21.15.6:0/3058557078 conn(0x7f5b000fec50 msgr2=0x7f5b000ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:23.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.468+0000 7f5b04e00640 1 -- 172.21.15.6:0/3058557078 shutdown_connections 2024-05-09T04:58:23.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.468+0000 7f5b04e00640 1 -- 172.21.15.6:0/3058557078 wait complete. 2024-05-09T04:58:23.496 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T04:58:23.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.782+0000 7f98e9000640 1 -- 172.21.15.6:0/2159014598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98e4104e60 msgr2=0x7f98e4105270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:23.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.782+0000 7f98e9000640 1 --2- 172.21.15.6:0/2159014598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98e4104e60 0x7f98e4105270 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f98d0009900 tx=0x7f98d002a3f0 comp rx=0 tx=0).stop 2024-05-09T04:58:23.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.782+0000 7f98e9000640 1 -- 172.21.15.6:0/2159014598 shutdown_connections 2024-05-09T04:58:23.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.782+0000 7f98e9000640 1 --2- 172.21.15.6:0/2159014598 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f98e4109b60 0x7f98e410bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:23.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.782+0000 7f98e9000640 1 --2- 172.21.15.6:0/2159014598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98e4104e60 0x7f98e4105270 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:23.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.782+0000 7f98e9000640 1 --2- 172.21.15.6:0/2159014598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98e4103c20 0x7f98e4104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:23.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.782+0000 7f98e9000640 1 -- 172.21.15.6:0/2159014598 >> 172.21.15.6:0/2159014598 conn(0x7f98e40fee10 msgr2=0x7f98e41012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:23.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.782+0000 7f98e9000640 1 -- 172.21.15.6:0/2159014598 shutdown_connections 2024-05-09T04:58:23.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.782+0000 7f98e9000640 1 -- 172.21.15.6:0/2159014598 wait complete. 2024-05-09T04:58:23.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.783+0000 7f98e9000640 1 Processor -- start 2024-05-09T04:58:23.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.783+0000 7f98e9000640 1 -- start start 2024-05-09T04:58:23.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.784+0000 7f98e9000640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f98e4103c20 0x7f98e419a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:23.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.784+0000 7f98e9000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98e4104e60 0x7f98e419a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:23.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.784+0000 7f98e9000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98e4109b60 0x7f98e41a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:23.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.784+0000 7f98e9000640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f98e410f740 con 0x7f98e4103c20 2024-05-09T04:58:23.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.784+0000 7f98e9000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f98e410f5c0 con 0x7f98e4109b60 2024-05-09T04:58:23.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.784+0000 7f98e9000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f98e410f8c0 con 0x7f98e4104e60 2024-05-09T04:58:23.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.784+0000 7f98e1e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f98e4103c20 0x7f98e419a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:23.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.784+0000 7f98e2800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98e4109b60 0x7f98e41a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:23.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.784+0000 7f98e1e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f98e4103c20 0x7f98e419a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39320/0 (socket says 172.21.15.6:39320) 2024-05-09T04:58:23.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.784+0000 7f98e1e00640 1 -- 172.21.15.6:0/3460649085 learned_addr learned my addr 172.21.15.6:0/3460649085 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:58:23.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.784+0000 7f98e1400640 1 --2- 172.21.15.6:0/3460649085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98e4104e60 0x7f98e419a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:23.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.785+0000 7f98e2800640 1 -- 172.21.15.6:0/3460649085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98e4104e60 msgr2=0x7f98e419a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:23.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.785+0000 7f98e2800640 1 --2- 172.21.15.6:0/3460649085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98e4104e60 0x7f98e419a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:23.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.785+0000 7f98e2800640 1 -- 172.21.15.6:0/3460649085 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f98e4103c20 msgr2=0x7f98e419a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:23.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.785+0000 7f98e2800640 1 --2- 172.21.15.6:0/3460649085 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f98e4103c20 0x7f98e419a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:23.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.785+0000 7f98e2800640 1 -- 172.21.15.6:0/3460649085 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e41a1e60 con 0x7f98e4109b60 2024-05-09T04:58:23.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.785+0000 7f98e1e00640 1 --2- 172.21.15.6:0/3460649085 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f98e4103c20 0x7f98e419a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:58:23.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.785+0000 7f98e2800640 1 --2- 172.21.15.6:0/3460649085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98e4109b60 0x7f98e41a17f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f98dc00e940 tx=0x7f98dc00ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:23.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.785+0000 7f98daa00640 1 -- 172.21.15.6:0/3460649085 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc004d20 con 0x7f98e4109b60 2024-05-09T04:58:23.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.785+0000 7f98daa00640 1 -- 172.21.15.6:0/3460649085 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98dc005020 con 0x7f98e4109b60 2024-05-09T04:58:23.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.785+0000 7f98daa00640 1 -- 172.21.15.6:0/3460649085 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc0116d0 con 0x7f98e4109b60 2024-05-09T04:58:23.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.785+0000 7f98e9000640 1 -- 172.21.15.6:0/3460649085 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98e41a2150 con 0x7f98e4109b60 2024-05-09T04:58:23.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.786+0000 7f98e9000640 1 -- 172.21.15.6:0/3460649085 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98e41a26a0 con 0x7f98e4109b60 2024-05-09T04:58:23.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.786+0000 7f98e9000640 1 -- 172.21.15.6:0/3460649085 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98e4108970 con 0x7f98e4109b60 2024-05-09T04:58:23.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.789+0000 7f98daa00640 1 -- 172.21.15.6:0/3460649085 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f98dc005460 con 0x7f98e4109b60 2024-05-09T04:58:23.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.791+0000 7f98daa00640 1 --2- 172.21.15.6:0/3460649085 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f98c80b2870 0x7f98c80b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:23.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.791+0000 7f98daa00640 1 -- 172.21.15.6:0/3460649085 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f98dc0e2990 con 0x7f98e4109b60 2024-05-09T04:58:23.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.791+0000 7f98e1e00640 1 --2- 172.21.15.6:0/3460649085 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f98c80b2870 0x7f98c80b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:23.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.792+0000 7f98e1e00640 1 --2- 172.21.15.6:0/3460649085 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f98c80b2870 0x7f98c80b4d80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009fd0 tx=0x7f98d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:23.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:23.793+0000 7f98daa00640 1 -- 172.21.15.6:0/3460649085 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f98dc016030 con 0x7f98e4109b60 2024-05-09T04:58:24.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.014+0000 7f98e9000640 1 -- 172.21.15.6:0/3460649085 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f98e40638d0 con 0x7f98e4109b60 2024-05-09T04:58:24.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.015+0000 7f98daa00640 1 -- 172.21.15.6:0/3460649085 <== mon.2 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f98dc0a9d00 con 0x7f98e4109b60 2024-05-09T04:58:24.016 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2024-05-09T04:58:24.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.018+0000 7f98e9000640 1 -- 172.21.15.6:0/3460649085 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f98c80b2870 msgr2=0x7f98c80b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:24.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.018+0000 7f98e9000640 1 --2- 172.21.15.6:0/3460649085 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f98c80b2870 0x7f98c80b4d80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009fd0 tx=0x7f98d4005f90 comp rx=0 tx=0).stop 2024-05-09T04:58:24.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.018+0000 7f98e9000640 1 -- 172.21.15.6:0/3460649085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98e4109b60 msgr2=0x7f98e41a17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:24.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.018+0000 7f98e9000640 1 --2- 172.21.15.6:0/3460649085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98e4109b60 0x7f98e41a17f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f98dc00e940 tx=0x7f98dc00ee00 comp rx=0 tx=0).stop 2024-05-09T04:58:24.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.018+0000 7f98e9000640 1 -- 172.21.15.6:0/3460649085 shutdown_connections 2024-05-09T04:58:24.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.018+0000 7f98e9000640 1 --2- 172.21.15.6:0/3460649085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98e4109b60 0x7f98e41a17f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:24.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.018+0000 7f98e9000640 1 --2- 172.21.15.6:0/3460649085 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f98c80b2870 0x7f98c80b4d80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:24.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.018+0000 7f98e9000640 1 --2- 172.21.15.6:0/3460649085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98e4104e60 0x7f98e419a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:24.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.018+0000 7f98e9000640 1 --2- 172.21.15.6:0/3460649085 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f98e4103c20 0x7f98e419a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:24.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.018+0000 7f98e9000640 1 -- 172.21.15.6:0/3460649085 >> 172.21.15.6:0/3460649085 conn(0x7f98e40fee10 msgr2=0x7f98e4105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:24.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.019+0000 7f98e9000640 1 -- 172.21.15.6:0/3460649085 shutdown_connections 2024-05-09T04:58:24.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.019+0000 7f98e9000640 1 -- 172.21.15.6:0/3460649085 wait complete. 2024-05-09T04:58:24.047 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T04:58:24.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:23 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/2964448869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T04:58:24.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:23 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/3058557078' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T04:58:24.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.334+0000 7fe6eac00640 1 -- 172.21.15.6:0/3669467046 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6e4105c60 msgr2=0x7fe6e4106960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:24.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.334+0000 7fe6eac00640 1 --2- 172.21.15.6:0/3669467046 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6e4105c60 0x7fe6e4106960 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8009ca0 tx=0x7fe6d802a8c0 comp rx=0 tx=0).stop 2024-05-09T04:58:24.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.335+0000 7fe6eac00640 1 -- 172.21.15.6:0/3669467046 shutdown_connections 2024-05-09T04:58:24.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.335+0000 7fe6eac00640 1 --2- 172.21.15.6:0/3669467046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6e410bba0 0x7fe6e410dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:24.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.335+0000 7fe6eac00640 1 --2- 172.21.15.6:0/3669467046 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6e4106ea0 0x7fe6e41072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:24.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.335+0000 7fe6eac00640 1 --2- 172.21.15.6:0/3669467046 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6e4105c60 0x7fe6e4106960 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:24.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.335+0000 7fe6eac00640 1 -- 172.21.15.6:0/3669467046 >> 172.21.15.6:0/3669467046 conn(0x7fe6e4100e50 msgr2=0x7fe6e4103300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:24.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.335+0000 7fe6eac00640 1 -- 172.21.15.6:0/3669467046 shutdown_connections 2024-05-09T04:58:24.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.335+0000 7fe6eac00640 1 -- 172.21.15.6:0/3669467046 wait complete. 2024-05-09T04:58:24.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.336+0000 7fe6eac00640 1 Processor -- start 2024-05-09T04:58:24.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.336+0000 7fe6eac00640 1 -- start start 2024-05-09T04:58:24.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.337+0000 7fe6eac00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6e4105c60 0x7fe6e419c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:24.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.337+0000 7fe6eac00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6e4106ea0 0x7fe6e419c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:24.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.337+0000 7fe6eac00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6e410bba0 0x7fe6e41a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:24.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.337+0000 7fe6eac00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e41117b0 con 0x7fe6e410bba0 2024-05-09T04:58:24.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.337+0000 7fe6eac00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e4111630 con 0x7fe6e4106ea0 2024-05-09T04:58:24.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.337+0000 7fe6eac00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e4111930 con 0x7fe6e4105c60 2024-05-09T04:58:24.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.337+0000 7fe6ea200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6e410bba0 0x7fe6e41a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:24.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.337+0000 7fe6e8e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6e4106ea0 0x7fe6e419c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:24.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.337+0000 7fe6e8e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6e4106ea0 0x7fe6e419c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:45052/0 (socket says 172.21.15.6:45052) 2024-05-09T04:58:24.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.337+0000 7fe6e8e00640 1 -- 172.21.15.6:0/4039212125 learned_addr learned my addr 172.21.15.6:0/4039212125 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:58:24.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.337+0000 7fe6e9800640 1 --2- 172.21.15.6:0/4039212125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6e4105c60 0x7fe6e419c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:24.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.338+0000 7fe6ea200640 1 -- 172.21.15.6:0/4039212125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6e4105c60 msgr2=0x7fe6e419c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:24.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.338+0000 7fe6ea200640 1 --2- 172.21.15.6:0/4039212125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6e4105c60 0x7fe6e419c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:24.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.338+0000 7fe6ea200640 1 -- 172.21.15.6:0/4039212125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6e4106ea0 msgr2=0x7fe6e419c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:24.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.338+0000 7fe6ea200640 1 --2- 172.21.15.6:0/4039212125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6e4106ea0 0x7fe6e419c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:24.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.338+0000 7fe6ea200640 1 -- 172.21.15.6:0/4039212125 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e41a3ed0 con 0x7fe6e410bba0 2024-05-09T04:58:24.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.338+0000 7fe6ea200640 1 --2- 172.21.15.6:0/4039212125 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6e410bba0 0x7fe6e41a3860 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc00e970 tx=0x7fe6dc00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:24.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.338+0000 7fe6e2000640 1 -- 172.21.15.6:0/4039212125 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc00ccc0 con 0x7fe6e410bba0 2024-05-09T04:58:24.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.338+0000 7fe6eac00640 1 -- 172.21.15.6:0/4039212125 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6e41a41c0 con 0x7fe6e410bba0 2024-05-09T04:58:24.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.338+0000 7fe6e2000640 1 -- 172.21.15.6:0/4039212125 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6dc005020 con 0x7fe6e410bba0 2024-05-09T04:58:24.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.338+0000 7fe6e2000640 1 -- 172.21.15.6:0/4039212125 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc004a00 con 0x7fe6e410bba0 2024-05-09T04:58:24.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.339+0000 7fe6eac00640 1 -- 172.21.15.6:0/4039212125 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6e41a4710 con 0x7fe6e410bba0 2024-05-09T04:58:24.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.340+0000 7fe6eac00640 1 -- 172.21.15.6:0/4039212125 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6e410a9b0 con 0x7fe6e410bba0 2024-05-09T04:58:24.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.343+0000 7fe6e2000640 1 -- 172.21.15.6:0/4039212125 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7fe6dc004ba0 con 0x7fe6e410bba0 2024-05-09T04:58:24.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.345+0000 7fe6e2000640 1 --2- 172.21.15.6:0/4039212125 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe6cc0b27e0 0x7fe6cc0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:24.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.345+0000 7fe6e2000640 1 -- 172.21.15.6:0/4039212125 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fe6dc0e2650 con 0x7fe6e410bba0 2024-05-09T04:58:24.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.346+0000 7fe6e9800640 1 --2- 172.21.15.6:0/4039212125 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe6cc0b27e0 0x7fe6cc0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:24.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.347+0000 7fe6e9800640 1 --2- 172.21.15.6:0/4039212125 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe6cc0b27e0 0x7fe6cc0b4cf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8000f80 tx=0x7fe6d8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:24.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.348+0000 7fe6e2000640 1 -- 172.21.15.6:0/4039212125 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe6dc0aa020 con 0x7fe6e410bba0 2024-05-09T04:58:24.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:23 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/2964448869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T04:58:24.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:23 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/3058557078' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T04:58:24.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:23 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/2964448869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T04:58:24.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:23 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/3058557078' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T04:58:24.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.560+0000 7fe6eac00640 1 -- 172.21.15.6:0/4039212125 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe6e40638d0 con 0x7fe6e410bba0 2024-05-09T04:58:24.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.561+0000 7fe6e2000640 1 -- 172.21.15.6:0/4039212125 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7fe6dc0a9920 con 0x7fe6e410bba0 2024-05-09T04:58:24.562 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2024-05-09T04:58:24.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.564+0000 7fe6eac00640 1 -- 172.21.15.6:0/4039212125 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe6cc0b27e0 msgr2=0x7fe6cc0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:24.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.564+0000 7fe6eac00640 1 --2- 172.21.15.6:0/4039212125 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe6cc0b27e0 0x7fe6cc0b4cf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8000f80 tx=0x7fe6d8005fd0 comp rx=0 tx=0).stop 2024-05-09T04:58:24.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.564+0000 7fe6eac00640 1 -- 172.21.15.6:0/4039212125 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6e410bba0 msgr2=0x7fe6e41a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:24.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.564+0000 7fe6eac00640 1 --2- 172.21.15.6:0/4039212125 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6e410bba0 0x7fe6e41a3860 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc00e970 tx=0x7fe6dc00ee30 comp rx=0 tx=0).stop 2024-05-09T04:58:24.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.564+0000 7fe6eac00640 1 -- 172.21.15.6:0/4039212125 shutdown_connections 2024-05-09T04:58:24.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.564+0000 7fe6eac00640 1 --2- 172.21.15.6:0/4039212125 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6e410bba0 0x7fe6e41a3860 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:24.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.564+0000 7fe6eac00640 1 --2- 172.21.15.6:0/4039212125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6e4106ea0 0x7fe6e419c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:24.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.564+0000 7fe6eac00640 1 --2- 172.21.15.6:0/4039212125 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fe6cc0b27e0 0x7fe6cc0b4cf0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:24.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.564+0000 7fe6eac00640 1 --2- 172.21.15.6:0/4039212125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6e4105c60 0x7fe6e419c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:24.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.564+0000 7fe6eac00640 1 -- 172.21.15.6:0/4039212125 >> 172.21.15.6:0/4039212125 conn(0x7fe6e4100e50 msgr2=0x7fe6e4107b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:24.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.565+0000 7fe6eac00640 1 -- 172.21.15.6:0/4039212125 shutdown_connections 2024-05-09T04:58:24.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.565+0000 7fe6eac00640 1 -- 172.21.15.6:0/4039212125 wait complete. 2024-05-09T04:58:24.591 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-09T04:58:24.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.869+0000 7f4b87200640 1 -- 172.21.15.6:0/2205422741 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b881039e0 msgr2=0x7f4b881046e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:24.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.869+0000 7f4b87200640 1 --2- 172.21.15.6:0/2205422741 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b881039e0 0x7f4b881046e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4b78009ca0 tx=0x7f4b7802a8c0 comp rx=0 tx=0).stop 2024-05-09T04:58:24.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.869+0000 7f4b87200640 1 -- 172.21.15.6:0/2205422741 shutdown_connections 2024-05-09T04:58:24.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.869+0000 7f4b87200640 1 --2- 172.21.15.6:0/2205422741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b88109920 0x7f4b8810bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:24.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.869+0000 7f4b87200640 1 --2- 172.21.15.6:0/2205422741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b88104c20 0x7f4b88105030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:24.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.869+0000 7f4b87200640 1 --2- 172.21.15.6:0/2205422741 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b881039e0 0x7f4b881046e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:24.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.869+0000 7f4b87200640 1 -- 172.21.15.6:0/2205422741 >> 172.21.15.6:0/2205422741 conn(0x7f4b880febb0 msgr2=0x7f4b88101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:24.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.870+0000 7f4b87200640 1 -- 172.21.15.6:0/2205422741 shutdown_connections 2024-05-09T04:58:24.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.870+0000 7f4b87200640 1 -- 172.21.15.6:0/2205422741 wait complete. 2024-05-09T04:58:24.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.871+0000 7f4b87200640 1 Processor -- start 2024-05-09T04:58:24.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.871+0000 7f4b87200640 1 -- start start 2024-05-09T04:58:24.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.871+0000 7f4b87200640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b881039e0 0x7f4b88076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:24.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.871+0000 7f4b87200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b88104c20 0x7f4b88075020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:24.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.871+0000 7f4b87200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b88109920 0x7f4b88075560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:24.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.871+0000 7f4b87200640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8810f4c0 con 0x7f4b881039e0 2024-05-09T04:58:24.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.871+0000 7f4b87200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8810f340 con 0x7f4b88109920 2024-05-09T04:58:24.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.871+0000 7f4b87200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8810f640 con 0x7f4b88104c20 2024-05-09T04:58:24.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.871+0000 7f4b85e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b881039e0 0x7f4b88076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:24.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.872+0000 7f4b85e00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b881039e0 0x7f4b88076910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39340/0 (socket says 172.21.15.6:39340) 2024-05-09T04:58:24.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.872+0000 7f4b85e00640 1 -- 172.21.15.6:0/1026281952 learned_addr learned my addr 172.21.15.6:0/1026281952 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:58:24.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.872+0000 7f4b7d400640 1 --2- 172.21.15.6:0/1026281952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b88104c20 0x7f4b88075020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:24.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.872+0000 7f4b86800640 1 --2- 172.21.15.6:0/1026281952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b88109920 0x7f4b88075560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:24.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.872+0000 7f4b85e00640 1 -- 172.21.15.6:0/1026281952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b88104c20 msgr2=0x7f4b88075020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:24.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.872+0000 7f4b85e00640 1 --2- 172.21.15.6:0/1026281952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b88104c20 0x7f4b88075020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:24.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.872+0000 7f4b85e00640 1 -- 172.21.15.6:0/1026281952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b88109920 msgr2=0x7f4b88075560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:24.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.872+0000 7f4b85e00640 1 --2- 172.21.15.6:0/1026281952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b88109920 0x7f4b88075560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:24.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.872+0000 7f4b85e00640 1 -- 172.21.15.6:0/1026281952 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b88075c90 con 0x7f4b881039e0 2024-05-09T04:58:24.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.872+0000 7f4b85e00640 1 --2- 172.21.15.6:0/1026281952 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b881039e0 0x7f4b88076910 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4b7802ade0 tx=0x7f4b780316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:24.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.873+0000 7f4b7f400640 1 -- 172.21.15.6:0/1026281952 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78003b70 con 0x7f4b881039e0 2024-05-09T04:58:24.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.873+0000 7f4b7f400640 1 -- 172.21.15.6:0/1026281952 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b78003d10 con 0x7f4b881039e0 2024-05-09T04:58:24.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.873+0000 7f4b87200640 1 -- 172.21.15.6:0/1026281952 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b881a4250 con 0x7f4b881039e0 2024-05-09T04:58:24.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.873+0000 7f4b7f400640 1 -- 172.21.15.6:0/1026281952 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7802fb90 con 0x7f4b881039e0 2024-05-09T04:58:24.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.873+0000 7f4b87200640 1 -- 172.21.15.6:0/1026281952 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b881a4650 con 0x7f4b881039e0 2024-05-09T04:58:24.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.874+0000 7f4b87200640 1 -- 172.21.15.6:0/1026281952 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b88069d80 con 0x7f4b881039e0 2024-05-09T04:58:24.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.876+0000 7f4b7f400640 1 -- 172.21.15.6:0/1026281952 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f4b7802fd30 con 0x7f4b881039e0 2024-05-09T04:58:24.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.878+0000 7f4b7f400640 1 --2- 172.21.15.6:0/1026281952 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4b6c0b27f0 0x7f4b6c0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:24.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.878+0000 7f4b7f400640 1 -- 172.21.15.6:0/1026281952 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f4b78103af0 con 0x7f4b881039e0 2024-05-09T04:58:24.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.878+0000 7f4b7d400640 1 --2- 172.21.15.6:0/1026281952 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4b6c0b27f0 0x7f4b6c0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:24.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.879+0000 7f4b7d400640 1 --2- 172.21.15.6:0/1026281952 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4b6c0b27f0 0x7f4b6c0b4d00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4b88076320 tx=0x7f4b7000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:24.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:24.880+0000 7f4b7f400640 1 -- 172.21.15.6:0/1026281952 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4b780cb430 con 0x7f4b881039e0 2024-05-09T04:58:25.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:25.149+0000 7f4b87200640 1 -- 172.21.15.6:0/1026281952 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f4b880638d0 con 0x7f4b881039e0 2024-05-09T04:58:25.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:25.165+0000 7f4b7f400640 1 -- 172.21.15.6:0/1026281952 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+815635 (secure 0 0 0) 0x7f4b780cad30 con 0x7f4b881039e0 2024-05-09T04:58:25.177 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:58:25.177 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-09T04:58:25.177 INFO:teuthology.orchestra.run.smithi006.stdout: "epoch": 20, 2024-05-09T04:58:25.177 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.178 INFO:teuthology.orchestra.run.smithi006.stdout: "active_gid": 14152, 2024-05-09T04:58:25.178 INFO:teuthology.orchestra.run.smithi006.stdout: "active_name": "x", 2024-05-09T04:58:25.178 INFO:teuthology.orchestra.run.smithi006.stdout: "active_addrs": { 2024-05-09T04:58:25.178 INFO:teuthology.orchestra.run.smithi006.stdout: "addrvec": [ 2024-05-09T04:58:25.178 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.178 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v2", 2024-05-09T04:58:25.178 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "172.21.15.6:6800", 2024-05-09T04:58:25.178 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 2997423326 2024-05-09T04:58:25.178 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.178 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.178 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v1", 2024-05-09T04:58:25.178 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "172.21.15.6:6801", 2024-05-09T04:58:25.178 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 2997423326 2024-05-09T04:58:25.178 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.179 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2024-05-09T04:58:25.179 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.179 INFO:teuthology.orchestra.run.smithi006.stdout: "active_addr": "172.21.15.6:6801/2997423326", 2024-05-09T04:58:25.179 INFO:teuthology.orchestra.run.smithi006.stdout: "active_change": "2024-05-09T04:40:55.357191+0000", 2024-05-09T04:58:25.179 INFO:teuthology.orchestra.run.smithi006.stdout: "active_mgr_features": 4540701547738038271, 2024-05-09T04:58:25.179 INFO:teuthology.orchestra.run.smithi006.stdout: "available": true, 2024-05-09T04:58:25.179 INFO:teuthology.orchestra.run.smithi006.stdout: "standbys": [ 2024-05-09T04:58:25.179 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.179 INFO:teuthology.orchestra.run.smithi006.stdout: "gid": 24112, 2024-05-09T04:58:25.179 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "z", 2024-05-09T04:58:25.179 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr_features": 4540701547738038271, 2024-05-09T04:58:25.179 INFO:teuthology.orchestra.run.smithi006.stdout: "available_modules": [ 2024-05-09T04:58:25.179 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.179 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "alerts", 2024-05-09T04:58:25.180 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.180 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.180 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.180 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-05-09T04:58:25.180 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-05-09T04:58:25.180 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.180 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.180 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.180 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-05-09T04:58:25.180 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.180 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.180 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.180 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T04:58:25.180 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.180 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.181 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.181 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.181 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.181 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.181 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.181 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.181 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.181 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.181 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.181 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.181 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.181 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.181 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.181 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.182 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.182 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.182 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.182 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.182 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.182 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.182 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.182 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.182 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.182 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.182 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.182 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.182 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.183 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.183 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.183 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.183 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.183 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.183 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.183 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.183 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.183 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.183 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.183 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.183 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.183 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.183 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.184 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.184 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.184 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.184 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.184 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.184 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.184 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.184 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.184 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.184 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.184 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.184 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.184 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.185 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.185 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.185 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.185 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.185 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.185 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.185 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.185 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.185 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.185 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.185 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.185 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.185 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.185 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.185 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.186 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.186 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.186 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.186 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_destination": { 2024-05-09T04:58:25.186 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_destination", 2024-05-09T04:58:25.186 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.186 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.186 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.186 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.186 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.186 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.186 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.186 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Email address to send alerts to", 2024-05-09T04:58:25.186 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.187 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.187 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.187 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.187 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_from_name": { 2024-05-09T04:58:25.187 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_from_name", 2024-05-09T04:58:25.187 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.187 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.187 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.187 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "Ceph", 2024-05-09T04:58:25.187 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.187 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.187 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.187 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Email From: name", 2024-05-09T04:58:25.187 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.187 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.187 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.188 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.188 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_host": { 2024-05-09T04:58:25.188 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_host", 2024-05-09T04:58:25.188 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.188 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.188 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.188 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.188 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.188 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.188 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.188 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP server", 2024-05-09T04:58:25.188 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.188 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.188 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.189 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.189 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_password": { 2024-05-09T04:58:25.189 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_password", 2024-05-09T04:58:25.189 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.189 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.189 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.189 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.189 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.189 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.189 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.189 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Password to authenticate with", 2024-05-09T04:58:25.189 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.189 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.189 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.189 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.190 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_port": { 2024-05-09T04:58:25.190 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_port", 2024-05-09T04:58:25.190 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.190 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.190 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.190 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "465", 2024-05-09T04:58:25.190 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.190 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.190 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.190 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP port", 2024-05-09T04:58:25.190 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.190 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.190 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.191 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.191 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_sender": { 2024-05-09T04:58:25.191 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_sender", 2024-05-09T04:58:25.191 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.191 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.191 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.191 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.191 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.191 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.191 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.191 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP envelope sender", 2024-05-09T04:58:25.191 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.191 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.192 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.192 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.192 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_ssl": { 2024-05-09T04:58:25.192 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_ssl", 2024-05-09T04:58:25.192 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.192 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.192 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.192 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.192 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.192 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.192 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.192 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T04:58:25.192 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.193 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.193 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.193 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.193 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_user": { 2024-05-09T04:58:25.193 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_user", 2024-05-09T04:58:25.193 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.193 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.193 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.193 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.193 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.193 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.193 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.193 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "User to authenticate as", 2024-05-09T04:58:25.194 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.194 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.194 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.194 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.194 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.194 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.194 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.194 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.194 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.194 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.194 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.194 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.194 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.195 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.195 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.195 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.195 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.195 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.195 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.195 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.195 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.195 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "balancer", 2024-05-09T04:58:25.195 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.195 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.195 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.196 INFO:teuthology.orchestra.run.smithi006.stdout: "active": { 2024-05-09T04:58:25.196 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "active", 2024-05-09T04:58:25.196 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.196 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.196 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.196 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.196 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.196 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.196 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.196 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T04:58:25.196 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.196 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.196 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.197 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.197 INFO:teuthology.orchestra.run.smithi006.stdout: "begin_time": { 2024-05-09T04:58:25.197 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "begin_time", 2024-05-09T04:58:25.197 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.197 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.197 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.197 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0000", 2024-05-09T04:58:25.197 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.197 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.197 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.197 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T04:58:25.197 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T04:58:25.198 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.198 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.198 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.198 INFO:teuthology.orchestra.run.smithi006.stdout: "begin_weekday": { 2024-05-09T04:58:25.198 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "begin_weekday", 2024-05-09T04:58:25.198 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-05-09T04:58:25.198 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.198 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.198 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.198 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0", 2024-05-09T04:58:25.198 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "6", 2024-05-09T04:58:25.198 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.198 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T04:58:25.199 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T04:58:25.199 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.199 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.199 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.199 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_max_iterations": { 2024-05-09T04:58:25.199 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_max_iterations", 2024-05-09T04:58:25.199 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-05-09T04:58:25.199 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.199 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.199 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "25", 2024-05-09T04:58:25.199 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-05-09T04:58:25.199 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "250", 2024-05-09T04:58:25.199 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.200 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T04:58:25.200 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.200 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.200 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.200 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.200 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_metrics": { 2024-05-09T04:58:25.200 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_metrics", 2024-05-09T04:58:25.200 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.200 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.200 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.200 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T04:58:25.200 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.201 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.201 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.201 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T04:58:25.201 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T04:58:25.201 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.201 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.201 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.201 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_step": { 2024-05-09T04:58:25.201 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_step", 2024-05-09T04:58:25.201 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-05-09T04:58:25.201 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.201 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.202 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0.5", 2024-05-09T04:58:25.202 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0.001", 2024-05-09T04:58:25.202 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "0.999", 2024-05-09T04:58:25.202 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.202 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "aggressiveness of optimization", 2024-05-09T04:58:25.202 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T04:58:25.202 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.202 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.202 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.202 INFO:teuthology.orchestra.run.smithi006.stdout: "end_time": { 2024-05-09T04:58:25.202 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "end_time", 2024-05-09T04:58:25.202 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.202 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.203 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.203 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "2359", 2024-05-09T04:58:25.203 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.203 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.203 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.203 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T04:58:25.203 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T04:58:25.203 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.203 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.203 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.203 INFO:teuthology.orchestra.run.smithi006.stdout: "end_weekday": { 2024-05-09T04:58:25.203 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "end_weekday", 2024-05-09T04:58:25.203 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-05-09T04:58:25.204 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.204 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.204 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.204 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0", 2024-05-09T04:58:25.204 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "6", 2024-05-09T04:58:25.204 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.204 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T04:58:25.204 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T04:58:25.204 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.204 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.204 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.204 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.204 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.204 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.205 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.205 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.205 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.205 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.205 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.205 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.205 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.205 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.205 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.205 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.205 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.205 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.206 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.206 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.206 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.206 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.206 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.206 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.206 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.206 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.206 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.206 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.206 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.206 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.207 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.207 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.207 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.207 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.207 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.207 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.207 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.207 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.207 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.207 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.207 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.207 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.208 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.208 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.208 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.208 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.208 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.208 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.208 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.208 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.208 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.208 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.208 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.209 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.209 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.209 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.209 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.209 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.209 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.209 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.209 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.209 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.209 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.209 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.209 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.209 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.210 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.210 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.210 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.210 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.210 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.210 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.210 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.210 INFO:teuthology.orchestra.run.smithi006.stdout: "min_score": { 2024-05-09T04:58:25.210 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "min_score", 2024-05-09T04:58:25.210 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-05-09T04:58:25.210 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.210 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.210 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.210 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.211 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.211 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.211 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T04:58:25.211 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.211 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.211 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.211 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.211 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": { 2024-05-09T04:58:25.211 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mode", 2024-05-09T04:58:25.211 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.211 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.211 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.212 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "upmap", 2024-05-09T04:58:25.212 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.212 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.212 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.212 INFO:teuthology.orchestra.run.smithi006.stdout: "crush-compat", 2024-05-09T04:58:25.212 INFO:teuthology.orchestra.run.smithi006.stdout: "none", 2024-05-09T04:58:25.212 INFO:teuthology.orchestra.run.smithi006.stdout: "read", 2024-05-09T04:58:25.212 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap", 2024-05-09T04:58:25.212 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap-read" 2024-05-09T04:58:25.212 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.212 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Balancer mode", 2024-05-09T04:58:25.212 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.213 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.213 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.213 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.213 INFO:teuthology.orchestra.run.smithi006.stdout: "pool_ids": { 2024-05-09T04:58:25.213 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pool_ids", 2024-05-09T04:58:25.213 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.213 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.213 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.213 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.213 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.213 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.213 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.214 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T04:58:25.214 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.214 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.214 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.214 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.214 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-05-09T04:58:25.214 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-05-09T04:58:25.214 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.214 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.214 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.214 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-05-09T04:58:25.214 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.214 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.214 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.215 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T04:58:25.215 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.215 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.215 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.215 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.215 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.215 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.215 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.215 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.215 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.215 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.215 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.216 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.216 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.216 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.216 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.216 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.216 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.216 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.216 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap_max_deviation": { 2024-05-09T04:58:25.216 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "upmap_max_deviation", 2024-05-09T04:58:25.216 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.216 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.217 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.217 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-05-09T04:58:25.217 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-05-09T04:58:25.217 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.217 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.217 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T04:58:25.217 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T04:58:25.217 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.217 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.217 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.217 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap_max_optimizations": { 2024-05-09T04:58:25.217 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "upmap_max_optimizations", 2024-05-09T04:58:25.217 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-05-09T04:58:25.217 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.218 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.218 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-05-09T04:58:25.218 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.218 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.218 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.218 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T04:58:25.218 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.218 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.218 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.218 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.218 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.218 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.218 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.218 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cephadm", 2024-05-09T04:58:25.219 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.219 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.219 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.219 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_down_multiplier": { 2024-05-09T04:58:25.219 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_down_multiplier", 2024-05-09T04:58:25.219 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-05-09T04:58:25.219 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.219 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.219 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3.0", 2024-05-09T04:58:25.219 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.219 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.219 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.219 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T04:58:25.219 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.220 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.220 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.220 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.220 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_refresh_rate": { 2024-05-09T04:58:25.220 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_refresh_rate", 2024-05-09T04:58:25.220 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.220 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.220 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.220 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "20", 2024-05-09T04:58:25.220 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.220 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.220 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.221 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T04:58:25.221 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.221 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.221 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.221 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.221 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_starting_port": { 2024-05-09T04:58:25.221 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_starting_port", 2024-05-09T04:58:25.221 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.221 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.221 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.221 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "4721", 2024-05-09T04:58:25.221 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.222 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.222 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.222 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T04:58:25.222 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.222 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.222 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.222 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.222 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_ptrace": { 2024-05-09T04:58:25.222 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_ptrace", 2024-05-09T04:58:25.222 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.222 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.222 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.222 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.223 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.223 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.223 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.223 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T04:58:25.223 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T04:58:25.223 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.223 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.223 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.223 INFO:teuthology.orchestra.run.smithi006.stdout: "autotune_interval": { 2024-05-09T04:58:25.223 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "autotune_interval", 2024-05-09T04:58:25.223 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.223 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.223 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.224 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-05-09T04:58:25.224 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.224 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.224 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.224 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T04:58:25.224 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.224 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.224 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.224 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.224 INFO:teuthology.orchestra.run.smithi006.stdout: "autotune_memory_target_ratio": { 2024-05-09T04:58:25.224 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T04:58:25.224 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-05-09T04:58:25.224 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.225 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.225 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0.7", 2024-05-09T04:58:25.225 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.225 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.225 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.225 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T04:58:25.225 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.225 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.225 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.225 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.225 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm_log_destination": { 2024-05-09T04:58:25.225 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cephadm_log_destination", 2024-05-09T04:58:25.226 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.226 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.226 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.226 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.226 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.226 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.226 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.226 INFO:teuthology.orchestra.run.smithi006.stdout: "file", 2024-05-09T04:58:25.226 INFO:teuthology.orchestra.run.smithi006.stdout: "file,syslog", 2024-05-09T04:58:25.226 INFO:teuthology.orchestra.run.smithi006.stdout: "syslog" 2024-05-09T04:58:25.226 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.227 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-09T04:58:25.227 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.227 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.227 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.227 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.227 INFO:teuthology.orchestra.run.smithi006.stdout: "cgroups_split": { 2024-05-09T04:58:25.227 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cgroups_split", 2024-05-09T04:58:25.227 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.227 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.227 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.227 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.227 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.228 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.228 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.228 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T04:58:25.228 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.228 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.228 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.228 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.228 INFO:teuthology.orchestra.run.smithi006.stdout: "config_checks_enabled": { 2024-05-09T04:58:25.228 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_checks_enabled", 2024-05-09T04:58:25.228 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.228 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.228 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.228 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.228 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.229 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.229 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.229 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T04:58:25.229 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.229 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.229 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.229 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.229 INFO:teuthology.orchestra.run.smithi006.stdout: "config_dashboard": { 2024-05-09T04:58:25.229 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_dashboard", 2024-05-09T04:58:25.229 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.229 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.229 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.229 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.229 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.229 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.229 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.230 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T04:58:25.230 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.230 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.230 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.230 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.230 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_alertmanager": { 2024-05-09T04:58:25.230 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_alertmanager", 2024-05-09T04:58:25.230 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.230 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.230 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.230 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T04:58:25.230 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.231 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.231 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.231 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-05-09T04:58:25.231 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.231 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.231 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.231 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.231 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_base": { 2024-05-09T04:58:25.231 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_base", 2024-05-09T04:58:25.231 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.231 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.232 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.232 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T04:58:25.232 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.232 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.232 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.232 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Container image name, without the tag", 2024-05-09T04:58:25.232 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.232 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.232 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.232 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.232 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_elasticsearch": { 2024-05-09T04:58:25.232 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_elasticsearch", 2024-05-09T04:58:25.233 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.233 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.233 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.233 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T04:58:25.233 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.233 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.233 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.233 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "elasticsearch container image", 2024-05-09T04:58:25.233 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.233 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.233 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.233 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.233 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_grafana": { 2024-05-09T04:58:25.234 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_grafana", 2024-05-09T04:58:25.234 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.234 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.234 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.234 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-09T04:58:25.234 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.234 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.234 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.234 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-05-09T04:58:25.234 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.234 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.234 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.234 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.234 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_haproxy": { 2024-05-09T04:58:25.235 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_haproxy", 2024-05-09T04:58:25.235 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.235 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.235 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.235 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T04:58:25.235 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.235 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.235 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.235 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "HAproxy container image", 2024-05-09T04:58:25.235 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.235 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.236 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.236 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.236 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_agent": { 2024-05-09T04:58:25.236 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_agent", 2024-05-09T04:58:25.236 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.236 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.236 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.236 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T04:58:25.236 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.236 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.236 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.236 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger agent container image", 2024-05-09T04:58:25.237 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.237 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.237 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.237 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.237 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_collector": { 2024-05-09T04:58:25.237 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_collector", 2024-05-09T04:58:25.237 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.237 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.237 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.237 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T04:58:25.238 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.238 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.238 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.238 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger collector container image", 2024-05-09T04:58:25.238 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.238 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.238 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.238 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.238 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_query": { 2024-05-09T04:58:25.238 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_query", 2024-05-09T04:58:25.238 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.239 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.239 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.239 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T04:58:25.239 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.239 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.239 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.239 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger query container image", 2024-05-09T04:58:25.239 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.239 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.239 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.239 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.239 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_keepalived": { 2024-05-09T04:58:25.240 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_keepalived", 2024-05-09T04:58:25.240 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.240 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.240 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.240 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T04:58:25.240 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.240 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.240 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.240 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Keepalived container image", 2024-05-09T04:58:25.240 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.240 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.241 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.241 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.241 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_loki": { 2024-05-09T04:58:25.241 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_loki", 2024-05-09T04:58:25.241 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.241 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.241 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.241 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T04:58:25.241 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.241 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.241 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.241 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Loki container image", 2024-05-09T04:58:25.241 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.242 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.242 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.242 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.242 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_node_exporter": { 2024-05-09T04:58:25.242 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_node_exporter", 2024-05-09T04:58:25.242 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.242 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.242 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.242 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T04:58:25.242 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.242 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.242 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.242 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-05-09T04:58:25.243 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.243 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.243 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.243 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.243 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_nvmeof": { 2024-05-09T04:58:25.243 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_nvmeof", 2024-05-09T04:58:25.243 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.243 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.243 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.243 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-09T04:58:25.243 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.243 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.244 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.244 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Nvme-of container image", 2024-05-09T04:58:25.244 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.244 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.244 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.244 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.244 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_prometheus": { 2024-05-09T04:58:25.244 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_prometheus", 2024-05-09T04:58:25.244 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.244 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.244 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.244 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T04:58:25.244 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.245 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.245 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.245 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-05-09T04:58:25.245 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.245 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.245 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.245 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.245 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_promtail": { 2024-05-09T04:58:25.245 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_promtail", 2024-05-09T04:58:25.245 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.245 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.245 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.246 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T04:58:25.246 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.246 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.246 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.246 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Promtail container image", 2024-05-09T04:58:25.246 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.246 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.246 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.246 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.246 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_samba": { 2024-05-09T04:58:25.246 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_samba", 2024-05-09T04:58:25.247 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.247 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.247 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.247 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-09T04:58:25.247 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.247 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.247 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.247 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Samba/SMB container image", 2024-05-09T04:58:25.247 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.247 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.247 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.248 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.248 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_snmp_gateway": { 2024-05-09T04:58:25.248 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_snmp_gateway", 2024-05-09T04:58:25.248 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.248 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.248 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.248 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T04:58:25.248 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.248 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.248 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.248 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SNMP Gateway container image", 2024-05-09T04:58:25.248 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.248 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.249 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.249 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.249 INFO:teuthology.orchestra.run.smithi006.stdout: "container_init": { 2024-05-09T04:58:25.249 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_init", 2024-05-09T04:58:25.249 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.249 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.249 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.249 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.249 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.249 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.249 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.249 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T04:58:25.249 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.250 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.250 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.250 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.250 INFO:teuthology.orchestra.run.smithi006.stdout: "daemon_cache_timeout": { 2024-05-09T04:58:25.250 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "daemon_cache_timeout", 2024-05-09T04:58:25.250 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.250 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.250 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.250 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-05-09T04:58:25.250 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.250 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.250 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.251 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T04:58:25.251 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.251 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.251 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.251 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.251 INFO:teuthology.orchestra.run.smithi006.stdout: "default_cephadm_command_timeout": { 2024-05-09T04:58:25.251 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T04:58:25.251 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.251 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.251 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.251 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "900", 2024-05-09T04:58:25.251 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.251 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.252 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.252 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T04:58:25.252 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.252 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.252 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.252 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.252 INFO:teuthology.orchestra.run.smithi006.stdout: "default_registry": { 2024-05-09T04:58:25.252 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "default_registry", 2024-05-09T04:58:25.252 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.252 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.252 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.252 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io", 2024-05-09T04:58:25.252 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.253 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.253 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.253 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T04:58:25.253 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.253 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.253 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.253 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.253 INFO:teuthology.orchestra.run.smithi006.stdout: "device_cache_timeout": { 2024-05-09T04:58:25.253 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_cache_timeout", 2024-05-09T04:58:25.253 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.253 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.253 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.253 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1800", 2024-05-09T04:58:25.254 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.254 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.254 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.254 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache device inventory", 2024-05-09T04:58:25.254 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.254 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.254 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.254 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.254 INFO:teuthology.orchestra.run.smithi006.stdout: "device_enhanced_scan": { 2024-05-09T04:58:25.254 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_enhanced_scan", 2024-05-09T04:58:25.254 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.254 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.254 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.255 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.255 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.255 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.255 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.255 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T04:58:25.255 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.255 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.255 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.255 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.255 INFO:teuthology.orchestra.run.smithi006.stdout: "facts_cache_timeout": { 2024-05-09T04:58:25.255 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "facts_cache_timeout", 2024-05-09T04:58:25.255 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.255 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.256 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.256 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-05-09T04:58:25.256 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.256 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.256 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.256 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache host facts data", 2024-05-09T04:58:25.256 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.256 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.256 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.256 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.256 INFO:teuthology.orchestra.run.smithi006.stdout: "grafana_dashboards_path": { 2024-05-09T04:58:25.256 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "grafana_dashboards_path", 2024-05-09T04:58:25.257 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.257 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.257 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.257 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-09T04:58:25.257 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.257 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.257 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.257 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-09T04:58:25.257 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.257 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.257 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.257 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.258 INFO:teuthology.orchestra.run.smithi006.stdout: "host_check_interval": { 2024-05-09T04:58:25.258 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "host_check_interval", 2024-05-09T04:58:25.258 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.258 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.258 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.258 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-05-09T04:58:25.258 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.258 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.258 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.258 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to perform a host check", 2024-05-09T04:58:25.258 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.258 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.259 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.259 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.259 INFO:teuthology.orchestra.run.smithi006.stdout: "hw_monitoring": { 2024-05-09T04:58:25.259 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "hw_monitoring", 2024-05-09T04:58:25.259 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.259 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.259 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.259 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.259 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.259 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.259 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.259 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T04:58:25.259 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.259 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.260 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.260 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.260 INFO:teuthology.orchestra.run.smithi006.stdout: "inventory_list_all": { 2024-05-09T04:58:25.260 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "inventory_list_all", 2024-05-09T04:58:25.260 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.260 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.260 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.260 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.260 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.260 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.260 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.260 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T04:58:25.260 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.261 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.261 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.261 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.261 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.261 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.261 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.261 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.261 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.261 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.261 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.261 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.261 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.261 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.262 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.262 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.262 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.262 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.262 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.262 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.262 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.262 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.262 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.262 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.262 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.262 INFO:teuthology.orchestra.run.smithi006.stdout: "log_refresh_metadata": { 2024-05-09T04:58:25.262 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_refresh_metadata", 2024-05-09T04:58:25.263 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.263 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.263 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.263 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.263 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.263 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.263 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.263 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T04:58:25.263 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.263 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.263 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.263 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.263 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.264 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.264 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.264 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.264 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.264 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.264 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.264 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.264 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.264 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T04:58:25.264 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.264 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.264 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.265 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.265 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.265 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.265 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.265 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.265 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.265 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.265 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.265 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.265 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.265 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.265 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.265 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.265 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.266 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.266 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.266 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.266 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.266 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.266 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.266 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.266 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.266 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.266 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.266 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.266 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.266 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.267 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.267 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.267 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.267 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.267 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.267 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.267 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.267 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.267 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.267 INFO:teuthology.orchestra.run.smithi006.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T04:58:25.267 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T04:58:25.267 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.267 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.267 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.268 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.268 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.268 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.268 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.268 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T04:58:25.268 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.268 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.268 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.268 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.268 INFO:teuthology.orchestra.run.smithi006.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T04:58:25.268 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T04:58:25.268 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.268 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.268 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.268 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "*", 2024-05-09T04:58:25.269 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.269 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.269 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.269 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T04:58:25.269 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.269 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.269 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.269 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.269 INFO:teuthology.orchestra.run.smithi006.stdout: "max_count_per_host": { 2024-05-09T04:58:25.269 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_count_per_host", 2024-05-09T04:58:25.269 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.269 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.269 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.269 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-05-09T04:58:25.269 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.270 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.270 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.270 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "max number of daemons per service per host", 2024-05-09T04:58:25.270 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.270 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.270 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.270 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.270 INFO:teuthology.orchestra.run.smithi006.stdout: "max_osd_draining_count": { 2024-05-09T04:58:25.270 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_osd_draining_count", 2024-05-09T04:58:25.270 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.270 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.270 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.270 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-05-09T04:58:25.270 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.270 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.270 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.271 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T04:58:25.271 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.271 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.271 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.271 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.271 INFO:teuthology.orchestra.run.smithi006.stdout: "migration_current": { 2024-05-09T04:58:25.271 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "migration_current", 2024-05-09T04:58:25.271 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.271 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.271 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.271 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.271 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.271 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.271 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.272 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "internal - do not modify", 2024-05-09T04:58:25.272 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.272 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.272 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.272 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.272 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": { 2024-05-09T04:58:25.272 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mode", 2024-05-09T04:58:25.272 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.272 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.272 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.272 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "root", 2024-05-09T04:58:25.272 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.272 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.272 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.272 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm-package", 2024-05-09T04:58:25.273 INFO:teuthology.orchestra.run.smithi006.stdout: "root" 2024-05-09T04:58:25.273 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.273 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T04:58:25.273 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.273 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.273 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.273 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.273 INFO:teuthology.orchestra.run.smithi006.stdout: "oob_default_addr": { 2024-05-09T04:58:25.273 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "oob_default_addr", 2024-05-09T04:58:25.273 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.273 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.273 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.273 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "169.254.1.1", 2024-05-09T04:58:25.273 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.273 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.274 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.274 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T04:58:25.274 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.274 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.274 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.274 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.274 INFO:teuthology.orchestra.run.smithi006.stdout: "prometheus_alerts_path": { 2024-05-09T04:58:25.274 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prometheus_alerts_path", 2024-05-09T04:58:25.274 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.274 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.274 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.274 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T04:58:25.274 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.275 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.275 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.275 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T04:58:25.275 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.275 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.275 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.275 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.275 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_insecure": { 2024-05-09T04:58:25.275 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_insecure", 2024-05-09T04:58:25.275 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.275 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.275 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.275 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.276 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.276 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.276 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.276 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T04:58:25.276 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.276 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.276 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.276 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.276 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_password": { 2024-05-09T04:58:25.276 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_password", 2024-05-09T04:58:25.276 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.276 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.276 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.276 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.276 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.276 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.277 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.277 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T04:58:25.277 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.277 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.277 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.277 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.277 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_url": { 2024-05-09T04:58:25.277 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_url", 2024-05-09T04:58:25.277 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.277 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.277 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.277 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.277 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.277 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.277 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.278 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T04:58:25.278 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.278 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.278 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.278 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.278 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_username": { 2024-05-09T04:58:25.278 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_username", 2024-05-09T04:58:25.278 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.278 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.278 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.278 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.278 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.278 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.278 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.279 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T04:58:25.279 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.279 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.279 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.279 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.279 INFO:teuthology.orchestra.run.smithi006.stdout: "secure_monitoring_stack": { 2024-05-09T04:58:25.279 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "secure_monitoring_stack", 2024-05-09T04:58:25.279 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.279 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.279 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.279 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.279 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.279 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.279 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.279 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T04:58:25.280 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.280 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.280 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.280 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.280 INFO:teuthology.orchestra.run.smithi006.stdout: "service_discovery_port": { 2024-05-09T04:58:25.280 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "service_discovery_port", 2024-05-09T04:58:25.280 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.280 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.280 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.280 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8765", 2024-05-09T04:58:25.280 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.280 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.280 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.280 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "cephadm service discovery port", 2024-05-09T04:58:25.280 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.280 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.280 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.281 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.281 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.281 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.281 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.281 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.281 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.281 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.281 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.281 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.281 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.281 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.281 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.281 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.281 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.281 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.282 INFO:teuthology.orchestra.run.smithi006.stdout: "ssh_config_file": { 2024-05-09T04:58:25.282 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssh_config_file", 2024-05-09T04:58:25.282 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.282 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.282 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.282 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.282 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.282 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.282 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.282 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T04:58:25.282 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.282 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.282 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.282 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.282 INFO:teuthology.orchestra.run.smithi006.stdout: "use_agent": { 2024-05-09T04:58:25.283 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "use_agent", 2024-05-09T04:58:25.283 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.283 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.283 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.283 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.283 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.283 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.283 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.283 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T04:58:25.283 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.283 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.283 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.283 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.283 INFO:teuthology.orchestra.run.smithi006.stdout: "use_repo_digest": { 2024-05-09T04:58:25.283 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "use_repo_digest", 2024-05-09T04:58:25.283 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.284 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.284 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.284 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.284 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.284 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.284 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.284 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T04:58:25.284 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.284 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.284 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.284 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.284 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_failed_host_check": { 2024-05-09T04:58:25.284 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_failed_host_check", 2024-05-09T04:58:25.284 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.284 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.285 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.285 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.285 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.285 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.285 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.285 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T04:58:25.285 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.285 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.285 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.285 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.285 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_stray_daemons": { 2024-05-09T04:58:25.285 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_stray_daemons", 2024-05-09T04:58:25.285 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.285 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.286 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.286 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.286 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.286 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.286 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.286 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T04:58:25.286 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.286 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.286 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.286 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.286 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_stray_hosts": { 2024-05-09T04:58:25.286 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_stray_hosts", 2024-05-09T04:58:25.286 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.286 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.287 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.287 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.287 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.287 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.287 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.287 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T04:58:25.287 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.287 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.287 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.287 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.287 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.287 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.287 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.288 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crash", 2024-05-09T04:58:25.288 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.288 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.288 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.288 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.288 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.288 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.288 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.288 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.288 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.288 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.288 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.288 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.288 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.288 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.288 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.289 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.289 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.289 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.289 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.289 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.289 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.289 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.289 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.289 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.289 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.289 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.289 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.289 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.289 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.289 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.289 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.290 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.290 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.290 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.290 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.290 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.290 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.290 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.290 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.290 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.290 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.290 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.290 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.290 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.290 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.291 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.291 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.291 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.291 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.291 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.291 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.291 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.291 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.291 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.291 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.291 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.291 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.292 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.292 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.292 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.292 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.292 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.292 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.292 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.292 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.292 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.292 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.292 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.292 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.292 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.293 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.293 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.293 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.293 INFO:teuthology.orchestra.run.smithi006.stdout: "retain_interval": { 2024-05-09T04:58:25.293 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "retain_interval", 2024-05-09T04:58:25.293 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.293 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.293 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.293 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "31536000", 2024-05-09T04:58:25.293 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.293 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.293 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.293 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T04:58:25.293 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.293 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.293 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.294 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.294 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.294 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.294 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.294 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.294 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.294 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.294 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.294 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.294 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.294 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.294 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.294 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.295 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.295 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.295 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_recent_interval": { 2024-05-09T04:58:25.295 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_recent_interval", 2024-05-09T04:58:25.295 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.295 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.295 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.295 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1209600", 2024-05-09T04:58:25.295 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.295 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.295 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.295 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T04:58:25.295 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.295 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.295 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.295 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.295 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.296 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.296 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.296 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "dashboard", 2024-05-09T04:58:25.296 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.296 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.296 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.296 INFO:teuthology.orchestra.run.smithi006.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T04:58:25.296 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T04:58:25.296 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.296 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.296 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.296 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-05-09T04:58:25.296 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.296 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.296 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.296 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.297 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.297 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.297 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.297 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.297 INFO:teuthology.orchestra.run.smithi006.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T04:58:25.297 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T04:58:25.297 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.297 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.297 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.297 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.297 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.297 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.297 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.297 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.298 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.298 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.298 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.298 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.298 INFO:teuthology.orchestra.run.smithi006.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T04:58:25.298 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T04:58:25.298 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.298 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.298 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.298 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.298 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.298 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.298 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.298 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.298 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.299 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.299 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.299 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.299 INFO:teuthology.orchestra.run.smithi006.stdout: "AUDIT_API_ENABLED": { 2024-05-09T04:58:25.299 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T04:58:25.299 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.299 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.299 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.299 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.299 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.299 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.299 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.299 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.299 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.299 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.299 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.300 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.300 INFO:teuthology.orchestra.run.smithi006.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T04:58:25.300 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T04:58:25.300 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.300 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.300 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.300 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.300 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.300 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.300 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.300 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.300 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.300 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.300 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.300 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.300 INFO:teuthology.orchestra.run.smithi006.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T04:58:25.301 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T04:58:25.301 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.301 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.301 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.301 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.301 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.301 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.301 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.301 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.301 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.301 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.301 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.301 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.301 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T04:58:25.301 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T04:58:25.301 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.302 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.302 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.302 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.302 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.302 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.302 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.302 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.302 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.302 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.302 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.302 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.302 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T04:58:25.302 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T04:58:25.302 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.302 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.303 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.303 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.303 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.303 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.303 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.303 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.303 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.303 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.303 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.303 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.303 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T04:58:25.303 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T04:58:25.303 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.303 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.303 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.303 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.304 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.304 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.304 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.304 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.304 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.304 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.304 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.304 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.304 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T04:58:25.304 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T04:58:25.304 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.304 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.304 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.304 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.304 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.304 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.305 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.305 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.305 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.305 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.305 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.305 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.305 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T04:58:25.305 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T04:58:25.305 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.305 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.305 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.305 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.305 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.305 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.305 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.305 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.306 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.306 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.306 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.306 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.306 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T04:58:25.306 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T04:58:25.306 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.306 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.306 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.306 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.306 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.306 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.306 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.306 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.306 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.307 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.307 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.307 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.307 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T04:58:25.307 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T04:58:25.307 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.307 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.307 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.307 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.307 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.307 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.307 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.307 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.307 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.307 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.308 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.308 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.308 INFO:teuthology.orchestra.run.smithi006.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T04:58:25.308 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T04:58:25.308 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.308 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.308 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.308 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.308 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.308 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.308 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.308 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.308 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.308 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.308 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.309 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.309 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T04:58:25.309 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T04:58:25.309 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.309 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.309 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.309 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2024-05-09T04:58:25.309 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.309 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.309 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.309 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.309 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.309 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.309 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.310 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.310 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T04:58:25.310 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T04:58:25.310 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.310 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.310 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.310 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.310 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.310 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.310 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.310 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.310 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.310 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.311 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.311 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.311 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_URL": { 2024-05-09T04:58:25.311 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_URL", 2024-05-09T04:58:25.311 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.311 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.311 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.311 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.311 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.311 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.311 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.311 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.311 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.312 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.312 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.312 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.312 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T04:58:25.312 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T04:58:25.312 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.312 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.312 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.312 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2024-05-09T04:58:25.312 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.312 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.312 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.312 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.312 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.312 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.313 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.313 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.313 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T04:58:25.313 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T04:58:25.313 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.313 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.313 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.313 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.313 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.313 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.313 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.313 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.313 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.313 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.313 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.314 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.314 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T04:58:25.314 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T04:58:25.314 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.314 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.314 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.314 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.314 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.314 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.314 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.314 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.314 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.314 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.314 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.314 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.314 INFO:teuthology.orchestra.run.smithi006.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T04:58:25.315 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T04:58:25.315 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.315 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.315 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.315 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.315 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.315 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.315 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.315 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.315 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.315 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.315 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.315 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.315 INFO:teuthology.orchestra.run.smithi006.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T04:58:25.315 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T04:58:25.315 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.316 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.316 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.316 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.316 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.316 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.316 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.316 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.316 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.316 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.316 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.316 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.316 INFO:teuthology.orchestra.run.smithi006.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-09T04:58:25.316 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-09T04:58:25.316 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.316 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.316 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.316 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "[]", 2024-05-09T04:58:25.317 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.317 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.317 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.317 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.317 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.317 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.317 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.317 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.317 INFO:teuthology.orchestra.run.smithi006.stdout: "MULTICLUSTER_CONFIG": { 2024-05-09T04:58:25.317 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-09T04:58:25.317 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.317 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.317 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.317 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "{}", 2024-05-09T04:58:25.317 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.317 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.318 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.318 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.318 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.318 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.318 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.318 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.318 INFO:teuthology.orchestra.run.smithi006.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T04:58:25.318 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T04:58:25.318 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.322 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.322 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.322 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.322 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.322 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.322 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.322 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.322 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.322 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.322 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.323 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.323 INFO:teuthology.orchestra.run.smithi006.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T04:58:25.323 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T04:58:25.323 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.323 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.323 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.323 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.323 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.323 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.323 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.323 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.323 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.323 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.323 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.324 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.324 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T04:58:25.324 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T04:58:25.324 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.324 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.324 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.324 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.324 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.324 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.324 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.324 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.324 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.324 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.324 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.324 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.325 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T04:58:25.325 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T04:58:25.325 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.325 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.325 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.325 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.325 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.325 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.325 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.325 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.325 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.325 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.325 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.325 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.326 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T04:58:25.326 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T04:58:25.326 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.326 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.326 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.326 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.326 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.326 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.326 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.326 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.326 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.326 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.326 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.326 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.326 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T04:58:25.327 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T04:58:25.327 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.327 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.327 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.327 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.327 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.327 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.327 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.327 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.327 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.327 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.327 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.327 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.327 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T04:58:25.328 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T04:58:25.328 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.328 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.328 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.328 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.328 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.328 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.328 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.328 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.328 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.328 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.328 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.328 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.328 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T04:58:25.328 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T04:58:25.328 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.329 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.329 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.329 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.329 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.329 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.329 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.329 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.329 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.329 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.329 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.329 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.329 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T04:58:25.329 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T04:58:25.329 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.329 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.329 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.330 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.330 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.330 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.330 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.330 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.330 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.330 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.330 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.330 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.330 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T04:58:25.330 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T04:58:25.330 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.330 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.330 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.330 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.330 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.331 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.331 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.331 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.331 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.331 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.331 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.331 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.331 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T04:58:25.331 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T04:58:25.331 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.331 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.331 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.331 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T04:58:25.331 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.331 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.332 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.332 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.332 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.332 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.332 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.332 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.332 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T04:58:25.332 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T04:58:25.332 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.332 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.332 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.332 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-05-09T04:58:25.332 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.332 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.332 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.333 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.333 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.333 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.333 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.333 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.333 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T04:58:25.333 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T04:58:25.333 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.333 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.333 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.333 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8", 2024-05-09T04:58:25.333 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.333 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.333 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.333 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.333 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.333 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.334 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.334 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.334 INFO:teuthology.orchestra.run.smithi006.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T04:58:25.334 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T04:58:25.334 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.334 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.334 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.334 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "45", 2024-05-09T04:58:25.334 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.334 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.334 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.334 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.334 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.334 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.334 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.335 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.335 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T04:58:25.335 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T04:58:25.335 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.335 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.335 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.335 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.335 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.335 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.335 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.335 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.335 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.335 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.335 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.335 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.335 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T04:58:25.336 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T04:58:25.336 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.336 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.336 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.336 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2024-05-09T04:58:25.336 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.336 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.336 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.336 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.336 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.336 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.336 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.336 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.336 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T04:58:25.336 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T04:58:25.336 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.337 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.337 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.337 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.337 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.337 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.337 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.337 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.337 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.337 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.337 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.337 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.337 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T04:58:25.337 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T04:58:25.337 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.337 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.338 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.338 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.338 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.338 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.338 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.338 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.338 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.338 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.338 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.338 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.338 INFO:teuthology.orchestra.run.smithi006.stdout: "UNSAFE_TLS_v1_2": { 2024-05-09T04:58:25.338 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-09T04:58:25.338 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.338 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.339 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.339 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.339 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.339 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.339 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.339 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.339 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.339 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.339 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.339 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.339 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T04:58:25.339 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T04:58:25.339 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.339 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.339 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.339 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.340 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.340 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.340 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.340 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.340 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.340 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.340 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.340 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.340 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T04:58:25.340 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T04:58:25.340 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.340 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.340 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.340 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-05-09T04:58:25.340 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.340 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.341 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.341 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.341 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.341 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.341 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.341 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.341 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T04:58:25.341 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T04:58:25.341 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.341 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.341 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.341 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-05-09T04:58:25.341 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.341 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.341 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.342 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.342 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.342 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.342 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.342 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.342 INFO:teuthology.orchestra.run.smithi006.stdout: "cross_origin_url": { 2024-05-09T04:58:25.342 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cross_origin_url", 2024-05-09T04:58:25.342 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.342 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.342 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.342 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.342 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.342 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.342 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.343 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.343 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.343 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.343 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.343 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.343 INFO:teuthology.orchestra.run.smithi006.stdout: "crt_file": { 2024-05-09T04:58:25.343 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crt_file", 2024-05-09T04:58:25.343 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.343 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.343 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.343 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.343 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.343 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.343 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.343 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.344 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.344 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.344 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.344 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.344 INFO:teuthology.orchestra.run.smithi006.stdout: "debug": { 2024-05-09T04:58:25.344 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "debug", 2024-05-09T04:58:25.344 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.344 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.344 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.344 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.344 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.344 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.344 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.344 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable/disable debug options", 2024-05-09T04:58:25.345 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.345 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.345 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.345 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.345 INFO:teuthology.orchestra.run.smithi006.stdout: "jwt_token_ttl": { 2024-05-09T04:58:25.345 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "jwt_token_ttl", 2024-05-09T04:58:25.345 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.345 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.345 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.345 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "28800", 2024-05-09T04:58:25.345 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.345 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.345 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.345 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.345 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.345 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.345 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.346 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.346 INFO:teuthology.orchestra.run.smithi006.stdout: "key_file": { 2024-05-09T04:58:25.346 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "key_file", 2024-05-09T04:58:25.346 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.346 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.346 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.346 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.346 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.346 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.346 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.346 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.346 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.346 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.346 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.346 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.346 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.347 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.347 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.347 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.347 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.347 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.347 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.347 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.347 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.347 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.347 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.347 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.347 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.347 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.347 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.347 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.348 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.348 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.348 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.348 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.348 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.348 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.348 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.348 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.348 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.348 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.348 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.348 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.348 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.348 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.348 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.349 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.349 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.349 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.349 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.349 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.349 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.349 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.349 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.349 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.349 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.349 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.349 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.349 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.349 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.350 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.350 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.350 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.350 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.350 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.350 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.350 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.350 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.350 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.350 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.350 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.350 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.350 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.350 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.351 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.351 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.351 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.351 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.351 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.351 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.351 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.351 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.351 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.351 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.351 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.351 INFO:teuthology.orchestra.run.smithi006.stdout: "motd": { 2024-05-09T04:58:25.351 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "motd", 2024-05-09T04:58:25.351 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.351 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.351 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.352 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.352 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.352 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.352 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.352 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "The message of the day", 2024-05-09T04:58:25.352 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.352 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.352 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.352 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.352 INFO:teuthology.orchestra.run.smithi006.stdout: "redirect_resolve_ip_addr": { 2024-05-09T04:58:25.352 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T04:58:25.352 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.352 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.352 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.352 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.353 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.353 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.353 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.353 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.353 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.353 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.353 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.353 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.353 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2024-05-09T04:58:25.353 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2024-05-09T04:58:25.353 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.353 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.353 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.353 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "::", 2024-05-09T04:58:25.353 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.353 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.354 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.354 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.354 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.354 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.354 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.354 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.354 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2024-05-09T04:58:25.354 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2024-05-09T04:58:25.354 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.354 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.354 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.354 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8080", 2024-05-09T04:58:25.354 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.354 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.354 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.355 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.355 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.355 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.355 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.355 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.355 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.355 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.355 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.355 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.355 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.355 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.355 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.355 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.355 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.355 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.356 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.356 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.356 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.356 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.356 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl": { 2024-05-09T04:58:25.356 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl", 2024-05-09T04:58:25.356 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.356 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.356 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.356 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.356 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.356 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.356 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.356 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.356 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.356 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.356 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.357 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.357 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl_server_port": { 2024-05-09T04:58:25.357 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl_server_port", 2024-05-09T04:58:25.357 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.357 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.357 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.357 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8443", 2024-05-09T04:58:25.357 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.357 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.357 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.357 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.357 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.357 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.357 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.357 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.357 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_behaviour": { 2024-05-09T04:58:25.358 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_behaviour", 2024-05-09T04:58:25.358 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.358 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.358 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.358 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "redirect", 2024-05-09T04:58:25.358 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.358 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.358 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.358 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.358 INFO:teuthology.orchestra.run.smithi006.stdout: "redirect" 2024-05-09T04:58:25.358 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.358 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.358 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.358 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.358 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.359 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.359 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_error_status_code": { 2024-05-09T04:58:25.359 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_error_status_code", 2024-05-09T04:58:25.359 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.359 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.359 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.359 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "500", 2024-05-09T04:58:25.359 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "400", 2024-05-09T04:58:25.359 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "599", 2024-05-09T04:58:25.359 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.359 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.359 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.359 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.359 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.359 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.360 INFO:teuthology.orchestra.run.smithi006.stdout: "url_prefix": { 2024-05-09T04:58:25.360 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "url_prefix", 2024-05-09T04:58:25.360 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.360 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.360 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.360 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.360 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.360 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.360 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.360 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.360 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.360 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.360 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.360 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.360 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.360 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.361 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.361 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "devicehealth", 2024-05-09T04:58:25.361 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.361 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.361 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.361 INFO:teuthology.orchestra.run.smithi006.stdout: "enable_monitoring": { 2024-05-09T04:58:25.361 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enable_monitoring", 2024-05-09T04:58:25.361 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.361 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.361 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.361 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.361 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.361 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.361 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.361 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "monitor device health metrics", 2024-05-09T04:58:25.361 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.362 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.362 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.362 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.362 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.362 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.362 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.362 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.362 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.362 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.362 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.362 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.362 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.362 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.363 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.363 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.363 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.363 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.363 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.363 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.363 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.363 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.363 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.363 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.363 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.363 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.363 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.363 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.363 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.364 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.364 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.364 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.364 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.364 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.364 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.364 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.364 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.364 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.364 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.364 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.364 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.364 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.364 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.364 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.364 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.364 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.365 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.365 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.365 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.365 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.365 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.365 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.365 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.365 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.365 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.365 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.365 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.365 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.365 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.365 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.365 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.366 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.366 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.366 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.366 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.366 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.366 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.366 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.366 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.366 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.366 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.366 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.366 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.366 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.366 INFO:teuthology.orchestra.run.smithi006.stdout: "mark_out_threshold": { 2024-05-09T04:58:25.367 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mark_out_threshold", 2024-05-09T04:58:25.367 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.367 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.367 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.367 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "2419200", 2024-05-09T04:58:25.367 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.367 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.367 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.367 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T04:58:25.367 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.367 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.367 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.367 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.367 INFO:teuthology.orchestra.run.smithi006.stdout: "pool_name": { 2024-05-09T04:58:25.367 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pool_name", 2024-05-09T04:58:25.367 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.368 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.368 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.368 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "device_health_metrics", 2024-05-09T04:58:25.368 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.368 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.368 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.368 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T04:58:25.368 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.368 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.368 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.368 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.368 INFO:teuthology.orchestra.run.smithi006.stdout: "retention_period": { 2024-05-09T04:58:25.368 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "retention_period", 2024-05-09T04:58:25.368 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.368 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.369 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.369 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15552000", 2024-05-09T04:58:25.369 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.369 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.369 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.369 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long to retain device health metrics", 2024-05-09T04:58:25.369 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.369 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.369 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.369 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.369 INFO:teuthology.orchestra.run.smithi006.stdout: "scrape_frequency": { 2024-05-09T04:58:25.369 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "scrape_frequency", 2024-05-09T04:58:25.369 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.369 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.369 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.369 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "86400", 2024-05-09T04:58:25.370 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.370 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.370 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.370 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T04:58:25.370 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.370 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.370 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.370 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.370 INFO:teuthology.orchestra.run.smithi006.stdout: "self_heal": { 2024-05-09T04:58:25.370 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "self_heal", 2024-05-09T04:58:25.370 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.370 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.370 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.370 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.370 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.370 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.371 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.371 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T04:58:25.371 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.371 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.371 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.371 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.371 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-05-09T04:58:25.371 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-05-09T04:58:25.371 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.371 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.371 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.371 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-05-09T04:58:25.371 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.371 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.371 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.371 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T04:58:25.372 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.372 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.372 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.372 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.372 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.372 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.372 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.372 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.372 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.372 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.372 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.372 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.372 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.372 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.372 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.373 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.373 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.373 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.373 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_threshold": { 2024-05-09T04:58:25.373 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_threshold", 2024-05-09T04:58:25.373 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.373 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.373 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.373 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "7257600", 2024-05-09T04:58:25.373 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.373 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.373 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.373 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T04:58:25.373 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.373 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.373 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.373 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.373 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.374 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.374 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.374 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "diskprediction_local", 2024-05-09T04:58:25.374 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.374 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.374 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.374 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.374 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.374 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.374 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.374 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.374 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.374 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.374 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.375 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.375 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.375 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.375 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.375 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.375 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.375 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.375 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.375 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.375 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.375 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.375 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.375 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.375 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.375 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.375 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.376 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.376 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.376 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.376 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.376 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.376 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.376 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.376 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.376 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.376 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.376 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.376 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.376 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.376 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.376 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.377 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.377 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.377 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.377 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.377 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.377 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.377 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.377 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.377 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.377 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.377 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.377 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.377 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.377 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.377 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.377 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.377 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.378 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.378 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.378 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.378 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.378 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.378 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.378 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.378 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.378 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.378 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.378 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.378 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.378 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.378 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.379 INFO:teuthology.orchestra.run.smithi006.stdout: "predict_interval": { 2024-05-09T04:58:25.379 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "predict_interval", 2024-05-09T04:58:25.379 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.379 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.379 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.379 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "86400", 2024-05-09T04:58:25.379 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.379 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.379 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.379 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.379 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.379 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.379 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.379 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.380 INFO:teuthology.orchestra.run.smithi006.stdout: "predictor_model": { 2024-05-09T04:58:25.380 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "predictor_model", 2024-05-09T04:58:25.380 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.380 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.380 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.380 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "prophetstor", 2024-05-09T04:58:25.380 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.380 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.380 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.380 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.380 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.380 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.380 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.380 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.380 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-05-09T04:58:25.380 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-05-09T04:58:25.381 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.381 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.381 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.381 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-05-09T04:58:25.381 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.381 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.381 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.381 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.381 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.381 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.381 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.381 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.381 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.381 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.381 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.382 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.382 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.382 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.382 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.382 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.382 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.382 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.382 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.382 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.382 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.382 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.382 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.382 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.382 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.382 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "influx", 2024-05-09T04:58:25.383 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": false, 2024-05-09T04:58:25.383 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "influxdb python module not found", 2024-05-09T04:58:25.383 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.383 INFO:teuthology.orchestra.run.smithi006.stdout: "batch_size": { 2024-05-09T04:58:25.383 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "batch_size", 2024-05-09T04:58:25.383 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.383 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.383 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.383 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5000", 2024-05-09T04:58:25.383 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.383 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.383 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.383 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T04:58:25.383 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.383 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.384 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.384 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.384 INFO:teuthology.orchestra.run.smithi006.stdout: "database": { 2024-05-09T04:58:25.384 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "database", 2024-05-09T04:58:25.384 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.384 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.384 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.384 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "ceph", 2024-05-09T04:58:25.384 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.384 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.384 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.384 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T04:58:25.384 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.384 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.385 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.385 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.385 INFO:teuthology.orchestra.run.smithi006.stdout: "hostname": { 2024-05-09T04:58:25.385 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "hostname", 2024-05-09T04:58:25.385 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.385 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.385 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.385 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.385 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.385 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.385 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.385 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB server hostname", 2024-05-09T04:58:25.385 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.385 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.385 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.385 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.386 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-05-09T04:58:25.386 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-05-09T04:58:25.386 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.386 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.386 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.386 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "30", 2024-05-09T04:58:25.386 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "5", 2024-05-09T04:58:25.386 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.386 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.386 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T04:58:25.386 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.386 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.386 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.387 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.387 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.387 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.387 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.387 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.387 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.387 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.387 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.387 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.387 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.387 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.387 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.387 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.387 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.388 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.388 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.388 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.388 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.388 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.388 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.388 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.388 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.388 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.388 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.388 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.388 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.388 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.388 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.389 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.389 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.389 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.389 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.389 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.389 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.389 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.389 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.389 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.389 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.389 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.389 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.389 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.389 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.389 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.389 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.390 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.390 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.390 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.390 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.390 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.390 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.390 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.390 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.390 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.390 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.390 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.390 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.390 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.390 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.391 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.391 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.391 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.391 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.391 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.391 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.391 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.391 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.391 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.391 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.391 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.391 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.391 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.391 INFO:teuthology.orchestra.run.smithi006.stdout: "password": { 2024-05-09T04:58:25.392 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "password", 2024-05-09T04:58:25.392 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.392 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.392 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.392 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.392 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.392 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.392 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.392 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "password of InfluxDB server user", 2024-05-09T04:58:25.392 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.392 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.392 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.392 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.392 INFO:teuthology.orchestra.run.smithi006.stdout: "port": { 2024-05-09T04:58:25.393 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "port", 2024-05-09T04:58:25.393 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.393 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.393 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.393 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8086", 2024-05-09T04:58:25.393 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.393 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.393 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.393 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB server port", 2024-05-09T04:58:25.393 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.393 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.393 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.393 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.394 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.394 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.394 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.394 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.394 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.394 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.394 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.394 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.394 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.394 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.394 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.394 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.394 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.394 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.394 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl": { 2024-05-09T04:58:25.395 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl", 2024-05-09T04:58:25.395 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.395 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.395 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.395 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "false", 2024-05-09T04:58:25.395 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.395 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.395 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.395 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T04:58:25.395 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.395 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.395 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.395 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.395 INFO:teuthology.orchestra.run.smithi006.stdout: "threads": { 2024-05-09T04:58:25.395 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "threads", 2024-05-09T04:58:25.396 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.396 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.396 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.396 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-05-09T04:58:25.396 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-05-09T04:58:25.396 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "32", 2024-05-09T04:58:25.396 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.396 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T04:58:25.396 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.396 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.396 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.396 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.396 INFO:teuthology.orchestra.run.smithi006.stdout: "username": { 2024-05-09T04:58:25.396 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "username", 2024-05-09T04:58:25.396 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.396 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.397 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.397 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.397 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.397 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.397 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.397 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "username of InfluxDB server user", 2024-05-09T04:58:25.397 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.397 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.397 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.397 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.397 INFO:teuthology.orchestra.run.smithi006.stdout: "verify_ssl": { 2024-05-09T04:58:25.397 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "verify_ssl", 2024-05-09T04:58:25.397 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.398 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.398 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.398 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "true", 2024-05-09T04:58:25.398 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.398 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.398 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.398 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T04:58:25.398 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.398 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.398 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.398 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.398 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.398 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.398 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.398 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "insights", 2024-05-09T04:58:25.399 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.399 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.399 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.399 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.399 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.399 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.399 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.399 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.399 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.399 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.399 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.399 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.399 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.399 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.399 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.400 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.400 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.400 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.400 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.400 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.400 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.400 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.400 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.400 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.400 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.400 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.400 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.400 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.400 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.401 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.401 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.401 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.401 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.401 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.401 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.401 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.401 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.401 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.401 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.401 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.401 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.401 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.402 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.402 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.402 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.402 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.402 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.402 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.402 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.402 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.402 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.402 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.402 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.402 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.402 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.402 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.403 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.403 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.403 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.403 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.403 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.403 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.403 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.403 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.403 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.403 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.403 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.403 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.403 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.403 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.403 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.404 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.404 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.404 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.404 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.404 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.404 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.404 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.404 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.404 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.404 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.404 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.404 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.404 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.404 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.404 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.405 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.405 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.405 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.405 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.405 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "iostat", 2024-05-09T04:58:25.405 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.405 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.405 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.405 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.405 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.405 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.405 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.405 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.405 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.405 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.405 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.406 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.406 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.406 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.406 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.406 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.406 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.406 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.406 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.406 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.406 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.406 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.406 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.406 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.407 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.407 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.407 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.407 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.407 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.407 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.407 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.407 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.407 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.407 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.407 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.407 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.407 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.407 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.407 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.408 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.408 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.408 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.408 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.408 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.408 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.408 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.408 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.408 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.408 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.408 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.408 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.408 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.408 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.408 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.409 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.409 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.409 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.409 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.409 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.409 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.409 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.409 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.409 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.409 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.409 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.409 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.409 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.409 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.409 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.409 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.410 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.410 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.410 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.410 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.410 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.410 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.410 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.410 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.410 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.410 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.410 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.410 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.410 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.410 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.411 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.411 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.411 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.411 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.411 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.411 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.411 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "k8sevents", 2024-05-09T04:58:25.411 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.411 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.411 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.411 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_event_retention_days": { 2024-05-09T04:58:25.411 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ceph_event_retention_days", 2024-05-09T04:58:25.411 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.412 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.412 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.412 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "7", 2024-05-09T04:58:25.412 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.412 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.412 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.412 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-09T04:58:25.412 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.412 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.412 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.412 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.412 INFO:teuthology.orchestra.run.smithi006.stdout: "config_check_secs": { 2024-05-09T04:58:25.412 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_check_secs", 2024-05-09T04:58:25.412 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.413 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.413 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.413 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-05-09T04:58:25.413 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "10", 2024-05-09T04:58:25.413 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.413 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.413 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-09T04:58:25.413 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.413 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.413 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.413 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.413 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.413 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.413 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.414 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.414 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.414 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.414 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.414 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.414 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.414 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.414 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.414 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.414 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.414 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.414 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.414 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.415 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.415 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.415 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.415 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.415 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.415 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.415 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.415 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.415 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.415 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.415 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.415 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.415 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.415 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.416 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.416 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.416 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.416 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.416 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.416 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.416 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.416 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.416 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.416 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.416 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.416 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.416 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.416 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.417 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.417 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.417 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.417 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.417 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.417 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.417 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.417 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.417 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.417 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.417 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.417 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.417 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.417 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.418 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.418 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.418 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.418 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.418 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.418 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.418 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.418 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.418 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.418 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.418 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.418 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.418 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.418 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.419 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.419 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.419 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.419 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.419 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.419 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.419 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.419 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.419 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.419 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.419 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.419 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.419 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.419 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.419 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.419 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "localpool", 2024-05-09T04:58:25.420 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.420 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.420 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.420 INFO:teuthology.orchestra.run.smithi006.stdout: "failure_domain": { 2024-05-09T04:58:25.420 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "failure_domain", 2024-05-09T04:58:25.420 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.420 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.420 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.420 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "host", 2024-05-09T04:58:25.420 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.420 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.420 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.420 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "failure domain for any created local pool", 2024-05-09T04:58:25.420 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T04:58:25.420 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.420 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.421 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.421 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.421 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.421 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.421 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.421 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.421 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.421 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.421 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.421 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.421 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.421 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.421 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.421 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.421 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.421 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.422 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.422 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.422 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.422 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.422 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.422 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.422 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.422 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.422 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.422 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.422 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.422 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.422 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.422 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.422 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.423 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.423 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.423 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.423 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.423 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.423 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.423 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.423 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.423 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.423 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.423 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.423 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.423 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.423 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.424 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.424 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.424 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.424 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.424 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.424 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.424 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.424 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.424 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.424 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.424 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.424 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.424 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.424 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.424 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.424 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.424 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.425 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.425 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.425 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.425 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.425 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.425 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.425 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.425 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.425 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.425 INFO:teuthology.orchestra.run.smithi006.stdout: "min_size": { 2024-05-09T04:58:25.425 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "min_size", 2024-05-09T04:58:25.425 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.425 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.425 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.425 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.425 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.426 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.426 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.426 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default min_size for any created local pool", 2024-05-09T04:58:25.426 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T04:58:25.426 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.426 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.426 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.426 INFO:teuthology.orchestra.run.smithi006.stdout: "num_rep": { 2024-05-09T04:58:25.426 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "num_rep", 2024-05-09T04:58:25.426 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.426 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.426 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.426 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3", 2024-05-09T04:58:25.427 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.427 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.427 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.427 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default replica count for any created local pool", 2024-05-09T04:58:25.427 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.427 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.427 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.427 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.427 INFO:teuthology.orchestra.run.smithi006.stdout: "pg_num": { 2024-05-09T04:58:25.427 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pg_num", 2024-05-09T04:58:25.427 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.427 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.427 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.427 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "128", 2024-05-09T04:58:25.427 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.427 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.428 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.428 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T04:58:25.428 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.428 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.428 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.428 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.428 INFO:teuthology.orchestra.run.smithi006.stdout: "prefix": { 2024-05-09T04:58:25.428 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prefix", 2024-05-09T04:58:25.428 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.428 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.428 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.428 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.428 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.428 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.428 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.428 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "name prefix for any created local pool", 2024-05-09T04:58:25.429 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.429 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.429 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.429 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.429 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.429 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.429 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.429 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.429 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.429 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.429 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.429 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.429 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.429 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.429 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.429 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.430 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.430 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.430 INFO:teuthology.orchestra.run.smithi006.stdout: "subtree": { 2024-05-09T04:58:25.430 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "subtree", 2024-05-09T04:58:25.430 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.430 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.430 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.430 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "rack", 2024-05-09T04:58:25.430 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.430 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.430 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.430 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T04:58:25.430 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T04:58:25.430 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.430 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.431 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.431 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.431 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.431 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.431 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mds_autoscaler", 2024-05-09T04:58:25.431 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.431 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.431 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.431 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.431 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.431 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.431 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.431 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.431 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.431 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.431 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.432 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.432 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.432 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.432 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.432 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.432 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.432 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.432 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.432 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.432 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.432 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.432 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.432 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.432 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.432 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.432 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.433 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.433 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.433 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.433 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.433 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.433 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.433 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.433 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.433 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.433 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.433 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.433 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.433 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.433 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.433 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.433 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.433 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.434 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.434 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.434 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.434 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.434 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.434 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.434 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.434 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.434 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.434 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.434 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.434 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.434 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.434 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.434 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.435 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.435 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.435 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.435 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.435 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.435 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.435 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.435 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.435 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.435 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.435 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.435 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.435 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.435 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.435 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.435 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.436 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.436 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.436 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.436 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.436 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.436 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.436 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.436 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.436 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.436 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.436 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.436 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.436 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.436 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.436 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.437 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mirroring", 2024-05-09T04:58:25.437 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.437 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.437 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.437 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.437 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.437 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.437 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.437 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.437 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.437 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.437 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.437 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.437 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.437 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.437 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.438 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.438 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.438 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.438 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.438 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.438 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.438 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.438 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.438 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.438 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.438 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.438 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.438 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.438 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.439 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.439 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.439 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.439 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.439 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.439 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.439 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.439 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.439 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.439 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.439 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.439 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.439 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.439 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.439 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.440 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.440 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.440 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.440 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.440 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.440 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.440 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.440 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.440 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.440 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.440 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.440 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.440 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.440 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.440 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.440 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.441 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.441 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.441 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.441 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.441 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.441 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.441 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.441 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.441 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.441 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.441 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.441 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.441 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.441 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.441 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.442 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.442 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.442 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.442 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.442 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.442 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.442 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.442 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.442 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.442 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.442 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.442 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.442 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.442 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.442 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.443 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "nfs", 2024-05-09T04:58:25.443 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.443 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.443 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.443 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.443 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.443 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.443 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.443 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.443 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.443 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.443 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.443 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.443 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.443 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.443 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.443 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.444 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.444 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.444 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.444 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.444 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.444 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.444 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.444 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.444 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.444 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.444 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.444 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.444 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.444 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.445 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.445 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.445 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.445 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.445 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.445 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.445 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.445 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.445 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.445 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.445 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.445 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.445 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.445 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.445 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.446 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.446 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.446 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.446 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.446 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.446 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.446 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.446 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.446 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.446 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.446 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.446 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.446 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.446 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.446 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.447 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.447 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.447 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.447 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.447 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.447 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.447 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.447 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.447 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.447 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.447 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.447 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.447 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.447 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.447 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.447 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.448 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.448 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.448 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.448 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.448 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.448 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.448 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.448 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.448 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.448 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.448 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.448 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.448 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.448 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.448 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "orchestrator", 2024-05-09T04:58:25.449 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.449 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.449 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.449 INFO:teuthology.orchestra.run.smithi006.stdout: "fail_fs": { 2024-05-09T04:58:25.449 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "fail_fs", 2024-05-09T04:58:25.449 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.449 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.449 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.449 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.449 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.449 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.449 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.449 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T04:58:25.449 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.449 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.449 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.449 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.450 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.450 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.450 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.450 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.450 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.450 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.450 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.450 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.450 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.450 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.450 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.450 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.450 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.450 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.450 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.451 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.451 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.451 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.451 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.451 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.451 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.451 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.451 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.451 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.451 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.451 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.451 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.451 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.451 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.452 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.452 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.452 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.452 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.452 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.452 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.452 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.452 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.452 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.452 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.452 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.452 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.452 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.452 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.452 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.452 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.453 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.453 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.453 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.453 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.453 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.453 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.453 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.453 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.453 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.453 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.453 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.453 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.453 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.453 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.453 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.453 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.454 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.454 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.454 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.454 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.454 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.454 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.454 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.454 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.454 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.454 INFO:teuthology.orchestra.run.smithi006.stdout: "orchestrator": { 2024-05-09T04:58:25.454 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "orchestrator", 2024-05-09T04:58:25.454 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.454 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.454 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.454 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.455 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.455 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.455 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.455 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm", 2024-05-09T04:58:25.455 INFO:teuthology.orchestra.run.smithi006.stdout: "rook", 2024-05-09T04:58:25.455 INFO:teuthology.orchestra.run.smithi006.stdout: "test_orchestrator" 2024-05-09T04:58:25.455 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.455 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Orchestrator backend", 2024-05-09T04:58:25.455 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.455 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.455 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.455 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.455 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.455 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.455 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.455 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.456 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.456 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.456 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.456 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.456 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.456 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.456 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.456 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.456 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.456 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.456 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.456 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.456 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.457 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "osd_perf_query", 2024-05-09T04:58:25.457 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.457 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.457 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.457 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.457 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.457 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.457 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.457 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.457 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.457 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.457 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.457 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.457 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.457 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.457 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.457 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.458 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.458 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.458 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.458 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.458 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.458 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.458 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.458 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.458 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.458 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.458 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.458 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.458 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.458 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.459 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.459 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.459 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.459 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.459 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.459 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.459 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.459 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.459 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.459 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.459 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.459 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.459 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.459 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.459 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.460 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.460 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.460 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.460 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.460 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.460 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.460 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.460 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.460 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.460 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.460 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.460 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.460 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.460 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.460 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.461 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.461 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.461 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.461 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.461 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.461 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.461 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.461 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.461 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.461 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.461 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.461 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.461 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.461 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.461 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.461 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.462 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.462 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.462 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.462 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.462 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.462 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.462 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.462 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.462 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.462 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.462 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.462 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.462 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.463 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.463 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "osd_support", 2024-05-09T04:58:25.463 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.463 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.463 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.463 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.463 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.463 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.463 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.463 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.463 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.463 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.463 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.463 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.463 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.464 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.464 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.464 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.464 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.464 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.464 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.464 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.464 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.464 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.464 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.464 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.464 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.464 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.464 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.464 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.465 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.465 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.465 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.465 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.465 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.465 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.465 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.465 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.465 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.465 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.465 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.465 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.465 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.465 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.465 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.466 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.466 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.466 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.466 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.466 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.466 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.466 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.466 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.466 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.466 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.466 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.467 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.467 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.467 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.467 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.467 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.467 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.467 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.467 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.467 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.467 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.468 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.468 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.468 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.468 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.468 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.468 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.468 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.468 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.469 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.469 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.469 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.469 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.469 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.469 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.469 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.469 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.469 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.470 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.470 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.470 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.470 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.470 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.470 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.470 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.470 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.470 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.470 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pg_autoscaler", 2024-05-09T04:58:25.470 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.470 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.471 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.471 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.471 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.471 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.471 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.471 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.471 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.471 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.471 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.471 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.471 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.471 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.472 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.472 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.472 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.472 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.472 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.472 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.472 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.472 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.472 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.472 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.472 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.473 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.473 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.473 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.473 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.473 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.473 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.473 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.473 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.473 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.473 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.473 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.473 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.473 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.474 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.474 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.474 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.474 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.474 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.474 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.474 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.474 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.474 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.474 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.474 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.474 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.474 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.475 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.475 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.475 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.475 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.475 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.475 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.475 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.475 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.475 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.475 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.475 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.475 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.475 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.476 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.476 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.476 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.476 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.476 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.476 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.476 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.476 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.476 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.476 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-05-09T04:58:25.476 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-05-09T04:58:25.476 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.476 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.477 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.477 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-05-09T04:58:25.477 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.477 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.477 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.477 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.477 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.477 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.477 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.477 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.477 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.477 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.478 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.478 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.478 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.478 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.478 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.478 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.478 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.478 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.478 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.478 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.479 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.479 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.479 INFO:teuthology.orchestra.run.smithi006.stdout: "threshold": { 2024-05-09T04:58:25.479 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "threshold", 2024-05-09T04:58:25.479 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-05-09T04:58:25.479 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.479 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.479 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3.0", 2024-05-09T04:58:25.479 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1.0", 2024-05-09T04:58:25.479 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.479 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.479 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "scaling threshold", 2024-05-09T04:58:25.480 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T04:58:25.480 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.480 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.480 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.480 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.480 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.480 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.480 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "progress", 2024-05-09T04:58:25.480 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.480 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.480 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.480 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_pg_recovery_event": { 2024-05-09T04:58:25.480 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_pg_recovery_event", 2024-05-09T04:58:25.481 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.481 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.481 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.481 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.481 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.481 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.481 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.481 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T04:58:25.482 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.482 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.482 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.482 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.482 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": { 2024-05-09T04:58:25.482 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enabled", 2024-05-09T04:58:25.482 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.482 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.482 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.482 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.483 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.483 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.483 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.483 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.483 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.483 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.483 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.483 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.483 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.483 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.484 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.484 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.484 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.484 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.484 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.484 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.484 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.484 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.484 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.484 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.484 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.484 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.484 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.485 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.485 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.485 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.485 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.485 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.485 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.485 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.485 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.485 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.486 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.486 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.486 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.486 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.486 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.486 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.486 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.486 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.486 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.486 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.486 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.486 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.486 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.487 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.487 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.487 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.487 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.487 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.487 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.487 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.487 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.487 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.487 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.488 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.488 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.488 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.488 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.488 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.488 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.488 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.488 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.488 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.488 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.488 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.488 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.489 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.489 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.489 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.489 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.489 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.489 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.489 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.489 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.489 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.489 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.489 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.490 INFO:teuthology.orchestra.run.smithi006.stdout: "max_completed_events": { 2024-05-09T04:58:25.490 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_completed_events", 2024-05-09T04:58:25.490 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.490 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.490 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.490 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "50", 2024-05-09T04:58:25.490 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.490 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.490 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.490 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "number of past completed events to remember", 2024-05-09T04:58:25.490 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.491 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.491 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.491 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.491 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-05-09T04:58:25.491 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-05-09T04:58:25.491 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.491 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.491 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.491 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-05-09T04:58:25.491 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.491 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.491 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.492 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long the module is going to sleep", 2024-05-09T04:58:25.492 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.492 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.492 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.492 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.492 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.492 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.492 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.492 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.492 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.492 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.492 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.492 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.493 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.493 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.493 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.493 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.493 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.493 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.493 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.493 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.493 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.493 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prometheus", 2024-05-09T04:58:25.493 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.493 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.493 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.494 INFO:teuthology.orchestra.run.smithi006.stdout: "cache": { 2024-05-09T04:58:25.494 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cache", 2024-05-09T04:58:25.494 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.494 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.494 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.494 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.494 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.494 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.494 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.494 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.494 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.494 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.495 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.495 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.495 INFO:teuthology.orchestra.run.smithi006.stdout: "exclude_perf_counters": { 2024-05-09T04:58:25.495 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "exclude_perf_counters", 2024-05-09T04:58:25.495 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.495 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.495 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.495 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.495 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.495 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.495 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.495 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T04:58:25.495 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T04:58:25.496 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.496 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.496 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.496 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.496 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.496 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.496 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.496 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.496 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.496 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.497 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.497 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.497 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.497 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.497 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.497 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.497 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.497 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.498 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.498 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.498 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.498 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.498 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.498 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.498 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.498 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.498 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.498 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.499 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.499 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.499 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.499 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.499 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.499 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.499 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.499 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.499 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.499 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.499 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.499 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.499 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.500 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.500 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.500 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.500 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.500 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.500 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.500 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.500 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.501 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.501 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.501 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.501 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.501 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.501 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.501 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.501 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.501 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.501 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.501 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.501 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.502 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.502 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.502 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.502 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.502 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.502 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.502 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.502 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.502 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.502 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.502 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.503 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.503 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_stats_pools": { 2024-05-09T04:58:25.503 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_stats_pools", 2024-05-09T04:58:25.503 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.503 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.503 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.503 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.503 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.503 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.503 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.503 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.503 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.503 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.504 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.504 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.504 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T04:58:25.504 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T04:58:25.504 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.504 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.504 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.504 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "300", 2024-05-09T04:58:25.504 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.504 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.504 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.504 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.504 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.505 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.505 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.505 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.505 INFO:teuthology.orchestra.run.smithi006.stdout: "scrape_interval": { 2024-05-09T04:58:25.505 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "scrape_interval", 2024-05-09T04:58:25.505 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-05-09T04:58:25.505 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.505 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.505 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15.0", 2024-05-09T04:58:25.506 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.506 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.506 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.506 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.506 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.506 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.506 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.506 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.506 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2024-05-09T04:58:25.506 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2024-05-09T04:58:25.507 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.507 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.507 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.507 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "::", 2024-05-09T04:58:25.507 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.507 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.507 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.507 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T04:58:25.507 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.507 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.507 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.507 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.507 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2024-05-09T04:58:25.508 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2024-05-09T04:58:25.508 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.508 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.508 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.508 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "9283", 2024-05-09T04:58:25.508 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.508 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.508 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.508 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T04:58:25.508 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.508 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.508 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.509 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.509 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.509 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.509 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.509 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.509 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.509 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.509 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.509 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.509 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.510 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.510 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.510 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.510 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.510 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.510 INFO:teuthology.orchestra.run.smithi006.stdout: "stale_cache_strategy": { 2024-05-09T04:58:25.510 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "stale_cache_strategy", 2024-05-09T04:58:25.510 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.510 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.510 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.510 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "log", 2024-05-09T04:58:25.511 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.511 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.511 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.511 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.511 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.511 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.511 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.511 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.511 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_behaviour": { 2024-05-09T04:58:25.511 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_behaviour", 2024-05-09T04:58:25.511 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.512 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.512 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.512 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "default", 2024-05-09T04:58:25.512 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.512 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.512 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.512 INFO:teuthology.orchestra.run.smithi006.stdout: "default", 2024-05-09T04:58:25.512 INFO:teuthology.orchestra.run.smithi006.stdout: "error" 2024-05-09T04:58:25.512 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.512 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.512 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.513 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.513 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.513 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.513 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_error_status_code": { 2024-05-09T04:58:25.513 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_error_status_code", 2024-05-09T04:58:25.513 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.513 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.513 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.513 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "500", 2024-05-09T04:58:25.513 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "400", 2024-05-09T04:58:25.514 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "599", 2024-05-09T04:58:25.514 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.514 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.514 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.514 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.514 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.514 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.514 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.515 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.515 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.515 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_support", 2024-05-09T04:58:25.515 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.515 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.515 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.515 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.515 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.515 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.515 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.515 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.516 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.516 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.516 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.516 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.516 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.516 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.516 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.516 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.516 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.516 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.517 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.517 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.517 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.517 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.517 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.517 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.517 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.517 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.517 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.517 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.517 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.517 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.518 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.518 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.518 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.518 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.518 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.518 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.518 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.518 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.518 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.518 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.519 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.519 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.519 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.519 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.519 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.519 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.519 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.519 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.519 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.519 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.519 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.520 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.520 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.520 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.520 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.520 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.520 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.520 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.520 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.520 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.520 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.520 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.520 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.521 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.521 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.521 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.521 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.521 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.521 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.521 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.521 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.521 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.521 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.521 INFO:teuthology.orchestra.run.smithi006.stdout: "max_concurrent_snap_create": { 2024-05-09T04:58:25.521 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_concurrent_snap_create", 2024-05-09T04:58:25.522 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.522 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.522 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.522 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-05-09T04:58:25.522 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.522 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.522 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.522 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.522 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.522 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.522 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.522 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.523 INFO:teuthology.orchestra.run.smithi006.stdout: "mirror_snapshot_schedule": { 2024-05-09T04:58:25.523 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T04:58:25.523 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.523 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.523 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.523 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.523 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.523 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.524 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.524 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.524 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.524 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.524 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.524 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.524 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.525 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.525 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.525 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.525 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.525 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.525 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.525 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.525 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.525 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.525 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.525 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.525 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.526 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.526 INFO:teuthology.orchestra.run.smithi006.stdout: "trash_purge_schedule": { 2024-05-09T04:58:25.526 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "trash_purge_schedule", 2024-05-09T04:58:25.526 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.526 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.526 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.526 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.526 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.526 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.526 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.526 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.526 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.527 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.527 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.527 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.527 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.527 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.527 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.527 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "restful", 2024-05-09T04:58:25.527 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.527 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.528 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.528 INFO:teuthology.orchestra.run.smithi006.stdout: "enable_auth": { 2024-05-09T04:58:25.528 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enable_auth", 2024-05-09T04:58:25.528 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.528 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.528 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.528 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.528 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.529 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.529 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.529 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.529 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.529 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.529 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.529 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.529 INFO:teuthology.orchestra.run.smithi006.stdout: "key_file": { 2024-05-09T04:58:25.529 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "key_file", 2024-05-09T04:58:25.529 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.529 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.529 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.530 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.530 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.530 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.530 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.530 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.530 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.530 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.530 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.530 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.530 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.530 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.530 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.531 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.531 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.531 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.531 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.531 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.531 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.531 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.531 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.531 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.531 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.531 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.531 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.532 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.532 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.532 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.532 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.532 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.532 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.532 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.532 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.532 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.532 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.532 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.533 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.533 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.533 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.533 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.533 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.533 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.533 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.533 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.533 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.533 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.533 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.533 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.534 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.534 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.534 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.534 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.534 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.534 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.534 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.534 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.534 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.534 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.534 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.535 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.535 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.535 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.535 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.535 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.535 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.535 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.535 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.535 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.535 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.535 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.535 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.535 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.536 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.536 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.536 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.536 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.536 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.536 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.536 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.536 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.536 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2024-05-09T04:58:25.536 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2024-05-09T04:58:25.536 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.536 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.536 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.536 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.537 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.537 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.537 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.537 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.537 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.537 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.537 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.537 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.537 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2024-05-09T04:58:25.537 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2024-05-09T04:58:25.537 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.537 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.538 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.538 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.538 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.538 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.538 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.538 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.538 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.538 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.538 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.538 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.538 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.538 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.539 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.539 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.539 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.539 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.539 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.539 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.539 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.539 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.539 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.539 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.539 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.539 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.539 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.540 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.540 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.540 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rgw", 2024-05-09T04:58:25.540 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.540 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.540 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.540 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.540 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.540 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.540 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.540 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.541 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.541 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.541 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.541 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.541 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.541 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.541 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.541 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.541 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.541 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.541 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.542 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.542 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.542 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.542 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.542 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.542 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.542 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.542 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.542 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.542 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.542 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.542 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.543 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.543 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.543 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.543 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.543 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.543 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.543 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.543 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.543 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.543 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.543 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.543 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.544 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.544 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.544 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.544 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.544 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.544 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.544 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.544 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.544 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.544 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.544 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.544 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.545 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.545 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.545 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.545 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.545 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.545 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.545 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.545 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.546 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.546 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.546 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.546 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.546 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.546 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.546 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.546 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.546 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.546 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.546 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.547 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.547 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.547 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.547 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.547 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.547 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.547 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.547 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.547 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.547 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.548 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.548 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.548 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.548 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.548 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.548 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.548 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rook", 2024-05-09T04:58:25.548 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.548 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.548 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.548 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.548 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.549 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.549 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.549 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.549 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.549 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.549 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.549 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.549 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.549 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.549 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.549 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.549 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.549 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.550 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.550 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.550 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.550 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.550 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.550 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.550 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.550 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.550 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.550 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.550 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.551 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.551 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.551 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.551 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.551 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.551 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.551 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.551 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.551 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.551 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.551 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.551 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.552 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.552 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.552 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.552 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.552 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.552 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.552 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.552 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.552 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.552 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.552 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.552 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.553 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.553 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.553 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.553 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.553 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.553 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.553 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.553 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.553 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.553 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.553 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.553 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.554 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.554 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.554 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.554 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.554 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.554 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.554 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.554 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.554 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.554 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.555 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.555 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.555 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.555 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.555 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.555 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.555 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.555 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.555 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.555 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.555 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.555 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.556 INFO:teuthology.orchestra.run.smithi006.stdout: "storage_class": { 2024-05-09T04:58:25.556 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "storage_class", 2024-05-09T04:58:25.556 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.556 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.556 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.556 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "local", 2024-05-09T04:58:25.556 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.556 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.556 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.556 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T04:58:25.556 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.556 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.557 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.557 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.557 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.557 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.557 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.557 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "selftest", 2024-05-09T04:58:25.557 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.557 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.557 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.557 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.557 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.557 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.557 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.558 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.558 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.558 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.558 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.558 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.558 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.558 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.558 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.558 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.558 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.558 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.558 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.559 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.559 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.559 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.559 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.559 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.559 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.559 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.559 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.559 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.560 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.560 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.560 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.560 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.560 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.560 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.560 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.560 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.560 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.560 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.560 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.560 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.560 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.561 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.561 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.561 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.561 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.561 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.561 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.561 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.561 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.561 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.561 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.562 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.562 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.562 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.562 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.562 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.562 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.562 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.563 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.563 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.563 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.563 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.563 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.563 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.563 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.563 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.563 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.563 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.564 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.564 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.564 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.564 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.564 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.564 INFO:teuthology.orchestra.run.smithi006.stdout: "roption1": { 2024-05-09T04:58:25.564 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "roption1", 2024-05-09T04:58:25.564 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.564 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.564 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.564 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.564 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.565 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.565 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.565 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.565 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.565 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.565 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.565 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.565 INFO:teuthology.orchestra.run.smithi006.stdout: "roption2": { 2024-05-09T04:58:25.565 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "roption2", 2024-05-09T04:58:25.565 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.565 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.565 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.565 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "xyz", 2024-05-09T04:58:25.566 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.566 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.566 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.566 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.566 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.566 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.566 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.566 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.566 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption1": { 2024-05-09T04:58:25.566 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption1", 2024-05-09T04:58:25.566 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.566 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.567 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.567 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.567 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.567 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.567 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.567 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.567 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.567 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.567 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.567 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.567 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption2": { 2024-05-09T04:58:25.567 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption2", 2024-05-09T04:58:25.568 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.568 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.568 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.568 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.568 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.568 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.568 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.568 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.568 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.568 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.568 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.569 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.569 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption3": { 2024-05-09T04:58:25.569 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption3", 2024-05-09T04:58:25.569 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-05-09T04:58:25.569 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.569 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.569 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.569 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.569 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.569 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.569 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.569 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.570 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.570 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.570 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.570 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption4": { 2024-05-09T04:58:25.570 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption4", 2024-05-09T04:58:25.570 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.570 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.570 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.570 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.571 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.571 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.571 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.571 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.571 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.571 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.571 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.571 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.571 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption5": { 2024-05-09T04:58:25.571 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption5", 2024-05-09T04:58:25.571 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.571 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.572 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.572 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.572 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.572 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.572 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.572 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.572 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.572 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.572 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.572 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.572 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption6": { 2024-05-09T04:58:25.572 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption6", 2024-05-09T04:58:25.573 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.573 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.573 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.573 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.573 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.573 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.573 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.573 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.573 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.573 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.573 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.573 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.573 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption7": { 2024-05-09T04:58:25.573 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption7", 2024-05-09T04:58:25.574 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.574 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.574 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.574 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.574 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-05-09T04:58:25.574 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "42", 2024-05-09T04:58:25.574 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.574 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.574 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.574 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.574 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.575 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.575 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.575 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.575 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.575 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.575 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.575 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.575 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.575 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.575 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.575 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.575 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.575 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.575 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.576 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.576 INFO:teuthology.orchestra.run.smithi006.stdout: "testkey": { 2024-05-09T04:58:25.576 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testkey", 2024-05-09T04:58:25.576 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.576 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.576 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.576 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.576 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.576 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.576 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.576 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.576 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.576 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.577 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.577 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.577 INFO:teuthology.orchestra.run.smithi006.stdout: "testlkey": { 2024-05-09T04:58:25.577 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testlkey", 2024-05-09T04:58:25.577 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.577 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.577 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.577 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.577 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.577 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.577 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.577 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.578 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.578 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.578 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.578 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.578 INFO:teuthology.orchestra.run.smithi006.stdout: "testnewline": { 2024-05-09T04:58:25.578 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testnewline", 2024-05-09T04:58:25.578 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.578 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.578 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.578 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.578 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.578 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.579 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.579 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.579 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.579 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.579 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.579 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.579 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.579 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.579 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.579 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smb", 2024-05-09T04:58:25.579 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.579 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.579 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.580 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.580 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.580 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.580 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.580 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.580 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.580 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.580 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.580 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.580 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.580 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.580 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.580 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.581 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.581 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.581 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.581 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.581 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.581 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.581 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.581 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.581 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.581 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.581 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.581 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.581 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.582 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.582 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.582 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.582 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.582 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.582 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.582 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.582 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.582 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.582 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.582 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.582 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.582 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.582 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.583 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.583 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.583 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.583 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.583 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.583 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.583 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.583 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.583 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.583 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.583 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.583 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.584 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.584 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.584 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.584 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.584 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.584 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.584 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.584 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.584 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.584 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.584 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.584 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.585 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.585 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.585 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.585 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.585 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.585 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.585 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.585 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.585 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.585 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.586 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.586 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.586 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.586 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.586 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.586 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.586 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.586 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.586 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.586 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.587 INFO:teuthology.orchestra.run.smithi006.stdout: "update_orchestration": { 2024-05-09T04:58:25.587 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "update_orchestration", 2024-05-09T04:58:25.587 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.587 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.587 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.587 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.587 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.587 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.587 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.587 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-09T04:58:25.587 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.588 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.588 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.588 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.588 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.588 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.588 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.588 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "snap_schedule", 2024-05-09T04:58:25.588 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.588 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.588 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.588 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_m_granularity": { 2024-05-09T04:58:25.588 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_m_granularity", 2024-05-09T04:58:25.589 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.589 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.589 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.589 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.589 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.589 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.589 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.589 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T04:58:25.589 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.589 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.589 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.589 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.589 INFO:teuthology.orchestra.run.smithi006.stdout: "dump_on_update": { 2024-05-09T04:58:25.590 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "dump_on_update", 2024-05-09T04:58:25.590 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.590 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.590 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.590 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.590 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.590 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.590 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.590 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "dump database to debug log on update", 2024-05-09T04:58:25.590 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.590 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.590 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.591 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.591 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.591 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.591 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.591 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.591 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.591 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.591 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.591 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.591 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.591 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.591 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.592 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.592 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.592 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.592 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.592 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.592 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.592 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.592 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.592 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.592 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.592 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.592 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.592 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.593 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.593 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.593 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.593 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.593 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.593 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.593 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.593 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.593 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.593 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.593 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.593 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.594 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.594 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.594 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.594 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.594 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.594 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.594 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.594 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.594 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.594 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.594 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.594 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.594 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.595 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.595 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.595 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.595 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.595 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.595 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.595 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.595 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.595 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.595 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.595 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.595 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.596 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.596 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.596 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.596 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.596 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.596 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.596 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.596 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.596 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.596 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.597 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.597 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.597 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.597 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.597 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.597 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.597 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.597 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.597 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.597 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.597 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.597 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.598 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.598 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.598 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.598 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.598 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "stats", 2024-05-09T04:58:25.598 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.598 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.598 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.598 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.598 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.598 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.598 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.599 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.599 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.599 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.599 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.599 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.599 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.599 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.599 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.599 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.599 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.599 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.600 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.600 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.600 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.600 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.600 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.600 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.600 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.600 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.600 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.600 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.600 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.601 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.601 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.601 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.601 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.601 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.601 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.601 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.601 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.601 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.602 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.602 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.602 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.602 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.602 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.602 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.602 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.602 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.602 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.602 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.602 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.602 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.603 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.603 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.603 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.603 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.603 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.603 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.603 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.603 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.603 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.603 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.603 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.604 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.604 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.604 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.604 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.604 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.604 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.604 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.604 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.604 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.604 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.604 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.605 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.605 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.605 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.605 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.605 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.605 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.605 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.605 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.605 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.605 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.605 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.605 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.606 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.606 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.606 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.606 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.606 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.606 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.606 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "status", 2024-05-09T04:58:25.606 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.606 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.606 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.606 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.606 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.607 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.607 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.607 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.607 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.607 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.607 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.607 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.607 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.607 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.607 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.607 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.607 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.607 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.608 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.608 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.608 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.608 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.608 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.608 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.608 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.608 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.608 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.608 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.608 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.608 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.608 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.609 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.609 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.609 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.609 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.609 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.609 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.609 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.609 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.609 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.609 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.609 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.609 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.609 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.609 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.610 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.610 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.610 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.610 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.610 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.610 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.610 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.610 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.610 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.610 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.611 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.611 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.611 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.611 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.611 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.611 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.611 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.611 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.611 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.611 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.611 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.611 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.612 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.612 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.612 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.612 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.612 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.612 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.612 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.612 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.612 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.612 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.612 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.612 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.612 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.612 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.613 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.613 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.613 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.613 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.613 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.613 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.613 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.613 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.613 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.613 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "telegraf", 2024-05-09T04:58:25.613 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.614 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.614 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.614 INFO:teuthology.orchestra.run.smithi006.stdout: "address": { 2024-05-09T04:58:25.614 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "address", 2024-05-09T04:58:25.614 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.614 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.614 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.614 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T04:58:25.614 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.614 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.614 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.614 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.614 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.615 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.615 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.615 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.615 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-05-09T04:58:25.615 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-05-09T04:58:25.615 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.615 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.615 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.615 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15", 2024-05-09T04:58:25.615 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.615 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.616 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.616 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.616 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.616 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.616 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.616 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.616 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.616 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.616 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.616 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.616 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.616 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.616 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.617 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.617 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.617 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.617 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.617 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.617 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.617 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.617 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.617 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.617 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.617 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.617 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.617 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.618 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.618 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.618 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.618 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.618 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.618 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.618 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.618 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.618 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.618 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.618 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.618 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.618 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.619 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.619 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.619 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.619 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.619 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.619 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.619 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.619 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.619 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.619 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.619 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.619 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.619 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.620 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.620 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.620 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.620 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.620 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.620 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.620 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.620 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.620 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.620 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.620 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.620 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.620 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.621 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.621 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.621 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.621 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.621 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.621 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.621 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.621 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.621 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.621 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.621 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.621 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.622 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.622 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.622 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.622 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.622 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.622 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.622 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.622 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.622 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.622 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.622 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.622 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.622 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.622 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.623 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.623 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.623 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "telemetry", 2024-05-09T04:58:25.623 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.623 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.623 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.623 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_basic": { 2024-05-09T04:58:25.623 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_basic", 2024-05-09T04:58:25.623 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.623 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.623 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.623 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.623 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.624 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.624 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.624 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T04:58:25.624 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.624 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.624 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.624 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.624 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_crash": { 2024-05-09T04:58:25.624 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_crash", 2024-05-09T04:58:25.624 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.625 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.625 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.625 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.625 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.625 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.625 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.625 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T04:58:25.625 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.625 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.625 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.625 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.626 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_device": { 2024-05-09T04:58:25.626 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_device", 2024-05-09T04:58:25.626 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.626 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.626 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.626 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.626 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.626 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.626 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.626 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T04:58:25.626 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.626 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.627 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.627 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.627 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_ident": { 2024-05-09T04:58:25.627 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_ident", 2024-05-09T04:58:25.627 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.627 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.627 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.627 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.627 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.627 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.627 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.627 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T04:58:25.627 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.628 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.628 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.628 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.628 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_perf": { 2024-05-09T04:58:25.628 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_perf", 2024-05-09T04:58:25.628 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.628 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.628 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.628 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.628 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.628 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.629 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.629 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T04:58:25.629 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.629 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.629 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.629 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.629 INFO:teuthology.orchestra.run.smithi006.stdout: "contact": { 2024-05-09T04:58:25.629 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "contact", 2024-05-09T04:58:25.629 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.629 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.629 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.629 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.630 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.630 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.630 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.630 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.630 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.630 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.630 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.630 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.630 INFO:teuthology.orchestra.run.smithi006.stdout: "description": { 2024-05-09T04:58:25.630 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "description", 2024-05-09T04:58:25.630 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.630 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.631 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.631 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.631 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.631 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.631 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.631 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.631 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.631 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.631 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.631 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.631 INFO:teuthology.orchestra.run.smithi006.stdout: "device_url": { 2024-05-09T04:58:25.631 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_url", 2024-05-09T04:58:25.632 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.632 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.632 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.632 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T04:58:25.632 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.632 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.632 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.632 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.632 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.632 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.632 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.632 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.633 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": { 2024-05-09T04:58:25.633 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enabled", 2024-05-09T04:58:25.633 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.633 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.633 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.633 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.633 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.633 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.633 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.633 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.633 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.633 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.633 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.634 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.634 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-05-09T04:58:25.634 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-05-09T04:58:25.634 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.634 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.634 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.634 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "24", 2024-05-09T04:58:25.634 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "8", 2024-05-09T04:58:25.634 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.634 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.634 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.634 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.635 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.635 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.635 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.635 INFO:teuthology.orchestra.run.smithi006.stdout: "last_opt_revision": { 2024-05-09T04:58:25.635 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "last_opt_revision", 2024-05-09T04:58:25.635 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.635 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.635 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.635 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1", 2024-05-09T04:58:25.635 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.635 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.635 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.635 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.636 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.636 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.636 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.636 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.636 INFO:teuthology.orchestra.run.smithi006.stdout: "leaderboard": { 2024-05-09T04:58:25.636 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "leaderboard", 2024-05-09T04:58:25.636 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.636 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.636 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.636 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.636 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.637 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.637 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.637 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.637 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.637 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.637 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.637 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.637 INFO:teuthology.orchestra.run.smithi006.stdout: "leaderboard_description": { 2024-05-09T04:58:25.637 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "leaderboard_description", 2024-05-09T04:58:25.637 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.637 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.637 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.638 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.638 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.638 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.638 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.638 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.638 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.638 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.638 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.638 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.638 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.638 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.638 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.639 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.639 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.639 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.639 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.639 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.639 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.639 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.639 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.639 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.639 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.639 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.639 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.639 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.639 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.640 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.640 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.640 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.640 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.640 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.640 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.640 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.640 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.640 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.640 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.640 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.640 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.640 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.641 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.641 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.641 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.641 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.641 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.641 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.641 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.641 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.641 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.641 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.641 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.641 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.641 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.642 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.642 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.642 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.642 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.642 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.642 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.642 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.642 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.642 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.642 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.642 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.642 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.643 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.643 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.643 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.643 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.643 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.643 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.643 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.643 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.643 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.643 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.643 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.644 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.644 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.644 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.644 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.644 INFO:teuthology.orchestra.run.smithi006.stdout: "organization": { 2024-05-09T04:58:25.644 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "organization", 2024-05-09T04:58:25.644 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.644 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.644 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.644 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.644 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.644 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.644 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.645 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.645 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.645 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.645 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.645 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.645 INFO:teuthology.orchestra.run.smithi006.stdout: "proxy": { 2024-05-09T04:58:25.645 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "proxy", 2024-05-09T04:58:25.645 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.645 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.645 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.645 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.646 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.646 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.646 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.646 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.646 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.646 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.646 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.646 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.646 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.646 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.646 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.646 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.647 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.647 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.647 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.647 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.647 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.647 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.647 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.647 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.647 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.647 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.647 INFO:teuthology.orchestra.run.smithi006.stdout: "url": { 2024-05-09T04:58:25.648 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "url", 2024-05-09T04:58:25.648 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.648 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.648 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.648 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T04:58:25.648 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.648 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.648 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.648 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.648 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.648 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.648 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.648 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.649 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.649 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.649 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.649 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "test_orchestrator", 2024-05-09T04:58:25.649 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.649 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.649 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.649 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.649 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.649 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.649 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.649 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.649 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.650 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.650 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.650 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.650 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.650 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.650 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.650 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.650 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.650 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.650 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.650 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.650 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.650 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.651 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.651 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.651 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.651 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.651 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.651 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.651 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.651 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.651 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.651 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.651 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.651 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.651 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.652 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.652 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.652 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.652 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.652 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.652 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.652 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.652 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.652 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.653 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.653 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.653 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.653 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.653 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.653 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.653 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.653 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.653 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.653 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.653 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.653 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.653 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.654 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.654 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.654 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.654 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.654 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.654 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.654 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.654 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.654 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.654 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.654 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.655 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.655 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.655 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.655 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.655 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.655 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.655 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.655 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.655 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.655 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.655 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.655 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.655 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.656 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.656 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.656 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.656 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.656 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.656 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.656 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.656 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.656 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.656 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "volumes", 2024-05-09T04:58:25.656 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.656 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.656 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.656 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.657 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.657 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.657 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.657 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.657 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.657 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.657 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.657 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.657 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.657 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.657 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.657 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.658 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.658 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.658 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.658 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.658 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.658 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.658 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.658 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.658 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.658 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.658 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.658 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.658 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.659 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.659 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.659 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.659 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.659 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.659 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.659 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.659 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.659 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.659 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.659 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.659 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.659 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.660 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.660 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.660 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.660 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.660 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.660 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.660 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.660 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.660 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.661 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.661 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.661 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.661 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.661 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.661 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.661 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.661 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.661 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.661 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.661 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.661 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.662 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.662 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.662 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.662 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.662 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.662 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.662 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.662 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.662 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.662 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.662 INFO:teuthology.orchestra.run.smithi006.stdout: "max_concurrent_clones": { 2024-05-09T04:58:25.662 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_concurrent_clones", 2024-05-09T04:58:25.662 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.663 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.663 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.663 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "4", 2024-05-09T04:58:25.663 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.663 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.663 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.663 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T04:58:25.663 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.663 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.663 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.663 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.664 INFO:teuthology.orchestra.run.smithi006.stdout: "periodic_async_work": { 2024-05-09T04:58:25.664 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "periodic_async_work", 2024-05-09T04:58:25.664 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.664 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.664 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.664 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.664 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.664 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.664 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.664 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Periodically check for async work", 2024-05-09T04:58:25.664 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.665 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.665 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.665 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.665 INFO:teuthology.orchestra.run.smithi006.stdout: "snapshot_clone_delay": { 2024-05-09T04:58:25.665 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "snapshot_clone_delay", 2024-05-09T04:58:25.665 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.665 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.665 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.665 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.665 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.665 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.665 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.666 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T04:58:25.666 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.666 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.666 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.666 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.666 INFO:teuthology.orchestra.run.smithi006.stdout: "snapshot_clone_no_wait": { 2024-05-09T04:58:25.666 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T04:58:25.666 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.666 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.666 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.666 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.666 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.667 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.667 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.667 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T04:58:25.667 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.667 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.667 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.667 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.667 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.667 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.667 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.667 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.667 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.668 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.668 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.668 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.668 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.668 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.668 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.668 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.668 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.668 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.668 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.668 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.668 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.668 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix", 2024-05-09T04:58:25.669 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.669 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.669 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.669 INFO:teuthology.orchestra.run.smithi006.stdout: "discovery_interval": { 2024-05-09T04:58:25.669 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "discovery_interval", 2024-05-09T04:58:25.669 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-05-09T04:58:25.669 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.669 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.669 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "100", 2024-05-09T04:58:25.669 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.669 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.669 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.669 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.670 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.670 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.670 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.670 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.670 INFO:teuthology.orchestra.run.smithi006.stdout: "identifier": { 2024-05-09T04:58:25.670 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "identifier", 2024-05-09T04:58:25.670 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.670 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.670 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.670 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.671 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.671 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.671 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.671 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.671 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.671 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.671 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.671 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.671 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-05-09T04:58:25.671 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-05-09T04:58:25.671 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.671 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.672 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.672 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-05-09T04:58:25.672 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.672 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.672 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.672 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.672 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.672 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.672 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.672 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.672 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.672 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.672 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.673 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.673 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.673 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.673 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.673 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.673 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.673 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.673 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.673 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.673 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.673 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.673 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.674 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.674 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.674 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.674 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.674 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.674 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.674 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.674 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.674 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.674 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.674 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.674 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.675 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.675 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.675 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.675 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.675 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.675 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.675 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.675 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.675 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.675 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.675 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.675 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.675 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.675 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.676 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.676 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.676 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.676 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.676 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.676 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.676 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.676 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.676 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.676 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.676 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.676 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.676 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.677 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.677 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.677 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.677 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.677 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.677 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.677 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.677 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.677 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.677 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.677 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.677 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.677 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.678 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.678 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.678 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.678 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.678 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.678 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.678 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.678 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.678 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.678 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.678 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.679 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.679 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.679 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.679 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.679 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.679 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.679 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_host": { 2024-05-09T04:58:25.679 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_host", 2024-05-09T04:58:25.679 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.679 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.679 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.679 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.679 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.680 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.680 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.680 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.680 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.680 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.680 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.680 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.680 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_port": { 2024-05-09T04:58:25.680 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_port", 2024-05-09T04:58:25.680 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.680 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.680 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.680 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10051", 2024-05-09T04:58:25.681 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.681 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.681 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.681 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.681 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.681 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.681 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.681 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.681 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_sender": { 2024-05-09T04:58:25.681 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_sender", 2024-05-09T04:58:25.681 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.682 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.682 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.682 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T04:58:25.682 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.682 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.682 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.682 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.682 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.682 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.682 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.682 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.682 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.683 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.683 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2024-05-09T04:58:25.683 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.683 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.683 INFO:teuthology.orchestra.run.smithi006.stdout: "gid": 24113, 2024-05-09T04:58:25.683 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "y", 2024-05-09T04:58:25.683 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr_features": 4540701547738038271, 2024-05-09T04:58:25.683 INFO:teuthology.orchestra.run.smithi006.stdout: "available_modules": [ 2024-05-09T04:58:25.683 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.683 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "alerts", 2024-05-09T04:58:25.683 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.683 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.683 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.683 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-05-09T04:58:25.683 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-05-09T04:58:25.684 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.684 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.684 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.684 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-05-09T04:58:25.684 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.684 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.684 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.684 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T04:58:25.684 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.684 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.684 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.684 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.685 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.685 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.685 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.685 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.685 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.685 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.685 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.685 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.685 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.685 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.685 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.685 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.686 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.686 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.686 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.686 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.686 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.686 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.686 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.686 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.686 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.686 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.687 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.687 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.687 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.687 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.687 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.687 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.687 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.687 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.687 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.687 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.687 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.687 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.687 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.687 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.687 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.688 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.688 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.688 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.688 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.688 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.688 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.688 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.688 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.688 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.688 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.688 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.688 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.689 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.689 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.689 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.689 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.689 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.689 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.689 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.689 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.689 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.689 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.689 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.689 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.689 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.689 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.690 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.690 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.690 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.690 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.690 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.690 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.690 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.690 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_destination": { 2024-05-09T04:58:25.690 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_destination", 2024-05-09T04:58:25.690 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.690 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.690 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.691 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.691 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.691 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.691 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.691 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Email address to send alerts to", 2024-05-09T04:58:25.691 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.691 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.691 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.691 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.691 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_from_name": { 2024-05-09T04:58:25.691 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_from_name", 2024-05-09T04:58:25.691 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.692 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.692 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.692 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "Ceph", 2024-05-09T04:58:25.692 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.692 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.692 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.692 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Email From: name", 2024-05-09T04:58:25.692 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.692 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.692 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.692 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.692 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_host": { 2024-05-09T04:58:25.692 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_host", 2024-05-09T04:58:25.693 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.693 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.693 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.693 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.693 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.693 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.693 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.693 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP server", 2024-05-09T04:58:25.693 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.693 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.693 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.693 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.693 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_password": { 2024-05-09T04:58:25.694 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_password", 2024-05-09T04:58:25.694 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.694 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.694 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.694 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.694 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.694 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.694 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.694 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Password to authenticate with", 2024-05-09T04:58:25.694 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.695 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.695 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.695 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.695 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_port": { 2024-05-09T04:58:25.695 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_port", 2024-05-09T04:58:25.695 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.695 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.695 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.695 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "465", 2024-05-09T04:58:25.695 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.695 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.695 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.695 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP port", 2024-05-09T04:58:25.696 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.696 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.696 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.696 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.696 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_sender": { 2024-05-09T04:58:25.696 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_sender", 2024-05-09T04:58:25.696 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.696 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.696 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.696 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.696 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.696 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.697 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.697 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP envelope sender", 2024-05-09T04:58:25.697 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.697 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.697 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.697 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.697 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_ssl": { 2024-05-09T04:58:25.697 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_ssl", 2024-05-09T04:58:25.697 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.698 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.698 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.698 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.698 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.698 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.698 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.698 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T04:58:25.698 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.698 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.698 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.698 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.699 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_user": { 2024-05-09T04:58:25.699 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_user", 2024-05-09T04:58:25.699 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.699 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.699 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.699 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.699 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.699 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.699 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.699 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "User to authenticate as", 2024-05-09T04:58:25.699 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.700 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.700 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.700 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.700 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.700 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.700 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.700 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.700 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.700 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.700 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.700 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.701 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.701 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.701 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.701 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.701 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.702 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.702 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.702 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.702 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.702 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "balancer", 2024-05-09T04:58:25.702 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.702 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.702 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.702 INFO:teuthology.orchestra.run.smithi006.stdout: "active": { 2024-05-09T04:58:25.702 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "active", 2024-05-09T04:58:25.702 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.703 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.703 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.703 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.703 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.703 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.703 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.703 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T04:58:25.703 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.703 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.703 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.703 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.703 INFO:teuthology.orchestra.run.smithi006.stdout: "begin_time": { 2024-05-09T04:58:25.704 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "begin_time", 2024-05-09T04:58:25.704 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.704 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.704 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.704 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0000", 2024-05-09T04:58:25.704 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.704 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.704 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.704 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T04:58:25.704 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T04:58:25.704 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.704 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.704 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.705 INFO:teuthology.orchestra.run.smithi006.stdout: "begin_weekday": { 2024-05-09T04:58:25.705 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "begin_weekday", 2024-05-09T04:58:25.705 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-05-09T04:58:25.705 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.705 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.705 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.705 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0", 2024-05-09T04:58:25.705 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "6", 2024-05-09T04:58:25.705 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.705 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T04:58:25.705 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T04:58:25.705 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.706 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.706 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.706 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_max_iterations": { 2024-05-09T04:58:25.706 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_max_iterations", 2024-05-09T04:58:25.706 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-05-09T04:58:25.706 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.706 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.706 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "25", 2024-05-09T04:58:25.706 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-05-09T04:58:25.706 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "250", 2024-05-09T04:58:25.706 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.706 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T04:58:25.707 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.707 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.707 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.707 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.707 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_metrics": { 2024-05-09T04:58:25.707 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_metrics", 2024-05-09T04:58:25.707 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.707 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.707 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.707 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T04:58:25.707 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.707 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.708 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.708 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T04:58:25.708 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T04:58:25.708 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.708 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.708 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.708 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_step": { 2024-05-09T04:58:25.708 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_step", 2024-05-09T04:58:25.708 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-05-09T04:58:25.708 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.708 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.708 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0.5", 2024-05-09T04:58:25.709 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0.001", 2024-05-09T04:58:25.709 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "0.999", 2024-05-09T04:58:25.709 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.709 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "aggressiveness of optimization", 2024-05-09T04:58:25.709 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T04:58:25.709 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.709 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.709 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.709 INFO:teuthology.orchestra.run.smithi006.stdout: "end_time": { 2024-05-09T04:58:25.709 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "end_time", 2024-05-09T04:58:25.710 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.710 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.710 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.710 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "2359", 2024-05-09T04:58:25.710 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.710 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.710 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.710 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T04:58:25.711 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T04:58:25.711 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.711 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.711 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.711 INFO:teuthology.orchestra.run.smithi006.stdout: "end_weekday": { 2024-05-09T04:58:25.711 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "end_weekday", 2024-05-09T04:58:25.711 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-05-09T04:58:25.711 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.711 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.711 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.712 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0", 2024-05-09T04:58:25.712 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "6", 2024-05-09T04:58:25.712 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.712 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T04:58:25.712 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T04:58:25.712 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.712 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.712 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.712 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.712 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.712 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.713 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.713 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.713 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.713 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.713 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.713 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.713 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.713 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.713 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.713 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.713 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.713 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.713 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.714 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.714 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.714 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.714 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.714 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.714 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.714 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.714 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.714 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.714 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.714 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.715 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.715 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.715 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.715 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.715 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.715 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.715 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.715 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.715 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.715 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.715 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.715 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.715 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.716 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.716 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.716 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.716 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.716 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.716 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.716 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.716 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.716 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.716 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.716 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.716 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.717 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.717 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.717 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.717 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.717 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.717 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.717 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.717 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.717 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.717 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.717 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.717 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.718 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.718 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.718 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.718 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.718 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.718 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.718 INFO:teuthology.orchestra.run.smithi006.stdout: "min_score": { 2024-05-09T04:58:25.718 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "min_score", 2024-05-09T04:58:25.718 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-05-09T04:58:25.718 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.718 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.719 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.719 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.719 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.719 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.719 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T04:58:25.719 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.719 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.719 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.719 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.719 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": { 2024-05-09T04:58:25.719 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mode", 2024-05-09T04:58:25.719 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.720 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.720 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.720 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "upmap", 2024-05-09T04:58:25.720 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.720 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.720 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.720 INFO:teuthology.orchestra.run.smithi006.stdout: "crush-compat", 2024-05-09T04:58:25.720 INFO:teuthology.orchestra.run.smithi006.stdout: "none", 2024-05-09T04:58:25.720 INFO:teuthology.orchestra.run.smithi006.stdout: "read", 2024-05-09T04:58:25.720 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap", 2024-05-09T04:58:25.720 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap-read" 2024-05-09T04:58:25.720 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.720 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Balancer mode", 2024-05-09T04:58:25.721 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.721 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.721 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.721 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.721 INFO:teuthology.orchestra.run.smithi006.stdout: "pool_ids": { 2024-05-09T04:58:25.721 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pool_ids", 2024-05-09T04:58:25.721 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.721 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.721 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.721 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.721 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.721 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.722 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.722 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T04:58:25.722 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.722 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.722 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.722 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.722 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-05-09T04:58:25.722 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-05-09T04:58:25.722 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.722 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.722 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.723 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-05-09T04:58:25.723 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.723 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.723 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.723 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T04:58:25.723 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.723 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.723 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.723 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.723 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.723 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.723 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.723 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.724 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.724 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.724 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.724 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.724 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.724 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.724 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.724 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.724 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.724 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.724 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap_max_deviation": { 2024-05-09T04:58:25.724 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "upmap_max_deviation", 2024-05-09T04:58:25.724 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.725 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.725 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.725 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-05-09T04:58:25.725 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-05-09T04:58:25.725 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.725 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.725 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T04:58:25.725 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T04:58:25.725 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.725 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.725 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.725 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap_max_optimizations": { 2024-05-09T04:58:25.726 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "upmap_max_optimizations", 2024-05-09T04:58:25.726 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-05-09T04:58:25.726 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.726 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.726 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-05-09T04:58:25.726 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.726 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.730 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.730 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T04:58:25.730 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.730 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.730 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.730 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.730 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.730 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.730 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.730 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cephadm", 2024-05-09T04:58:25.731 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.731 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.731 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.731 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_down_multiplier": { 2024-05-09T04:58:25.731 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_down_multiplier", 2024-05-09T04:58:25.731 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-05-09T04:58:25.731 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.731 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.731 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3.0", 2024-05-09T04:58:25.731 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.732 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.732 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.732 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T04:58:25.732 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.732 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.732 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.732 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.732 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_refresh_rate": { 2024-05-09T04:58:25.732 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_refresh_rate", 2024-05-09T04:58:25.732 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.732 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.732 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.733 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "20", 2024-05-09T04:58:25.733 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.733 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.733 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.733 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T04:58:25.733 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.733 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.733 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.733 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.733 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_starting_port": { 2024-05-09T04:58:25.733 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_starting_port", 2024-05-09T04:58:25.733 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.734 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.734 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.734 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "4721", 2024-05-09T04:58:25.734 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.734 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.734 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.734 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T04:58:25.734 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.734 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.734 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.734 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.735 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_ptrace": { 2024-05-09T04:58:25.735 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_ptrace", 2024-05-09T04:58:25.735 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.735 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.735 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.735 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.735 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.735 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.735 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.735 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T04:58:25.735 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T04:58:25.735 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.736 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.736 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.736 INFO:teuthology.orchestra.run.smithi006.stdout: "autotune_interval": { 2024-05-09T04:58:25.736 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "autotune_interval", 2024-05-09T04:58:25.736 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.736 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.736 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.736 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-05-09T04:58:25.736 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.736 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.736 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.736 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T04:58:25.736 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.737 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.737 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.737 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.737 INFO:teuthology.orchestra.run.smithi006.stdout: "autotune_memory_target_ratio": { 2024-05-09T04:58:25.737 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T04:58:25.737 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-05-09T04:58:25.737 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.737 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.737 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0.7", 2024-05-09T04:58:25.737 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.737 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.737 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.738 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T04:58:25.738 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.738 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.738 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.738 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.738 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm_log_destination": { 2024-05-09T04:58:25.738 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cephadm_log_destination", 2024-05-09T04:58:25.738 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.738 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.738 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.738 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.739 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.739 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.739 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.739 INFO:teuthology.orchestra.run.smithi006.stdout: "file", 2024-05-09T04:58:25.739 INFO:teuthology.orchestra.run.smithi006.stdout: "file,syslog", 2024-05-09T04:58:25.739 INFO:teuthology.orchestra.run.smithi006.stdout: "syslog" 2024-05-09T04:58:25.739 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.739 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-09T04:58:25.739 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.739 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.739 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.740 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.740 INFO:teuthology.orchestra.run.smithi006.stdout: "cgroups_split": { 2024-05-09T04:58:25.740 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cgroups_split", 2024-05-09T04:58:25.740 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.740 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.740 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.740 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.740 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.740 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.740 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.740 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T04:58:25.740 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.740 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.741 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.741 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.741 INFO:teuthology.orchestra.run.smithi006.stdout: "config_checks_enabled": { 2024-05-09T04:58:25.741 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_checks_enabled", 2024-05-09T04:58:25.741 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.741 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.741 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.741 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.741 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.741 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.741 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.741 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T04:58:25.742 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.742 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.742 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.742 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.742 INFO:teuthology.orchestra.run.smithi006.stdout: "config_dashboard": { 2024-05-09T04:58:25.742 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_dashboard", 2024-05-09T04:58:25.742 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.742 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.742 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.742 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.742 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.743 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.743 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.743 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T04:58:25.743 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.743 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.743 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.743 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.743 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_alertmanager": { 2024-05-09T04:58:25.743 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_alertmanager", 2024-05-09T04:58:25.743 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.743 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.744 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.744 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T04:58:25.744 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.744 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.744 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.744 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-05-09T04:58:25.744 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.744 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.744 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.744 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.744 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_base": { 2024-05-09T04:58:25.744 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_base", 2024-05-09T04:58:25.745 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.745 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.745 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.745 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T04:58:25.745 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.745 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.745 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.745 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Container image name, without the tag", 2024-05-09T04:58:25.745 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.745 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.746 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.746 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.746 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_elasticsearch": { 2024-05-09T04:58:25.746 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_elasticsearch", 2024-05-09T04:58:25.746 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.746 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.746 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.746 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T04:58:25.746 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.746 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.746 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.747 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "elasticsearch container image", 2024-05-09T04:58:25.747 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.747 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.747 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.747 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.747 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_grafana": { 2024-05-09T04:58:25.747 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_grafana", 2024-05-09T04:58:25.747 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.747 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.747 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.747 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-09T04:58:25.748 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.748 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.748 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.748 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-05-09T04:58:25.748 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.748 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.748 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.748 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.748 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_haproxy": { 2024-05-09T04:58:25.748 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_haproxy", 2024-05-09T04:58:25.748 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.748 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.749 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.749 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T04:58:25.749 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.749 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.749 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.749 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "HAproxy container image", 2024-05-09T04:58:25.749 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.749 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.749 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.749 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.749 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_agent": { 2024-05-09T04:58:25.750 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_agent", 2024-05-09T04:58:25.750 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.750 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.750 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.750 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T04:58:25.750 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.750 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.750 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.750 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger agent container image", 2024-05-09T04:58:25.750 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.750 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.750 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.751 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.751 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_collector": { 2024-05-09T04:58:25.751 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_collector", 2024-05-09T04:58:25.751 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.751 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.751 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.751 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T04:58:25.751 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.751 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.751 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.751 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger collector container image", 2024-05-09T04:58:25.751 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.751 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.752 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.752 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.752 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_query": { 2024-05-09T04:58:25.752 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_query", 2024-05-09T04:58:25.752 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.752 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.752 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.752 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T04:58:25.752 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.752 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.752 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.752 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger query container image", 2024-05-09T04:58:25.753 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.753 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.753 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.753 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.753 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_keepalived": { 2024-05-09T04:58:25.753 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_keepalived", 2024-05-09T04:58:25.753 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.753 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.753 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.753 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T04:58:25.753 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.753 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.754 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.754 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Keepalived container image", 2024-05-09T04:58:25.754 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.754 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.754 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.754 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.754 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_loki": { 2024-05-09T04:58:25.754 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_loki", 2024-05-09T04:58:25.754 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.754 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.754 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.755 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T04:58:25.755 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.755 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.755 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.755 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Loki container image", 2024-05-09T04:58:25.755 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.755 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.755 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.755 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.755 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_node_exporter": { 2024-05-09T04:58:25.755 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_node_exporter", 2024-05-09T04:58:25.755 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.756 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.756 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.756 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T04:58:25.756 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.756 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.756 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.756 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-05-09T04:58:25.756 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.756 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.756 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.756 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.757 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_nvmeof": { 2024-05-09T04:58:25.757 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_nvmeof", 2024-05-09T04:58:25.757 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.757 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.757 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.757 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-09T04:58:25.757 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.757 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.757 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.757 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Nvme-of container image", 2024-05-09T04:58:25.757 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.757 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.757 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.758 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.758 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_prometheus": { 2024-05-09T04:58:25.758 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_prometheus", 2024-05-09T04:58:25.758 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.758 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.758 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.758 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T04:58:25.758 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.758 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.758 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.758 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-05-09T04:58:25.759 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.759 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.759 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.759 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.759 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_promtail": { 2024-05-09T04:58:25.759 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_promtail", 2024-05-09T04:58:25.759 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.759 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.759 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.759 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T04:58:25.759 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.760 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.760 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.760 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Promtail container image", 2024-05-09T04:58:25.760 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.760 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.760 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.760 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.760 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_samba": { 2024-05-09T04:58:25.760 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_samba", 2024-05-09T04:58:25.760 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.760 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.761 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.761 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-09T04:58:25.761 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.761 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.761 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.761 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Samba/SMB container image", 2024-05-09T04:58:25.761 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.761 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.761 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.761 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.761 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_snmp_gateway": { 2024-05-09T04:58:25.762 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_snmp_gateway", 2024-05-09T04:58:25.762 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.762 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.762 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.762 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T04:58:25.762 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.762 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.762 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.762 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SNMP Gateway container image", 2024-05-09T04:58:25.762 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.762 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.762 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.763 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.763 INFO:teuthology.orchestra.run.smithi006.stdout: "container_init": { 2024-05-09T04:58:25.763 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_init", 2024-05-09T04:58:25.763 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.763 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.763 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.763 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.763 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.763 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.763 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.763 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T04:58:25.763 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.764 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.764 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.764 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.764 INFO:teuthology.orchestra.run.smithi006.stdout: "daemon_cache_timeout": { 2024-05-09T04:58:25.764 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "daemon_cache_timeout", 2024-05-09T04:58:25.764 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.764 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.764 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.764 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-05-09T04:58:25.764 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.764 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.764 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.765 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T04:58:25.765 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.765 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.765 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.765 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.765 INFO:teuthology.orchestra.run.smithi006.stdout: "default_cephadm_command_timeout": { 2024-05-09T04:58:25.765 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T04:58:25.765 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.765 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.765 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.765 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "900", 2024-05-09T04:58:25.765 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.765 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.766 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.766 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T04:58:25.766 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.766 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.766 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.766 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.766 INFO:teuthology.orchestra.run.smithi006.stdout: "default_registry": { 2024-05-09T04:58:25.766 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "default_registry", 2024-05-09T04:58:25.766 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.766 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.766 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.766 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io", 2024-05-09T04:58:25.767 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.767 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.767 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.767 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T04:58:25.767 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.767 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.767 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.767 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.767 INFO:teuthology.orchestra.run.smithi006.stdout: "device_cache_timeout": { 2024-05-09T04:58:25.767 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_cache_timeout", 2024-05-09T04:58:25.767 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.767 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.768 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.768 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1800", 2024-05-09T04:58:25.768 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.768 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.768 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.768 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache device inventory", 2024-05-09T04:58:25.768 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.768 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.768 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.768 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.768 INFO:teuthology.orchestra.run.smithi006.stdout: "device_enhanced_scan": { 2024-05-09T04:58:25.768 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_enhanced_scan", 2024-05-09T04:58:25.769 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.769 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.769 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.769 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.769 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.769 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.769 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.769 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T04:58:25.769 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.769 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.769 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.769 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.770 INFO:teuthology.orchestra.run.smithi006.stdout: "facts_cache_timeout": { 2024-05-09T04:58:25.770 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "facts_cache_timeout", 2024-05-09T04:58:25.770 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.770 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.770 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.770 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-05-09T04:58:25.770 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.770 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.770 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.770 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache host facts data", 2024-05-09T04:58:25.770 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.770 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.771 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.771 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.771 INFO:teuthology.orchestra.run.smithi006.stdout: "grafana_dashboards_path": { 2024-05-09T04:58:25.771 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "grafana_dashboards_path", 2024-05-09T04:58:25.771 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.771 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.771 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.771 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-09T04:58:25.771 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.771 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.771 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.771 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-09T04:58:25.772 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.772 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.772 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.772 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.772 INFO:teuthology.orchestra.run.smithi006.stdout: "host_check_interval": { 2024-05-09T04:58:25.772 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "host_check_interval", 2024-05-09T04:58:25.772 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.772 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.772 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.772 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-05-09T04:58:25.773 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.773 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.773 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.773 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to perform a host check", 2024-05-09T04:58:25.773 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.773 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.773 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.773 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.773 INFO:teuthology.orchestra.run.smithi006.stdout: "hw_monitoring": { 2024-05-09T04:58:25.773 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "hw_monitoring", 2024-05-09T04:58:25.773 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.773 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.773 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.774 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.774 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.774 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.774 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.774 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T04:58:25.774 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.774 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.774 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.774 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.774 INFO:teuthology.orchestra.run.smithi006.stdout: "inventory_list_all": { 2024-05-09T04:58:25.774 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "inventory_list_all", 2024-05-09T04:58:25.775 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.775 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.775 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.775 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.775 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.775 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.775 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.775 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T04:58:25.775 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.775 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.775 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.775 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.776 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.776 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.776 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.776 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.776 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.776 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.776 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.776 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.776 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.776 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.776 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.777 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.777 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.777 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.777 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.777 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.777 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.777 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.777 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.777 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.777 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.777 INFO:teuthology.orchestra.run.smithi006.stdout: "log_refresh_metadata": { 2024-05-09T04:58:25.777 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_refresh_metadata", 2024-05-09T04:58:25.778 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.778 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.778 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.778 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.778 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.778 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.778 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.778 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T04:58:25.778 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.778 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.778 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.779 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.779 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.779 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.779 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.779 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.779 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.779 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.779 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.779 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.779 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.779 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T04:58:25.779 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.779 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.780 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.780 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.780 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.780 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.780 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.780 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.780 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.780 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.780 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.780 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.780 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.780 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.781 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.781 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.781 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.781 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.781 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.781 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.781 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.781 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.781 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.781 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.781 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.781 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.781 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.782 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.782 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.782 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.782 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.782 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.782 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.782 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.782 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.782 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.782 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.782 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.782 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.783 INFO:teuthology.orchestra.run.smithi006.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T04:58:25.783 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T04:58:25.783 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.783 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.783 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.783 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.783 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.783 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.783 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.783 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T04:58:25.783 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.783 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.783 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.784 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.784 INFO:teuthology.orchestra.run.smithi006.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T04:58:25.784 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T04:58:25.784 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.784 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.784 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.784 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "*", 2024-05-09T04:58:25.784 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.784 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.784 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.784 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T04:58:25.784 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.785 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.785 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.785 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.785 INFO:teuthology.orchestra.run.smithi006.stdout: "max_count_per_host": { 2024-05-09T04:58:25.785 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_count_per_host", 2024-05-09T04:58:25.785 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.785 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.785 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.785 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-05-09T04:58:25.785 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.785 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.785 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.786 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "max number of daemons per service per host", 2024-05-09T04:58:25.786 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.786 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.786 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.786 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.786 INFO:teuthology.orchestra.run.smithi006.stdout: "max_osd_draining_count": { 2024-05-09T04:58:25.786 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_osd_draining_count", 2024-05-09T04:58:25.786 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.786 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.786 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.786 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-05-09T04:58:25.786 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.787 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.787 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.787 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T04:58:25.787 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.787 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.787 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.787 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.787 INFO:teuthology.orchestra.run.smithi006.stdout: "migration_current": { 2024-05-09T04:58:25.787 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "migration_current", 2024-05-09T04:58:25.787 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.787 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.788 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.788 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.788 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.788 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.788 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.788 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "internal - do not modify", 2024-05-09T04:58:25.788 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.788 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.788 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.788 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.788 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": { 2024-05-09T04:58:25.788 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mode", 2024-05-09T04:58:25.789 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.789 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.789 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.789 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "root", 2024-05-09T04:58:25.789 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.789 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.789 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.789 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm-package", 2024-05-09T04:58:25.789 INFO:teuthology.orchestra.run.smithi006.stdout: "root" 2024-05-09T04:58:25.789 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.789 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T04:58:25.789 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.790 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.790 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.790 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.790 INFO:teuthology.orchestra.run.smithi006.stdout: "oob_default_addr": { 2024-05-09T04:58:25.790 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "oob_default_addr", 2024-05-09T04:58:25.790 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.790 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.790 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.790 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "169.254.1.1", 2024-05-09T04:58:25.790 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.790 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.790 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.791 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T04:58:25.791 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.791 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.791 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.791 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.791 INFO:teuthology.orchestra.run.smithi006.stdout: "prometheus_alerts_path": { 2024-05-09T04:58:25.791 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prometheus_alerts_path", 2024-05-09T04:58:25.791 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.791 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.791 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.791 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T04:58:25.791 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.792 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.792 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.792 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T04:58:25.792 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.792 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.792 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.792 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.792 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_insecure": { 2024-05-09T04:58:25.792 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_insecure", 2024-05-09T04:58:25.792 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.792 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.792 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.793 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.793 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.793 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.793 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.793 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T04:58:25.793 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.793 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.793 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.793 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.793 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_password": { 2024-05-09T04:58:25.793 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_password", 2024-05-09T04:58:25.793 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.794 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.794 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.794 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.794 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.794 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.794 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.794 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T04:58:25.794 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.794 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.794 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.794 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.794 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_url": { 2024-05-09T04:58:25.795 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_url", 2024-05-09T04:58:25.795 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.795 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.795 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.795 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.795 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.795 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.795 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.795 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T04:58:25.795 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.795 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.795 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.796 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.796 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_username": { 2024-05-09T04:58:25.796 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_username", 2024-05-09T04:58:25.796 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.796 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.796 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.796 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.796 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.796 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.796 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.796 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T04:58:25.796 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.796 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.797 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.797 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.797 INFO:teuthology.orchestra.run.smithi006.stdout: "secure_monitoring_stack": { 2024-05-09T04:58:25.797 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "secure_monitoring_stack", 2024-05-09T04:58:25.797 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.797 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.797 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.797 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.797 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.797 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.797 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.797 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T04:58:25.798 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.798 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.798 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.798 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.798 INFO:teuthology.orchestra.run.smithi006.stdout: "service_discovery_port": { 2024-05-09T04:58:25.798 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "service_discovery_port", 2024-05-09T04:58:25.798 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.798 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.798 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.798 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8765", 2024-05-09T04:58:25.798 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.798 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.799 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.799 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "cephadm service discovery port", 2024-05-09T04:58:25.799 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.799 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.799 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.799 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.799 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.799 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.799 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.799 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.799 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.799 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.800 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.800 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.800 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.800 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.800 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.800 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.800 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.800 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.800 INFO:teuthology.orchestra.run.smithi006.stdout: "ssh_config_file": { 2024-05-09T04:58:25.800 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssh_config_file", 2024-05-09T04:58:25.800 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.800 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.800 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.801 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.801 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.801 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.801 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.801 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T04:58:25.801 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.801 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.801 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.801 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.801 INFO:teuthology.orchestra.run.smithi006.stdout: "use_agent": { 2024-05-09T04:58:25.801 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "use_agent", 2024-05-09T04:58:25.802 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.802 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.802 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.802 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.802 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.802 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.802 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.802 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T04:58:25.802 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.802 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.802 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.803 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.803 INFO:teuthology.orchestra.run.smithi006.stdout: "use_repo_digest": { 2024-05-09T04:58:25.803 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "use_repo_digest", 2024-05-09T04:58:25.803 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.803 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.803 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.803 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.803 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.803 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.803 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.803 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T04:58:25.803 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.803 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.804 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.804 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.804 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_failed_host_check": { 2024-05-09T04:58:25.804 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_failed_host_check", 2024-05-09T04:58:25.804 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.804 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.804 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.804 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.804 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.804 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.804 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.805 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T04:58:25.805 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.805 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.805 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.805 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.805 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_stray_daemons": { 2024-05-09T04:58:25.805 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_stray_daemons", 2024-05-09T04:58:25.805 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.805 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.805 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.805 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.805 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.805 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.806 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.806 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T04:58:25.806 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.806 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.806 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.806 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.806 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_stray_hosts": { 2024-05-09T04:58:25.806 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_stray_hosts", 2024-05-09T04:58:25.806 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.806 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.806 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.806 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.807 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.807 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.807 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.807 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T04:58:25.807 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.807 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.807 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.807 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.807 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.807 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.807 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.808 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crash", 2024-05-09T04:58:25.808 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.808 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.808 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.808 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.808 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.808 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.808 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.808 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.808 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.808 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.808 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.808 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.809 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.809 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.809 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.809 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.809 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.809 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.809 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.809 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.809 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.809 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.809 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.809 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.810 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.810 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.810 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.810 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.810 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.810 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.810 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.810 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.810 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.810 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.810 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.810 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.811 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.811 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.811 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.811 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.811 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.811 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.811 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.811 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.811 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.811 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.811 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.811 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.811 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.812 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.812 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.812 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.812 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.812 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.812 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.812 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.812 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.812 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.812 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.812 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.812 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.813 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.813 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.813 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.813 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.813 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.813 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.813 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.813 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.813 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.813 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.813 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.813 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.814 INFO:teuthology.orchestra.run.smithi006.stdout: "retain_interval": { 2024-05-09T04:58:25.814 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "retain_interval", 2024-05-09T04:58:25.814 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.814 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.814 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.814 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "31536000", 2024-05-09T04:58:25.814 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.814 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.814 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.814 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T04:58:25.814 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.814 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.815 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.815 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.815 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.815 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.815 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.815 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.815 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.815 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.815 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.815 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.815 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.815 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.815 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.816 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.816 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.816 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.816 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_recent_interval": { 2024-05-09T04:58:25.816 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_recent_interval", 2024-05-09T04:58:25.816 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.816 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.816 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.816 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1209600", 2024-05-09T04:58:25.816 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.816 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.816 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.816 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T04:58:25.817 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.817 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.817 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.817 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.817 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.817 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.817 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.817 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "dashboard", 2024-05-09T04:58:25.817 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.817 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.817 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.817 INFO:teuthology.orchestra.run.smithi006.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T04:58:25.817 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T04:58:25.818 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.818 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.818 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.818 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-05-09T04:58:25.818 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.818 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.818 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.818 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.818 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.818 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.818 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.818 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.819 INFO:teuthology.orchestra.run.smithi006.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T04:58:25.819 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T04:58:25.819 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.819 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.819 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.819 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.819 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.819 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.819 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.819 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.819 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.819 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.820 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.820 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.820 INFO:teuthology.orchestra.run.smithi006.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T04:58:25.820 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T04:58:25.820 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.820 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.820 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.820 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.820 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.820 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.820 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.820 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.821 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.821 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.821 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.821 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.821 INFO:teuthology.orchestra.run.smithi006.stdout: "AUDIT_API_ENABLED": { 2024-05-09T04:58:25.821 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T04:58:25.821 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.821 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.821 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.821 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.821 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.822 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.822 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.822 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.822 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.822 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.822 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.822 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.822 INFO:teuthology.orchestra.run.smithi006.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T04:58:25.822 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T04:58:25.822 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.822 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.822 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.823 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.823 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.823 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.823 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.823 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.823 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.823 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.823 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.823 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.823 INFO:teuthology.orchestra.run.smithi006.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T04:58:25.823 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T04:58:25.823 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.824 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.824 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.824 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.824 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.824 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.824 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.824 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.824 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.824 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.824 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.824 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.824 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T04:58:25.825 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T04:58:25.825 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.825 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.825 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.825 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.825 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.825 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.825 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.825 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.825 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.825 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.825 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.825 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.825 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T04:58:25.826 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T04:58:25.826 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.826 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.826 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.826 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.826 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.826 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.826 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.826 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.826 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.826 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.826 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.827 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.827 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T04:58:25.827 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T04:58:25.827 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.827 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.827 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.827 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.827 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.827 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.827 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.827 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.827 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.827 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.828 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.828 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.828 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T04:58:25.828 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T04:58:25.828 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.828 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.828 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.828 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.828 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.828 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.828 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.828 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.828 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.828 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.829 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.829 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.829 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T04:58:25.829 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T04:58:25.829 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.829 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.829 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.829 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.829 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.829 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.829 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.829 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.829 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.830 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.830 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.830 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.830 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T04:58:25.830 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T04:58:25.830 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.830 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.830 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.830 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.830 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.830 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.830 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.831 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.831 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.831 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.831 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.831 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.831 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T04:58:25.831 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T04:58:25.831 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.831 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.831 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.831 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.831 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.832 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.832 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.832 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.832 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.832 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.832 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.832 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.832 INFO:teuthology.orchestra.run.smithi006.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T04:58:25.832 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T04:58:25.832 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.832 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.832 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.832 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.832 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.833 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.833 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.833 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.833 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.833 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.833 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.833 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.833 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T04:58:25.833 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T04:58:25.833 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.833 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.833 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.833 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2024-05-09T04:58:25.834 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.834 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.834 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.834 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.834 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.834 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.834 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.834 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.834 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T04:58:25.834 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T04:58:25.834 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.834 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.835 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.835 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.835 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.835 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.835 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.835 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.835 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.835 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.835 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.835 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.835 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_URL": { 2024-05-09T04:58:25.836 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_URL", 2024-05-09T04:58:25.836 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.836 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.836 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.836 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.836 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.836 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.836 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.836 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.836 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.836 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.836 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.837 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.837 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T04:58:25.837 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T04:58:25.837 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.837 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.837 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.837 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2024-05-09T04:58:25.837 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.837 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.837 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.837 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.837 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.837 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.838 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.838 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.838 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T04:58:25.838 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T04:58:25.838 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.838 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.838 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.838 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.838 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.838 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.838 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.839 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.839 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.839 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.839 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.839 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.839 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T04:58:25.839 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T04:58:25.839 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.839 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.839 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.839 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.839 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.839 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.840 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.840 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.840 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.840 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.840 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.840 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.840 INFO:teuthology.orchestra.run.smithi006.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T04:58:25.840 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T04:58:25.840 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.840 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.840 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.840 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.840 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.841 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.841 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.841 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.841 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.841 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.841 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.841 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.841 INFO:teuthology.orchestra.run.smithi006.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T04:58:25.841 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T04:58:25.841 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.841 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.841 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.841 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.842 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.842 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.842 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.842 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.842 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.842 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.842 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.842 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.842 INFO:teuthology.orchestra.run.smithi006.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-09T04:58:25.842 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-09T04:58:25.842 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.842 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.843 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.843 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "[]", 2024-05-09T04:58:25.843 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.843 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.843 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.843 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.843 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.843 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.843 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.843 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.843 INFO:teuthology.orchestra.run.smithi006.stdout: "MULTICLUSTER_CONFIG": { 2024-05-09T04:58:25.843 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-09T04:58:25.844 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.844 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.844 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.844 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "{}", 2024-05-09T04:58:25.844 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.844 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.844 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.844 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.844 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.844 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.844 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.844 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.845 INFO:teuthology.orchestra.run.smithi006.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T04:58:25.845 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T04:58:25.845 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.845 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.845 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.845 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.845 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.845 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.845 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.845 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.845 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.846 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.846 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.846 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.846 INFO:teuthology.orchestra.run.smithi006.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T04:58:25.846 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T04:58:25.846 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.846 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.846 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.846 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.846 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.846 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.846 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.847 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.847 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.847 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.847 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.847 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.847 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T04:58:25.847 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T04:58:25.847 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.847 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.847 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.847 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.848 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.848 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.848 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.848 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.848 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.848 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.848 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.848 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.848 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T04:58:25.848 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T04:58:25.848 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.848 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.849 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.849 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.849 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.849 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.849 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.849 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.849 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.849 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.849 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.849 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.849 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T04:58:25.849 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T04:58:25.849 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.850 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.850 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.850 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.850 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.850 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.850 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.850 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.850 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.850 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.850 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.850 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.851 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T04:58:25.851 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T04:58:25.851 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.851 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.851 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.851 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.851 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.851 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.851 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.851 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.851 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.851 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.852 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.852 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.852 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T04:58:25.852 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T04:58:25.852 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.852 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.852 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.852 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.852 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.852 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.852 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.852 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.852 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.853 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.853 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.853 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.853 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T04:58:25.853 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T04:58:25.853 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.853 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.853 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.853 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.853 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.853 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.853 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.854 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.854 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.854 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.854 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.854 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.854 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T04:58:25.854 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T04:58:25.854 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.854 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.854 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.854 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.854 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.855 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.855 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.855 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.855 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.855 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.855 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.855 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.855 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T04:58:25.855 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T04:58:25.855 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.855 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.855 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.855 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.855 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.856 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.856 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.856 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.856 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.856 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.856 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.856 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.856 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T04:58:25.856 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T04:58:25.856 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.856 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.856 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.857 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T04:58:25.857 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.857 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.857 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.857 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.857 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.857 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.857 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.857 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.857 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T04:58:25.857 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T04:58:25.857 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.858 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.858 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.858 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-05-09T04:58:25.858 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.858 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.858 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.858 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.858 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.858 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.858 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.858 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.858 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T04:58:25.858 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T04:58:25.859 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.859 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.859 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.859 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8", 2024-05-09T04:58:25.859 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.859 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.859 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.859 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.859 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.859 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.859 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.859 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.860 INFO:teuthology.orchestra.run.smithi006.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T04:58:25.860 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T04:58:25.860 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.860 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.860 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.860 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "45", 2024-05-09T04:58:25.860 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.860 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.860 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.860 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.860 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.861 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.861 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.861 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.861 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T04:58:25.861 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T04:58:25.861 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.861 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.861 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.861 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.861 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.861 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.862 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.862 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.862 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.862 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.862 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.862 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.862 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T04:58:25.862 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T04:58:25.862 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.862 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.862 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.862 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2024-05-09T04:58:25.863 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.863 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.863 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.863 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.863 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.863 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.863 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.863 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.863 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T04:58:25.863 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T04:58:25.863 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.864 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.864 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.864 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.864 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.864 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.864 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.864 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.864 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.864 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.864 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.864 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.864 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T04:58:25.865 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T04:58:25.865 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.865 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.865 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.865 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.865 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.865 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.865 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.865 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.865 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.865 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.865 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.866 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.866 INFO:teuthology.orchestra.run.smithi006.stdout: "UNSAFE_TLS_v1_2": { 2024-05-09T04:58:25.866 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-09T04:58:25.866 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.866 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.866 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.866 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.866 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.866 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.866 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.866 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.867 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.867 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.867 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.867 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.867 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T04:58:25.867 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T04:58:25.867 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.867 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.867 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.867 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.867 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.867 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.867 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.868 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.868 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.868 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.868 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.868 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.868 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T04:58:25.868 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T04:58:25.868 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.868 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.868 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.868 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-05-09T04:58:25.868 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.868 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.868 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.869 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.869 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.869 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.869 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.869 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.869 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T04:58:25.869 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T04:58:25.869 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.869 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.869 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.869 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-05-09T04:58:25.869 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.870 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.870 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.870 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.870 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.870 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.870 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.870 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.870 INFO:teuthology.orchestra.run.smithi006.stdout: "cross_origin_url": { 2024-05-09T04:58:25.870 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cross_origin_url", 2024-05-09T04:58:25.870 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.870 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.871 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.871 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.871 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.871 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.871 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.871 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.871 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.871 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.871 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.871 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.871 INFO:teuthology.orchestra.run.smithi006.stdout: "crt_file": { 2024-05-09T04:58:25.871 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crt_file", 2024-05-09T04:58:25.871 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.871 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.872 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.872 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.872 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.872 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.872 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.872 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.872 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.872 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.872 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.872 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.872 INFO:teuthology.orchestra.run.smithi006.stdout: "debug": { 2024-05-09T04:58:25.872 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "debug", 2024-05-09T04:58:25.872 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.872 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.873 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.873 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.873 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.873 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.873 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.873 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable/disable debug options", 2024-05-09T04:58:25.873 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.873 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.873 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.873 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.873 INFO:teuthology.orchestra.run.smithi006.stdout: "jwt_token_ttl": { 2024-05-09T04:58:25.873 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "jwt_token_ttl", 2024-05-09T04:58:25.873 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.874 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.874 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.874 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "28800", 2024-05-09T04:58:25.874 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.874 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.874 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.874 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.874 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.874 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.874 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.874 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.874 INFO:teuthology.orchestra.run.smithi006.stdout: "key_file": { 2024-05-09T04:58:25.874 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "key_file", 2024-05-09T04:58:25.874 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.875 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.875 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.875 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.875 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.875 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.875 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.875 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.875 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.875 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.875 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.875 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.875 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.875 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.875 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.876 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.876 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.876 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.876 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.876 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.876 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.876 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.876 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.876 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.876 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.876 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.876 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.876 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.876 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.877 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.877 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.877 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.877 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.877 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.877 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.877 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.877 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.877 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.877 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.877 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.877 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.877 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.877 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.878 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.878 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.878 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.878 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.878 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.878 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.878 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.878 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.878 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.878 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.878 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.878 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.878 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.879 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.879 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.879 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.879 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.879 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.879 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.879 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.879 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.879 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.879 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.879 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.879 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.879 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.880 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.880 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.880 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.880 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.880 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.880 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.880 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.880 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.880 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.880 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.880 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.880 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.880 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.881 INFO:teuthology.orchestra.run.smithi006.stdout: "motd": { 2024-05-09T04:58:25.881 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "motd", 2024-05-09T04:58:25.881 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.881 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.881 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.881 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.881 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.881 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.881 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.881 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "The message of the day", 2024-05-09T04:58:25.881 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.881 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.882 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.882 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.882 INFO:teuthology.orchestra.run.smithi006.stdout: "redirect_resolve_ip_addr": { 2024-05-09T04:58:25.882 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T04:58:25.882 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.882 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.882 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.882 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.882 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.882 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.882 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.883 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.883 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.883 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.883 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.883 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.883 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2024-05-09T04:58:25.883 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2024-05-09T04:58:25.883 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.883 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.883 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.883 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "::", 2024-05-09T04:58:25.883 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.883 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.884 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.884 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.884 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.884 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.884 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.884 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.884 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2024-05-09T04:58:25.884 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2024-05-09T04:58:25.884 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.884 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.884 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.884 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8080", 2024-05-09T04:58:25.884 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.885 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.885 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.885 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.885 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.885 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.885 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.885 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.885 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.885 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.885 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.885 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.885 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.885 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.886 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.886 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.886 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.886 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.886 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.886 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.886 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.886 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.886 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl": { 2024-05-09T04:58:25.886 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl", 2024-05-09T04:58:25.886 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.886 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.887 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.887 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.887 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.887 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.887 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.887 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.887 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.887 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.887 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.887 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.887 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl_server_port": { 2024-05-09T04:58:25.887 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl_server_port", 2024-05-09T04:58:25.887 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.888 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.888 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.888 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8443", 2024-05-09T04:58:25.888 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.888 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.888 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.888 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.888 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.888 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.888 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.888 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.888 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_behaviour": { 2024-05-09T04:58:25.889 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_behaviour", 2024-05-09T04:58:25.889 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.889 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.889 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.889 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "redirect", 2024-05-09T04:58:25.889 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.889 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.889 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.889 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.889 INFO:teuthology.orchestra.run.smithi006.stdout: "redirect" 2024-05-09T04:58:25.889 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.889 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.889 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.890 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.890 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.890 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.890 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_error_status_code": { 2024-05-09T04:58:25.890 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_error_status_code", 2024-05-09T04:58:25.890 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.890 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.890 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.890 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "500", 2024-05-09T04:58:25.890 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "400", 2024-05-09T04:58:25.890 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "599", 2024-05-09T04:58:25.891 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.891 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.891 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.891 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.891 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.891 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.891 INFO:teuthology.orchestra.run.smithi006.stdout: "url_prefix": { 2024-05-09T04:58:25.891 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "url_prefix", 2024-05-09T04:58:25.891 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.891 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.891 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.891 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.891 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.892 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.892 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.892 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.892 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.892 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.892 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.892 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.892 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.892 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.892 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.892 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "devicehealth", 2024-05-09T04:58:25.892 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.892 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.893 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.893 INFO:teuthology.orchestra.run.smithi006.stdout: "enable_monitoring": { 2024-05-09T04:58:25.893 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enable_monitoring", 2024-05-09T04:58:25.893 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.893 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.893 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.893 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.893 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.893 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.893 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.893 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "monitor device health metrics", 2024-05-09T04:58:25.893 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.894 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.894 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.894 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.894 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.894 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.894 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.894 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.894 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.894 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.894 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.894 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.894 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.895 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.895 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.895 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.895 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.895 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.895 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.895 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.895 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.895 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.895 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.895 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.895 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.896 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.896 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.896 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.896 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.896 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.896 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.896 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.896 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.896 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.896 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.896 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.896 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.897 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.897 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.897 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.897 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.897 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.897 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.897 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.897 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.897 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.897 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.897 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.897 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.898 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.898 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.898 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.898 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.898 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.898 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.898 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.898 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.898 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.898 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.898 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.898 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.898 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.899 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.899 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.899 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.899 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.899 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.899 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.899 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.899 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.899 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.899 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.899 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.899 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.900 INFO:teuthology.orchestra.run.smithi006.stdout: "mark_out_threshold": { 2024-05-09T04:58:25.900 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mark_out_threshold", 2024-05-09T04:58:25.900 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.900 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.900 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.900 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "2419200", 2024-05-09T04:58:25.900 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.900 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.900 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.900 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T04:58:25.900 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.900 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.901 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.901 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.901 INFO:teuthology.orchestra.run.smithi006.stdout: "pool_name": { 2024-05-09T04:58:25.901 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pool_name", 2024-05-09T04:58:25.901 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.901 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.901 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.901 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "device_health_metrics", 2024-05-09T04:58:25.901 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.901 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.901 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.901 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T04:58:25.902 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.902 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.902 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.902 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.902 INFO:teuthology.orchestra.run.smithi006.stdout: "retention_period": { 2024-05-09T04:58:25.902 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "retention_period", 2024-05-09T04:58:25.902 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.902 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.902 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.902 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15552000", 2024-05-09T04:58:25.902 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.902 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.902 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.903 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long to retain device health metrics", 2024-05-09T04:58:25.903 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.903 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.903 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.903 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.903 INFO:teuthology.orchestra.run.smithi006.stdout: "scrape_frequency": { 2024-05-09T04:58:25.903 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "scrape_frequency", 2024-05-09T04:58:25.903 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.903 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.903 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.903 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "86400", 2024-05-09T04:58:25.903 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.904 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.904 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.904 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T04:58:25.904 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.904 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.904 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.904 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.904 INFO:teuthology.orchestra.run.smithi006.stdout: "self_heal": { 2024-05-09T04:58:25.904 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "self_heal", 2024-05-09T04:58:25.904 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.904 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.904 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.904 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:25.905 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.905 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.905 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.905 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T04:58:25.905 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.905 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.905 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.905 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.905 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-05-09T04:58:25.905 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-05-09T04:58:25.905 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.905 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.905 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.905 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-05-09T04:58:25.906 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.906 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.906 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.906 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T04:58:25.906 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.906 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.906 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.906 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.906 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.906 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.907 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.907 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.907 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.907 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.907 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.907 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.907 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.907 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.907 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.907 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.907 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.908 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.908 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_threshold": { 2024-05-09T04:58:25.908 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_threshold", 2024-05-09T04:58:25.908 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.908 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.908 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.908 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "7257600", 2024-05-09T04:58:25.908 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.908 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.908 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.908 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T04:58:25.908 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.908 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.908 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.909 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.909 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.909 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.909 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.909 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "diskprediction_local", 2024-05-09T04:58:25.909 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.909 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.909 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.909 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.909 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.909 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.909 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.909 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.909 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.910 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.910 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.910 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.910 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.910 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.910 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.910 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.910 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.910 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.910 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.910 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.911 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.911 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.911 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.911 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.911 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.911 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.911 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.911 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.911 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.911 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.911 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.911 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.911 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.912 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.912 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.912 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.912 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.912 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.912 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.912 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.912 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.912 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.912 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.912 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.912 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.913 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.913 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.913 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.913 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.913 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.913 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.913 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.913 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.913 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.913 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.913 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.913 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.914 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.914 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.914 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.914 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.914 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.914 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.914 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.914 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.914 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.914 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.914 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.914 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.914 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.915 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.915 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.915 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.915 INFO:teuthology.orchestra.run.smithi006.stdout: "predict_interval": { 2024-05-09T04:58:25.915 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "predict_interval", 2024-05-09T04:58:25.915 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.915 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.915 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.915 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "86400", 2024-05-09T04:58:25.915 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.915 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.915 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.915 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.915 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.916 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.916 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.916 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.916 INFO:teuthology.orchestra.run.smithi006.stdout: "predictor_model": { 2024-05-09T04:58:25.916 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "predictor_model", 2024-05-09T04:58:25.916 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.916 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.916 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.916 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "prophetstor", 2024-05-09T04:58:25.916 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.916 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.917 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.917 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.917 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.917 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.917 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.917 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.917 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-05-09T04:58:25.917 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-05-09T04:58:25.917 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.917 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.917 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.917 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-05-09T04:58:25.917 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.918 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.918 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.918 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.918 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.918 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.918 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.918 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.918 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.918 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.918 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.918 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.919 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.919 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.919 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.919 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.919 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.919 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.919 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.919 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.919 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.919 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.919 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.920 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.920 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.920 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "influx", 2024-05-09T04:58:25.920 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": false, 2024-05-09T04:58:25.920 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "influxdb python module not found", 2024-05-09T04:58:25.920 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.920 INFO:teuthology.orchestra.run.smithi006.stdout: "batch_size": { 2024-05-09T04:58:25.920 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "batch_size", 2024-05-09T04:58:25.920 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.920 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.920 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.920 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5000", 2024-05-09T04:58:25.920 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.921 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.921 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.921 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T04:58:25.921 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.921 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.921 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.921 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.921 INFO:teuthology.orchestra.run.smithi006.stdout: "database": { 2024-05-09T04:58:25.921 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "database", 2024-05-09T04:58:25.921 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.921 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.921 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.922 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "ceph", 2024-05-09T04:58:25.922 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.922 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.922 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.922 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T04:58:25.922 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.923 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.923 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.923 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.923 INFO:teuthology.orchestra.run.smithi006.stdout: "hostname": { 2024-05-09T04:58:25.923 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "hostname", 2024-05-09T04:58:25.923 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.924 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.924 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.924 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.924 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.924 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.924 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.924 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB server hostname", 2024-05-09T04:58:25.925 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.925 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.925 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.925 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.925 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-05-09T04:58:25.925 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-05-09T04:58:25.926 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:25.926 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.926 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.926 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "30", 2024-05-09T04:58:25.926 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "5", 2024-05-09T04:58:25.926 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.926 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.927 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T04:58:25.927 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.927 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.927 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.927 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.927 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.927 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.927 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.927 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.927 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.927 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.928 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.928 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.928 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.928 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.928 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.928 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.928 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.928 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.928 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.928 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.928 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.928 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.928 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.929 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.929 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.929 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.929 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.929 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.929 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.929 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.929 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.929 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.929 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.929 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.930 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.930 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.930 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.930 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.930 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.930 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.930 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.930 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.930 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.930 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.930 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.931 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.931 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.931 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.931 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.931 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.931 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.931 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.931 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.931 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.931 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.931 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.931 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.932 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.932 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.932 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.932 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.932 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.932 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.932 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.932 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.932 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.932 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.934 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.934 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.934 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.934 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.934 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.934 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.934 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.934 INFO:teuthology.orchestra.run.smithi006.stdout: "password": { 2024-05-09T04:58:25.934 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "password", 2024-05-09T04:58:25.934 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.935 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.935 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.935 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.935 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.935 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.935 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.935 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "password of InfluxDB server user", 2024-05-09T04:58:25.935 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.935 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.935 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.935 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.935 INFO:teuthology.orchestra.run.smithi006.stdout: "port": { 2024-05-09T04:58:25.935 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "port", 2024-05-09T04:58:25.935 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.936 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.936 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.936 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8086", 2024-05-09T04:58:25.936 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.936 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.936 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.936 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB server port", 2024-05-09T04:58:25.936 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.936 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.936 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.936 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.937 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.937 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.937 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.937 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.937 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.937 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.937 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.937 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.937 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.937 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.937 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.937 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.938 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.938 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.938 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl": { 2024-05-09T04:58:25.938 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl", 2024-05-09T04:58:25.938 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.938 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.938 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.938 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "false", 2024-05-09T04:58:25.938 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.938 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.938 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.938 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T04:58:25.938 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.938 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.939 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.939 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.939 INFO:teuthology.orchestra.run.smithi006.stdout: "threads": { 2024-05-09T04:58:25.939 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "threads", 2024-05-09T04:58:25.939 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.939 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.939 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.939 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-05-09T04:58:25.939 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-05-09T04:58:25.939 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "32", 2024-05-09T04:58:25.939 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.939 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T04:58:25.939 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.939 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.940 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.940 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.940 INFO:teuthology.orchestra.run.smithi006.stdout: "username": { 2024-05-09T04:58:25.940 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "username", 2024-05-09T04:58:25.940 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.940 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.940 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.940 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.940 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.940 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.940 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.940 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "username of InfluxDB server user", 2024-05-09T04:58:25.940 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.941 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.941 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.941 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.941 INFO:teuthology.orchestra.run.smithi006.stdout: "verify_ssl": { 2024-05-09T04:58:25.941 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "verify_ssl", 2024-05-09T04:58:25.941 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.941 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.941 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.941 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "true", 2024-05-09T04:58:25.941 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.941 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.941 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.942 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T04:58:25.942 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.942 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.942 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.942 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.942 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.942 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.942 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.942 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "insights", 2024-05-09T04:58:25.942 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.942 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.942 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.942 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.943 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.943 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.943 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.943 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.943 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.943 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.943 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.943 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.943 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.943 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.943 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.943 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.943 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.944 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.944 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.944 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.944 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.944 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.944 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.944 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.944 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.944 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.944 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.944 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.944 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.945 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.945 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.945 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.945 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.945 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.945 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.945 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.945 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.945 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.945 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.945 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.946 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.946 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.946 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.946 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.946 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.946 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.946 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.946 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.946 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.946 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.946 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.946 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.947 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.947 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.947 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.947 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.947 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.947 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.947 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.947 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.947 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.947 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.947 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.948 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.948 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.948 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.948 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.948 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.948 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.948 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.948 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.948 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.948 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.948 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.948 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.949 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.949 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.949 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.949 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.949 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.949 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.949 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.949 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.949 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.949 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.949 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.949 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.949 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.950 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.950 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.950 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "iostat", 2024-05-09T04:58:25.950 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.950 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.950 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.950 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.950 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.950 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.950 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.950 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.951 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.951 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.951 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.951 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.951 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.951 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.951 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.951 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.951 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.951 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.951 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.951 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.952 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.952 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.952 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.952 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.952 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.952 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.952 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.952 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.952 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.952 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.952 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.952 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.952 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.953 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.953 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.953 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.953 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.953 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.953 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.953 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.953 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.953 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.953 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.953 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.953 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.953 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.953 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.954 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.954 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.954 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.954 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.954 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.954 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.954 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.954 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.954 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.954 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.954 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.955 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.955 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.955 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.955 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.955 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.955 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.955 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.955 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.955 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.955 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.955 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.955 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.955 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.956 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.956 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.956 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.956 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.956 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.956 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.956 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.956 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.956 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.956 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.956 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.956 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.956 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.957 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.957 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.957 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.957 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.957 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.957 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.957 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "k8sevents", 2024-05-09T04:58:25.957 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.957 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.957 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.957 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_event_retention_days": { 2024-05-09T04:58:25.958 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ceph_event_retention_days", 2024-05-09T04:58:25.958 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.958 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.958 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.958 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "7", 2024-05-09T04:58:25.958 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.958 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.958 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.958 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-09T04:58:25.958 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.958 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.958 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.959 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.959 INFO:teuthology.orchestra.run.smithi006.stdout: "config_check_secs": { 2024-05-09T04:58:25.959 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_check_secs", 2024-05-09T04:58:25.959 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.959 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.959 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:25.959 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-05-09T04:58:25.959 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "10", 2024-05-09T04:58:25.959 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.959 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.959 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-09T04:58:25.959 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.959 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.960 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.960 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.960 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.960 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.960 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.960 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.960 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.960 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.960 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.960 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.960 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.960 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.961 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.961 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.961 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.961 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.961 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.961 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.961 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.961 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.961 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.961 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.961 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.961 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.962 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.962 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.962 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.962 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.962 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.962 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.962 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.962 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.962 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.962 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.962 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.962 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.963 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.963 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.963 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.963 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.963 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.963 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.963 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.963 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.963 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.963 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.963 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.963 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.964 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.964 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.964 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.964 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.964 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.964 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.964 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.964 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.964 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.964 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.964 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.965 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.965 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.965 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.965 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.965 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.965 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.965 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.965 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.965 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.965 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.965 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.965 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.965 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.966 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.966 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.966 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.966 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.966 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.966 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.966 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.966 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.966 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.966 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.966 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.966 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.967 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.967 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.967 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.967 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.967 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.967 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "localpool", 2024-05-09T04:58:25.967 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.967 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.967 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.967 INFO:teuthology.orchestra.run.smithi006.stdout: "failure_domain": { 2024-05-09T04:58:25.967 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "failure_domain", 2024-05-09T04:58:25.967 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.968 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.968 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.968 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "host", 2024-05-09T04:58:25.968 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.968 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.968 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.968 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "failure domain for any created local pool", 2024-05-09T04:58:25.968 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T04:58:25.968 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.968 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.968 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.968 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.968 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.968 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.969 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.969 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.969 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.969 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.969 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.969 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.969 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.969 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.969 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.969 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.969 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.969 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.970 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.970 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.970 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.970 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.970 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.970 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.970 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.970 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.970 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.970 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.970 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.970 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.971 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.971 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.971 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.971 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.971 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.971 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.971 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.971 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.971 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.971 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.972 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.972 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.972 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.972 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.972 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.972 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.972 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.972 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.972 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.972 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.972 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.972 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.973 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.973 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.973 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.973 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.973 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.973 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.973 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.973 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.973 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.973 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.973 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.974 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.974 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.974 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.974 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.974 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.974 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.974 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.974 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.974 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.974 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.974 INFO:teuthology.orchestra.run.smithi006.stdout: "min_size": { 2024-05-09T04:58:25.974 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "min_size", 2024-05-09T04:58:25.974 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.975 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.975 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.975 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.975 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.975 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.975 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.975 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default min_size for any created local pool", 2024-05-09T04:58:25.975 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T04:58:25.975 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.975 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.975 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.975 INFO:teuthology.orchestra.run.smithi006.stdout: "num_rep": { 2024-05-09T04:58:25.976 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "num_rep", 2024-05-09T04:58:25.976 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.976 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.976 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.976 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3", 2024-05-09T04:58:25.976 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.976 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.976 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.976 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default replica count for any created local pool", 2024-05-09T04:58:25.976 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.976 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.976 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.976 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.977 INFO:teuthology.orchestra.run.smithi006.stdout: "pg_num": { 2024-05-09T04:58:25.977 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pg_num", 2024-05-09T04:58:25.977 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.977 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.977 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.977 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "128", 2024-05-09T04:58:25.977 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.977 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.977 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.977 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T04:58:25.977 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.977 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.978 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.978 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.978 INFO:teuthology.orchestra.run.smithi006.stdout: "prefix": { 2024-05-09T04:58:25.978 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prefix", 2024-05-09T04:58:25.978 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.978 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.978 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.978 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.978 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.978 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.978 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.978 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "name prefix for any created local pool", 2024-05-09T04:58:25.979 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.979 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.979 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.979 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.979 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.979 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.979 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.979 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.979 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.979 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.979 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.979 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.979 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.980 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.980 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.980 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.980 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.980 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.980 INFO:teuthology.orchestra.run.smithi006.stdout: "subtree": { 2024-05-09T04:58:25.980 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "subtree", 2024-05-09T04:58:25.980 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.980 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.980 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.980 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "rack", 2024-05-09T04:58:25.980 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.981 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.981 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.981 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T04:58:25.981 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T04:58:25.981 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.981 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.981 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.981 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.981 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.981 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.981 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mds_autoscaler", 2024-05-09T04:58:25.981 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.982 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.982 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.982 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.982 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.982 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.982 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.982 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.982 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.982 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.982 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.982 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.982 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.982 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.983 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.983 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.983 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.983 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.983 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.983 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.983 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.983 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.983 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.983 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.983 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.983 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.983 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.984 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.984 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.984 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.984 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.984 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.984 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.984 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.984 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.984 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.984 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.984 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.984 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.984 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.985 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.985 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.985 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.985 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.985 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.985 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.985 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.985 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.985 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.985 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.985 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.985 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.986 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.986 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.986 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.986 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.986 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.986 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.986 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.986 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.986 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.986 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.986 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.986 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.986 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.987 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.987 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.987 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.987 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.987 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.987 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.987 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.987 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.987 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.987 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.987 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.987 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.988 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.988 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.988 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.988 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.988 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.988 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.988 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.988 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.988 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.988 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.988 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.989 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.989 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.989 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mirroring", 2024-05-09T04:58:25.989 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.989 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.989 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.989 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.989 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.989 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.989 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.989 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.989 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.990 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.990 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.990 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.990 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.990 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.990 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.990 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.990 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.990 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.990 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.990 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.990 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.991 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.991 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.991 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.991 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.991 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.991 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.991 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.991 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.991 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.991 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.991 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.991 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.991 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.992 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.992 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.992 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.992 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.992 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.992 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.992 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.992 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.992 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.992 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:25.992 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.992 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.993 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.993 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.993 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.993 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.993 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.993 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.993 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.993 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.993 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.993 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.993 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.994 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.994 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.994 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:25.994 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:25.994 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.994 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.994 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.994 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.994 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.994 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.994 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.994 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.994 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.995 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.995 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.995 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.995 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:25.995 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:25.995 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:25.995 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:25.995 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.995 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:25.995 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.995 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.995 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.995 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.995 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.996 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.996 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.996 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.996 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:25.996 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.996 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:25.996 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "nfs", 2024-05-09T04:58:25.996 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:25.996 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:25.996 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:25.996 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:25.996 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:25.997 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.997 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.997 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.997 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:25.997 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.997 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.997 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:25.997 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:25.997 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:25.997 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:25.997 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:25.997 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:25.997 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:25.997 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:25.998 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.998 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.998 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.998 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.998 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.998 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:25.998 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:25.998 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:25.998 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.998 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.998 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:25.998 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:25.999 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:25.999 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:25.999 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:25.999 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:25.999 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:25.999 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:25.999 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:25.999 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:25.999 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:25.999 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:25.999 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:25.999 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:25.999 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.000 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.000 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.000 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.000 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.000 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.000 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.000 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.000 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.000 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.000 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.000 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.000 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.000 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.001 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.001 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.001 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.001 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.001 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.001 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.001 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.001 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.001 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.001 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.001 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.001 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.001 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.002 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.002 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.002 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.002 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.002 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.002 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.002 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.002 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.002 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.002 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.002 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.002 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.003 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.003 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.003 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.003 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.003 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.003 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.003 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.003 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.003 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "orchestrator", 2024-05-09T04:58:26.003 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.003 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.003 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.003 INFO:teuthology.orchestra.run.smithi006.stdout: "fail_fs": { 2024-05-09T04:58:26.003 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "fail_fs", 2024-05-09T04:58:26.004 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.004 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.004 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.004 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.004 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.004 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.004 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.004 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T04:58:26.004 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.004 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.004 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.005 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.005 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.005 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.005 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.005 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.005 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.005 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.005 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.005 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.005 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.005 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.005 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.006 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.006 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.006 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.006 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.006 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.006 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.006 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.006 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.006 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.006 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.006 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.006 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.007 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.007 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.007 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.007 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.007 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.007 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.007 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.007 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.007 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.007 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.007 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.008 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.008 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.008 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.008 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.008 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.008 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.008 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.008 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.008 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.008 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.008 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.008 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.008 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.009 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.009 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.009 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.009 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.009 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.009 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.009 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.009 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.009 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.009 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.009 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.009 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.009 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.010 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.010 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.010 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.010 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.010 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.010 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.010 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.010 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.010 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.010 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.010 INFO:teuthology.orchestra.run.smithi006.stdout: "orchestrator": { 2024-05-09T04:58:26.010 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "orchestrator", 2024-05-09T04:58:26.011 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.011 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.011 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.011 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.011 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.011 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.011 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.011 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm", 2024-05-09T04:58:26.011 INFO:teuthology.orchestra.run.smithi006.stdout: "rook", 2024-05-09T04:58:26.011 INFO:teuthology.orchestra.run.smithi006.stdout: "test_orchestrator" 2024-05-09T04:58:26.011 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.011 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Orchestrator backend", 2024-05-09T04:58:26.012 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.012 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.012 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.012 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.012 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.012 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.012 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.012 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.012 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.012 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.012 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.012 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.012 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.013 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.013 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.013 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.013 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.013 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.013 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.013 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.013 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.013 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "osd_perf_query", 2024-05-09T04:58:26.013 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.013 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.013 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.013 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.014 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.014 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.014 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.014 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.014 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.014 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.014 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.014 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.014 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.014 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.014 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.014 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.015 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.015 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.015 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.015 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.015 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.015 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.015 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.015 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.015 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.015 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.015 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.015 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.016 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.016 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.016 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.016 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.016 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.016 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.016 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.016 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.016 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.016 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.016 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.016 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.016 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.017 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.017 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.017 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.017 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.017 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.017 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.017 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.017 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.017 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.017 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.017 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.017 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.017 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.018 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.018 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.018 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.018 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.018 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.018 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.018 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.018 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.018 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.018 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.018 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.018 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.019 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.019 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.019 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.019 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.019 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.019 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.019 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.019 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.019 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.019 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.019 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.019 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.020 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.020 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.020 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.020 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.020 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.020 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.020 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.020 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.020 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.020 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.020 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.020 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.021 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "osd_support", 2024-05-09T04:58:26.021 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.021 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.021 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.021 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.021 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.021 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.021 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.021 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.021 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.021 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.022 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.022 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.022 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.022 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.022 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.022 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.022 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.022 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.022 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.022 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.022 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.022 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.023 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.023 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.023 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.023 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.023 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.023 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.023 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.023 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.023 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.023 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.023 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.023 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.024 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.024 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.024 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.024 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.024 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.024 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.024 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.024 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.024 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.024 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.024 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.024 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.024 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.024 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.025 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.025 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.025 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.025 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.025 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.025 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.025 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.025 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.025 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.025 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.025 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.025 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.025 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.026 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.026 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.026 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.026 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.026 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.026 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.026 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.026 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.026 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.026 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.026 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.027 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.027 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.027 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.027 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.027 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.027 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.027 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.027 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.027 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.027 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.027 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.027 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.027 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.028 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.028 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.028 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.028 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.028 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.028 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pg_autoscaler", 2024-05-09T04:58:26.028 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.028 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.028 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.028 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.028 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.028 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.028 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.029 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.029 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.029 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.029 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.029 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.029 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.029 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.029 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.029 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.029 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.029 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.029 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.030 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.030 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.030 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.030 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.030 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.030 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.030 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.030 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.030 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.030 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.030 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.030 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.031 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.031 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.031 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.031 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.031 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.031 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.031 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.031 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.031 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.031 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.031 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.031 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.032 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.032 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.032 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.032 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.032 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.032 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.032 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.032 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.032 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.032 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.032 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.032 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.033 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.033 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.033 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.033 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.033 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.033 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.033 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.033 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.033 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.033 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.033 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.033 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.033 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.034 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.034 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.034 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.034 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.034 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.034 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-05-09T04:58:26.034 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-05-09T04:58:26.034 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:26.034 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.034 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.034 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-05-09T04:58:26.034 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.034 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.035 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.035 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.035 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.035 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.035 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.035 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.035 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.035 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.035 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.035 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.035 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.036 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.036 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.036 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.036 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.036 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.036 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.036 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.036 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.036 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.036 INFO:teuthology.orchestra.run.smithi006.stdout: "threshold": { 2024-05-09T04:58:26.036 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "threshold", 2024-05-09T04:58:26.036 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-05-09T04:58:26.037 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.037 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.037 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3.0", 2024-05-09T04:58:26.037 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1.0", 2024-05-09T04:58:26.037 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.037 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.037 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "scaling threshold", 2024-05-09T04:58:26.037 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T04:58:26.037 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.037 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.037 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.038 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.038 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.038 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.038 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "progress", 2024-05-09T04:58:26.038 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.038 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.038 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.038 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_pg_recovery_event": { 2024-05-09T04:58:26.038 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_pg_recovery_event", 2024-05-09T04:58:26.038 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.038 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.038 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.038 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.039 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.039 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.039 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.039 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T04:58:26.039 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.039 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.039 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.039 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.039 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": { 2024-05-09T04:58:26.039 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enabled", 2024-05-09T04:58:26.040 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.040 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.040 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.040 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.040 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.040 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.040 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.040 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.040 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.040 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.040 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.041 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.041 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.041 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.041 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.041 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.041 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.041 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.041 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.041 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.041 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.041 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.041 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.042 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.042 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.042 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.042 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.042 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.042 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.042 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.042 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.042 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.042 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.042 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.042 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.042 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.042 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.043 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.043 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.043 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.043 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.043 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.043 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.043 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.043 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.043 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.043 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.043 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.043 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.043 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.044 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.044 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.044 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.044 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.044 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.044 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.044 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.044 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.044 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.044 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.044 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.044 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.044 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.045 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.045 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.045 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.045 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.045 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.045 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.045 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.045 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.045 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.045 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.045 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.045 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.045 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.046 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.046 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.046 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.046 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.046 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.046 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.046 INFO:teuthology.orchestra.run.smithi006.stdout: "max_completed_events": { 2024-05-09T04:58:26.046 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_completed_events", 2024-05-09T04:58:26.046 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.046 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.046 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.046 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "50", 2024-05-09T04:58:26.046 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.047 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.047 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.047 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "number of past completed events to remember", 2024-05-09T04:58:26.047 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.047 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.047 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.047 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.047 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-05-09T04:58:26.047 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-05-09T04:58:26.047 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:26.047 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.047 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.048 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-05-09T04:58:26.048 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.048 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.048 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.048 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long the module is going to sleep", 2024-05-09T04:58:26.048 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.048 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.048 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.048 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.048 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.048 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.048 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.048 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.048 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.049 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.049 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.049 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.049 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.049 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.049 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.049 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.049 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.049 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.049 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.049 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.049 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.050 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prometheus", 2024-05-09T04:58:26.050 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.050 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.050 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.050 INFO:teuthology.orchestra.run.smithi006.stdout: "cache": { 2024-05-09T04:58:26.050 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cache", 2024-05-09T04:58:26.050 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.050 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.050 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.050 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.050 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.050 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.051 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.051 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.051 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.051 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.051 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.051 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.051 INFO:teuthology.orchestra.run.smithi006.stdout: "exclude_perf_counters": { 2024-05-09T04:58:26.051 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "exclude_perf_counters", 2024-05-09T04:58:26.051 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.051 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.051 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.051 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.051 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.052 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.052 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.052 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T04:58:26.052 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T04:58:26.052 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.052 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.052 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.052 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.052 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.052 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.052 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.053 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.053 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.053 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.053 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.053 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.053 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.053 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.053 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.053 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.053 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.053 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.053 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.053 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.054 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.054 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.054 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.054 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.054 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.054 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.054 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.054 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.054 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.054 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.054 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.054 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.055 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.055 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.055 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.055 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.055 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.055 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.055 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.055 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.055 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.055 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.055 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.055 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.055 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.056 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.056 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.056 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.056 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.056 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.056 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.056 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.056 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.056 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.056 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.056 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.056 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.056 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.057 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.057 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.057 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.057 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.057 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.057 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.057 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.057 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.057 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.057 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.057 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.057 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.058 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.058 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.058 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.058 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_stats_pools": { 2024-05-09T04:58:26.058 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_stats_pools", 2024-05-09T04:58:26.058 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.058 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.058 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.058 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.058 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.058 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.058 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.058 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.058 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.059 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.059 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.059 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.059 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T04:58:26.059 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T04:58:26.059 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.059 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.059 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.059 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "300", 2024-05-09T04:58:26.059 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.059 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.059 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.059 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.059 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.060 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.060 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.060 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.060 INFO:teuthology.orchestra.run.smithi006.stdout: "scrape_interval": { 2024-05-09T04:58:26.060 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "scrape_interval", 2024-05-09T04:58:26.060 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-05-09T04:58:26.060 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.060 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.060 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15.0", 2024-05-09T04:58:26.060 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.060 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.060 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.061 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.061 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.061 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.061 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.061 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.061 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2024-05-09T04:58:26.061 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2024-05-09T04:58:26.061 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.061 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.061 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.061 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "::", 2024-05-09T04:58:26.061 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.062 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.062 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.062 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T04:58:26.062 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.062 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.062 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.062 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.062 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2024-05-09T04:58:26.062 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2024-05-09T04:58:26.062 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.062 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.062 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.062 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "9283", 2024-05-09T04:58:26.063 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.063 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.063 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.063 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T04:58:26.063 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.063 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.063 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.063 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.063 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.063 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.063 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.063 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.064 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.064 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.064 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.064 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.064 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.064 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.064 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.064 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.064 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.064 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.064 INFO:teuthology.orchestra.run.smithi006.stdout: "stale_cache_strategy": { 2024-05-09T04:58:26.064 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "stale_cache_strategy", 2024-05-09T04:58:26.065 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.065 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.065 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.065 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "log", 2024-05-09T04:58:26.065 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.065 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.065 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.065 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.065 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.065 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.065 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.065 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.065 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_behaviour": { 2024-05-09T04:58:26.066 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_behaviour", 2024-05-09T04:58:26.066 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.066 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.066 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.066 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "default", 2024-05-09T04:58:26.066 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.066 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.066 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.066 INFO:teuthology.orchestra.run.smithi006.stdout: "default", 2024-05-09T04:58:26.066 INFO:teuthology.orchestra.run.smithi006.stdout: "error" 2024-05-09T04:58:26.066 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.066 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.066 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.067 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.067 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.067 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.067 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_error_status_code": { 2024-05-09T04:58:26.067 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_error_status_code", 2024-05-09T04:58:26.067 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.067 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.067 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.067 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "500", 2024-05-09T04:58:26.067 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "400", 2024-05-09T04:58:26.067 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "599", 2024-05-09T04:58:26.067 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.067 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.067 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.068 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.068 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.068 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.068 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.068 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.068 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.068 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_support", 2024-05-09T04:58:26.068 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.068 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.068 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.068 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.068 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.068 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.069 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.069 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.069 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.069 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.069 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.069 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.069 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.069 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.069 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.069 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.069 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.069 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.069 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.070 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.070 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.070 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.070 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.070 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.070 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.070 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.070 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.070 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.070 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.070 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.070 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.071 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.071 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.071 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.071 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.071 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.071 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.071 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.071 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.071 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.071 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.071 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.071 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.072 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.072 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.072 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.072 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.072 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.072 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.072 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.072 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.072 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.072 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.072 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.072 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.072 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.073 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.073 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.073 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.073 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.073 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.073 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.073 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.073 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.073 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.073 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.073 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.073 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.073 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.074 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.074 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.074 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.074 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.074 INFO:teuthology.orchestra.run.smithi006.stdout: "max_concurrent_snap_create": { 2024-05-09T04:58:26.074 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_concurrent_snap_create", 2024-05-09T04:58:26.074 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.074 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.074 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.074 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-05-09T04:58:26.074 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.074 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.074 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.074 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.075 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.075 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.075 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.075 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.075 INFO:teuthology.orchestra.run.smithi006.stdout: "mirror_snapshot_schedule": { 2024-05-09T04:58:26.075 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T04:58:26.075 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.075 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.075 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.075 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.075 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.075 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.075 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.076 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.076 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.076 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.076 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.076 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.076 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.076 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.076 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.076 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.076 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.076 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.076 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.076 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.077 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.077 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.077 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.077 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.077 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.077 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.077 INFO:teuthology.orchestra.run.smithi006.stdout: "trash_purge_schedule": { 2024-05-09T04:58:26.077 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "trash_purge_schedule", 2024-05-09T04:58:26.077 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.077 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.077 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.077 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.077 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.077 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.078 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.078 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.078 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.078 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.078 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.078 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.078 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.078 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.078 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.078 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "restful", 2024-05-09T04:58:26.078 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.078 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.078 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.079 INFO:teuthology.orchestra.run.smithi006.stdout: "enable_auth": { 2024-05-09T04:58:26.079 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enable_auth", 2024-05-09T04:58:26.079 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.079 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.079 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.079 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.079 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.079 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.079 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.079 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.079 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.079 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.079 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.080 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.080 INFO:teuthology.orchestra.run.smithi006.stdout: "key_file": { 2024-05-09T04:58:26.080 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "key_file", 2024-05-09T04:58:26.080 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.080 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.080 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.080 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.080 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.080 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.080 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.080 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.080 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.081 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.081 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.081 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.081 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.081 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.081 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.081 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.081 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.081 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.081 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.081 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.081 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.082 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.082 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.082 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.082 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.082 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.082 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.082 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.082 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.082 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.082 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.082 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.082 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.083 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.083 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.083 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.083 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.083 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.083 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.083 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.083 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.083 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.083 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.083 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.083 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.084 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.084 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.084 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.084 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.084 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.084 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.084 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.084 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.084 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.084 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.084 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.084 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.084 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.085 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.085 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.085 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.085 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.085 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.085 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.085 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.085 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.085 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.085 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.085 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.085 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.085 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.086 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.086 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.086 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.086 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.086 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.086 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.086 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.086 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.086 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.086 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.086 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.086 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2024-05-09T04:58:26.087 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2024-05-09T04:58:26.087 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.087 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.087 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.087 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.087 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.087 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.087 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.087 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.087 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.087 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.087 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.088 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.088 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2024-05-09T04:58:26.088 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2024-05-09T04:58:26.088 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.088 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.088 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.088 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.088 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.088 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.088 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.088 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.088 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.088 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.089 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.089 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.089 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.089 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.089 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.089 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.089 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.089 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.089 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.089 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.089 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.089 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.089 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.090 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.090 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.090 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.090 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.090 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.090 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.090 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rgw", 2024-05-09T04:58:26.090 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.090 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.090 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.090 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.090 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.091 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.091 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.091 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.091 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.091 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.091 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.091 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.091 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.091 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.091 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.091 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.091 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.092 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.092 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.092 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.092 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.092 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.092 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.092 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.092 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.092 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.092 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.092 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.092 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.093 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.093 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.093 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.093 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.093 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.093 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.093 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.093 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.093 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.093 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.093 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.093 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.093 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.094 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.094 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.094 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.094 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.094 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.094 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.094 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.094 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.094 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.094 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.094 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.094 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.094 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.095 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.095 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.095 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.095 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.095 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.095 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.095 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.095 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.095 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.095 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.095 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.095 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.096 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.096 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.096 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.096 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.096 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.096 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.096 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.096 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.096 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.096 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.096 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.097 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.097 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.097 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.097 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.097 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.097 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.097 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.097 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.097 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.097 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.097 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.097 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.097 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rook", 2024-05-09T04:58:26.098 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.098 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.098 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.098 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.098 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.098 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.098 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.098 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.098 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.098 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.098 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.098 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.099 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.099 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.099 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.099 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.099 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.099 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.099 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.099 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.099 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.099 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.099 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.099 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.099 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.100 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.100 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.100 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.100 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.100 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.100 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.100 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.100 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.100 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.100 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.100 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.100 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.101 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.101 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.101 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.101 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.101 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.101 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.101 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.101 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.101 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.101 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.101 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.101 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.102 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.102 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.102 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.102 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.102 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.102 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.102 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.102 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.102 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.102 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.102 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.102 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.103 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.103 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.103 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.103 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.103 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.103 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.103 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.103 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.103 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.103 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.103 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.103 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.103 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.104 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.104 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.104 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.104 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.104 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.104 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.104 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.104 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.104 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.104 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.104 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.104 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.105 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.105 INFO:teuthology.orchestra.run.smithi006.stdout: "storage_class": { 2024-05-09T04:58:26.105 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "storage_class", 2024-05-09T04:58:26.105 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.105 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.105 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.105 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "local", 2024-05-09T04:58:26.105 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.105 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.105 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.105 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T04:58:26.105 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.106 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.106 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.106 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.106 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.106 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.106 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.106 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "selftest", 2024-05-09T04:58:26.106 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.106 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.106 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.106 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.106 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.107 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.107 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.107 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.107 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.107 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.107 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.107 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.107 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.107 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.107 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.107 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.108 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.108 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.108 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.108 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.108 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.108 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.108 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.108 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.108 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.108 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.108 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.108 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.108 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.109 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.109 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.109 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.109 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.109 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.109 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.109 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.109 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.109 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.109 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.109 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.109 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.109 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.110 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.110 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.110 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.110 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.110 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.110 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.110 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.110 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.110 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.110 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.110 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.110 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.110 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.111 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.111 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.111 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.111 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.111 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.111 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.111 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.111 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.111 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.111 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.111 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.111 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.111 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.112 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.112 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.112 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.112 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.112 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.112 INFO:teuthology.orchestra.run.smithi006.stdout: "roption1": { 2024-05-09T04:58:26.112 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "roption1", 2024-05-09T04:58:26.112 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.112 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.112 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.112 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.113 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.113 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.113 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.113 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.113 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.113 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.113 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.113 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.113 INFO:teuthology.orchestra.run.smithi006.stdout: "roption2": { 2024-05-09T04:58:26.113 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "roption2", 2024-05-09T04:58:26.113 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.113 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.113 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.114 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "xyz", 2024-05-09T04:58:26.114 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.114 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.114 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.114 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.114 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.114 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.114 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.114 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.114 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption1": { 2024-05-09T04:58:26.114 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption1", 2024-05-09T04:58:26.114 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.114 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.115 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.115 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.115 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.115 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.115 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.115 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.115 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.115 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.115 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.115 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.115 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption2": { 2024-05-09T04:58:26.115 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption2", 2024-05-09T04:58:26.115 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.116 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.116 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.116 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.116 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.116 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.116 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.116 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.116 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.116 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.116 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.116 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.116 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption3": { 2024-05-09T04:58:26.116 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption3", 2024-05-09T04:58:26.117 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-05-09T04:58:26.117 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.117 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.117 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.117 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.117 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.117 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.117 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.117 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.117 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.117 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.117 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.117 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption4": { 2024-05-09T04:58:26.117 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption4", 2024-05-09T04:58:26.118 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.118 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.118 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.118 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.118 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.118 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.118 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.118 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.118 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.118 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.118 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.118 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.119 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption5": { 2024-05-09T04:58:26.119 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption5", 2024-05-09T04:58:26.119 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.119 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.119 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.119 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.119 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.119 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.119 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.119 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.119 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.119 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.119 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.120 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.120 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption6": { 2024-05-09T04:58:26.120 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption6", 2024-05-09T04:58:26.120 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.120 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.120 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.120 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.120 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.120 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.120 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.120 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.120 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.121 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.121 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.121 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.121 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption7": { 2024-05-09T04:58:26.121 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption7", 2024-05-09T04:58:26.121 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.121 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.121 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.121 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.121 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-05-09T04:58:26.121 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "42", 2024-05-09T04:58:26.122 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.122 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.122 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.122 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.122 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.122 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.122 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.122 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.122 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.122 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.122 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.122 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.123 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.123 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.123 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.123 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.123 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.123 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.123 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.123 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.123 INFO:teuthology.orchestra.run.smithi006.stdout: "testkey": { 2024-05-09T04:58:26.123 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testkey", 2024-05-09T04:58:26.123 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.123 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.124 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.124 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.124 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.124 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.124 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.124 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.124 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.124 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.124 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.124 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.124 INFO:teuthology.orchestra.run.smithi006.stdout: "testlkey": { 2024-05-09T04:58:26.124 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testlkey", 2024-05-09T04:58:26.124 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.124 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.125 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.125 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.125 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.125 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.125 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.125 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.125 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.125 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.125 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.125 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.125 INFO:teuthology.orchestra.run.smithi006.stdout: "testnewline": { 2024-05-09T04:58:26.125 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testnewline", 2024-05-09T04:58:26.125 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.125 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.126 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.126 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.126 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.126 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.126 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.126 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.126 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.126 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.126 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.126 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.126 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.126 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.126 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.127 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smb", 2024-05-09T04:58:26.127 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.127 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.127 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.127 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.127 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.127 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.127 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.127 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.127 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.127 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.127 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.127 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.127 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.128 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.128 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.128 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.128 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.128 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.128 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.128 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.128 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.128 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.128 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.128 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.128 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.128 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.128 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.129 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.129 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.129 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.129 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.129 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.129 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.129 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.129 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.129 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.129 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.129 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.129 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.129 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.129 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.130 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.130 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.130 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.130 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.130 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.130 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.130 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.130 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.130 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.130 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.130 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.130 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.130 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.130 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.131 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.131 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.131 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.131 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.131 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.131 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.131 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.131 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.131 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.131 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.131 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.131 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.131 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.132 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.132 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.132 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.132 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.132 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.132 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.132 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.132 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.132 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.132 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.132 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.132 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.132 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.133 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.133 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.133 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.133 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.133 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.133 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.133 INFO:teuthology.orchestra.run.smithi006.stdout: "update_orchestration": { 2024-05-09T04:58:26.133 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "update_orchestration", 2024-05-09T04:58:26.133 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.133 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.133 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.133 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.133 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.134 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.134 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.134 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-09T04:58:26.134 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.134 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.134 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.134 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.134 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.134 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.134 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.134 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "snap_schedule", 2024-05-09T04:58:26.134 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.134 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.135 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.135 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_m_granularity": { 2024-05-09T04:58:26.135 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_m_granularity", 2024-05-09T04:58:26.135 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.135 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.135 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.135 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.135 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.135 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.135 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.135 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T04:58:26.135 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.135 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.136 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.136 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.138 INFO:teuthology.orchestra.run.smithi006.stdout: "dump_on_update": { 2024-05-09T04:58:26.138 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "dump_on_update", 2024-05-09T04:58:26.138 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.138 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.138 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.138 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.138 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.138 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.138 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.139 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "dump database to debug log on update", 2024-05-09T04:58:26.139 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.139 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.139 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.139 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.139 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.139 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.139 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.139 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.139 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.139 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.140 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.140 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.140 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.140 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.140 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.140 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.140 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.140 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.140 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.140 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.140 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.140 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.140 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.141 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.141 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.141 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.141 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.141 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.141 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.141 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.141 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.142 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.142 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.142 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.142 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.142 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.142 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.142 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.142 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.142 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.142 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.142 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.142 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.143 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.143 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.143 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.143 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.143 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.143 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.143 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.143 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.143 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.143 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.143 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.143 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.144 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.144 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.144 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.144 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.144 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.144 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.144 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.144 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.144 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.144 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.144 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.144 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.144 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.145 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.145 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.145 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.145 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.145 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.145 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.145 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.145 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.145 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.145 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.145 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.145 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.146 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.146 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.146 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.146 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.146 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.146 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.146 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.146 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.146 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.146 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.146 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.147 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "stats", 2024-05-09T04:58:26.147 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.147 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.147 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.147 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.147 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.147 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.147 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.147 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.147 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.147 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.147 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.148 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.148 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.148 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.148 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.148 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.148 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.148 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.148 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.148 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.148 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.148 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.149 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.149 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.149 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.149 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.149 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.149 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.149 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.149 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.149 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.149 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.149 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.150 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.150 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.150 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.150 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.150 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.150 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.150 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.150 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.150 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.150 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.150 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.151 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.151 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.151 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.151 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.151 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.151 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.151 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.151 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.151 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.151 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.152 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.152 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.152 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.152 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.152 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.152 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.152 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.152 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.152 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.152 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.152 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.152 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.153 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.153 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.153 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.153 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.153 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.153 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.153 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.153 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.153 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.153 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.153 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.153 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.154 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.154 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.154 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.154 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.154 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.154 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.154 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.154 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.154 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.154 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.154 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.155 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.155 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "status", 2024-05-09T04:58:26.155 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.155 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.155 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.155 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.155 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.155 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.155 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.155 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.155 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.155 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.156 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.156 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.156 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.156 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.156 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.156 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.156 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.156 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.156 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.156 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.156 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.156 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.156 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.157 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.157 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.157 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.157 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.157 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.157 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.157 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.157 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.157 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.157 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.157 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.157 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.158 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.158 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.158 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.158 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.158 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.158 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.158 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.158 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.158 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.158 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.158 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.158 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.159 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.159 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.159 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.159 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.159 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.159 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.159 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.159 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.159 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.159 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.159 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.159 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.159 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.160 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.160 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.160 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.160 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.160 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.160 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.160 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.160 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.160 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.160 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.160 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.160 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.160 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.161 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.161 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.161 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.161 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.161 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.161 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.161 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.161 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.161 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.161 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.161 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.161 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.162 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.162 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.162 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.162 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.162 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.162 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "telegraf", 2024-05-09T04:58:26.162 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.162 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.162 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.162 INFO:teuthology.orchestra.run.smithi006.stdout: "address": { 2024-05-09T04:58:26.162 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "address", 2024-05-09T04:58:26.162 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.163 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.163 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.163 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T04:58:26.163 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.163 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.163 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.163 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.163 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.163 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.163 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.163 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.163 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-05-09T04:58:26.163 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-05-09T04:58:26.164 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:26.164 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.164 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.164 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15", 2024-05-09T04:58:26.164 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.164 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.164 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.164 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.164 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.164 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.164 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.164 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.164 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.165 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.165 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.165 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.165 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.165 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.165 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.165 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.165 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.165 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.165 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.165 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.165 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.165 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.165 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.166 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.166 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.166 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.166 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.166 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.166 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.166 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.166 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.166 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.166 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.166 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.166 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.167 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.167 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.167 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.167 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.167 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.167 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.167 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.167 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.167 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.167 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.167 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.167 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.167 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.168 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.168 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.168 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.168 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.168 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.168 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.168 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.168 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.168 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.168 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.168 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.168 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.168 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.168 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.169 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.169 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.169 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.169 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.169 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.169 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.169 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.169 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.169 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.169 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.169 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.169 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.169 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.170 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.170 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.170 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.170 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.170 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.170 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.170 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.170 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.170 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.170 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.170 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.170 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.170 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.171 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.171 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.171 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.171 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.171 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.171 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.171 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.171 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "telemetry", 2024-05-09T04:58:26.171 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.171 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.171 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.172 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_basic": { 2024-05-09T04:58:26.172 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_basic", 2024-05-09T04:58:26.172 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.172 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.172 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.172 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.172 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.172 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.172 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.172 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T04:58:26.172 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.172 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.173 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.173 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.173 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_crash": { 2024-05-09T04:58:26.173 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_crash", 2024-05-09T04:58:26.173 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.173 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.173 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.173 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.173 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.173 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.173 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.173 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T04:58:26.173 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.173 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.173 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.174 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.174 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_device": { 2024-05-09T04:58:26.174 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_device", 2024-05-09T04:58:26.174 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.174 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.174 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.174 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.174 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.174 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.174 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.174 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T04:58:26.174 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.174 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.175 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.175 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.175 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_ident": { 2024-05-09T04:58:26.175 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_ident", 2024-05-09T04:58:26.175 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.175 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.175 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.175 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.175 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.175 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.175 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.175 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T04:58:26.175 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.176 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.176 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.176 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.176 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_perf": { 2024-05-09T04:58:26.176 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_perf", 2024-05-09T04:58:26.176 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.176 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.176 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.176 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.176 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.176 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.176 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.177 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T04:58:26.177 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.177 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.177 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.177 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.177 INFO:teuthology.orchestra.run.smithi006.stdout: "contact": { 2024-05-09T04:58:26.177 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "contact", 2024-05-09T04:58:26.177 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.177 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.177 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.177 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.177 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.178 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.178 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.178 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.178 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.178 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.178 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.178 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.178 INFO:teuthology.orchestra.run.smithi006.stdout: "description": { 2024-05-09T04:58:26.178 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "description", 2024-05-09T04:58:26.178 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.178 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.179 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.179 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.179 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.179 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.179 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.179 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.179 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.179 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.179 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.179 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.179 INFO:teuthology.orchestra.run.smithi006.stdout: "device_url": { 2024-05-09T04:58:26.179 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_url", 2024-05-09T04:58:26.180 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.180 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.180 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.180 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T04:58:26.180 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.180 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.180 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.180 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.180 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.180 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.180 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.180 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.180 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": { 2024-05-09T04:58:26.181 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enabled", 2024-05-09T04:58:26.181 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.181 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.181 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.181 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.181 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.181 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.181 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.181 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.181 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.181 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.182 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.182 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.182 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-05-09T04:58:26.182 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-05-09T04:58:26.182 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.182 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.182 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.182 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "24", 2024-05-09T04:58:26.182 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "8", 2024-05-09T04:58:26.182 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.182 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.182 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.182 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.183 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.183 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.183 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.183 INFO:teuthology.orchestra.run.smithi006.stdout: "last_opt_revision": { 2024-05-09T04:58:26.183 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "last_opt_revision", 2024-05-09T04:58:26.183 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.183 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.183 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.183 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1", 2024-05-09T04:58:26.183 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.183 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.183 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.183 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.184 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.184 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.184 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.184 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.184 INFO:teuthology.orchestra.run.smithi006.stdout: "leaderboard": { 2024-05-09T04:58:26.184 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "leaderboard", 2024-05-09T04:58:26.184 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.184 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.184 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.184 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.184 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.184 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.184 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.185 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.185 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.185 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.185 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.185 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.185 INFO:teuthology.orchestra.run.smithi006.stdout: "leaderboard_description": { 2024-05-09T04:58:26.185 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "leaderboard_description", 2024-05-09T04:58:26.185 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.185 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.185 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.185 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.185 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.186 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.186 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.186 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.186 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.186 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.186 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.186 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.186 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.186 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.186 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.186 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.186 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.187 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.187 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.187 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.187 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.187 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.187 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.187 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.187 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.187 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.187 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.188 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.188 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.188 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.188 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.188 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.188 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.188 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.188 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.188 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.188 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.188 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.188 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.188 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.189 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.189 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.189 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.189 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.189 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.189 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.189 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.189 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.189 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.189 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.189 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.189 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.190 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.190 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.190 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.190 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.190 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.190 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.190 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.190 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.190 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.190 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.190 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.190 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.191 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.191 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.191 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.191 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.191 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.191 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.191 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.191 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.191 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.191 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.191 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.191 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.192 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.192 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.192 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.192 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.192 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.192 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.192 INFO:teuthology.orchestra.run.smithi006.stdout: "organization": { 2024-05-09T04:58:26.192 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "organization", 2024-05-09T04:58:26.192 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.192 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.192 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.192 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.193 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.193 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.193 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.193 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.193 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.193 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.193 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.193 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.193 INFO:teuthology.orchestra.run.smithi006.stdout: "proxy": { 2024-05-09T04:58:26.193 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "proxy", 2024-05-09T04:58:26.193 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.193 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.194 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.194 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.194 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.194 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.194 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.194 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.194 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.194 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.194 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.194 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.194 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.194 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.194 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.195 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.195 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.195 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.195 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.195 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.195 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.195 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.195 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.195 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.195 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.195 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.195 INFO:teuthology.orchestra.run.smithi006.stdout: "url": { 2024-05-09T04:58:26.196 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "url", 2024-05-09T04:58:26.196 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.196 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.196 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.196 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T04:58:26.196 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.196 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.196 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.196 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.196 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.196 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.196 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.196 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.197 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.197 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.197 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.197 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "test_orchestrator", 2024-05-09T04:58:26.197 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.197 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.197 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.197 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.197 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.197 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.197 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.198 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.198 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.198 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.198 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.198 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.198 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.198 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.198 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.198 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.198 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.198 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.198 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.198 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.198 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.199 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.199 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.199 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.199 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.199 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.199 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.199 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.199 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.199 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.199 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.199 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.200 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.200 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.200 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.200 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.200 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.200 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.200 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.200 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.200 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.200 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.200 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.200 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.201 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.201 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.201 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.201 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.201 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.201 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.201 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.201 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.201 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.201 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.201 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.202 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.202 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.202 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.202 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.202 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.202 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.202 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.202 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.202 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.202 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.202 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.202 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.203 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.203 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.203 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.203 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.203 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.203 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.203 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.203 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.203 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.203 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.203 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.203 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.203 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.203 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.204 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.204 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.204 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.204 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.204 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.204 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.204 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.204 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.204 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.204 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "volumes", 2024-05-09T04:58:26.204 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.204 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.205 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.205 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.205 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.205 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.205 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.205 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.205 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.205 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.205 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.205 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.205 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.205 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.205 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.205 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.206 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.206 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.206 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.206 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.206 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.206 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.206 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.206 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.206 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.206 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.206 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.206 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.206 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.207 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.207 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.207 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.207 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.207 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.207 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.207 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.207 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.207 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.207 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.207 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.207 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.207 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.208 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.208 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.208 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.208 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.208 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.208 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.208 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.208 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.208 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.208 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.208 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.208 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.209 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.209 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.209 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.209 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.209 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.209 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.209 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.209 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.209 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.209 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.209 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.209 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.210 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.210 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.210 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.210 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.210 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.210 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.210 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.210 INFO:teuthology.orchestra.run.smithi006.stdout: "max_concurrent_clones": { 2024-05-09T04:58:26.210 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_concurrent_clones", 2024-05-09T04:58:26.210 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.210 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.210 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.211 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "4", 2024-05-09T04:58:26.211 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.211 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.211 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.211 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T04:58:26.211 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.211 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.211 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.211 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.211 INFO:teuthology.orchestra.run.smithi006.stdout: "periodic_async_work": { 2024-05-09T04:58:26.211 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "periodic_async_work", 2024-05-09T04:58:26.212 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.212 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.212 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.212 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.212 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.212 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.212 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.212 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Periodically check for async work", 2024-05-09T04:58:26.212 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.212 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.212 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.212 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.212 INFO:teuthology.orchestra.run.smithi006.stdout: "snapshot_clone_delay": { 2024-05-09T04:58:26.213 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "snapshot_clone_delay", 2024-05-09T04:58:26.213 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.213 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.213 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.213 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.213 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.213 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.213 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.213 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T04:58:26.213 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.213 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.213 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.213 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.214 INFO:teuthology.orchestra.run.smithi006.stdout: "snapshot_clone_no_wait": { 2024-05-09T04:58:26.214 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T04:58:26.214 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.214 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.214 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.214 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.214 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.214 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.214 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.214 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T04:58:26.214 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.214 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.214 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.215 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.215 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.215 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.215 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.215 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.215 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.215 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.215 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.215 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.215 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.215 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.215 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.216 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.216 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.216 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.216 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.216 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.216 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.216 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix", 2024-05-09T04:58:26.216 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.216 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.216 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.216 INFO:teuthology.orchestra.run.smithi006.stdout: "discovery_interval": { 2024-05-09T04:58:26.216 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "discovery_interval", 2024-05-09T04:58:26.216 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-05-09T04:58:26.217 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.217 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.217 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "100", 2024-05-09T04:58:26.217 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.217 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.217 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.217 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.217 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.217 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.217 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.217 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.218 INFO:teuthology.orchestra.run.smithi006.stdout: "identifier": { 2024-05-09T04:58:26.218 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "identifier", 2024-05-09T04:58:26.218 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.218 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.218 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.218 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.218 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.218 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.218 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.218 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.218 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.219 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.219 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.219 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.219 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-05-09T04:58:26.219 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-05-09T04:58:26.219 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:26.219 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.219 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.219 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-05-09T04:58:26.219 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.219 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.219 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.220 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.220 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.220 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.220 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.220 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.220 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.220 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.220 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.220 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.220 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.220 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.221 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.221 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.221 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.221 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.221 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.221 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.221 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.221 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.221 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.221 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.221 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.221 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.222 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.222 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.222 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.222 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.222 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.222 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.222 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.222 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.222 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.222 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.222 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.222 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.223 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.223 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.223 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.223 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.223 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.223 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.223 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.223 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.223 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.223 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.223 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.224 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.224 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.224 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.224 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.224 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.224 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.224 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.224 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.224 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.224 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.224 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.225 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.225 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.225 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.225 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.225 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.225 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.225 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.225 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.225 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.225 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.225 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.225 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.226 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.226 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.226 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.226 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.226 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.226 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.226 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.226 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.226 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.226 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.226 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.226 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.227 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.227 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.227 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.227 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.227 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.227 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.227 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.227 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.227 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_host": { 2024-05-09T04:58:26.227 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_host", 2024-05-09T04:58:26.227 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.227 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.228 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.228 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.228 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.228 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.228 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.228 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.228 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.228 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.228 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.228 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.228 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_port": { 2024-05-09T04:58:26.228 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_port", 2024-05-09T04:58:26.229 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.229 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.229 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.229 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10051", 2024-05-09T04:58:26.229 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.229 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.229 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.229 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.229 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.229 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.229 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.229 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.230 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_sender": { 2024-05-09T04:58:26.230 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_sender", 2024-05-09T04:58:26.230 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.230 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.230 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.230 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T04:58:26.230 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.230 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.230 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.230 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.230 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.230 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.231 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.231 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.231 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.231 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.231 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2024-05-09T04:58:26.231 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.231 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.231 INFO:teuthology.orchestra.run.smithi006.stdout: "modules": [ 2024-05-09T04:58:26.231 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm", 2024-05-09T04:58:26.231 INFO:teuthology.orchestra.run.smithi006.stdout: "dashboard", 2024-05-09T04:58:26.231 INFO:teuthology.orchestra.run.smithi006.stdout: "iostat", 2024-05-09T04:58:26.232 INFO:teuthology.orchestra.run.smithi006.stdout: "nfs", 2024-05-09T04:58:26.232 INFO:teuthology.orchestra.run.smithi006.stdout: "restful" 2024-05-09T04:58:26.232 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.232 INFO:teuthology.orchestra.run.smithi006.stdout: "available_modules": [ 2024-05-09T04:58:26.232 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.232 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "alerts", 2024-05-09T04:58:26.232 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.232 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.232 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.232 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-05-09T04:58:26.232 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-05-09T04:58:26.232 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:26.232 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.232 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.233 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-05-09T04:58:26.233 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.233 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.233 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.233 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T04:58:26.233 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.233 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.233 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.233 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.233 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.233 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.233 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.234 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.234 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.234 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.234 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.234 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.234 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.234 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.234 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.234 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.234 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.234 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.234 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.234 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.235 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.235 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.235 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.235 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.235 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.235 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.235 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.235 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.235 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.235 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.235 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.235 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.235 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.236 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.236 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.236 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.236 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.236 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.236 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.236 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.236 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.236 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.236 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.236 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.236 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.236 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.237 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.237 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.237 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.237 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.237 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.237 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.237 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.237 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.237 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.237 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.237 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.237 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.237 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.237 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.238 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.238 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.238 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.238 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.238 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.238 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.238 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.238 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.238 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.238 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.238 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.238 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.239 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.239 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.239 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_destination": { 2024-05-09T04:58:26.239 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_destination", 2024-05-09T04:58:26.239 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.239 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.239 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.239 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.239 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.239 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.239 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.239 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Email address to send alerts to", 2024-05-09T04:58:26.240 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.240 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.240 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.240 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.240 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_from_name": { 2024-05-09T04:58:26.240 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_from_name", 2024-05-09T04:58:26.240 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.240 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.240 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.240 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "Ceph", 2024-05-09T04:58:26.240 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.240 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.240 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.241 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Email From: name", 2024-05-09T04:58:26.241 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.241 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.241 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.241 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.241 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_host": { 2024-05-09T04:58:26.241 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_host", 2024-05-09T04:58:26.241 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.241 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.241 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.241 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.241 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.242 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.242 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.242 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP server", 2024-05-09T04:58:26.242 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.242 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.242 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.242 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.242 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_password": { 2024-05-09T04:58:26.242 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_password", 2024-05-09T04:58:26.242 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.242 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.242 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.242 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.243 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.243 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.243 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.243 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Password to authenticate with", 2024-05-09T04:58:26.243 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.243 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.243 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.243 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.243 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_port": { 2024-05-09T04:58:26.243 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_port", 2024-05-09T04:58:26.243 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.243 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.244 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.244 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "465", 2024-05-09T04:58:26.244 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.244 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.244 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.244 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP port", 2024-05-09T04:58:26.244 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.244 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.244 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.244 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.244 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_sender": { 2024-05-09T04:58:26.244 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_sender", 2024-05-09T04:58:26.244 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.245 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.245 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.245 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.245 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.245 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.245 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.245 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP envelope sender", 2024-05-09T04:58:26.245 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.245 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.245 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.245 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.245 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_ssl": { 2024-05-09T04:58:26.245 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_ssl", 2024-05-09T04:58:26.246 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.246 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.246 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.246 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.246 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.246 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.246 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.246 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T04:58:26.246 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.246 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.246 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.246 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.247 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_user": { 2024-05-09T04:58:26.247 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_user", 2024-05-09T04:58:26.247 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.247 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.247 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.247 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.247 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.247 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.247 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.247 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "User to authenticate as", 2024-05-09T04:58:26.247 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.247 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.248 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.248 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.248 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.248 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.248 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.248 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.248 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.248 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.248 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.248 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.248 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.248 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.249 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.249 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.249 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.249 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.249 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.249 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.249 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.249 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "balancer", 2024-05-09T04:58:26.249 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.249 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.249 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.249 INFO:teuthology.orchestra.run.smithi006.stdout: "active": { 2024-05-09T04:58:26.250 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "active", 2024-05-09T04:58:26.250 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.250 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.250 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.250 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.250 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.250 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.250 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.250 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T04:58:26.250 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.250 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.250 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.251 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.251 INFO:teuthology.orchestra.run.smithi006.stdout: "begin_time": { 2024-05-09T04:58:26.251 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "begin_time", 2024-05-09T04:58:26.251 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.251 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.251 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.251 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0000", 2024-05-09T04:58:26.251 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.251 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.251 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.251 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T04:58:26.252 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T04:58:26.252 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.252 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.252 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.252 INFO:teuthology.orchestra.run.smithi006.stdout: "begin_weekday": { 2024-05-09T04:58:26.252 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "begin_weekday", 2024-05-09T04:58:26.252 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-05-09T04:58:26.252 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.252 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.252 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.252 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0", 2024-05-09T04:58:26.252 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "6", 2024-05-09T04:58:26.252 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.253 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T04:58:26.253 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T04:58:26.253 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.253 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.253 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.253 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_max_iterations": { 2024-05-09T04:58:26.253 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_max_iterations", 2024-05-09T04:58:26.253 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-05-09T04:58:26.253 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.253 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.253 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "25", 2024-05-09T04:58:26.254 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-05-09T04:58:26.254 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "250", 2024-05-09T04:58:26.254 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.254 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T04:58:26.254 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.254 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.254 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.254 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.254 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_metrics": { 2024-05-09T04:58:26.254 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_metrics", 2024-05-09T04:58:26.254 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.254 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.254 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.255 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T04:58:26.255 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.255 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.255 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.255 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T04:58:26.255 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T04:58:26.255 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.255 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.255 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.255 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_step": { 2024-05-09T04:58:26.255 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_step", 2024-05-09T04:58:26.255 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-05-09T04:58:26.255 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.256 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.256 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0.5", 2024-05-09T04:58:26.256 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0.001", 2024-05-09T04:58:26.256 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "0.999", 2024-05-09T04:58:26.256 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.256 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "aggressiveness of optimization", 2024-05-09T04:58:26.256 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T04:58:26.256 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.256 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.256 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.256 INFO:teuthology.orchestra.run.smithi006.stdout: "end_time": { 2024-05-09T04:58:26.256 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "end_time", 2024-05-09T04:58:26.256 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.257 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.257 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.257 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "2359", 2024-05-09T04:58:26.257 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.257 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.257 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.257 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T04:58:26.257 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T04:58:26.257 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.257 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.257 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.257 INFO:teuthology.orchestra.run.smithi006.stdout: "end_weekday": { 2024-05-09T04:58:26.257 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "end_weekday", 2024-05-09T04:58:26.258 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-05-09T04:58:26.258 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.258 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.258 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.258 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0", 2024-05-09T04:58:26.258 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "6", 2024-05-09T04:58:26.258 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.258 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T04:58:26.260 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:24 smithi169 ceph-mon[34075]: pgmap v603: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 MiB/s wr, 20 op/s 2024-05-09T04:58:26.260 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:24 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/3460649085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T04:58:26.260 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:24 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/4039212125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T04:58:26.260 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:24 smithi186 ceph-mon[34668]: pgmap v603: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 MiB/s wr, 20 op/s 2024-05-09T04:58:26.260 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:24 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/3460649085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T04:58:26.260 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:24 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/4039212125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T04:58:26.263 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T04:58:26.263 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.263 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.263 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.263 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.263 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.263 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.263 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.263 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.264 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.264 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.264 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.264 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.264 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.264 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.264 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.264 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.264 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.264 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.264 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.264 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.265 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.265 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.265 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.265 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.265 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.265 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.265 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.265 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.265 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.265 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.265 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.265 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.265 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.266 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.266 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.266 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.266 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.266 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.266 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.266 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.266 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.266 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.266 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.266 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.266 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.266 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.267 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.267 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.267 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.267 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.267 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.267 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.267 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.267 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.267 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.267 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.267 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.267 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.267 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.268 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.268 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.268 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.268 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.268 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.268 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.268 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.268 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.268 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.268 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.268 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.268 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.269 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.269 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.269 INFO:teuthology.orchestra.run.smithi006.stdout: "min_score": { 2024-05-09T04:58:26.269 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "min_score", 2024-05-09T04:58:26.269 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-05-09T04:58:26.269 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.269 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.269 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.269 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.269 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.269 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.269 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T04:58:26.270 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.270 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.270 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.270 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.270 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": { 2024-05-09T04:58:26.270 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mode", 2024-05-09T04:58:26.270 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.270 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.270 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.270 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "upmap", 2024-05-09T04:58:26.270 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.271 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.271 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.271 INFO:teuthology.orchestra.run.smithi006.stdout: "crush-compat", 2024-05-09T04:58:26.271 INFO:teuthology.orchestra.run.smithi006.stdout: "none", 2024-05-09T04:58:26.271 INFO:teuthology.orchestra.run.smithi006.stdout: "read", 2024-05-09T04:58:26.271 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap", 2024-05-09T04:58:26.271 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap-read" 2024-05-09T04:58:26.271 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.271 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Balancer mode", 2024-05-09T04:58:26.271 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.271 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.271 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.272 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.272 INFO:teuthology.orchestra.run.smithi006.stdout: "pool_ids": { 2024-05-09T04:58:26.272 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pool_ids", 2024-05-09T04:58:26.272 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.272 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.272 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.272 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.272 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.272 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.272 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.273 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T04:58:26.273 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.273 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.273 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.273 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.273 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-05-09T04:58:26.273 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-05-09T04:58:26.273 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:26.273 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.273 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.273 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-05-09T04:58:26.273 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.273 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.274 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.274 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T04:58:26.274 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.274 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.274 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.274 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.274 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.274 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.274 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.274 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.274 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.275 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.275 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.275 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.275 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.275 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.275 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.275 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.275 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.275 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.275 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap_max_deviation": { 2024-05-09T04:58:26.275 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "upmap_max_deviation", 2024-05-09T04:58:26.276 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.276 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.276 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.276 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-05-09T04:58:26.276 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-05-09T04:58:26.276 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.276 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.276 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T04:58:26.276 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T04:58:26.276 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.276 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.276 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.276 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap_max_optimizations": { 2024-05-09T04:58:26.277 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "upmap_max_optimizations", 2024-05-09T04:58:26.277 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-05-09T04:58:26.277 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.277 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.277 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-05-09T04:58:26.277 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.277 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.277 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.277 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T04:58:26.277 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.277 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.277 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.277 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.277 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.278 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.278 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.278 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cephadm", 2024-05-09T04:58:26.278 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.278 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.278 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.278 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_down_multiplier": { 2024-05-09T04:58:26.278 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_down_multiplier", 2024-05-09T04:58:26.278 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-05-09T04:58:26.278 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.278 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.278 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3.0", 2024-05-09T04:58:26.279 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.279 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.279 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.279 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T04:58:26.279 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.279 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.279 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.279 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.279 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_refresh_rate": { 2024-05-09T04:58:26.279 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_refresh_rate", 2024-05-09T04:58:26.279 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:26.279 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.280 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.280 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "20", 2024-05-09T04:58:26.280 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.280 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.280 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.280 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T04:58:26.280 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.280 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.280 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.280 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.280 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_starting_port": { 2024-05-09T04:58:26.280 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_starting_port", 2024-05-09T04:58:26.281 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.281 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.281 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.281 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "4721", 2024-05-09T04:58:26.281 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.281 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.281 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.281 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T04:58:26.281 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.281 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.281 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.281 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.282 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_ptrace": { 2024-05-09T04:58:26.282 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_ptrace", 2024-05-09T04:58:26.282 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.282 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.282 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.282 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.282 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.282 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.282 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.282 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T04:58:26.282 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T04:58:26.282 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.283 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.283 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.283 INFO:teuthology.orchestra.run.smithi006.stdout: "autotune_interval": { 2024-05-09T04:58:26.283 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "autotune_interval", 2024-05-09T04:58:26.283 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:26.283 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.283 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.283 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-05-09T04:58:26.283 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.283 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.283 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.283 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T04:58:26.283 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.284 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.284 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.284 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.284 INFO:teuthology.orchestra.run.smithi006.stdout: "autotune_memory_target_ratio": { 2024-05-09T04:58:26.284 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T04:58:26.284 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-05-09T04:58:26.284 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.284 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.284 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0.7", 2024-05-09T04:58:26.284 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.284 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.284 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.285 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T04:58:26.285 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.285 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.285 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.285 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.285 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm_log_destination": { 2024-05-09T04:58:26.285 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cephadm_log_destination", 2024-05-09T04:58:26.285 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.285 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.285 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.285 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.286 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.286 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.286 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.286 INFO:teuthology.orchestra.run.smithi006.stdout: "file", 2024-05-09T04:58:26.286 INFO:teuthology.orchestra.run.smithi006.stdout: "file,syslog", 2024-05-09T04:58:26.286 INFO:teuthology.orchestra.run.smithi006.stdout: "syslog" 2024-05-09T04:58:26.286 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.286 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-09T04:58:26.286 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.286 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.286 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.286 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.287 INFO:teuthology.orchestra.run.smithi006.stdout: "cgroups_split": { 2024-05-09T04:58:26.287 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cgroups_split", 2024-05-09T04:58:26.287 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.287 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.287 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.287 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.287 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.287 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.287 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.287 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T04:58:26.287 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.288 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.288 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.288 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.288 INFO:teuthology.orchestra.run.smithi006.stdout: "config_checks_enabled": { 2024-05-09T04:58:26.288 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_checks_enabled", 2024-05-09T04:58:26.288 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.288 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.288 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.288 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.288 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.288 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.288 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.288 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T04:58:26.289 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.289 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.289 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.289 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.289 INFO:teuthology.orchestra.run.smithi006.stdout: "config_dashboard": { 2024-05-09T04:58:26.289 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_dashboard", 2024-05-09T04:58:26.289 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.289 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.289 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.289 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.289 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.289 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.289 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.290 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T04:58:26.290 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.290 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.290 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.290 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.290 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_alertmanager": { 2024-05-09T04:58:26.290 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_alertmanager", 2024-05-09T04:58:26.290 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.290 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.290 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.290 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T04:58:26.290 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.291 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.291 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.291 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-05-09T04:58:26.291 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.291 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.291 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.291 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.291 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_base": { 2024-05-09T04:58:26.291 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_base", 2024-05-09T04:58:26.291 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.292 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.292 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.292 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T04:58:26.292 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.292 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.292 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.292 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Container image name, without the tag", 2024-05-09T04:58:26.292 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.292 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.292 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.292 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.292 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_elasticsearch": { 2024-05-09T04:58:26.293 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_elasticsearch", 2024-05-09T04:58:26.293 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.293 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.293 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.293 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T04:58:26.293 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.293 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.293 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.293 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "elasticsearch container image", 2024-05-09T04:58:26.293 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.293 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.293 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.294 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.294 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_grafana": { 2024-05-09T04:58:26.294 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_grafana", 2024-05-09T04:58:26.294 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.294 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.294 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.294 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-09T04:58:26.294 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.294 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.294 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.294 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-05-09T04:58:26.295 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.295 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.295 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.295 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.295 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_haproxy": { 2024-05-09T04:58:26.295 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_haproxy", 2024-05-09T04:58:26.295 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.295 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.295 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.295 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T04:58:26.295 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.295 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.295 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.296 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "HAproxy container image", 2024-05-09T04:58:26.296 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.296 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.296 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.296 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.296 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_agent": { 2024-05-09T04:58:26.296 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_agent", 2024-05-09T04:58:26.296 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.296 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.296 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.296 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T04:58:26.296 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.297 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.297 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.297 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger agent container image", 2024-05-09T04:58:26.297 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.297 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.297 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.297 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.297 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_collector": { 2024-05-09T04:58:26.297 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_collector", 2024-05-09T04:58:26.297 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.297 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.298 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.298 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T04:58:26.298 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.298 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.298 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.298 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger collector container image", 2024-05-09T04:58:26.298 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.298 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.298 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.298 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.298 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_query": { 2024-05-09T04:58:26.298 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_query", 2024-05-09T04:58:26.299 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.299 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.299 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.299 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T04:58:26.299 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.299 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.299 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.299 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger query container image", 2024-05-09T04:58:26.299 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.299 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.299 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.299 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.299 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_keepalived": { 2024-05-09T04:58:26.300 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_keepalived", 2024-05-09T04:58:26.300 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.300 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.300 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.300 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T04:58:26.300 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.300 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.300 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.300 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Keepalived container image", 2024-05-09T04:58:26.300 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.300 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.300 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.300 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.301 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_loki": { 2024-05-09T04:58:26.301 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_loki", 2024-05-09T04:58:26.301 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.301 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.301 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.301 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T04:58:26.301 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.301 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.301 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.301 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Loki container image", 2024-05-09T04:58:26.301 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.301 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.302 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.302 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.302 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_node_exporter": { 2024-05-09T04:58:26.302 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_node_exporter", 2024-05-09T04:58:26.302 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.302 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.302 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.302 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T04:58:26.302 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.302 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.302 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.303 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-05-09T04:58:26.303 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.303 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.303 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.303 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.303 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_nvmeof": { 2024-05-09T04:58:26.303 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_nvmeof", 2024-05-09T04:58:26.303 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.303 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.303 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.303 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-09T04:58:26.303 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.303 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.304 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.304 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Nvme-of container image", 2024-05-09T04:58:26.304 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.304 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.304 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.304 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.304 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_prometheus": { 2024-05-09T04:58:26.304 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_prometheus", 2024-05-09T04:58:26.304 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.304 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.304 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.304 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T04:58:26.304 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.305 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.305 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.305 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2024-05-09T04:58:26.305 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.305 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.305 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.305 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.305 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_promtail": { 2024-05-09T04:58:26.305 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_promtail", 2024-05-09T04:58:26.305 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.305 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.305 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.305 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T04:58:26.306 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.306 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.306 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.306 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Promtail container image", 2024-05-09T04:58:26.306 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.306 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.306 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.306 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.306 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_samba": { 2024-05-09T04:58:26.306 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_samba", 2024-05-09T04:58:26.306 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.306 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.306 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.307 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-09T04:58:26.307 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.307 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.307 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.307 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Samba/SMB container image", 2024-05-09T04:58:26.307 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.307 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.307 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.307 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.307 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_snmp_gateway": { 2024-05-09T04:58:26.307 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_snmp_gateway", 2024-05-09T04:58:26.307 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.307 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.308 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.308 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T04:58:26.308 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.308 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.308 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.308 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SNMP Gateway container image", 2024-05-09T04:58:26.308 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.308 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.308 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.308 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.308 INFO:teuthology.orchestra.run.smithi006.stdout: "container_init": { 2024-05-09T04:58:26.308 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_init", 2024-05-09T04:58:26.309 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.309 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.309 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.309 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.309 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.309 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.309 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.309 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T04:58:26.309 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.309 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.309 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.309 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.309 INFO:teuthology.orchestra.run.smithi006.stdout: "daemon_cache_timeout": { 2024-05-09T04:58:26.310 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "daemon_cache_timeout", 2024-05-09T04:58:26.310 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:26.310 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.310 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.310 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-05-09T04:58:26.310 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.310 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.310 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.310 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T04:58:26.310 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.310 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.311 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.311 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.311 INFO:teuthology.orchestra.run.smithi006.stdout: "default_cephadm_command_timeout": { 2024-05-09T04:58:26.311 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T04:58:26.311 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.311 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.311 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.311 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "900", 2024-05-09T04:58:26.311 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.311 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.311 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.312 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T04:58:26.312 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.312 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.312 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.312 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.312 INFO:teuthology.orchestra.run.smithi006.stdout: "default_registry": { 2024-05-09T04:58:26.312 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "default_registry", 2024-05-09T04:58:26.312 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.312 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.312 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.312 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io", 2024-05-09T04:58:26.312 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.313 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.313 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.313 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T04:58:26.313 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.313 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.313 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.313 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.313 INFO:teuthology.orchestra.run.smithi006.stdout: "device_cache_timeout": { 2024-05-09T04:58:26.313 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_cache_timeout", 2024-05-09T04:58:26.313 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:26.313 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.313 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.314 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1800", 2024-05-09T04:58:26.314 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.314 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.314 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.314 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache device inventory", 2024-05-09T04:58:26.314 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.314 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.314 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.314 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.314 INFO:teuthology.orchestra.run.smithi006.stdout: "device_enhanced_scan": { 2024-05-09T04:58:26.314 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_enhanced_scan", 2024-05-09T04:58:26.314 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.315 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.315 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.315 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.315 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.315 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.315 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.315 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T04:58:26.315 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.315 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.315 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.315 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.315 INFO:teuthology.orchestra.run.smithi006.stdout: "facts_cache_timeout": { 2024-05-09T04:58:26.315 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "facts_cache_timeout", 2024-05-09T04:58:26.316 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:26.316 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.316 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.316 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-05-09T04:58:26.316 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.316 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.316 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.316 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache host facts data", 2024-05-09T04:58:26.316 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.316 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.316 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.316 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.317 INFO:teuthology.orchestra.run.smithi006.stdout: "grafana_dashboards_path": { 2024-05-09T04:58:26.317 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "grafana_dashboards_path", 2024-05-09T04:58:26.317 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.317 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.317 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.317 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-09T04:58:26.317 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.317 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.317 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.317 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-09T04:58:26.317 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.317 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.318 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.318 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.318 INFO:teuthology.orchestra.run.smithi006.stdout: "host_check_interval": { 2024-05-09T04:58:26.318 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "host_check_interval", 2024-05-09T04:58:26.318 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:26.318 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.318 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.318 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-05-09T04:58:26.318 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.318 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.318 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.318 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to perform a host check", 2024-05-09T04:58:26.319 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.319 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.319 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.319 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.319 INFO:teuthology.orchestra.run.smithi006.stdout: "hw_monitoring": { 2024-05-09T04:58:26.319 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "hw_monitoring", 2024-05-09T04:58:26.319 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.319 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.319 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.319 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.319 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.319 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.320 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.320 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T04:58:26.320 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.320 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.320 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.320 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.320 INFO:teuthology.orchestra.run.smithi006.stdout: "inventory_list_all": { 2024-05-09T04:58:26.320 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "inventory_list_all", 2024-05-09T04:58:26.320 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.320 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.320 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.321 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.321 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.321 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.321 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.321 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T04:58:26.321 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.321 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.321 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.321 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.321 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.321 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.322 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.322 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.322 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.322 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.322 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.322 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.322 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.322 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.322 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.322 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.322 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.323 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.323 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.323 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.323 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.323 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.323 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.323 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.323 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.323 INFO:teuthology.orchestra.run.smithi006.stdout: "log_refresh_metadata": { 2024-05-09T04:58:26.323 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_refresh_metadata", 2024-05-09T04:58:26.323 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.323 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.324 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.324 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.324 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.324 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.324 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.324 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T04:58:26.324 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.324 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.324 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.324 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.324 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.324 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.324 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.325 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.325 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.325 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.325 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.325 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.325 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.325 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T04:58:26.325 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.325 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.325 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.325 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.325 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.325 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.326 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.326 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.326 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.326 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.326 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.326 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.326 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.326 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.326 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.326 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.326 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.326 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.327 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.327 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.327 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.327 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.327 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.327 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.327 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.327 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.327 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.327 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.327 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.327 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.328 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.328 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.328 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.328 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.328 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.328 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.328 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.328 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.328 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.328 INFO:teuthology.orchestra.run.smithi006.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T04:58:26.328 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T04:58:26.328 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.329 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.329 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.329 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.329 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.329 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.329 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.329 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T04:58:26.329 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.329 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.329 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.329 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.329 INFO:teuthology.orchestra.run.smithi006.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T04:58:26.330 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T04:58:26.330 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.330 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.330 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.330 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "*", 2024-05-09T04:58:26.330 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.330 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.330 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.330 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T04:58:26.330 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.330 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.330 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.331 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.331 INFO:teuthology.orchestra.run.smithi006.stdout: "max_count_per_host": { 2024-05-09T04:58:26.331 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_count_per_host", 2024-05-09T04:58:26.331 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.331 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.331 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.331 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-05-09T04:58:26.331 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.331 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.331 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.331 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "max number of daemons per service per host", 2024-05-09T04:58:26.332 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.332 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.332 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.332 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.332 INFO:teuthology.orchestra.run.smithi006.stdout: "max_osd_draining_count": { 2024-05-09T04:58:26.332 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_osd_draining_count", 2024-05-09T04:58:26.332 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.332 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.332 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.333 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-05-09T04:58:26.333 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.333 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.333 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.333 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T04:58:26.333 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.333 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.333 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.333 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.333 INFO:teuthology.orchestra.run.smithi006.stdout: "migration_current": { 2024-05-09T04:58:26.333 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "migration_current", 2024-05-09T04:58:26.334 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.334 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.334 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.334 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.334 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.334 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.334 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.334 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "internal - do not modify", 2024-05-09T04:58:26.334 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.334 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.334 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.334 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.334 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": { 2024-05-09T04:58:26.335 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mode", 2024-05-09T04:58:26.335 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.335 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.335 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.335 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "root", 2024-05-09T04:58:26.335 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.335 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.335 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.335 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm-package", 2024-05-09T04:58:26.335 INFO:teuthology.orchestra.run.smithi006.stdout: "root" 2024-05-09T04:58:26.335 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.335 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T04:58:26.335 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.336 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.336 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.336 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.336 INFO:teuthology.orchestra.run.smithi006.stdout: "oob_default_addr": { 2024-05-09T04:58:26.336 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "oob_default_addr", 2024-05-09T04:58:26.336 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.336 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.336 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.336 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "169.254.1.1", 2024-05-09T04:58:26.336 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.336 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.336 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.337 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T04:58:26.337 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.337 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.337 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.337 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.337 INFO:teuthology.orchestra.run.smithi006.stdout: "prometheus_alerts_path": { 2024-05-09T04:58:26.337 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prometheus_alerts_path", 2024-05-09T04:58:26.337 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.337 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.337 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.337 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T04:58:26.337 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.337 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.338 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.338 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T04:58:26.338 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.338 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.338 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.338 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.338 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_insecure": { 2024-05-09T04:58:26.338 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_insecure", 2024-05-09T04:58:26.338 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.338 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.338 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.338 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.339 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.339 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.339 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.342 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T04:58:26.342 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.342 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.342 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.343 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.343 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_password": { 2024-05-09T04:58:26.343 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_password", 2024-05-09T04:58:26.343 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.343 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.343 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.343 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.343 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.343 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.343 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.343 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T04:58:26.343 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.343 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.344 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.344 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.344 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_url": { 2024-05-09T04:58:26.344 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_url", 2024-05-09T04:58:26.344 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.344 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.344 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.344 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.344 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.344 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.344 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.344 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T04:58:26.344 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.344 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.345 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.345 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.345 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_username": { 2024-05-09T04:58:26.345 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_username", 2024-05-09T04:58:26.345 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.345 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.345 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.345 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.345 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.345 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.345 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.345 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T04:58:26.346 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.346 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.346 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.346 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.346 INFO:teuthology.orchestra.run.smithi006.stdout: "secure_monitoring_stack": { 2024-05-09T04:58:26.346 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "secure_monitoring_stack", 2024-05-09T04:58:26.346 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.346 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.346 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.346 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.346 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.346 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.346 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.347 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T04:58:26.347 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.347 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.347 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.347 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.347 INFO:teuthology.orchestra.run.smithi006.stdout: "service_discovery_port": { 2024-05-09T04:58:26.347 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "service_discovery_port", 2024-05-09T04:58:26.347 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.347 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.347 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.347 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8765", 2024-05-09T04:58:26.347 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.348 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.348 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.348 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "cephadm service discovery port", 2024-05-09T04:58:26.348 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.348 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.348 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.348 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.348 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.348 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.348 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.348 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.349 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.349 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.349 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.349 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.349 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.349 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.349 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.349 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.349 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.349 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.349 INFO:teuthology.orchestra.run.smithi006.stdout: "ssh_config_file": { 2024-05-09T04:58:26.349 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssh_config_file", 2024-05-09T04:58:26.350 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.350 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.350 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.350 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.350 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.350 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.350 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.350 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T04:58:26.350 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.350 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.350 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.351 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.351 INFO:teuthology.orchestra.run.smithi006.stdout: "use_agent": { 2024-05-09T04:58:26.351 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "use_agent", 2024-05-09T04:58:26.351 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.351 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.351 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.351 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.351 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.351 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.351 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.351 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T04:58:26.352 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.352 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.352 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.352 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.352 INFO:teuthology.orchestra.run.smithi006.stdout: "use_repo_digest": { 2024-05-09T04:58:26.352 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "use_repo_digest", 2024-05-09T04:58:26.352 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.352 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.352 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.352 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.352 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.352 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.352 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.352 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T04:58:26.353 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.353 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.353 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.353 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.353 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_failed_host_check": { 2024-05-09T04:58:26.353 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_failed_host_check", 2024-05-09T04:58:26.353 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.353 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.353 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.353 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.353 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.353 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.354 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.354 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T04:58:26.354 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.354 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.354 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.354 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.354 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_stray_daemons": { 2024-05-09T04:58:26.354 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_stray_daemons", 2024-05-09T04:58:26.354 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.354 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.354 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.354 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.354 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.355 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.355 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.355 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T04:58:26.355 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.355 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.355 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.355 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.355 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_stray_hosts": { 2024-05-09T04:58:26.355 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_stray_hosts", 2024-05-09T04:58:26.355 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.355 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.355 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.355 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.355 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.356 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.356 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.356 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T04:58:26.356 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.356 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.356 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.356 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.356 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.356 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.356 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.356 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crash", 2024-05-09T04:58:26.356 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.357 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.357 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.357 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.357 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.357 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.357 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.357 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.357 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.357 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.357 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.357 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.357 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.357 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.357 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.358 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.358 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.358 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.358 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.358 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.358 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.358 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.358 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.358 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.358 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.358 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.358 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.359 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.359 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.359 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.359 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.359 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.359 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.359 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.359 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.359 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.359 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.359 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.360 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.360 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.360 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.360 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.360 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.360 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.360 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.360 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.360 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.360 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.360 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.360 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.361 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.361 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.361 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.361 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.361 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.361 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.361 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.361 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.361 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.361 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.361 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.362 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.362 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.362 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.362 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.362 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.362 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.362 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.362 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.362 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.362 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.362 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.362 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.363 INFO:teuthology.orchestra.run.smithi006.stdout: "retain_interval": { 2024-05-09T04:58:26.363 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "retain_interval", 2024-05-09T04:58:26.363 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:26.363 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.363 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.363 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "31536000", 2024-05-09T04:58:26.363 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.363 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.363 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.363 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T04:58:26.363 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.363 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.364 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.364 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.364 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.364 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.364 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.364 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.364 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.364 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.364 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.364 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.364 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.364 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.365 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.365 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.365 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.365 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.365 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_recent_interval": { 2024-05-09T04:58:26.365 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_recent_interval", 2024-05-09T04:58:26.365 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:26.365 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.365 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.365 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1209600", 2024-05-09T04:58:26.365 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.365 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.365 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.366 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T04:58:26.366 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.366 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.366 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.366 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.366 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.366 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.366 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.366 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "dashboard", 2024-05-09T04:58:26.366 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.366 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.366 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.367 INFO:teuthology.orchestra.run.smithi006.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T04:58:26.367 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T04:58:26.367 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.367 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.367 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.367 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-05-09T04:58:26.367 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.367 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.367 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.367 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.367 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.367 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.367 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.368 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.368 INFO:teuthology.orchestra.run.smithi006.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T04:58:26.368 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T04:58:26.368 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.368 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.368 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.368 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.368 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.368 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.368 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.368 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.368 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.369 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.369 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.369 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.369 INFO:teuthology.orchestra.run.smithi006.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T04:58:26.369 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T04:58:26.369 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.369 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.369 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.369 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.370 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.370 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.370 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.370 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.370 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.370 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.370 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.370 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.370 INFO:teuthology.orchestra.run.smithi006.stdout: "AUDIT_API_ENABLED": { 2024-05-09T04:58:26.371 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T04:58:26.371 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.371 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.371 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.371 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.371 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.371 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.371 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.371 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.371 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.372 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.372 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.372 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.372 INFO:teuthology.orchestra.run.smithi006.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T04:58:26.372 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T04:58:26.372 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.372 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.372 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.372 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.372 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.372 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.373 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.373 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.373 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.373 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.373 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.373 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.373 INFO:teuthology.orchestra.run.smithi006.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T04:58:26.373 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T04:58:26.373 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.373 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.373 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.373 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.373 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.374 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.374 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.374 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.374 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.374 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.374 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.374 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.374 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T04:58:26.374 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T04:58:26.374 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.374 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.374 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.375 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.375 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.375 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.375 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.375 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.375 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.375 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.375 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.375 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.375 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T04:58:26.375 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T04:58:26.375 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.375 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.376 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.376 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.376 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.376 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.376 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.376 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.376 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.376 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.376 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.376 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.376 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T04:58:26.376 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T04:58:26.376 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.377 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.377 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.377 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.377 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.377 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.377 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.377 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.377 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.377 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.377 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.377 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.377 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T04:58:26.377 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T04:58:26.377 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.378 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.378 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.378 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.378 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.378 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.378 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.378 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.378 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.378 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.378 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.378 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.378 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T04:58:26.379 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T04:58:26.379 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.379 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.379 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.379 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.379 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.379 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.379 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.379 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.379 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.379 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.379 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.379 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.379 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T04:58:26.380 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T04:58:26.380 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.380 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.380 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.380 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.380 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.380 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.380 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.380 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.380 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.380 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.380 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.380 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.381 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T04:58:26.381 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T04:58:26.381 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.381 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.381 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.381 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.381 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.381 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.381 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.381 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.382 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.382 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.382 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.382 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.382 INFO:teuthology.orchestra.run.smithi006.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T04:58:26.382 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T04:58:26.382 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.382 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.382 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.382 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.382 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.383 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.383 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.383 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.383 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.383 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.383 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.383 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.383 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T04:58:26.383 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T04:58:26.383 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.383 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.383 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.384 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2024-05-09T04:58:26.384 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.384 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.384 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.384 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.384 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.384 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.384 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.384 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.384 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T04:58:26.384 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T04:58:26.384 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.384 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.385 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.385 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.385 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.385 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.385 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.385 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.385 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.385 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.385 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.385 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.385 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_URL": { 2024-05-09T04:58:26.385 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_URL", 2024-05-09T04:58:26.385 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.386 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.386 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.386 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.386 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.386 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.386 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.386 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.386 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.386 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.386 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.386 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.386 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T04:58:26.386 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T04:58:26.386 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.387 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.387 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.387 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2024-05-09T04:58:26.387 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.387 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.387 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.387 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.387 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.387 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.387 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.387 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.387 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T04:58:26.387 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T04:58:26.387 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.388 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.388 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.388 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.388 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.388 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.388 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.388 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.388 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.388 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.388 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.388 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.388 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T04:58:26.388 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T04:58:26.388 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.389 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.389 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.389 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.389 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.389 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.389 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.389 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.389 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.389 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.389 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.389 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.389 INFO:teuthology.orchestra.run.smithi006.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T04:58:26.389 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T04:58:26.389 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.390 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.390 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.390 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.390 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.390 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.390 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.390 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.390 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.390 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.390 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.390 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.391 INFO:teuthology.orchestra.run.smithi006.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T04:58:26.391 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T04:58:26.391 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.391 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.391 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.391 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.391 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.391 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.391 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.391 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.391 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.391 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.391 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.391 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.392 INFO:teuthology.orchestra.run.smithi006.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-09T04:58:26.392 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-09T04:58:26.392 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.392 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.392 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.392 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "[]", 2024-05-09T04:58:26.392 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.392 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.392 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.392 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.392 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.392 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.392 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.392 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.393 INFO:teuthology.orchestra.run.smithi006.stdout: "MULTICLUSTER_CONFIG": { 2024-05-09T04:58:26.393 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-09T04:58:26.393 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.393 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.393 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.393 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "{}", 2024-05-09T04:58:26.393 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.393 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.393 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.393 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.393 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.393 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.394 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.394 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.394 INFO:teuthology.orchestra.run.smithi006.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T04:58:26.394 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T04:58:26.394 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.394 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.394 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.394 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.394 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.394 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.394 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.394 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.394 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.395 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.395 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.395 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.395 INFO:teuthology.orchestra.run.smithi006.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T04:58:26.395 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T04:58:26.395 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.395 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.395 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.395 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.395 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.395 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.395 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.396 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.396 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.396 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.396 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.396 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.396 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T04:58:26.396 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T04:58:26.396 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.396 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.396 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.396 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.396 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.396 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.397 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.397 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.397 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.397 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.397 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.397 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.397 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T04:58:26.397 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T04:58:26.397 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.397 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.397 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.397 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.397 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.398 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.398 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.398 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.398 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.398 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.398 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.398 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.398 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T04:58:26.398 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T04:58:26.398 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.398 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.398 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.399 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.399 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.399 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.399 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.399 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.399 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.399 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.399 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.399 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.399 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T04:58:26.399 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T04:58:26.399 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.399 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.400 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.400 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.400 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.400 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.400 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.400 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.400 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.400 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.400 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.400 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.400 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T04:58:26.400 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T04:58:26.400 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.401 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.401 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.401 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.401 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.401 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.401 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.401 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.401 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.401 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.401 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.401 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.401 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T04:58:26.402 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T04:58:26.402 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.402 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.402 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.402 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.402 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.402 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.402 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.402 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.402 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.402 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.402 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.403 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.403 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T04:58:26.403 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T04:58:26.403 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.403 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.403 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.403 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.403 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.403 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.403 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.403 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.403 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.403 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.404 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.404 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.404 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T04:58:26.404 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T04:58:26.404 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.404 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.404 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.404 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.404 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.404 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.404 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.404 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.405 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.405 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.405 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.405 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.405 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T04:58:26.405 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T04:58:26.405 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.405 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.405 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.405 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T04:58:26.405 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.405 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.406 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.406 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.406 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.406 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.406 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.406 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.406 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T04:58:26.406 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T04:58:26.406 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.406 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.406 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.406 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-05-09T04:58:26.406 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.407 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.407 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.407 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.407 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.407 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.407 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.407 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.407 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T04:58:26.407 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T04:58:26.407 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.407 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.408 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.408 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8", 2024-05-09T04:58:26.408 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.408 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.408 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.408 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.408 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.408 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.408 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.408 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.408 INFO:teuthology.orchestra.run.smithi006.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T04:58:26.408 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T04:58:26.408 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.409 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.409 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.409 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "45", 2024-05-09T04:58:26.409 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.409 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.409 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.409 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.409 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.409 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.409 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.409 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.409 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T04:58:26.409 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T04:58:26.409 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.410 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.410 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.410 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.410 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.410 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.410 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.410 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.410 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.410 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.410 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.410 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.410 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T04:58:26.410 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T04:58:26.411 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.411 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.411 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.411 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2024-05-09T04:58:26.411 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.411 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.411 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.411 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.411 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.411 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.411 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.412 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.412 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T04:58:26.412 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T04:58:26.412 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.412 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.412 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.412 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.412 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.412 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.412 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.412 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.412 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.412 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.413 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.413 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.413 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T04:58:26.413 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T04:58:26.413 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.413 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.413 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.413 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.413 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.413 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.413 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.413 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.414 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.414 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.414 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.414 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.414 INFO:teuthology.orchestra.run.smithi006.stdout: "UNSAFE_TLS_v1_2": { 2024-05-09T04:58:26.414 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-09T04:58:26.414 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.414 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.414 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.414 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.414 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.414 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.414 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.415 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.415 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.415 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.415 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.415 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.415 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T04:58:26.415 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T04:58:26.415 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.415 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.415 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.415 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.415 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.415 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.416 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.416 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.416 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.416 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.416 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.416 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.416 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T04:58:26.416 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T04:58:26.416 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.416 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.416 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.416 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-05-09T04:58:26.416 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.417 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.417 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.417 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.417 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.417 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.417 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.417 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.417 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T04:58:26.417 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T04:58:26.417 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.417 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.417 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.417 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-05-09T04:58:26.418 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.418 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.418 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.418 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.418 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.418 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.418 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.418 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.418 INFO:teuthology.orchestra.run.smithi006.stdout: "cross_origin_url": { 2024-05-09T04:58:26.418 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cross_origin_url", 2024-05-09T04:58:26.418 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.418 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.418 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.419 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.419 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.419 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.419 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.419 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.419 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.419 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.419 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.419 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.419 INFO:teuthology.orchestra.run.smithi006.stdout: "crt_file": { 2024-05-09T04:58:26.419 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crt_file", 2024-05-09T04:58:26.420 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.420 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.420 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.420 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.420 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.420 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.420 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.420 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.420 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.420 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.420 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.420 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.420 INFO:teuthology.orchestra.run.smithi006.stdout: "debug": { 2024-05-09T04:58:26.420 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "debug", 2024-05-09T04:58:26.421 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.421 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.421 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.421 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.421 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.421 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.421 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.421 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable/disable debug options", 2024-05-09T04:58:26.421 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.421 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.421 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.421 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.422 INFO:teuthology.orchestra.run.smithi006.stdout: "jwt_token_ttl": { 2024-05-09T04:58:26.422 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "jwt_token_ttl", 2024-05-09T04:58:26.422 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.422 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.422 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.422 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "28800", 2024-05-09T04:58:26.422 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.422 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.422 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.422 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.422 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.422 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.423 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.423 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.423 INFO:teuthology.orchestra.run.smithi006.stdout: "key_file": { 2024-05-09T04:58:26.423 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "key_file", 2024-05-09T04:58:26.423 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.423 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.423 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.423 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.423 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.423 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.423 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.423 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.423 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.423 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.424 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.424 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.424 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.424 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.424 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.424 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.424 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.424 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.424 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.424 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.424 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.424 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.424 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.425 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.425 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.425 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.425 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.425 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.425 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.425 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.425 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.425 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.425 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.425 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.425 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.425 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.426 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.426 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.426 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.426 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.426 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.426 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.426 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.426 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.426 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.426 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.426 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.426 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.426 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.427 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.427 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.427 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.427 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.427 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.427 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.427 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.427 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.427 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.427 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.427 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.427 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.428 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.428 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.428 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.428 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.428 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.428 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.428 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.428 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.428 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.428 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.428 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.428 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.429 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.429 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.429 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.429 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.429 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.429 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.429 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.429 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.429 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.429 INFO:teuthology.orchestra.run.smithi006.stdout: "motd": { 2024-05-09T04:58:26.429 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "motd", 2024-05-09T04:58:26.429 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.429 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.430 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.430 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.430 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.430 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.430 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.430 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "The message of the day", 2024-05-09T04:58:26.430 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.430 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.430 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.430 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.431 INFO:teuthology.orchestra.run.smithi006.stdout: "redirect_resolve_ip_addr": { 2024-05-09T04:58:26.431 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T04:58:26.431 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.431 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.431 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.431 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.431 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.431 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.431 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.431 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.431 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.431 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.432 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.432 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.432 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2024-05-09T04:58:26.432 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2024-05-09T04:58:26.432 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.432 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.432 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.432 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "::", 2024-05-09T04:58:26.432 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.432 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.432 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.432 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.433 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.433 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.433 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.433 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.433 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2024-05-09T04:58:26.433 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2024-05-09T04:58:26.433 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.433 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.433 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.433 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8080", 2024-05-09T04:58:26.433 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.433 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.434 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.434 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.434 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.434 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.434 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.434 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.434 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.434 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.434 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.434 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.434 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.434 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.435 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.435 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.435 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.435 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.435 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.435 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.435 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.435 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.435 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl": { 2024-05-09T04:58:26.435 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl", 2024-05-09T04:58:26.435 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.435 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.435 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.436 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.436 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.436 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.436 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.436 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.436 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.436 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.436 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.436 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.436 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl_server_port": { 2024-05-09T04:58:26.436 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl_server_port", 2024-05-09T04:58:26.436 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.437 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.437 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.437 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8443", 2024-05-09T04:58:26.437 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.437 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.437 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.437 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.437 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.437 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.437 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.437 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.437 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_behaviour": { 2024-05-09T04:58:26.438 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_behaviour", 2024-05-09T04:58:26.438 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.438 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.438 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.438 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "redirect", 2024-05-09T04:58:26.438 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.438 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.438 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.438 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.438 INFO:teuthology.orchestra.run.smithi006.stdout: "redirect" 2024-05-09T04:58:26.438 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.438 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.439 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.439 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.439 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.439 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.439 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_error_status_code": { 2024-05-09T04:58:26.439 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_error_status_code", 2024-05-09T04:58:26.439 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.439 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.439 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.439 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "500", 2024-05-09T04:58:26.439 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "400", 2024-05-09T04:58:26.439 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "599", 2024-05-09T04:58:26.440 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.440 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.440 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.440 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.440 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.440 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.440 INFO:teuthology.orchestra.run.smithi006.stdout: "url_prefix": { 2024-05-09T04:58:26.440 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "url_prefix", 2024-05-09T04:58:26.440 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.440 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.440 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.440 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.441 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.441 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.441 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.441 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.441 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.441 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.441 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.441 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.441 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.441 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.441 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.442 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "devicehealth", 2024-05-09T04:58:26.442 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.442 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.442 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.442 INFO:teuthology.orchestra.run.smithi006.stdout: "enable_monitoring": { 2024-05-09T04:58:26.442 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enable_monitoring", 2024-05-09T04:58:26.442 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.442 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.442 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.442 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.443 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.443 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.443 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.443 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "monitor device health metrics", 2024-05-09T04:58:26.443 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.443 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.443 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.443 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.443 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.443 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.443 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.443 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.444 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.444 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.444 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.444 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.444 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.444 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.444 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.444 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.444 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.444 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.444 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.444 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.444 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.444 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.445 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.445 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.445 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.445 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.445 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.445 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.445 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.445 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.445 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.445 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.445 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.445 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.446 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.446 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.446 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.446 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.446 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.446 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.446 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.446 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.446 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.446 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.446 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.447 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.447 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.447 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.447 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.447 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.447 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.447 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.447 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.447 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.447 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.447 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.447 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.448 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.448 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.448 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.448 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.448 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.448 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.448 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.448 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.448 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.448 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.449 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.449 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.449 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.449 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.449 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.449 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.449 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.449 INFO:teuthology.orchestra.run.smithi006.stdout: "mark_out_threshold": { 2024-05-09T04:58:26.449 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mark_out_threshold", 2024-05-09T04:58:26.449 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:26.449 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.449 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.450 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "2419200", 2024-05-09T04:58:26.450 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.450 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.450 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.450 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T04:58:26.450 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.450 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.450 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.450 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.450 INFO:teuthology.orchestra.run.smithi006.stdout: "pool_name": { 2024-05-09T04:58:26.450 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pool_name", 2024-05-09T04:58:26.451 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.451 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.451 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.451 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "device_health_metrics", 2024-05-09T04:58:26.451 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.451 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.451 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.451 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T04:58:26.451 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.451 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.452 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.452 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.452 INFO:teuthology.orchestra.run.smithi006.stdout: "retention_period": { 2024-05-09T04:58:26.452 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "retention_period", 2024-05-09T04:58:26.452 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:26.452 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.452 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.452 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15552000", 2024-05-09T04:58:26.452 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.452 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.452 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.452 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long to retain device health metrics", 2024-05-09T04:58:26.453 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.453 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.453 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.453 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.453 INFO:teuthology.orchestra.run.smithi006.stdout: "scrape_frequency": { 2024-05-09T04:58:26.453 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "scrape_frequency", 2024-05-09T04:58:26.453 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:26.453 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.453 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.453 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "86400", 2024-05-09T04:58:26.453 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.453 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.453 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.453 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T04:58:26.454 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.454 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.454 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.454 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.454 INFO:teuthology.orchestra.run.smithi006.stdout: "self_heal": { 2024-05-09T04:58:26.454 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "self_heal", 2024-05-09T04:58:26.454 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.454 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.454 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.454 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.454 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.454 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.454 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.455 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T04:58:26.455 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.455 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.455 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.455 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.455 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-05-09T04:58:26.455 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-05-09T04:58:26.455 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:26.455 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.455 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.455 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-05-09T04:58:26.455 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.456 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.456 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.456 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T04:58:26.456 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.456 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.456 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.456 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.456 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.456 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.456 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.456 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.456 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.456 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.457 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.457 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.457 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.457 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.457 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.457 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.457 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.457 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.457 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_threshold": { 2024-05-09T04:58:26.457 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_threshold", 2024-05-09T04:58:26.457 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:26.457 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.458 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.458 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "7257600", 2024-05-09T04:58:26.458 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.458 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.458 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.458 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T04:58:26.458 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.458 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.458 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.458 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.458 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.458 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.458 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.459 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "diskprediction_local", 2024-05-09T04:58:26.459 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.459 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.459 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.459 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.459 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.459 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.459 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.459 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.459 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.459 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.459 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.460 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.460 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.460 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.460 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.460 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.460 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.460 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.460 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.460 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.460 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.460 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.460 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.460 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.461 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.461 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.461 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.461 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.461 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.461 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.461 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.461 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.461 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.461 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.461 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.461 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.462 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.462 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.462 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.462 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.462 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.462 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.462 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.462 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.462 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.462 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.462 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.462 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.463 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.463 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.463 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.463 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.463 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.463 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.463 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.463 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.463 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.463 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.463 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.463 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.463 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.463 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.464 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.464 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.464 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.464 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.464 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.464 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.464 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.464 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.464 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.464 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.464 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.465 INFO:teuthology.orchestra.run.smithi006.stdout: "predict_interval": { 2024-05-09T04:58:26.465 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "predict_interval", 2024-05-09T04:58:26.465 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.465 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.465 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.465 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "86400", 2024-05-09T04:58:26.465 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.465 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.465 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.465 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.465 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.465 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.466 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.466 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.466 INFO:teuthology.orchestra.run.smithi006.stdout: "predictor_model": { 2024-05-09T04:58:26.466 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "predictor_model", 2024-05-09T04:58:26.466 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.466 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.466 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.466 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "prophetstor", 2024-05-09T04:58:26.466 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.466 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.466 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.466 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.467 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.467 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.467 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.467 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.467 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-05-09T04:58:26.467 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-05-09T04:58:26.467 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.467 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.467 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.467 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2024-05-09T04:58:26.467 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.467 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.467 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.468 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.468 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.468 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.468 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.468 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.468 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.468 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.468 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.468 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.468 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.468 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.468 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.469 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.469 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.469 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.469 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.469 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.469 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.469 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.469 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.469 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.469 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.469 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "influx", 2024-05-09T04:58:26.469 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": false, 2024-05-09T04:58:26.469 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "influxdb python module not found", 2024-05-09T04:58:26.470 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.470 INFO:teuthology.orchestra.run.smithi006.stdout: "batch_size": { 2024-05-09T04:58:26.470 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "batch_size", 2024-05-09T04:58:26.470 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.470 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.470 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.470 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5000", 2024-05-09T04:58:26.470 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.470 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.470 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.470 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T04:58:26.470 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.471 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.471 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.471 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.471 INFO:teuthology.orchestra.run.smithi006.stdout: "database": { 2024-05-09T04:58:26.471 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "database", 2024-05-09T04:58:26.471 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.471 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.471 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.471 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "ceph", 2024-05-09T04:58:26.471 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.471 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.471 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.472 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T04:58:26.472 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.472 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.472 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.472 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.472 INFO:teuthology.orchestra.run.smithi006.stdout: "hostname": { 2024-05-09T04:58:26.472 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "hostname", 2024-05-09T04:58:26.472 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.472 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.472 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.472 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.472 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.473 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.473 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.473 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB server hostname", 2024-05-09T04:58:26.473 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.473 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.473 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.473 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.473 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-05-09T04:58:26.473 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-05-09T04:58:26.473 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:26.473 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.473 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.474 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "30", 2024-05-09T04:58:26.474 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "5", 2024-05-09T04:58:26.474 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.474 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.474 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T04:58:26.474 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.474 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.474 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.474 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.474 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.474 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.474 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.475 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.475 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.475 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.475 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.475 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.475 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.475 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.475 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.475 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.475 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.475 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.475 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.476 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.476 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.476 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.476 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.476 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.476 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.476 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.476 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.476 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.476 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.476 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.476 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.477 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.477 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.477 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.477 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.477 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.477 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.477 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.477 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.477 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.477 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.477 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.477 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.478 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.478 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.478 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.478 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.478 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.478 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.478 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.478 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.478 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.478 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.478 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.478 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.478 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.478 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.479 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.479 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.479 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.479 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.479 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.479 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.479 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.479 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.479 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.479 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.479 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.479 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.479 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.480 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.480 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.480 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.480 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.480 INFO:teuthology.orchestra.run.smithi006.stdout: "password": { 2024-05-09T04:58:26.480 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "password", 2024-05-09T04:58:26.480 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.480 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.480 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.480 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.480 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.480 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.480 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.481 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "password of InfluxDB server user", 2024-05-09T04:58:26.481 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.481 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.481 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.481 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.481 INFO:teuthology.orchestra.run.smithi006.stdout: "port": { 2024-05-09T04:58:26.481 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "port", 2024-05-09T04:58:26.481 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.481 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.481 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.481 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8086", 2024-05-09T04:58:26.481 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.481 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.482 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.482 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB server port", 2024-05-09T04:58:26.482 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.482 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.482 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.482 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.482 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.482 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.482 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.482 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.482 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.482 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.482 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.483 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.483 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.483 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.483 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.483 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.483 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.483 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.483 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl": { 2024-05-09T04:58:26.483 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl", 2024-05-09T04:58:26.483 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.483 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.483 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.483 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "false", 2024-05-09T04:58:26.483 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.484 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.484 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.484 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T04:58:26.484 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.484 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.484 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.484 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.484 INFO:teuthology.orchestra.run.smithi006.stdout: "threads": { 2024-05-09T04:58:26.484 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "threads", 2024-05-09T04:58:26.484 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.484 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.485 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.485 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-05-09T04:58:26.485 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-05-09T04:58:26.485 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "32", 2024-05-09T04:58:26.485 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.485 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T04:58:26.485 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.485 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.485 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.485 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.485 INFO:teuthology.orchestra.run.smithi006.stdout: "username": { 2024-05-09T04:58:26.486 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "username", 2024-05-09T04:58:26.486 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.486 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.486 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.486 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.486 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.486 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.486 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.486 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "username of InfluxDB server user", 2024-05-09T04:58:26.486 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.486 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.487 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.487 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.487 INFO:teuthology.orchestra.run.smithi006.stdout: "verify_ssl": { 2024-05-09T04:58:26.487 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "verify_ssl", 2024-05-09T04:58:26.487 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.487 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.487 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.487 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "true", 2024-05-09T04:58:26.487 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.487 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.487 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.487 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T04:58:26.488 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.488 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.488 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.488 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.488 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.488 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.488 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.488 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "insights", 2024-05-09T04:58:26.488 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.488 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.488 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.488 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.489 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.489 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.489 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.489 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.489 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.489 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.489 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.489 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.489 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.489 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.489 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.490 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.490 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.490 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.490 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.490 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.490 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.490 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.490 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.490 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.490 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.490 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.491 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.491 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.491 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.491 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.491 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.491 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.491 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.491 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.491 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.491 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.491 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.491 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.492 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.492 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.492 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.492 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.492 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.492 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.492 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.492 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.492 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.492 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.492 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.492 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.492 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.493 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.493 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.493 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.493 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.493 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.493 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.493 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.493 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.493 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.493 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.493 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.493 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.493 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.494 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.494 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.494 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.494 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.494 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.494 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.494 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.494 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.494 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.494 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.494 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.495 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.495 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.495 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.495 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.495 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.495 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.495 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.495 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.495 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.495 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.495 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.495 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.496 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.496 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.496 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.496 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "iostat", 2024-05-09T04:58:26.496 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.496 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.496 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.496 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.496 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.496 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.496 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.496 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.497 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.497 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.497 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.497 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.497 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.497 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.497 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.497 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.497 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.497 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.497 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.497 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.498 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.498 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.498 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.498 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.498 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.498 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.498 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.498 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.498 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.498 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.498 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.499 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.499 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.499 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.499 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.499 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.499 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.499 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.499 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.499 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.499 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.499 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.499 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.499 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.500 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.500 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.500 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.500 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.500 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.500 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.500 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.500 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.500 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.500 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.500 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.500 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.500 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.501 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.501 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.501 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.501 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.501 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.501 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.501 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.501 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.501 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.501 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.501 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.502 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.502 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.502 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.502 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.502 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.502 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.502 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.502 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.502 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.502 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.502 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.502 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.503 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.503 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.503 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.503 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.503 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.503 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.503 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.503 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.503 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.503 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.503 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "k8sevents", 2024-05-09T04:58:26.503 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.503 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.504 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.504 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_event_retention_days": { 2024-05-09T04:58:26.504 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ceph_event_retention_days", 2024-05-09T04:58:26.504 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.504 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.504 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.504 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "7", 2024-05-09T04:58:26.504 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.504 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.504 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.504 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-09T04:58:26.504 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.505 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.505 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.505 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.505 INFO:teuthology.orchestra.run.smithi006.stdout: "config_check_secs": { 2024-05-09T04:58:26.505 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_check_secs", 2024-05-09T04:58:26.505 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.505 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.505 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.505 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-05-09T04:58:26.505 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "10", 2024-05-09T04:58:26.505 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.505 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.505 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-09T04:58:26.506 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.506 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.506 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.506 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.506 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.506 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.506 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.506 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.506 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.506 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.506 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.506 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.506 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.507 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.507 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.507 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.507 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.507 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.507 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.507 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.507 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.507 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.507 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.507 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.507 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.508 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.508 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.508 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.508 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.508 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.508 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.508 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.508 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.508 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.508 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.508 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.508 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.509 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.509 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.509 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.509 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.509 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.509 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.509 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.509 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.509 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.509 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.509 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.509 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.509 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.510 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.510 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.510 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.510 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.510 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.510 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.510 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.510 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.510 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.510 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.510 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.510 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.511 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.511 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.511 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.511 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.511 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.511 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.511 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.511 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.511 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.511 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.511 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.511 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.512 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.512 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.512 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.512 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.512 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.512 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.512 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.512 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.512 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.512 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.512 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.513 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.513 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.513 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.513 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.513 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.513 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.513 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "localpool", 2024-05-09T04:58:26.513 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.513 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.513 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.513 INFO:teuthology.orchestra.run.smithi006.stdout: "failure_domain": { 2024-05-09T04:58:26.513 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "failure_domain", 2024-05-09T04:58:26.514 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.514 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.514 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.514 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "host", 2024-05-09T04:58:26.514 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.514 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.514 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.514 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "failure domain for any created local pool", 2024-05-09T04:58:26.514 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T04:58:26.514 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.514 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.515 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.515 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.515 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.515 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.515 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.515 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.515 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.515 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.515 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.515 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.515 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.515 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.515 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.516 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.516 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.516 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.516 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.516 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.516 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.516 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.516 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.516 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.516 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.516 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.516 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.517 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.517 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.517 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.517 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.517 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.517 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.517 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.517 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.517 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.517 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.517 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.517 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.517 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.517 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.518 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.518 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.518 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.518 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.518 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.518 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.518 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.518 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.518 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.518 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.518 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.518 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.519 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.519 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.519 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.519 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.519 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.519 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.519 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.519 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.519 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.519 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.519 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.519 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.519 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.520 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.520 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.520 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.520 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.520 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.520 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.520 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.520 INFO:teuthology.orchestra.run.smithi006.stdout: "min_size": { 2024-05-09T04:58:26.520 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "min_size", 2024-05-09T04:58:26.520 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.520 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.520 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.520 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.521 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.521 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.521 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.521 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default min_size for any created local pool", 2024-05-09T04:58:26.521 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T04:58:26.521 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.521 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.521 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.521 INFO:teuthology.orchestra.run.smithi006.stdout: "num_rep": { 2024-05-09T04:58:26.521 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "num_rep", 2024-05-09T04:58:26.521 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.522 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.522 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.522 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3", 2024-05-09T04:58:26.522 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.522 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.522 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.522 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default replica count for any created local pool", 2024-05-09T04:58:26.522 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.522 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.522 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.522 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.523 INFO:teuthology.orchestra.run.smithi006.stdout: "pg_num": { 2024-05-09T04:58:26.523 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pg_num", 2024-05-09T04:58:26.523 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.523 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.523 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.523 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "128", 2024-05-09T04:58:26.523 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.523 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.523 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.523 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T04:58:26.523 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.523 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.524 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.524 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.524 INFO:teuthology.orchestra.run.smithi006.stdout: "prefix": { 2024-05-09T04:58:26.524 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prefix", 2024-05-09T04:58:26.524 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.524 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.524 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.524 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.524 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.525 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.525 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.525 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "name prefix for any created local pool", 2024-05-09T04:58:26.525 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.525 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.525 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.525 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.525 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.525 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.525 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.525 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.526 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.526 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.526 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.526 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.526 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.526 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.526 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.526 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.526 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.526 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.526 INFO:teuthology.orchestra.run.smithi006.stdout: "subtree": { 2024-05-09T04:58:26.526 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "subtree", 2024-05-09T04:58:26.527 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.527 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.527 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.527 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "rack", 2024-05-09T04:58:26.527 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.527 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.527 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.527 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T04:58:26.527 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T04:58:26.527 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.527 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.527 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.528 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.528 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.528 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.528 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mds_autoscaler", 2024-05-09T04:58:26.528 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.528 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.528 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.528 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.528 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.528 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.528 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.528 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.529 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.529 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.529 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.529 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.529 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.529 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.529 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.529 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.529 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.529 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.529 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.529 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.530 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.530 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.530 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.530 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.530 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.530 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.530 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.530 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.530 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.530 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.530 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.530 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.531 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.531 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.531 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.531 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.531 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.531 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.531 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.531 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.531 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.531 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.531 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.531 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.532 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.532 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.532 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.532 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.532 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.532 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.532 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.532 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.532 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.532 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.532 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.532 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.533 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.533 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.533 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.533 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.533 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.533 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.533 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.533 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.533 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.533 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.534 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.534 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.534 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.534 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.534 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.534 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.534 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.534 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.534 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.534 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.534 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.534 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.535 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.535 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.535 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.535 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.535 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.535 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.535 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.535 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.535 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.535 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.535 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.535 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.535 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mirroring", 2024-05-09T04:58:26.536 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.536 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.536 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.536 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.536 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.536 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.536 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.536 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.536 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.536 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.536 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.536 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.537 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.537 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.537 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.537 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.537 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.537 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.537 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.537 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.537 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.537 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.537 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.537 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.538 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.538 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.538 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.538 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.538 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.538 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.538 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.538 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.538 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.538 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.538 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.539 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.539 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.539 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.539 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.539 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.539 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.539 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.539 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.539 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.539 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.539 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.539 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.539 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.540 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.540 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.540 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.540 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.540 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.540 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.540 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.540 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.540 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.540 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.540 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.541 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.541 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.541 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.541 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.541 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.541 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.541 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.541 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.541 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.541 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.541 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.542 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.542 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.542 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.542 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.542 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.542 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.542 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.542 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.542 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.542 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.542 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.542 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.543 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.543 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.543 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.543 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.543 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.543 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.543 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.546 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.546 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "nfs", 2024-05-09T04:58:26.547 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.547 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.547 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.547 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.547 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.547 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.547 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.547 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.547 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.547 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.547 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.548 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.548 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.548 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.548 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.548 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.548 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.548 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.548 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.548 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.548 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.548 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.548 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.549 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.549 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.549 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.549 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.549 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.549 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.549 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.549 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.549 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.549 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.549 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.550 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.550 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.550 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.550 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.550 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.550 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.550 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.550 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.550 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.550 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.550 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.550 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.551 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.551 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.551 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.551 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.551 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.551 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.551 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.551 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.551 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.551 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.551 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.552 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.552 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.552 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.552 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.552 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.552 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.552 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.552 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.552 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.552 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.552 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.553 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.553 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.553 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.553 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.553 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.553 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.553 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.553 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.553 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.553 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.553 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.554 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.554 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.554 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.554 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.554 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.554 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.554 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.554 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.554 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.554 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.554 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.554 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "orchestrator", 2024-05-09T04:58:26.555 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.555 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.555 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.555 INFO:teuthology.orchestra.run.smithi006.stdout: "fail_fs": { 2024-05-09T04:58:26.555 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "fail_fs", 2024-05-09T04:58:26.555 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.555 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.555 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.555 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.555 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.555 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.556 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.556 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T04:58:26.556 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.556 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.556 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.556 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.556 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.556 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.556 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.556 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.557 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.557 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.557 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.557 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.557 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.557 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.557 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.557 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.557 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.557 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.557 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.557 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.558 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.558 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.558 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.558 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.558 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.558 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.558 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.558 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.558 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.558 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.558 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.559 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.559 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.559 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.559 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.559 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.559 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.559 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.559 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.559 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.559 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.559 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.559 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.559 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.560 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.560 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.560 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.560 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.560 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.560 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.560 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.560 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.560 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.560 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.560 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.560 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.561 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.561 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.561 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.561 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.561 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.561 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.561 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.561 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.561 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.561 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.561 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.561 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.562 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.562 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.562 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.562 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.562 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.562 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.562 INFO:teuthology.orchestra.run.smithi006.stdout: "orchestrator": { 2024-05-09T04:58:26.562 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "orchestrator", 2024-05-09T04:58:26.562 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.562 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.562 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.563 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.563 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.563 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.563 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.563 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm", 2024-05-09T04:58:26.563 INFO:teuthology.orchestra.run.smithi006.stdout: "rook", 2024-05-09T04:58:26.563 INFO:teuthology.orchestra.run.smithi006.stdout: "test_orchestrator" 2024-05-09T04:58:26.563 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.563 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Orchestrator backend", 2024-05-09T04:58:26.563 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.563 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.563 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.563 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.564 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.564 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.564 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.564 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.564 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.564 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.564 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.564 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.564 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.564 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.564 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.564 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.564 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.564 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.565 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.565 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.565 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.565 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "osd_perf_query", 2024-05-09T04:58:26.565 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.565 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.565 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.565 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.565 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.565 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.565 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.565 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.566 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.566 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.566 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.566 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.566 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.566 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.566 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.566 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.566 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.566 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.566 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.566 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.566 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.566 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.567 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.567 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.567 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.567 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.567 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.567 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.567 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.567 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.567 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.567 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.567 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.567 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.567 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.567 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.568 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.568 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.568 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.568 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.568 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.568 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.568 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.568 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.568 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.568 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.568 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.568 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.568 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.569 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.569 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.569 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.569 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.569 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.569 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.569 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.569 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.569 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.569 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.569 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.570 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.570 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.570 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.570 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.570 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.570 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.570 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.570 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.570 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.570 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.570 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.570 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.571 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.571 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.571 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.571 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.571 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.571 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.571 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.571 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.571 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.571 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.571 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.572 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.572 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.572 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.572 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.572 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.572 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.572 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.572 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "osd_support", 2024-05-09T04:58:26.572 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.572 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.572 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.572 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.573 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.573 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.573 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.573 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.573 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.573 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.573 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.573 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.573 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.573 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.573 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.574 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.574 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.574 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.574 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.574 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.574 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.574 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.574 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.574 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.574 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.574 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.574 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.574 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.575 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.575 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.575 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.575 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.575 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.575 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.575 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.575 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.575 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.575 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.575 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.575 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.576 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.576 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.576 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.576 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.576 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.576 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.576 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.576 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.576 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.576 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.576 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.576 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.577 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.577 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.577 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.577 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.577 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.577 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.577 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.577 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.577 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.577 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.577 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.577 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.577 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.578 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.578 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.578 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.578 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.578 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.578 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.578 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.578 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.578 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.578 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.578 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.578 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.579 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.579 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.579 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.579 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.579 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.579 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.579 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.579 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.579 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.579 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.579 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.579 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.579 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.580 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pg_autoscaler", 2024-05-09T04:58:26.580 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.580 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.580 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.580 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.580 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.580 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.580 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.580 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.580 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.580 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.580 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.581 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.581 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.581 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.581 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.581 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.581 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.581 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.581 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.581 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.581 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.581 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.582 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.582 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.582 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.582 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.582 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.582 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.582 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.582 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.582 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.582 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.582 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.582 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.582 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.583 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.583 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.583 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.583 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.583 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.583 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.583 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.583 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.583 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.583 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.583 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.583 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.584 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.584 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.584 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.584 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.584 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.584 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.584 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.584 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.584 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.584 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.584 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.584 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.585 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.585 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.585 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.585 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.585 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.585 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.585 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.585 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.585 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.585 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.585 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.585 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.585 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.586 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.586 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-05-09T04:58:26.586 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-05-09T04:58:26.586 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:26.586 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.586 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.586 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-05-09T04:58:26.586 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.586 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.586 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.586 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.587 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.587 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.587 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.587 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.587 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.587 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.587 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.587 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.587 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.587 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.587 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.587 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.587 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.588 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.588 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.588 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.588 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.588 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.588 INFO:teuthology.orchestra.run.smithi006.stdout: "threshold": { 2024-05-09T04:58:26.588 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "threshold", 2024-05-09T04:58:26.588 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-05-09T04:58:26.588 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.588 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.589 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3.0", 2024-05-09T04:58:26.589 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1.0", 2024-05-09T04:58:26.589 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.589 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.589 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "scaling threshold", 2024-05-09T04:58:26.589 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T04:58:26.589 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.589 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.589 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.589 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.589 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.590 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.590 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "progress", 2024-05-09T04:58:26.590 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.590 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.590 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.590 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_pg_recovery_event": { 2024-05-09T04:58:26.590 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_pg_recovery_event", 2024-05-09T04:58:26.590 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.590 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.590 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.590 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.591 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.591 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.591 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.591 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T04:58:26.591 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.591 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.591 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.591 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.591 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": { 2024-05-09T04:58:26.591 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enabled", 2024-05-09T04:58:26.592 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.592 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.592 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.592 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.592 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.592 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.592 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.592 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.592 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.592 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.592 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.592 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.593 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.593 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.593 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.593 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.593 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.593 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.593 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.593 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.593 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.593 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.593 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.593 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.593 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.594 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.594 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.594 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.594 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.594 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.594 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.594 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.594 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.594 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.594 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.594 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.594 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.595 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.595 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.595 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.595 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.595 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.595 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.595 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.595 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.595 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.595 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.595 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.595 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.596 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.596 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.596 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.596 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.596 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.596 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.596 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.596 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.596 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.596 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.596 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.596 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.596 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.596 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.597 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.597 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.597 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.597 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.597 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.597 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.597 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.597 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.597 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.597 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.597 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.597 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.597 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.597 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.598 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.598 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.598 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.598 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.598 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.598 INFO:teuthology.orchestra.run.smithi006.stdout: "max_completed_events": { 2024-05-09T04:58:26.598 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_completed_events", 2024-05-09T04:58:26.598 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.598 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.598 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.598 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "50", 2024-05-09T04:58:26.598 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.598 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.598 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.599 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "number of past completed events to remember", 2024-05-09T04:58:26.599 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.599 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.599 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.599 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.599 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2024-05-09T04:58:26.599 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2024-05-09T04:58:26.599 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:26.599 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.599 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.599 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2024-05-09T04:58:26.599 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.599 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.599 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.600 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long the module is going to sleep", 2024-05-09T04:58:26.600 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.600 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.600 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.600 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.600 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.600 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.600 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.600 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.600 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.600 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.600 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.600 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.601 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.601 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.601 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.601 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.601 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.601 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.601 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.601 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.601 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.601 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prometheus", 2024-05-09T04:58:26.601 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.601 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.602 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.602 INFO:teuthology.orchestra.run.smithi006.stdout: "cache": { 2024-05-09T04:58:26.602 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cache", 2024-05-09T04:58:26.602 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.602 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.602 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.602 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.602 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.602 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.602 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.602 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.602 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.602 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.602 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.603 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.603 INFO:teuthology.orchestra.run.smithi006.stdout: "exclude_perf_counters": { 2024-05-09T04:58:26.603 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "exclude_perf_counters", 2024-05-09T04:58:26.603 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.603 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.603 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.603 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.603 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.603 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.603 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.603 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T04:58:26.603 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T04:58:26.603 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.604 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.604 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.604 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.604 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.604 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.604 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.604 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.604 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.604 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.604 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.604 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.605 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.605 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.605 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.605 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.605 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.605 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.605 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.605 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.605 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.605 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.605 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.605 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.605 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.605 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.606 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.606 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.606 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.606 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.606 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.606 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.606 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.606 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.606 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.606 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.606 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.607 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.607 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.607 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.607 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.607 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.607 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.607 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.607 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.607 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.607 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.607 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.607 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.607 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.608 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.608 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.608 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.608 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.608 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.608 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.608 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.608 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.608 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.608 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.608 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.608 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.609 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.609 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.609 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.609 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.609 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.609 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.609 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.609 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.609 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.609 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.609 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.609 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_stats_pools": { 2024-05-09T04:58:26.609 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_stats_pools", 2024-05-09T04:58:26.609 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.610 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.610 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.610 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.610 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.610 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.610 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.610 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.610 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.610 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.610 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.611 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.611 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T04:58:26.611 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T04:58:26.611 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.611 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.611 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.611 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "300", 2024-05-09T04:58:26.611 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.611 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.611 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.611 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.611 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.612 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.612 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.612 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.612 INFO:teuthology.orchestra.run.smithi006.stdout: "scrape_interval": { 2024-05-09T04:58:26.612 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "scrape_interval", 2024-05-09T04:58:26.612 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-05-09T04:58:26.612 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.612 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.612 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15.0", 2024-05-09T04:58:26.612 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.612 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.612 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.613 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.613 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.613 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.613 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.613 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.613 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2024-05-09T04:58:26.613 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2024-05-09T04:58:26.613 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.613 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.613 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.613 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "::", 2024-05-09T04:58:26.613 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.613 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.614 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.614 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T04:58:26.614 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.614 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.614 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.614 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.614 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2024-05-09T04:58:26.614 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2024-05-09T04:58:26.614 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.614 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.614 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.614 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "9283", 2024-05-09T04:58:26.615 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.615 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.615 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.615 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T04:58:26.615 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.615 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.615 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.615 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.615 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.615 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.615 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.615 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.616 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.616 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.616 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.616 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.616 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.616 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.616 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.616 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.616 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.616 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.616 INFO:teuthology.orchestra.run.smithi006.stdout: "stale_cache_strategy": { 2024-05-09T04:58:26.616 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "stale_cache_strategy", 2024-05-09T04:58:26.616 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.617 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.617 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.617 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "log", 2024-05-09T04:58:26.617 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.617 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.617 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.617 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.617 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.617 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.617 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.617 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.617 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_behaviour": { 2024-05-09T04:58:26.617 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_behaviour", 2024-05-09T04:58:26.618 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.618 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.618 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.618 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "default", 2024-05-09T04:58:26.618 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.618 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.618 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.618 INFO:teuthology.orchestra.run.smithi006.stdout: "default", 2024-05-09T04:58:26.618 INFO:teuthology.orchestra.run.smithi006.stdout: "error" 2024-05-09T04:58:26.618 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.618 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.618 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.619 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.619 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.619 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.619 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_error_status_code": { 2024-05-09T04:58:26.619 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_error_status_code", 2024-05-09T04:58:26.619 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.619 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.619 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.619 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "500", 2024-05-09T04:58:26.619 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "400", 2024-05-09T04:58:26.619 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "599", 2024-05-09T04:58:26.619 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.620 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.620 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.620 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.620 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.620 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.620 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.620 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.620 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.620 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_support", 2024-05-09T04:58:26.620 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.620 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.621 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.621 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.621 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.621 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.621 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.621 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.621 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.621 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.621 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.621 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.621 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.622 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.622 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.622 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.622 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.622 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.622 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.622 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.622 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.622 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.622 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.622 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.622 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.623 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.623 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.623 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.623 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.623 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.623 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.623 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.623 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.623 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.623 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.623 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.624 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.624 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.624 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.624 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.624 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.624 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.624 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.624 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.624 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.624 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.624 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.624 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.624 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.625 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.625 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.625 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.625 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.625 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.625 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.625 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.625 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.625 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.625 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.625 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.625 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.626 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.626 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.626 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.626 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.626 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.626 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.626 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.626 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.626 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.626 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.626 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.626 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.627 INFO:teuthology.orchestra.run.smithi006.stdout: "max_concurrent_snap_create": { 2024-05-09T04:58:26.627 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_concurrent_snap_create", 2024-05-09T04:58:26.627 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.627 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.627 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.627 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2024-05-09T04:58:26.627 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.627 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.627 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.627 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.627 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.627 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.627 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.628 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.628 INFO:teuthology.orchestra.run.smithi006.stdout: "mirror_snapshot_schedule": { 2024-05-09T04:58:26.628 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T04:58:26.628 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.628 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.628 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.628 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.628 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.628 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.628 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.628 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.628 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.629 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.629 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.629 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.629 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.629 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.629 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.629 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.629 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.629 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.629 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.629 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.629 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.629 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.630 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.630 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.630 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.630 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.630 INFO:teuthology.orchestra.run.smithi006.stdout: "trash_purge_schedule": { 2024-05-09T04:58:26.630 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "trash_purge_schedule", 2024-05-09T04:58:26.630 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.630 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.630 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.630 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.630 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.630 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.631 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.631 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.631 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.631 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.631 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.631 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.631 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.631 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.631 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.631 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "restful", 2024-05-09T04:58:26.631 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.632 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.632 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.632 INFO:teuthology.orchestra.run.smithi006.stdout: "enable_auth": { 2024-05-09T04:58:26.632 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enable_auth", 2024-05-09T04:58:26.632 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.632 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.632 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.632 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.632 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.632 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.632 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.632 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.632 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.632 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.633 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.633 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.633 INFO:teuthology.orchestra.run.smithi006.stdout: "key_file": { 2024-05-09T04:58:26.633 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "key_file", 2024-05-09T04:58:26.633 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.633 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.633 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.633 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.633 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.633 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.633 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.633 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.633 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.634 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.634 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.634 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.634 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.634 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.634 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.634 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.634 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.634 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.634 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.634 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.634 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.635 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.635 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.635 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.635 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.635 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.635 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.635 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.635 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.635 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.635 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.635 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.635 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.636 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.636 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.636 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.636 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.636 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.636 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.636 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.636 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.636 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.636 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.636 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.636 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.636 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.636 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.637 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.637 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.637 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.637 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.637 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.637 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.637 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.637 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.637 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.637 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.637 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.637 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.638 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.638 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.638 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.638 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.638 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.638 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.638 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.638 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.638 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.638 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.638 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.638 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.639 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.639 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.639 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.639 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.639 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.639 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.639 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.639 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.639 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.639 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.639 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.639 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2024-05-09T04:58:26.640 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2024-05-09T04:58:26.640 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.640 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.640 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.640 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.640 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.640 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.640 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.640 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.640 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.640 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.640 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.641 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.641 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2024-05-09T04:58:26.641 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2024-05-09T04:58:26.641 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.641 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.641 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.641 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.641 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.641 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.641 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.641 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.641 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.642 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.642 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.642 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.642 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.642 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.642 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.642 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.642 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.642 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.642 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.642 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.642 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.643 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.643 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.643 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.643 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.643 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.643 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.643 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.643 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.643 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rgw", 2024-05-09T04:58:26.643 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.643 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.644 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.644 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.644 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.644 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.644 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.644 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.644 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.644 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.644 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.644 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.644 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.644 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.644 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.645 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.645 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.645 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.645 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.645 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.645 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.645 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.645 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.645 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.645 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.645 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.645 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.646 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.646 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.646 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.646 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.646 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.646 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.646 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.646 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.646 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.646 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.646 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.646 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.647 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.647 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.647 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.647 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.647 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.647 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.647 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.647 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.647 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.647 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.647 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.647 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.648 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.648 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.648 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.648 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.648 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.648 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.648 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.648 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.648 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.648 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.648 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.648 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.648 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.649 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.649 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.649 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.649 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.649 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.649 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.649 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.649 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.649 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.649 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.649 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.650 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.650 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.650 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.650 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.650 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.650 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.650 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.650 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.650 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.650 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.650 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.650 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.651 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.651 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.651 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.651 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rook", 2024-05-09T04:58:26.651 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.651 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.651 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.651 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.651 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.651 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.651 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.651 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.652 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.652 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.652 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.652 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.652 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.652 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.652 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.652 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.652 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.652 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.652 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.652 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.653 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.653 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.653 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.653 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.653 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.653 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.653 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.653 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.653 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.653 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.653 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.654 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.654 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.654 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.654 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.654 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.654 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.654 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.654 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.654 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.654 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.654 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.654 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.655 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.655 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.655 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.655 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.655 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.655 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.655 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.655 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.655 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.655 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.655 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.655 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.655 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.655 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.656 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.656 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.656 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.656 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.656 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.656 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.656 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.656 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.656 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.656 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.656 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.657 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.657 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.657 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.657 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.657 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.657 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.657 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.657 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.657 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.657 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.657 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.657 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.657 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.657 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.658 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.658 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.658 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.658 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.658 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.658 INFO:teuthology.orchestra.run.smithi006.stdout: "storage_class": { 2024-05-09T04:58:26.658 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "storage_class", 2024-05-09T04:58:26.658 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.658 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.658 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.658 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "local", 2024-05-09T04:58:26.658 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.658 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.658 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.659 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T04:58:26.659 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.659 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.659 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.659 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.659 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.659 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.659 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.659 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "selftest", 2024-05-09T04:58:26.659 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.659 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.659 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.659 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.660 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.660 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.660 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.660 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.660 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.660 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.660 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.660 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.660 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.660 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.660 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.660 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.660 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.661 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.661 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.661 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.661 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.661 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.661 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.661 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.661 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.661 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.661 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.661 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.661 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.662 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.662 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.662 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.662 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.662 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.662 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.662 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.662 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.662 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.662 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.662 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.662 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.662 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.663 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.663 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.663 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.663 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.663 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.663 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.663 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.663 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.663 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.663 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.663 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.663 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.663 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.664 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.664 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.664 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.664 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.664 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.664 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.664 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.664 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.664 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.664 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.664 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.664 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.664 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.664 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.665 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.665 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.665 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.665 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.665 INFO:teuthology.orchestra.run.smithi006.stdout: "roption1": { 2024-05-09T04:58:26.665 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "roption1", 2024-05-09T04:58:26.665 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.665 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.665 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.665 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.665 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.665 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.665 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.666 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.666 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.666 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.666 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.666 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.666 INFO:teuthology.orchestra.run.smithi006.stdout: "roption2": { 2024-05-09T04:58:26.666 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "roption2", 2024-05-09T04:58:26.666 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.666 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.666 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.666 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "xyz", 2024-05-09T04:58:26.666 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.666 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.667 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.667 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.667 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.667 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.667 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.667 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.667 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption1": { 2024-05-09T04:58:26.667 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption1", 2024-05-09T04:58:26.667 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.667 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.667 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.667 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.667 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.667 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.668 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.668 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.668 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.668 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.668 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.668 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.668 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption2": { 2024-05-09T04:58:26.668 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption2", 2024-05-09T04:58:26.668 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.668 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.668 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.668 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.668 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.668 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.669 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.669 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.669 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.669 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.669 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.669 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.669 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption3": { 2024-05-09T04:58:26.669 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption3", 2024-05-09T04:58:26.669 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2024-05-09T04:58:26.669 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.669 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.669 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.669 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.670 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.670 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.670 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.670 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.670 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.670 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.670 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.670 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption4": { 2024-05-09T04:58:26.670 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption4", 2024-05-09T04:58:26.670 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.670 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.670 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.671 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.671 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.671 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.671 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.671 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.671 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.671 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.671 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.671 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.671 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption5": { 2024-05-09T04:58:26.671 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption5", 2024-05-09T04:58:26.671 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.672 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.672 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.672 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.672 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.672 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.672 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.672 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.672 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.672 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.672 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.672 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.672 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption6": { 2024-05-09T04:58:26.673 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption6", 2024-05-09T04:58:26.673 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.673 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.673 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.673 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.673 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.673 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.673 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.673 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.673 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.673 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.673 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.674 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.674 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption7": { 2024-05-09T04:58:26.674 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption7", 2024-05-09T04:58:26.674 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.674 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.674 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.674 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.674 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2024-05-09T04:58:26.674 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "42", 2024-05-09T04:58:26.674 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.674 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.674 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.674 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.675 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.675 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.675 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.675 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.675 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.675 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.675 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.675 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.675 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.675 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.675 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.675 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.676 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.676 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.676 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.676 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.676 INFO:teuthology.orchestra.run.smithi006.stdout: "testkey": { 2024-05-09T04:58:26.676 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testkey", 2024-05-09T04:58:26.676 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.676 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.676 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.676 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.676 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.676 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.677 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.677 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.677 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.677 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.677 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.677 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.677 INFO:teuthology.orchestra.run.smithi006.stdout: "testlkey": { 2024-05-09T04:58:26.677 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testlkey", 2024-05-09T04:58:26.677 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.677 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.677 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.677 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.677 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.678 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.678 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.678 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.678 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.678 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.678 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.678 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.678 INFO:teuthology.orchestra.run.smithi006.stdout: "testnewline": { 2024-05-09T04:58:26.678 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testnewline", 2024-05-09T04:58:26.678 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.678 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.678 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.679 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.679 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.679 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.679 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.679 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.679 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.679 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.679 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.679 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.679 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.679 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.679 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.679 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smb", 2024-05-09T04:58:26.680 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.680 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.680 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.680 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.680 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.680 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.680 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.680 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.680 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.680 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.680 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.680 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.680 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.681 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.681 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.681 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.681 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.681 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.681 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.681 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.681 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.681 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.681 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.681 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.681 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.682 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.682 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.682 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.682 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.682 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.682 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.682 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.682 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.682 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.682 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.683 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.683 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.683 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.683 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.683 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.683 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.683 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.683 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.683 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.683 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.683 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.683 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.684 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.684 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.684 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.684 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.684 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.684 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.684 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.684 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.684 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.684 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.684 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.684 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.684 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.685 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.685 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.685 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.685 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.685 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.685 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.685 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.685 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.685 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.685 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.685 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.685 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.686 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.686 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.686 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.686 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.686 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.686 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.686 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.686 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.686 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.686 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.687 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.687 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.687 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.687 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.687 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.687 INFO:teuthology.orchestra.run.smithi006.stdout: "update_orchestration": { 2024-05-09T04:58:26.687 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "update_orchestration", 2024-05-09T04:58:26.687 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.687 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.687 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.687 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.687 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.688 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.688 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.688 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-09T04:58:26.688 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.688 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.688 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.688 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.688 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.688 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.688 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.688 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "snap_schedule", 2024-05-09T04:58:26.688 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.689 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.689 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.689 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_m_granularity": { 2024-05-09T04:58:26.689 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_m_granularity", 2024-05-09T04:58:26.689 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.689 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.689 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.689 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.689 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.689 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.689 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.690 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T04:58:26.690 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.690 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.690 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.690 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.690 INFO:teuthology.orchestra.run.smithi006.stdout: "dump_on_update": { 2024-05-09T04:58:26.690 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "dump_on_update", 2024-05-09T04:58:26.690 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.690 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.690 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.690 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.691 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.691 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.691 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.691 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "dump database to debug log on update", 2024-05-09T04:58:26.691 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.691 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.691 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.691 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.691 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.691 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.691 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.691 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.691 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.692 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.692 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.692 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.692 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.692 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.692 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.692 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.692 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.692 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.692 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.692 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.692 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.693 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.693 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.693 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.693 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.693 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.693 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.693 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.693 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.693 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.693 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.693 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.693 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.694 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.694 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.694 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.694 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.694 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.694 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.694 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.694 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.694 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.694 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.694 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.694 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.695 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.695 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.695 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.695 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.695 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.695 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.695 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.695 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.695 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.695 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.695 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.695 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.695 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.696 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.696 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.696 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.696 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.696 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.696 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.696 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.696 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.696 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.696 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.696 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.696 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.696 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.697 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.697 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.697 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.697 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.697 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.697 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.697 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.697 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.697 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.697 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.697 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.697 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.697 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.697 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.698 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.698 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.698 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.698 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.698 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.698 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.698 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "stats", 2024-05-09T04:58:26.698 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.698 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.698 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.698 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.698 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.698 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.699 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.699 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.699 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.699 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.699 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.699 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.699 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.699 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.699 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.699 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.699 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.699 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.699 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.699 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.700 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.700 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.700 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.700 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.700 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.700 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.700 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.700 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.700 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.700 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.700 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.700 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.701 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.701 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.701 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.701 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.701 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.701 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.701 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.701 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.701 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.701 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.701 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.702 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.702 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.702 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.702 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.702 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.702 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.702 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.702 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.702 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.702 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.702 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.702 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.702 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.703 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.703 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.703 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.703 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.703 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.703 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.703 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.703 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.703 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.703 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.703 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.703 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.704 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.704 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.704 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.704 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.704 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.704 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.704 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.704 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.704 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.704 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.704 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.704 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.705 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.705 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.705 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.705 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.705 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.705 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.705 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.705 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.705 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.705 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.705 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "status", 2024-05-09T04:58:26.705 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.705 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.706 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.706 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.706 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.706 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.706 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.706 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.706 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.706 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.706 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.706 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.706 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.706 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.706 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.706 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.707 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.707 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.707 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.707 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.707 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.707 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.707 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.707 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.707 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.707 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.707 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.707 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.708 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.708 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.708 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.708 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.708 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.708 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.708 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.708 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.708 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.708 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.708 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.708 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.708 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.709 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.709 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.709 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.709 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.709 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.709 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.709 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.709 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.709 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.709 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.709 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.709 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.709 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.710 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.710 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.710 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.710 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.710 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.710 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.710 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.710 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.710 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.710 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.710 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.710 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.711 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.711 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.711 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.711 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.711 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.711 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.711 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.711 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.711 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.711 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.711 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.712 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.712 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.712 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.712 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.712 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.712 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.712 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.712 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.712 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.712 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.712 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.712 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.712 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.713 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "telegraf", 2024-05-09T04:58:26.713 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.713 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.713 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.713 INFO:teuthology.orchestra.run.smithi006.stdout: "address": { 2024-05-09T04:58:26.713 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "address", 2024-05-09T04:58:26.713 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.713 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.713 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.713 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T04:58:26.713 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.713 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.714 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.714 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.714 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.714 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.714 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.714 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.714 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-05-09T04:58:26.714 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-05-09T04:58:26.714 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:26.714 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.714 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.714 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15", 2024-05-09T04:58:26.715 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.715 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.715 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.715 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.715 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.715 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.715 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.715 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.715 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.715 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.715 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.715 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.716 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.716 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.716 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.716 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.716 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.716 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.716 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.716 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.716 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.716 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.716 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.716 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.717 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.717 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.717 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.717 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.717 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.717 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.717 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.717 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.717 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.717 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.717 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.717 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.717 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.717 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.718 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.718 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.718 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.718 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.718 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.718 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.718 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.718 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.718 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.718 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.718 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.719 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.719 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.719 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.719 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.719 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.719 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.719 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.719 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.719 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.719 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.719 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.719 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.719 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.720 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.720 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.720 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.720 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.720 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.720 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.720 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.720 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.720 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.720 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.720 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.720 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.721 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.721 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.721 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.721 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.721 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.721 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.721 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.721 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.721 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.721 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.721 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.722 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.722 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.722 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.722 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.722 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.722 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.722 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.722 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.722 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.722 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.722 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "telemetry", 2024-05-09T04:58:26.722 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.723 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.723 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.723 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_basic": { 2024-05-09T04:58:26.723 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_basic", 2024-05-09T04:58:26.723 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.723 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.723 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.723 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.723 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.723 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.723 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.723 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T04:58:26.724 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.724 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.724 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.724 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.724 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_crash": { 2024-05-09T04:58:26.724 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_crash", 2024-05-09T04:58:26.724 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.724 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.724 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.724 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.724 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.724 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.724 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.725 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T04:58:26.725 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.725 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.725 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.725 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.725 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_device": { 2024-05-09T04:58:26.725 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_device", 2024-05-09T04:58:26.725 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.725 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.725 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.725 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.725 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.725 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.726 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.726 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T04:58:26.726 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.726 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.726 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.726 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.726 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_ident": { 2024-05-09T04:58:26.726 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_ident", 2024-05-09T04:58:26.726 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.726 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.726 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.727 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.727 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.727 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.727 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.727 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T04:58:26.727 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.727 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.727 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.727 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.727 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_perf": { 2024-05-09T04:58:26.727 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_perf", 2024-05-09T04:58:26.727 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.727 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.727 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.728 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.728 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.728 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.728 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.728 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T04:58:26.728 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.728 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.728 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.728 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.728 INFO:teuthology.orchestra.run.smithi006.stdout: "contact": { 2024-05-09T04:58:26.728 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "contact", 2024-05-09T04:58:26.728 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.728 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.728 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.729 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.729 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.729 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.729 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.729 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.729 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.729 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.729 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.729 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.729 INFO:teuthology.orchestra.run.smithi006.stdout: "description": { 2024-05-09T04:58:26.729 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "description", 2024-05-09T04:58:26.730 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.730 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.730 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.730 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.730 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.730 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.730 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.730 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.730 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.730 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.730 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.730 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.730 INFO:teuthology.orchestra.run.smithi006.stdout: "device_url": { 2024-05-09T04:58:26.731 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_url", 2024-05-09T04:58:26.731 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.731 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.731 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.731 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T04:58:26.731 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.731 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.731 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.731 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.731 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.731 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.732 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.732 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.732 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": { 2024-05-09T04:58:26.732 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enabled", 2024-05-09T04:58:26.732 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.732 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.732 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.732 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.732 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.732 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.732 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.732 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.732 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.732 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.733 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.733 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.733 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-05-09T04:58:26.733 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-05-09T04:58:26.733 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.733 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.733 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.733 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "24", 2024-05-09T04:58:26.733 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "8", 2024-05-09T04:58:26.733 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.733 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.733 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.733 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.733 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.734 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.734 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.734 INFO:teuthology.orchestra.run.smithi006.stdout: "last_opt_revision": { 2024-05-09T04:58:26.734 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "last_opt_revision", 2024-05-09T04:58:26.734 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.734 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.734 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.734 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1", 2024-05-09T04:58:26.734 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.734 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.734 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.734 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.735 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.735 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.735 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.735 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.735 INFO:teuthology.orchestra.run.smithi006.stdout: "leaderboard": { 2024-05-09T04:58:26.735 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "leaderboard", 2024-05-09T04:58:26.735 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.735 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.735 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.735 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.735 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.735 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.735 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.735 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.736 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.736 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.736 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.736 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.736 INFO:teuthology.orchestra.run.smithi006.stdout: "leaderboard_description": { 2024-05-09T04:58:26.736 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "leaderboard_description", 2024-05-09T04:58:26.736 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.736 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.736 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.736 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.736 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.736 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.736 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.736 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.737 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.737 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.737 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.737 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.737 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.737 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.737 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.737 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.737 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.737 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.737 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.737 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.737 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.737 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.738 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.738 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.738 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.738 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.738 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.738 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.738 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.738 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.738 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.738 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.738 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.738 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.738 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.739 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.739 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.739 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.739 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.739 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.739 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.739 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.739 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.739 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.739 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.739 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.739 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.739 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.739 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.740 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.740 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.740 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.740 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.740 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.740 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.740 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.740 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.740 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.740 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.740 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.740 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.740 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.741 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.741 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.741 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.741 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.741 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.741 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.741 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.741 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.741 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.741 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.741 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.742 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.742 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.742 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.742 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.742 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.742 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.742 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.742 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.742 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.742 INFO:teuthology.orchestra.run.smithi006.stdout: "organization": { 2024-05-09T04:58:26.742 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "organization", 2024-05-09T04:58:26.743 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.743 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.743 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.743 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.743 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.743 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.743 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.743 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.743 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.743 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.743 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.744 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.744 INFO:teuthology.orchestra.run.smithi006.stdout: "proxy": { 2024-05-09T04:58:26.744 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "proxy", 2024-05-09T04:58:26.744 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.744 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.744 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.744 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.744 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.744 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.744 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.744 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.744 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.745 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.745 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.745 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.745 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.745 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.745 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.745 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.745 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.745 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.745 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.745 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.745 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.746 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.746 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.746 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.746 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.746 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.746 INFO:teuthology.orchestra.run.smithi006.stdout: "url": { 2024-05-09T04:58:26.746 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "url", 2024-05-09T04:58:26.746 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.746 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.746 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.746 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T04:58:26.746 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.747 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.747 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.747 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.750 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.750 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.750 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.750 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.750 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.750 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.750 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.751 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "test_orchestrator", 2024-05-09T04:58:26.751 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.751 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.751 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.751 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.751 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.751 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.751 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.751 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.751 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.751 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.752 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.752 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.752 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.752 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.752 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.752 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.752 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.752 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.752 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.752 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.752 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.752 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.753 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.753 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.753 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.753 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.753 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.753 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.753 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.753 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.753 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.753 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.753 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.753 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.754 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.754 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.754 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.754 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.754 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.754 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.754 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.754 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.754 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.754 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.754 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.754 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.754 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.755 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.755 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.755 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.755 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.755 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.755 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.755 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.755 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.755 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.755 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.755 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.755 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.755 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.756 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.756 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.756 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.756 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.756 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.756 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.756 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.756 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.756 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.756 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.756 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.756 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.757 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.757 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.757 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.757 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.757 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.757 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.757 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.757 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.757 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.757 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.757 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.758 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.758 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.758 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.758 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.758 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.758 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.758 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.758 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "volumes", 2024-05-09T04:58:26.758 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.758 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.758 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.758 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.759 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.759 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.759 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.759 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.759 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.759 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.759 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.759 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.759 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.759 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.759 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.759 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.760 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.760 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.760 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.760 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.760 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.760 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.760 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.760 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.760 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.760 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.760 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.760 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.761 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:24 smithi006 ceph-mon[29835]: pgmap v603: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 MiB/s wr, 20 op/s 2024-05-09T04:58:26.761 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:24 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/3460649085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T04:58:26.761 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:24 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/4039212125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T04:58:26.762 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.762 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.762 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.762 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.762 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.762 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.762 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.762 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.762 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.762 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.763 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.763 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.763 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.763 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.763 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.763 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.763 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.763 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.763 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.763 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.763 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.764 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.764 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.764 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.764 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.764 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.764 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.764 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.764 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.764 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.764 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.764 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.764 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.765 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.765 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.765 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.765 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.765 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.765 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.765 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.765 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.765 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.765 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.765 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.765 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.765 INFO:teuthology.orchestra.run.smithi006.stdout: "max_concurrent_clones": { 2024-05-09T04:58:26.766 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_concurrent_clones", 2024-05-09T04:58:26.766 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.766 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.766 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.766 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "4", 2024-05-09T04:58:26.766 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.766 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.766 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.766 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T04:58:26.766 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.766 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.766 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.767 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.767 INFO:teuthology.orchestra.run.smithi006.stdout: "periodic_async_work": { 2024-05-09T04:58:26.767 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "periodic_async_work", 2024-05-09T04:58:26.767 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.767 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.767 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.767 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.767 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.767 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.767 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.767 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Periodically check for async work", 2024-05-09T04:58:26.767 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.767 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.768 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.768 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.768 INFO:teuthology.orchestra.run.smithi006.stdout: "snapshot_clone_delay": { 2024-05-09T04:58:26.768 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "snapshot_clone_delay", 2024-05-09T04:58:26.768 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.768 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.768 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.768 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.768 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.768 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.768 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.769 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T04:58:26.769 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.769 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.769 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.769 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.769 INFO:teuthology.orchestra.run.smithi006.stdout: "snapshot_clone_no_wait": { 2024-05-09T04:58:26.769 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T04:58:26.769 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.769 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.769 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.769 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2024-05-09T04:58:26.769 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.770 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.770 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.770 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T04:58:26.770 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.770 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.770 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.770 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.770 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.770 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.770 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.770 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.771 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.771 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.771 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.771 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.771 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.771 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.771 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.771 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.771 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.771 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.771 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.771 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.772 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.772 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix", 2024-05-09T04:58:26.772 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2024-05-09T04:58:26.772 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2024-05-09T04:58:26.772 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2024-05-09T04:58:26.772 INFO:teuthology.orchestra.run.smithi006.stdout: "discovery_interval": { 2024-05-09T04:58:26.772 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "discovery_interval", 2024-05-09T04:58:26.772 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2024-05-09T04:58:26.772 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.772 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.772 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "100", 2024-05-09T04:58:26.772 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.773 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.773 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.773 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.773 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.773 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.773 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.773 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.773 INFO:teuthology.orchestra.run.smithi006.stdout: "identifier": { 2024-05-09T04:58:26.773 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "identifier", 2024-05-09T04:58:26.773 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.773 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.773 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.774 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.774 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.774 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.774 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.774 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.774 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.774 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.774 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.774 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.774 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2024-05-09T04:58:26.774 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2024-05-09T04:58:26.774 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2024-05-09T04:58:26.775 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.775 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.775 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2024-05-09T04:58:26.775 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.775 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.775 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.775 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.775 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.775 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.775 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.775 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.775 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2024-05-09T04:58:26.776 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2024-05-09T04:58:26.776 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.776 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.776 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.776 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.776 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.776 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.776 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.776 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.776 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.776 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.776 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.777 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.777 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.777 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.777 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.777 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.777 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.777 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.777 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.777 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2024-05-09T04:58:26.777 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2024-05-09T04:58:26.777 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.777 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.778 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.778 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.778 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.778 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.778 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.778 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.778 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.778 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.778 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.778 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.778 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2024-05-09T04:58:26.778 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2024-05-09T04:58:26.778 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.779 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.779 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.779 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2024-05-09T04:58:26.779 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.779 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.779 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2024-05-09T04:58:26.779 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2024-05-09T04:58:26.779 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2024-05-09T04:58:26.779 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2024-05-09T04:58:26.779 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2024-05-09T04:58:26.779 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2024-05-09T04:58:26.779 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2024-05-09T04:58:26.779 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.780 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.780 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.780 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.780 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.780 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.780 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2024-05-09T04:58:26.780 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2024-05-09T04:58:26.780 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2024-05-09T04:58:26.780 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.780 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.780 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2024-05-09T04:58:26.780 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.781 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.781 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.781 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.781 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.781 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.781 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.781 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.781 INFO:teuthology.orchestra.run.smithi006.stdout: "sqlite3_killpoint": { 2024-05-09T04:58:26.781 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sqlite3_killpoint", 2024-05-09T04:58:26.781 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.781 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "dev", 2024-05-09T04:58:26.781 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2024-05-09T04:58:26.781 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2024-05-09T04:58:26.782 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.782 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.782 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.782 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.782 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.782 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.782 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.782 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.782 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_host": { 2024-05-09T04:58:26.782 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_host", 2024-05-09T04:58:26.783 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.783 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.783 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.783 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2024-05-09T04:58:26.783 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.783 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.783 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.783 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.783 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.783 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.783 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.783 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.783 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_port": { 2024-05-09T04:58:26.784 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_port", 2024-05-09T04:58:26.784 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2024-05-09T04:58:26.784 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.784 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.784 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10051", 2024-05-09T04:58:26.784 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.784 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.784 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.784 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.784 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.784 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.784 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.784 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.785 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_sender": { 2024-05-09T04:58:26.785 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_sender", 2024-05-09T04:58:26.785 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2024-05-09T04:58:26.785 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2024-05-09T04:58:26.785 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2024-05-09T04:58:26.785 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T04:58:26.785 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2024-05-09T04:58:26.785 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2024-05-09T04:58:26.785 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2024-05-09T04:58:26.785 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2024-05-09T04:58:26.785 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2024-05-09T04:58:26.785 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-09T04:58:26.786 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2024-05-09T04:58:26.786 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.786 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.786 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.786 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.786 INFO:teuthology.orchestra.run.smithi006.stdout: "services": { 2024-05-09T04:58:26.786 INFO:teuthology.orchestra.run.smithi006.stdout: "dashboard": "https://172.21.15.6:8443/" 2024-05-09T04:58:26.786 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.786 INFO:teuthology.orchestra.run.smithi006.stdout: "always_on_modules": { 2024-05-09T04:58:26.786 INFO:teuthology.orchestra.run.smithi006.stdout: "octopus": [ 2024-05-09T04:58:26.786 INFO:teuthology.orchestra.run.smithi006.stdout: "balancer", 2024-05-09T04:58:26.786 INFO:teuthology.orchestra.run.smithi006.stdout: "crash", 2024-05-09T04:58:26.787 INFO:teuthology.orchestra.run.smithi006.stdout: "devicehealth", 2024-05-09T04:58:26.787 INFO:teuthology.orchestra.run.smithi006.stdout: "orchestrator", 2024-05-09T04:58:26.787 INFO:teuthology.orchestra.run.smithi006.stdout: "pg_autoscaler", 2024-05-09T04:58:26.787 INFO:teuthology.orchestra.run.smithi006.stdout: "progress", 2024-05-09T04:58:26.787 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_support", 2024-05-09T04:58:26.787 INFO:teuthology.orchestra.run.smithi006.stdout: "status", 2024-05-09T04:58:26.787 INFO:teuthology.orchestra.run.smithi006.stdout: "telemetry", 2024-05-09T04:58:26.787 INFO:teuthology.orchestra.run.smithi006.stdout: "volumes" 2024-05-09T04:58:26.787 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.787 INFO:teuthology.orchestra.run.smithi006.stdout: "pacific": [ 2024-05-09T04:58:26.787 INFO:teuthology.orchestra.run.smithi006.stdout: "balancer", 2024-05-09T04:58:26.787 INFO:teuthology.orchestra.run.smithi006.stdout: "crash", 2024-05-09T04:58:26.787 INFO:teuthology.orchestra.run.smithi006.stdout: "devicehealth", 2024-05-09T04:58:26.788 INFO:teuthology.orchestra.run.smithi006.stdout: "orchestrator", 2024-05-09T04:58:26.788 INFO:teuthology.orchestra.run.smithi006.stdout: "pg_autoscaler", 2024-05-09T04:58:26.788 INFO:teuthology.orchestra.run.smithi006.stdout: "progress", 2024-05-09T04:58:26.788 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_support", 2024-05-09T04:58:26.788 INFO:teuthology.orchestra.run.smithi006.stdout: "status", 2024-05-09T04:58:26.788 INFO:teuthology.orchestra.run.smithi006.stdout: "telemetry", 2024-05-09T04:58:26.788 INFO:teuthology.orchestra.run.smithi006.stdout: "volumes" 2024-05-09T04:58:26.788 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.788 INFO:teuthology.orchestra.run.smithi006.stdout: "quincy": [ 2024-05-09T04:58:26.788 INFO:teuthology.orchestra.run.smithi006.stdout: "balancer", 2024-05-09T04:58:26.788 INFO:teuthology.orchestra.run.smithi006.stdout: "crash", 2024-05-09T04:58:26.788 INFO:teuthology.orchestra.run.smithi006.stdout: "devicehealth", 2024-05-09T04:58:26.788 INFO:teuthology.orchestra.run.smithi006.stdout: "orchestrator", 2024-05-09T04:58:26.789 INFO:teuthology.orchestra.run.smithi006.stdout: "pg_autoscaler", 2024-05-09T04:58:26.789 INFO:teuthology.orchestra.run.smithi006.stdout: "progress", 2024-05-09T04:58:26.789 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_support", 2024-05-09T04:58:26.789 INFO:teuthology.orchestra.run.smithi006.stdout: "status", 2024-05-09T04:58:26.789 INFO:teuthology.orchestra.run.smithi006.stdout: "telemetry", 2024-05-09T04:58:26.789 INFO:teuthology.orchestra.run.smithi006.stdout: "volumes" 2024-05-09T04:58:26.789 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.789 INFO:teuthology.orchestra.run.smithi006.stdout: "reef": [ 2024-05-09T04:58:26.789 INFO:teuthology.orchestra.run.smithi006.stdout: "balancer", 2024-05-09T04:58:26.789 INFO:teuthology.orchestra.run.smithi006.stdout: "crash", 2024-05-09T04:58:26.789 INFO:teuthology.orchestra.run.smithi006.stdout: "devicehealth", 2024-05-09T04:58:26.789 INFO:teuthology.orchestra.run.smithi006.stdout: "orchestrator", 2024-05-09T04:58:26.789 INFO:teuthology.orchestra.run.smithi006.stdout: "pg_autoscaler", 2024-05-09T04:58:26.790 INFO:teuthology.orchestra.run.smithi006.stdout: "progress", 2024-05-09T04:58:26.790 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_support", 2024-05-09T04:58:26.790 INFO:teuthology.orchestra.run.smithi006.stdout: "status", 2024-05-09T04:58:26.790 INFO:teuthology.orchestra.run.smithi006.stdout: "telemetry", 2024-05-09T04:58:26.790 INFO:teuthology.orchestra.run.smithi006.stdout: "volumes" 2024-05-09T04:58:26.790 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-09T04:58:26.790 INFO:teuthology.orchestra.run.smithi006.stdout: "squid": [ 2024-05-09T04:58:26.790 INFO:teuthology.orchestra.run.smithi006.stdout: "balancer", 2024-05-09T04:58:26.790 INFO:teuthology.orchestra.run.smithi006.stdout: "crash", 2024-05-09T04:58:26.790 INFO:teuthology.orchestra.run.smithi006.stdout: "devicehealth", 2024-05-09T04:58:26.790 INFO:teuthology.orchestra.run.smithi006.stdout: "orchestrator", 2024-05-09T04:58:26.791 INFO:teuthology.orchestra.run.smithi006.stdout: "pg_autoscaler", 2024-05-09T04:58:26.791 INFO:teuthology.orchestra.run.smithi006.stdout: "progress", 2024-05-09T04:58:26.791 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_support", 2024-05-09T04:58:26.791 INFO:teuthology.orchestra.run.smithi006.stdout: "status", 2024-05-09T04:58:26.791 INFO:teuthology.orchestra.run.smithi006.stdout: "telemetry", 2024-05-09T04:58:26.791 INFO:teuthology.orchestra.run.smithi006.stdout: "volumes" 2024-05-09T04:58:26.791 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2024-05-09T04:58:26.791 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.791 INFO:teuthology.orchestra.run.smithi006.stdout: "last_failure_osd_epoch": 3, 2024-05-09T04:58:26.791 INFO:teuthology.orchestra.run.smithi006.stdout: "active_clients": [ 2024-05-09T04:58:26.791 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.791 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "devicehealth", 2024-05-09T04:58:26.792 INFO:teuthology.orchestra.run.smithi006.stdout: "addrvec": [ 2024-05-09T04:58:26.792 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.792 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v2", 2024-05-09T04:58:26.792 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "172.21.15.6:0", 2024-05-09T04:58:26.792 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 3701138119 2024-05-09T04:58:26.792 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.792 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2024-05-09T04:58:26.792 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.792 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.792 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "libcephsqlite", 2024-05-09T04:58:26.793 INFO:teuthology.orchestra.run.smithi006.stdout: "addrvec": [ 2024-05-09T04:58:26.793 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.793 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v2", 2024-05-09T04:58:26.793 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "172.21.15.6:0", 2024-05-09T04:58:26.793 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 1900352833 2024-05-09T04:58:26.793 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.793 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2024-05-09T04:58:26.793 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.793 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.793 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_support", 2024-05-09T04:58:26.793 INFO:teuthology.orchestra.run.smithi006.stdout: "addrvec": [ 2024-05-09T04:58:26.794 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.794 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v2", 2024-05-09T04:58:26.794 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "172.21.15.6:0", 2024-05-09T04:58:26.794 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 2114267993 2024-05-09T04:58:26.794 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.794 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2024-05-09T04:58:26.794 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:26.794 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.794 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "volumes", 2024-05-09T04:58:26.794 INFO:teuthology.orchestra.run.smithi006.stdout: "addrvec": [ 2024-05-09T04:58:26.794 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-09T04:58:26.794 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v2", 2024-05-09T04:58:26.795 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "172.21.15.6:0", 2024-05-09T04:58:26.795 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 3649951131 2024-05-09T04:58:26.795 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.795 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2024-05-09T04:58:26.795 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:26.795 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2024-05-09T04:58:26.795 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-09T04:58:26.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:25.182+0000 7f4b87200640 1 -- 172.21.15.6:0/1026281952 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4b6c0b27f0 msgr2=0x7f4b6c0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:26.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:25.182+0000 7f4b87200640 1 --2- 172.21.15.6:0/1026281952 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4b6c0b27f0 0x7f4b6c0b4d00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4b88076320 tx=0x7f4b7000b390 comp rx=0 tx=0).stop 2024-05-09T04:58:26.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:25.182+0000 7f4b87200640 1 -- 172.21.15.6:0/1026281952 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b881039e0 msgr2=0x7f4b88076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:26.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:25.182+0000 7f4b87200640 1 --2- 172.21.15.6:0/1026281952 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b881039e0 0x7f4b88076910 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4b7802ade0 tx=0x7f4b780316e0 comp rx=0 tx=0).stop 2024-05-09T04:58:26.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:25.182+0000 7f4b87200640 1 -- 172.21.15.6:0/1026281952 shutdown_connections 2024-05-09T04:58:26.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:25.182+0000 7f4b87200640 1 --2- 172.21.15.6:0/1026281952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b88109920 0x7f4b88075560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:26.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:25.182+0000 7f4b87200640 1 --2- 172.21.15.6:0/1026281952 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f4b6c0b27f0 0x7f4b6c0b4d00 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:26.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:25.182+0000 7f4b87200640 1 --2- 172.21.15.6:0/1026281952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b88104c20 0x7f4b88075020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:26.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:25.182+0000 7f4b87200640 1 --2- 172.21.15.6:0/1026281952 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b881039e0 0x7f4b88076910 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:26.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:25.182+0000 7f4b87200640 1 -- 172.21.15.6:0/1026281952 >> 172.21.15.6:0/1026281952 conn(0x7f4b880febb0 msgr2=0x7f4b88105900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:26.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:25.182+0000 7f4b87200640 1 -- 172.21.15.6:0/1026281952 shutdown_connections 2024-05-09T04:58:26.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:58:25.182+0000 7f4b87200640 1 -- 172.21.15.6:0/1026281952 wait complete. 2024-05-09T04:58:26.798 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:25 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/1026281952' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-09T04:58:26.798 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:25 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/1026281952' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-09T04:58:26.804 DEBUG:tasks.check_counter:Getting stats from a 2024-05-09T04:58:26.805 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-09T04:58:26.805 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:25 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1026281952' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-09T04:58:27.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:26 smithi006 ceph-mon[29835]: pgmap v604: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 13 op/s 2024-05-09T04:58:27.277 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:58:27.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:26 smithi186 ceph-mon[34668]: pgmap v604: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 13 op/s 2024-05-09T04:58:27.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:26 smithi169 ceph-mon[34075]: pgmap v604: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 13 op/s 2024-05-09T04:58:29.133 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:28 smithi006 ceph-mon[29835]: pgmap v605: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-05-09T04:58:29.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:28 smithi186 ceph-mon[34668]: pgmap v605: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-05-09T04:58:29.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:28 smithi169 ceph-mon[34075]: pgmap v605: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-05-09T04:58:29.909 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-09T04:58:29.909 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T04:58:29.909 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 0, 2024-05-09T04:58:29.910 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 2, 2024-05-09T04:58:29.910 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 0, 2024-05-09T04:58:29.910 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 0, 2024-05-09T04:58:29.910 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 4, 2024-05-09T04:58:29.910 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 0, 2024-05-09T04:58:29.910 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.004664543, 2024-05-09T04:58:29.910 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.000000000, 2024-05-09T04:58:29.910 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.001159499, 2024-05-09T04:58:29.910 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T04:58:29.910 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:58:29.910 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:29.910 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:29.910 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:29.910 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:29.911 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:58:29.911 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:29.911 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:29.911 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:29.911 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:29.911 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T04:58:29.911 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T04:58:29.911 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:29.911 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T04:58:29.911 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 157, 2024-05-09T04:58:29.911 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 152, 2024-05-09T04:58:29.911 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 795232, 2024-05-09T04:58:29.911 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 92848, 2024-05-09T04:58:29.911 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 2, 2024-05-09T04:58:29.911 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 1, 2024-05-09T04:58:29.912 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.065597629, 2024-05-09T04:58:29.912 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.024170788, 2024-05-09T04:58:29.912 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.047218132, 2024-05-09T04:58:29.912 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.006114924, 2024-05-09T04:58:29.912 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:58:29.912 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 151, 2024-05-09T04:58:29.912 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.013857078, 2024-05-09T04:58:29.912 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000091768 2024-05-09T04:58:29.912 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:29.912 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:58:29.912 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:29.912 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:29.912 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:29.912 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:29.912 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 795232, 2024-05-09T04:58:29.912 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 92848 2024-05-09T04:58:29.913 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:29.913 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T04:58:29.913 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 5, 2024-05-09T04:58:29.913 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 117, 2024-05-09T04:58:29.913 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 3200, 2024-05-09T04:58:29.913 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 99328, 2024-05-09T04:58:29.913 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 3, 2024-05-09T04:58:29.913 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 2, 2024-05-09T04:58:29.913 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.024128269, 2024-05-09T04:58:29.913 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.018451697, 2024-05-09T04:58:29.913 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.002870493, 2024-05-09T04:58:29.913 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T04:58:29.913 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:58:29.913 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 116, 2024-05-09T04:58:29.913 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.012975386, 2024-05-09T04:58:29.914 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000111856 2024-05-09T04:58:29.914 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:29.914 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:58:29.914 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:29.914 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:29.914 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:29.914 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:29.914 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-05-09T04:58:29.914 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 99328 2024-05-09T04:58:29.914 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:29.914 INFO:teuthology.orchestra.run.smithi006.stdout: "cct": { 2024-05-09T04:58:29.914 INFO:teuthology.orchestra.run.smithi006.stdout: "total_workers": 0, 2024-05-09T04:58:29.914 INFO:teuthology.orchestra.run.smithi006.stdout: "unhealthy_workers": 0 2024-05-09T04:58:29.915 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:29.915 INFO:teuthology.orchestra.run.smithi006.stdout: "mempool": { 2024-05-09T04:58:29.915 INFO:teuthology.orchestra.run.smithi006.stdout: "bloom_filter_bytes": 0, 2024-05-09T04:58:29.915 INFO:teuthology.orchestra.run.smithi006.stdout: "bloom_filter_items": 0, 2024-05-09T04:58:29.915 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T04:58:29.915 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_alloc_items": 0, 2024-05-09T04:58:29.915 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T04:58:29.915 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_data_items": 0, 2024-05-09T04:58:29.915 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T04:58:29.915 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T04:58:29.915 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T04:58:29.915 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T04:58:29.915 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T04:58:29.915 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_other_items": 0, 2024-05-09T04:58:29.915 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T04:58:29.915 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T04:58:29.916 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_bytes": 0, 2024-05-09T04:58:29.916 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_items": 0, 2024-05-09T04:58:29.916 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_blob_bytes": 0, 2024-05-09T04:58:29.916 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_blob_items": 0, 2024-05-09T04:58:29.916 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T04:58:29.916 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T04:58:29.916 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T04:58:29.916 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T04:58:29.916 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T04:58:29.916 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_items": 0, 2024-05-09T04:58:29.916 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_txc_bytes": 0, 2024-05-09T04:58:29.916 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_txc_items": 0, 2024-05-09T04:58:29.916 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T04:58:29.916 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T04:58:29.916 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_bytes": 0, 2024-05-09T04:58:29.916 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_items": 0, 2024-05-09T04:58:29.917 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_bytes": 0, 2024-05-09T04:58:29.917 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_items": 0, 2024-05-09T04:58:29.917 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T04:58:29.917 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_reader_items": 0, 2024-05-09T04:58:29.917 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T04:58:29.917 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_writer_items": 0, 2024-05-09T04:58:29.917 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_anon_bytes": 149998, 2024-05-09T04:58:29.917 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_anon_items": 30, 2024-05-09T04:58:29.917 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_meta_bytes": 0, 2024-05-09T04:58:29.917 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_meta_items": 0, 2024-05-09T04:58:29.917 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_bytes": 0, 2024-05-09T04:58:29.917 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_items": 0, 2024-05-09T04:58:29.917 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_mapbl_bytes": 0, 2024-05-09T04:58:29.917 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_mapbl_items": 0, 2024-05-09T04:58:29.917 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_pglog_bytes": 0, 2024-05-09T04:58:29.918 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_pglog_items": 0, 2024-05-09T04:58:29.918 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_bytes": 0, 2024-05-09T04:58:29.918 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_items": 0, 2024-05-09T04:58:29.918 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T04:58:29.918 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_mapping_items": 0, 2024-05-09T04:58:29.918 INFO:teuthology.orchestra.run.smithi006.stdout: "pgmap_bytes": 0, 2024-05-09T04:58:29.918 INFO:teuthology.orchestra.run.smithi006.stdout: "pgmap_items": 0, 2024-05-09T04:58:29.918 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_co_bytes": 928, 2024-05-09T04:58:29.918 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_co_items": 2, 2024-05-09T04:58:29.918 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_1_bytes": 0, 2024-05-09T04:58:29.918 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_1_items": 0, 2024-05-09T04:58:29.918 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_2_bytes": 0, 2024-05-09T04:58:29.918 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_2_items": 0 2024-05-09T04:58:29.918 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:29.918 INFO:teuthology.orchestra.run.smithi006.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T04:58:29.919 INFO:teuthology.orchestra.run.smithi006.stdout: "val": 0, 2024-05-09T04:58:29.919 INFO:teuthology.orchestra.run.smithi006.stdout: "max": 104857600, 2024-05-09T04:58:29.919 INFO:teuthology.orchestra.run.smithi006.stdout: "get_started": 0, 2024-05-09T04:58:29.919 INFO:teuthology.orchestra.run.smithi006.stdout: "get": 158, 2024-05-09T04:58:29.919 INFO:teuthology.orchestra.run.smithi006.stdout: "get_sum": 773061, 2024-05-09T04:58:29.919 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:29.919 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_success": 158, 2024-05-09T04:58:29.919 INFO:teuthology.orchestra.run.smithi006.stdout: "take": 0, 2024-05-09T04:58:29.919 INFO:teuthology.orchestra.run.smithi006.stdout: "take_sum": 0, 2024-05-09T04:58:29.919 INFO:teuthology.orchestra.run.smithi006.stdout: "put": 158, 2024-05-09T04:58:29.919 INFO:teuthology.orchestra.run.smithi006.stdout: "put_sum": 773061, 2024-05-09T04:58:29.919 INFO:teuthology.orchestra.run.smithi006.stdout: "wait": { 2024-05-09T04:58:29.919 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:29.919 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:29.919 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:29.920 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:29.920 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:29.920 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-09T04:58:30.905 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-05-09T04:58:30.905 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-05-09T04:58:30.905 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-05-09T04:58:30.906 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-05-09T04:58:30.906 DEBUG:tasks.check_counter:Getting stats from d 2024-05-09T04:58:30.906 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-09T04:58:31.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:30 smithi006 ceph-mon[29835]: pgmap v606: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-05-09T04:58:31.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:30 smithi186 ceph-mon[34668]: pgmap v606: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-05-09T04:58:31.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:30 smithi169 ceph-mon[34075]: pgmap v606: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-05-09T04:58:31.375 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:58:33.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:32 smithi006 ceph-mon[29835]: pgmap v607: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-05-09T04:58:33.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:32 smithi186 ceph-mon[34668]: pgmap v607: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-05-09T04:58:33.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:32 smithi169 ceph-mon[34075]: pgmap v607: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-05-09T04:58:33.844 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-09T04:58:33.844 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T04:58:33.844 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 456, 2024-05-09T04:58:33.844 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 458, 2024-05-09T04:58:33.844 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 120889, 2024-05-09T04:58:33.844 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 145071, 2024-05-09T04:58:33.844 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 6, 2024-05-09T04:58:33.845 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 2, 2024-05-09T04:58:33.845 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.092284908, 2024-05-09T04:58:33.845 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.032000555, 2024-05-09T04:58:33.845 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.024557760, 2024-05-09T04:58:33.845 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.023157205, 2024-05-09T04:58:33.845 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:58:33.845 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 456, 2024-05-09T04:58:33.845 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.037076071, 2024-05-09T04:58:33.845 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000081307 2024-05-09T04:58:33.845 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.845 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:58:33.845 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.845 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.846 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.846 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.846 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T04:58:33.846 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T04:58:33.846 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.846 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T04:58:33.846 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 3, 2024-05-09T04:58:33.846 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 123, 2024-05-09T04:58:33.846 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 1262, 2024-05-09T04:58:33.846 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 255298, 2024-05-09T04:58:33.846 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 5, 2024-05-09T04:58:33.846 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 4, 2024-05-09T04:58:33.846 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.024868082, 2024-05-09T04:58:33.846 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.018178430, 2024-05-09T04:58:33.847 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.003978653, 2024-05-09T04:58:33.847 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.000046944, 2024-05-09T04:58:33.847 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:58:33.847 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 121, 2024-05-09T04:58:33.847 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.013178599, 2024-05-09T04:58:33.847 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000108914 2024-05-09T04:58:33.847 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.847 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:58:33.847 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.847 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.847 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.847 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.847 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T04:58:33.847 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 254656 2024-05-09T04:58:33.848 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.848 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T04:58:33.848 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 961, 2024-05-09T04:58:33.848 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 952, 2024-05-09T04:58:33.848 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 1063894, 2024-05-09T04:58:33.848 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 348556, 2024-05-09T04:58:33.848 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 4, 2024-05-09T04:58:33.848 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 3, 2024-05-09T04:58:33.848 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.267519331, 2024-05-09T04:58:33.848 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.103943222, 2024-05-09T04:58:33.848 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.110884776, 2024-05-09T04:58:33.848 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.055792881, 2024-05-09T04:58:33.848 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:58:33.848 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 952, 2024-05-09T04:58:33.849 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.088197079, 2024-05-09T04:58:33.849 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000092643 2024-05-09T04:58:33.849 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.849 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:58:33.849 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.849 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.849 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.849 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.849 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 830128, 2024-05-09T04:58:33.849 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 98608 2024-05-09T04:58:33.849 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.849 INFO:teuthology.orchestra.run.smithi006.stdout: "cct": { 2024-05-09T04:58:33.849 INFO:teuthology.orchestra.run.smithi006.stdout: "total_workers": 1, 2024-05-09T04:58:33.850 INFO:teuthology.orchestra.run.smithi006.stdout: "unhealthy_workers": 0 2024-05-09T04:58:33.850 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.850 INFO:teuthology.orchestra.run.smithi006.stdout: "finisher-MDSRank": { 2024-05-09T04:58:33.850 INFO:teuthology.orchestra.run.smithi006.stdout: "queue_len": 0, 2024-05-09T04:58:33.850 INFO:teuthology.orchestra.run.smithi006.stdout: "complete_latency": { 2024-05-09T04:58:33.850 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 2114, 2024-05-09T04:58:33.850 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.111814255, 2024-05-09T04:58:33.850 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000052892 2024-05-09T04:58:33.850 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:33.850 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.850 INFO:teuthology.orchestra.run.smithi006.stdout: "finisher-PurgeQueue": { 2024-05-09T04:58:33.850 INFO:teuthology.orchestra.run.smithi006.stdout: "queue_len": 0, 2024-05-09T04:58:33.850 INFO:teuthology.orchestra.run.smithi006.stdout: "complete_latency": { 2024-05-09T04:58:33.851 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.851 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.851 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.851 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:33.851 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.851 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": { 2024-05-09T04:58:33.851 INFO:teuthology.orchestra.run.smithi006.stdout: "request": 0, 2024-05-09T04:58:33.851 INFO:teuthology.orchestra.run.smithi006.stdout: "reply": 0, 2024-05-09T04:58:33.851 INFO:teuthology.orchestra.run.smithi006.stdout: "reply_latency": { 2024-05-09T04:58:33.851 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.851 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.851 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.851 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.851 INFO:teuthology.orchestra.run.smithi006.stdout: "slow_reply": 0, 2024-05-09T04:58:33.852 INFO:teuthology.orchestra.run.smithi006.stdout: "forward": 0, 2024-05-09T04:58:33.852 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_fetch_complete": 0, 2024-05-09T04:58:33.852 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_fetch_keys": 0, 2024-05-09T04:58:33.852 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_commit": 0, 2024-05-09T04:58:33.852 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_split": 0, 2024-05-09T04:58:33.852 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_merge": 0, 2024-05-09T04:58:33.852 INFO:teuthology.orchestra.run.smithi006.stdout: "inodes": 0, 2024-05-09T04:58:33.852 INFO:teuthology.orchestra.run.smithi006.stdout: "inodes_top": 0, 2024-05-09T04:58:33.852 INFO:teuthology.orchestra.run.smithi006.stdout: "inodes_bottom": 0, 2024-05-09T04:58:33.852 INFO:teuthology.orchestra.run.smithi006.stdout: "inodes_pin_tail": 0, 2024-05-09T04:58:33.852 INFO:teuthology.orchestra.run.smithi006.stdout: "inodes_pinned": 0, 2024-05-09T04:58:33.852 INFO:teuthology.orchestra.run.smithi006.stdout: "inodes_expired": 0, 2024-05-09T04:58:33.852 INFO:teuthology.orchestra.run.smithi006.stdout: "inodes_with_caps": 0, 2024-05-09T04:58:33.852 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": 0, 2024-05-09T04:58:33.852 INFO:teuthology.orchestra.run.smithi006.stdout: "subtrees": 1, 2024-05-09T04:58:33.853 INFO:teuthology.orchestra.run.smithi006.stdout: "traverse": 0, 2024-05-09T04:58:33.853 INFO:teuthology.orchestra.run.smithi006.stdout: "traverse_hit": 0, 2024-05-09T04:58:33.853 INFO:teuthology.orchestra.run.smithi006.stdout: "traverse_forward": 0, 2024-05-09T04:58:33.853 INFO:teuthology.orchestra.run.smithi006.stdout: "traverse_discover": 0, 2024-05-09T04:58:33.853 INFO:teuthology.orchestra.run.smithi006.stdout: "traverse_dir_fetch": 0, 2024-05-09T04:58:33.853 INFO:teuthology.orchestra.run.smithi006.stdout: "traverse_remote_ino": 0, 2024-05-09T04:58:33.853 INFO:teuthology.orchestra.run.smithi006.stdout: "traverse_lock": 0, 2024-05-09T04:58:33.853 INFO:teuthology.orchestra.run.smithi006.stdout: "load_cent": 0, 2024-05-09T04:58:33.853 INFO:teuthology.orchestra.run.smithi006.stdout: "q": 0, 2024-05-09T04:58:33.853 INFO:teuthology.orchestra.run.smithi006.stdout: "exported": 0, 2024-05-09T04:58:33.853 INFO:teuthology.orchestra.run.smithi006.stdout: "exported_inodes": 0, 2024-05-09T04:58:33.853 INFO:teuthology.orchestra.run.smithi006.stdout: "imported": 0, 2024-05-09T04:58:33.853 INFO:teuthology.orchestra.run.smithi006.stdout: "imported_inodes": 0, 2024-05-09T04:58:33.853 INFO:teuthology.orchestra.run.smithi006.stdout: "openino_dir_fetch": 0, 2024-05-09T04:58:33.854 INFO:teuthology.orchestra.run.smithi006.stdout: "openino_backtrace_fetch": 0, 2024-05-09T04:58:33.854 INFO:teuthology.orchestra.run.smithi006.stdout: "openino_peer_discover": 0, 2024-05-09T04:58:33.854 INFO:teuthology.orchestra.run.smithi006.stdout: "root_rfiles": 0, 2024-05-09T04:58:33.854 INFO:teuthology.orchestra.run.smithi006.stdout: "root_rbytes": 0, 2024-05-09T04:58:33.854 INFO:teuthology.orchestra.run.smithi006.stdout: "root_rsnaps": 0, 2024-05-09T04:58:33.854 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T04:58:33.854 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_set_tag": 0, 2024-05-09T04:58:33.854 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T04:58:33.854 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_inotable_repaired": 0, 2024-05-09T04:58:33.854 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_dir_inodes": 0, 2024-05-09T04:58:33.854 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T04:58:33.854 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T04:58:33.854 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_file_inodes": 0, 2024-05-09T04:58:33.855 INFO:teuthology.orchestra.run.smithi006.stdout: "handle_inode_file_caps": 0, 2024-05-09T04:58:33.855 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T04:58:33.855 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_cap_op_grant": 0, 2024-05-09T04:58:33.855 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T04:58:33.855 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T04:58:33.855 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T04:58:33.855 INFO:teuthology.orchestra.run.smithi006.stdout: "handle_client_caps": 0, 2024-05-09T04:58:33.855 INFO:teuthology.orchestra.run.smithi006.stdout: "handle_client_caps_dirty": 0, 2024-05-09T04:58:33.855 INFO:teuthology.orchestra.run.smithi006.stdout: "handle_client_cap_release": 0, 2024-05-09T04:58:33.855 INFO:teuthology.orchestra.run.smithi006.stdout: "process_request_cap_release": 0 2024-05-09T04:58:33.855 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.855 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_cache": { 2024-05-09T04:58:33.855 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_update": 0, 2024-05-09T04:58:33.855 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_update_receipt": 0, 2024-05-09T04:58:33.855 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_try_discover": 0, 2024-05-09T04:58:33.856 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_send_discover": 0, 2024-05-09T04:58:33.856 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_handle_discover": 0, 2024-05-09T04:58:33.856 INFO:teuthology.orchestra.run.smithi006.stdout: "num_strays": 0, 2024-05-09T04:58:33.856 INFO:teuthology.orchestra.run.smithi006.stdout: "num_strays_delayed": 0, 2024-05-09T04:58:33.856 INFO:teuthology.orchestra.run.smithi006.stdout: "num_strays_enqueuing": 0, 2024-05-09T04:58:33.856 INFO:teuthology.orchestra.run.smithi006.stdout: "strays_created": 0, 2024-05-09T04:58:33.856 INFO:teuthology.orchestra.run.smithi006.stdout: "strays_enqueued": 0, 2024-05-09T04:58:33.856 INFO:teuthology.orchestra.run.smithi006.stdout: "strays_reintegrated": 0, 2024-05-09T04:58:33.856 INFO:teuthology.orchestra.run.smithi006.stdout: "strays_migrated": 0, 2024-05-09T04:58:33.856 INFO:teuthology.orchestra.run.smithi006.stdout: "num_recovering_processing": 0, 2024-05-09T04:58:33.856 INFO:teuthology.orchestra.run.smithi006.stdout: "num_recovering_enqueued": 0, 2024-05-09T04:58:33.856 INFO:teuthology.orchestra.run.smithi006.stdout: "num_recovering_prioritized": 0, 2024-05-09T04:58:33.856 INFO:teuthology.orchestra.run.smithi006.stdout: "recovery_started": 0, 2024-05-09T04:58:33.856 INFO:teuthology.orchestra.run.smithi006.stdout: "recovery_completed": 0, 2024-05-09T04:58:33.856 INFO:teuthology.orchestra.run.smithi006.stdout: "ireq_quiesce_path": 0, 2024-05-09T04:58:33.857 INFO:teuthology.orchestra.run.smithi006.stdout: "ireq_quiesce_inode": 0, 2024-05-09T04:58:33.857 INFO:teuthology.orchestra.run.smithi006.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T04:58:33.857 INFO:teuthology.orchestra.run.smithi006.stdout: "ireq_exportdir": 0, 2024-05-09T04:58:33.857 INFO:teuthology.orchestra.run.smithi006.stdout: "ireq_flush": 0, 2024-05-09T04:58:33.857 INFO:teuthology.orchestra.run.smithi006.stdout: "ireq_fragmentdir": 0, 2024-05-09T04:58:33.857 INFO:teuthology.orchestra.run.smithi006.stdout: "ireq_fragstats": 0, 2024-05-09T04:58:33.857 INFO:teuthology.orchestra.run.smithi006.stdout: "ireq_inodestats": 0 2024-05-09T04:58:33.857 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.857 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_log": { 2024-05-09T04:58:33.857 INFO:teuthology.orchestra.run.smithi006.stdout: "evlrg": 0, 2024-05-09T04:58:33.857 INFO:teuthology.orchestra.run.smithi006.stdout: "evadd": 0, 2024-05-09T04:58:33.857 INFO:teuthology.orchestra.run.smithi006.stdout: "evex": 0, 2024-05-09T04:58:33.857 INFO:teuthology.orchestra.run.smithi006.stdout: "evtrm": 0, 2024-05-09T04:58:33.857 INFO:teuthology.orchestra.run.smithi006.stdout: "ev": 5, 2024-05-09T04:58:33.858 INFO:teuthology.orchestra.run.smithi006.stdout: "evexg": 0, 2024-05-09T04:58:33.858 INFO:teuthology.orchestra.run.smithi006.stdout: "evexd": 0, 2024-05-09T04:58:33.858 INFO:teuthology.orchestra.run.smithi006.stdout: "segadd": 0, 2024-05-09T04:58:33.858 INFO:teuthology.orchestra.run.smithi006.stdout: "segex": 0, 2024-05-09T04:58:33.858 INFO:teuthology.orchestra.run.smithi006.stdout: "segtrm": 0, 2024-05-09T04:58:33.858 INFO:teuthology.orchestra.run.smithi006.stdout: "seg": 2, 2024-05-09T04:58:33.858 INFO:teuthology.orchestra.run.smithi006.stdout: "segmjr": 2, 2024-05-09T04:58:33.858 INFO:teuthology.orchestra.run.smithi006.stdout: "segexg": 0, 2024-05-09T04:58:33.858 INFO:teuthology.orchestra.run.smithi006.stdout: "segexd": 0, 2024-05-09T04:58:33.858 INFO:teuthology.orchestra.run.smithi006.stdout: "expos": 4194304, 2024-05-09T04:58:33.858 INFO:teuthology.orchestra.run.smithi006.stdout: "wrpos": 4195734, 2024-05-09T04:58:33.858 INFO:teuthology.orchestra.run.smithi006.stdout: "rdpos": 4195424, 2024-05-09T04:58:33.858 INFO:teuthology.orchestra.run.smithi006.stdout: "jlat": { 2024-05-09T04:58:33.858 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.859 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.859 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.859 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.859 INFO:teuthology.orchestra.run.smithi006.stdout: "replayed": 5 2024-05-09T04:58:33.859 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.859 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_mem": { 2024-05-09T04:58:33.859 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 3, 2024-05-09T04:58:33.859 INFO:teuthology.orchestra.run.smithi006.stdout: "ino+": 3, 2024-05-09T04:58:33.859 INFO:teuthology.orchestra.run.smithi006.stdout: "ino-": 0, 2024-05-09T04:58:33.859 INFO:teuthology.orchestra.run.smithi006.stdout: "dir": 1, 2024-05-09T04:58:33.859 INFO:teuthology.orchestra.run.smithi006.stdout: "dir+": 1, 2024-05-09T04:58:33.859 INFO:teuthology.orchestra.run.smithi006.stdout: "dir-": 0, 2024-05-09T04:58:33.859 INFO:teuthology.orchestra.run.smithi006.stdout: "dn": 0, 2024-05-09T04:58:33.859 INFO:teuthology.orchestra.run.smithi006.stdout: "dn+": 0, 2024-05-09T04:58:33.860 INFO:teuthology.orchestra.run.smithi006.stdout: "dn-": 0, 2024-05-09T04:58:33.860 INFO:teuthology.orchestra.run.smithi006.stdout: "cap": 0, 2024-05-09T04:58:33.860 INFO:teuthology.orchestra.run.smithi006.stdout: "cap+": 0, 2024-05-09T04:58:33.860 INFO:teuthology.orchestra.run.smithi006.stdout: "cap-": 0, 2024-05-09T04:58:33.860 INFO:teuthology.orchestra.run.smithi006.stdout: "rss": 117232, 2024-05-09T04:58:33.860 INFO:teuthology.orchestra.run.smithi006.stdout: "heap": 215440 2024-05-09T04:58:33.860 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.860 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_server": { 2024-05-09T04:58:33.860 INFO:teuthology.orchestra.run.smithi006.stdout: "dispatch_client_request": 0, 2024-05-09T04:58:33.860 INFO:teuthology.orchestra.run.smithi006.stdout: "dispatch_server_request": 0, 2024-05-09T04:58:33.860 INFO:teuthology.orchestra.run.smithi006.stdout: "handle_client_request": 0, 2024-05-09T04:58:33.860 INFO:teuthology.orchestra.run.smithi006.stdout: "handle_client_session": 0, 2024-05-09T04:58:33.860 INFO:teuthology.orchestra.run.smithi006.stdout: "handle_peer_request": 0, 2024-05-09T04:58:33.860 INFO:teuthology.orchestra.run.smithi006.stdout: "req_create_latency": { 2024-05-09T04:58:33.860 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.861 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.861 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.861 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.861 INFO:teuthology.orchestra.run.smithi006.stdout: "req_getattr_latency": { 2024-05-09T04:58:33.861 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.861 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.861 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.861 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.861 INFO:teuthology.orchestra.run.smithi006.stdout: "req_getfilelock_latency": { 2024-05-09T04:58:33.861 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.861 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.861 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.861 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.861 INFO:teuthology.orchestra.run.smithi006.stdout: "req_link_latency": { 2024-05-09T04:58:33.861 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.862 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.862 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.862 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.862 INFO:teuthology.orchestra.run.smithi006.stdout: "req_lookup_latency": { 2024-05-09T04:58:33.862 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.862 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.862 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.862 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.862 INFO:teuthology.orchestra.run.smithi006.stdout: "req_lookuphash_latency": { 2024-05-09T04:58:33.862 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.862 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.862 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.862 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.862 INFO:teuthology.orchestra.run.smithi006.stdout: "req_lookupino_latency": { 2024-05-09T04:58:33.863 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.863 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.863 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.863 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.863 INFO:teuthology.orchestra.run.smithi006.stdout: "req_lookupname_latency": { 2024-05-09T04:58:33.863 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.863 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.863 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.863 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.863 INFO:teuthology.orchestra.run.smithi006.stdout: "req_lookupparent_latency": { 2024-05-09T04:58:33.863 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.863 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.863 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.863 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.864 INFO:teuthology.orchestra.run.smithi006.stdout: "req_lookupsnap_latency": { 2024-05-09T04:58:33.864 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.864 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.864 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.864 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.864 INFO:teuthology.orchestra.run.smithi006.stdout: "req_lssnap_latency": { 2024-05-09T04:58:33.864 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.864 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.864 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.864 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.864 INFO:teuthology.orchestra.run.smithi006.stdout: "req_mkdir_latency": { 2024-05-09T04:58:33.864 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.865 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.865 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.865 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.865 INFO:teuthology.orchestra.run.smithi006.stdout: "req_mknod_latency": { 2024-05-09T04:58:33.865 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.865 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.865 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.865 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.866 INFO:teuthology.orchestra.run.smithi006.stdout: "req_mksnap_latency": { 2024-05-09T04:58:33.866 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.866 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.866 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.866 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.866 INFO:teuthology.orchestra.run.smithi006.stdout: "req_open_latency": { 2024-05-09T04:58:33.866 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.866 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.866 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.866 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.866 INFO:teuthology.orchestra.run.smithi006.stdout: "req_readdir_latency": { 2024-05-09T04:58:33.866 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.866 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.867 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.867 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.867 INFO:teuthology.orchestra.run.smithi006.stdout: "req_rename_latency": { 2024-05-09T04:58:33.867 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.867 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.867 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.867 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.867 INFO:teuthology.orchestra.run.smithi006.stdout: "req_renamesnap_latency": { 2024-05-09T04:58:33.867 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.867 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.867 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.867 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.867 INFO:teuthology.orchestra.run.smithi006.stdout: "req_snapdiff_latency": { 2024-05-09T04:58:33.868 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.868 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.868 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.868 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.868 INFO:teuthology.orchestra.run.smithi006.stdout: "req_rmdir_latency": { 2024-05-09T04:58:33.868 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.868 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.868 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.868 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.868 INFO:teuthology.orchestra.run.smithi006.stdout: "req_rmsnap_latency": { 2024-05-09T04:58:33.868 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.868 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.868 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.868 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.869 INFO:teuthology.orchestra.run.smithi006.stdout: "req_rmxattr_latency": { 2024-05-09T04:58:33.869 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.869 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.869 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.869 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.869 INFO:teuthology.orchestra.run.smithi006.stdout: "req_setattr_latency": { 2024-05-09T04:58:33.869 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.869 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.869 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.869 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.869 INFO:teuthology.orchestra.run.smithi006.stdout: "req_setdirlayout_latency": { 2024-05-09T04:58:33.869 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.869 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.869 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.870 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.870 INFO:teuthology.orchestra.run.smithi006.stdout: "req_setfilelock_latency": { 2024-05-09T04:58:33.870 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.870 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.870 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.870 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.870 INFO:teuthology.orchestra.run.smithi006.stdout: "req_setlayout_latency": { 2024-05-09T04:58:33.870 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.870 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.870 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.870 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.870 INFO:teuthology.orchestra.run.smithi006.stdout: "req_setxattr_latency": { 2024-05-09T04:58:33.870 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.871 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.871 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.871 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.871 INFO:teuthology.orchestra.run.smithi006.stdout: "req_symlink_latency": { 2024-05-09T04:58:33.871 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.871 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.871 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.871 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.871 INFO:teuthology.orchestra.run.smithi006.stdout: "req_unlink_latency": { 2024-05-09T04:58:33.871 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.871 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.871 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.871 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.871 INFO:teuthology.orchestra.run.smithi006.stdout: "cap_revoke_eviction": 0, 2024-05-09T04:58:33.872 INFO:teuthology.orchestra.run.smithi006.stdout: "cap_acquisition_throttle": 0, 2024-05-09T04:58:33.872 INFO:teuthology.orchestra.run.smithi006.stdout: "req_getvxattr_latency": { 2024-05-09T04:58:33.872 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.872 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.872 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.872 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:33.872 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.872 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_sessions": { 2024-05-09T04:58:33.872 INFO:teuthology.orchestra.run.smithi006.stdout: "session_count": 1, 2024-05-09T04:58:33.872 INFO:teuthology.orchestra.run.smithi006.stdout: "session_add": 2, 2024-05-09T04:58:33.872 INFO:teuthology.orchestra.run.smithi006.stdout: "session_remove": 1, 2024-05-09T04:58:33.872 INFO:teuthology.orchestra.run.smithi006.stdout: "sessions_open": 1, 2024-05-09T04:58:33.872 INFO:teuthology.orchestra.run.smithi006.stdout: "sessions_stale": 0, 2024-05-09T04:58:33.873 INFO:teuthology.orchestra.run.smithi006.stdout: "total_load": 0, 2024-05-09T04:58:33.873 INFO:teuthology.orchestra.run.smithi006.stdout: "average_load": 0, 2024-05-09T04:58:33.873 INFO:teuthology.orchestra.run.smithi006.stdout: "avg_session_uptime": 1767, 2024-05-09T04:58:33.873 INFO:teuthology.orchestra.run.smithi006.stdout: "mdthresh_evicted": 0 2024-05-09T04:58:33.873 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.873 INFO:teuthology.orchestra.run.smithi006.stdout: "mempool": { 2024-05-09T04:58:33.873 INFO:teuthology.orchestra.run.smithi006.stdout: "bloom_filter_bytes": 0, 2024-05-09T04:58:33.873 INFO:teuthology.orchestra.run.smithi006.stdout: "bloom_filter_items": 0, 2024-05-09T04:58:33.873 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T04:58:33.873 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_alloc_items": 0, 2024-05-09T04:58:33.873 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T04:58:33.873 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_data_items": 0, 2024-05-09T04:58:33.873 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T04:58:33.873 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T04:58:33.873 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T04:58:33.874 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T04:58:33.874 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T04:58:33.874 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_other_items": 0, 2024-05-09T04:58:33.874 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T04:58:33.874 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T04:58:33.874 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_bytes": 0, 2024-05-09T04:58:33.874 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_items": 0, 2024-05-09T04:58:33.874 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_blob_bytes": 0, 2024-05-09T04:58:33.874 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_blob_items": 0, 2024-05-09T04:58:33.874 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T04:58:33.874 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T04:58:33.874 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T04:58:33.874 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T04:58:33.874 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T04:58:33.875 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_items": 0, 2024-05-09T04:58:33.875 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_txc_bytes": 0, 2024-05-09T04:58:33.875 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_txc_items": 0, 2024-05-09T04:58:33.875 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T04:58:33.875 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T04:58:33.875 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_bytes": 0, 2024-05-09T04:58:33.875 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_items": 0, 2024-05-09T04:58:33.875 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_bytes": 0, 2024-05-09T04:58:33.875 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_items": 0, 2024-05-09T04:58:33.875 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T04:58:33.875 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_reader_items": 0, 2024-05-09T04:58:33.875 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T04:58:33.875 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_writer_items": 0, 2024-05-09T04:58:33.875 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_anon_bytes": 198264, 2024-05-09T04:58:33.876 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_anon_items": 57, 2024-05-09T04:58:33.876 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_meta_bytes": 0, 2024-05-09T04:58:33.876 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_meta_items": 0, 2024-05-09T04:58:33.876 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_bytes": 0, 2024-05-09T04:58:33.876 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_items": 0, 2024-05-09T04:58:33.876 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_mapbl_bytes": 0, 2024-05-09T04:58:33.876 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_mapbl_items": 0, 2024-05-09T04:58:33.876 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_pglog_bytes": 0, 2024-05-09T04:58:33.876 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_pglog_items": 0, 2024-05-09T04:58:33.876 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_bytes": 8800, 2024-05-09T04:58:33.876 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_items": 169, 2024-05-09T04:58:33.876 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T04:58:33.876 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_mapping_items": 0, 2024-05-09T04:58:33.876 INFO:teuthology.orchestra.run.smithi006.stdout: "pgmap_bytes": 0, 2024-05-09T04:58:33.876 INFO:teuthology.orchestra.run.smithi006.stdout: "pgmap_items": 0, 2024-05-09T04:58:33.877 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_co_bytes": 8512, 2024-05-09T04:58:33.877 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_co_items": 13, 2024-05-09T04:58:33.877 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_1_bytes": 0, 2024-05-09T04:58:33.877 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_1_items": 0, 2024-05-09T04:58:33.877 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_2_bytes": 0, 2024-05-09T04:58:33.877 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_2_items": 0 2024-05-09T04:58:33.877 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.877 INFO:teuthology.orchestra.run.smithi006.stdout: "objecter": { 2024-05-09T04:58:33.877 INFO:teuthology.orchestra.run.smithi006.stdout: "op_active": 0, 2024-05-09T04:58:33.877 INFO:teuthology.orchestra.run.smithi006.stdout: "op_laggy": 0, 2024-05-09T04:58:33.877 INFO:teuthology.orchestra.run.smithi006.stdout: "op_send": 1244, 2024-05-09T04:58:33.877 INFO:teuthology.orchestra.run.smithi006.stdout: "op_send_bytes": 21, 2024-05-09T04:58:33.877 INFO:teuthology.orchestra.run.smithi006.stdout: "op_resend": 3, 2024-05-09T04:58:33.877 INFO:teuthology.orchestra.run.smithi006.stdout: "op_reply": 1244, 2024-05-09T04:58:33.877 INFO:teuthology.orchestra.run.smithi006.stdout: "op_latency": { 2024-05-09T04:58:33.878 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 1244, 2024-05-09T04:58:33.878 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 1.128986887, 2024-05-09T04:58:33.878 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000907545 2024-05-09T04:58:33.878 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.878 INFO:teuthology.orchestra.run.smithi006.stdout: "op_inflight": 0, 2024-05-09T04:58:33.878 INFO:teuthology.orchestra.run.smithi006.stdout: "oplen_avg": { 2024-05-09T04:58:33.878 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 1244, 2024-05-09T04:58:33.878 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 1245 2024-05-09T04:58:33.878 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.878 INFO:teuthology.orchestra.run.smithi006.stdout: "op": 1244, 2024-05-09T04:58:33.878 INFO:teuthology.orchestra.run.smithi006.stdout: "op_r": 1244, 2024-05-09T04:58:33.878 INFO:teuthology.orchestra.run.smithi006.stdout: "op_w": 0, 2024-05-09T04:58:33.878 INFO:teuthology.orchestra.run.smithi006.stdout: "op_rmw": 0, 2024-05-09T04:58:33.879 INFO:teuthology.orchestra.run.smithi006.stdout: "op_pg": 0, 2024-05-09T04:58:33.879 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_stat": 786, 2024-05-09T04:58:33.879 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_create": 0, 2024-05-09T04:58:33.879 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_read": 456, 2024-05-09T04:58:33.879 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_write": 0, 2024-05-09T04:58:33.879 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_writefull": 0, 2024-05-09T04:58:33.879 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_writesame": 0, 2024-05-09T04:58:33.879 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_append": 0, 2024-05-09T04:58:33.879 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_zero": 0, 2024-05-09T04:58:33.879 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_truncate": 0, 2024-05-09T04:58:33.879 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_delete": 0, 2024-05-09T04:58:33.879 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_mapext": 0, 2024-05-09T04:58:33.879 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_sparse_read": 0, 2024-05-09T04:58:33.880 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_clonerange": 0, 2024-05-09T04:58:33.880 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_getxattr": 1, 2024-05-09T04:58:33.880 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_setxattr": 0, 2024-05-09T04:58:33.880 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_cmpxattr": 0, 2024-05-09T04:58:33.880 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_rmxattr": 0, 2024-05-09T04:58:33.880 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_resetxattrs": 0, 2024-05-09T04:58:33.880 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_call": 0, 2024-05-09T04:58:33.880 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_watch": 0, 2024-05-09T04:58:33.880 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_notify": 0, 2024-05-09T04:58:33.880 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T04:58:33.880 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_pgls": 0, 2024-05-09T04:58:33.880 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_pgls_filter": 0, 2024-05-09T04:58:33.880 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_other": 0, 2024-05-09T04:58:33.880 INFO:teuthology.orchestra.run.smithi006.stdout: "linger_active": 0, 2024-05-09T04:58:33.880 INFO:teuthology.orchestra.run.smithi006.stdout: "linger_send": 0, 2024-05-09T04:58:33.881 INFO:teuthology.orchestra.run.smithi006.stdout: "linger_resend": 0, 2024-05-09T04:58:33.881 INFO:teuthology.orchestra.run.smithi006.stdout: "linger_ping": 0, 2024-05-09T04:58:33.881 INFO:teuthology.orchestra.run.smithi006.stdout: "poolop_active": 0, 2024-05-09T04:58:33.881 INFO:teuthology.orchestra.run.smithi006.stdout: "poolop_send": 0, 2024-05-09T04:58:33.881 INFO:teuthology.orchestra.run.smithi006.stdout: "poolop_resend": 0, 2024-05-09T04:58:33.881 INFO:teuthology.orchestra.run.smithi006.stdout: "poolstat_active": 0, 2024-05-09T04:58:33.881 INFO:teuthology.orchestra.run.smithi006.stdout: "poolstat_send": 0, 2024-05-09T04:58:33.881 INFO:teuthology.orchestra.run.smithi006.stdout: "poolstat_resend": 0, 2024-05-09T04:58:33.881 INFO:teuthology.orchestra.run.smithi006.stdout: "statfs_active": 0, 2024-05-09T04:58:33.881 INFO:teuthology.orchestra.run.smithi006.stdout: "statfs_send": 0, 2024-05-09T04:58:33.881 INFO:teuthology.orchestra.run.smithi006.stdout: "statfs_resend": 0, 2024-05-09T04:58:33.881 INFO:teuthology.orchestra.run.smithi006.stdout: "command_active": 0, 2024-05-09T04:58:33.881 INFO:teuthology.orchestra.run.smithi006.stdout: "command_send": 0, 2024-05-09T04:58:33.881 INFO:teuthology.orchestra.run.smithi006.stdout: "command_resend": 0, 2024-05-09T04:58:33.882 INFO:teuthology.orchestra.run.smithi006.stdout: "map_epoch": 0, 2024-05-09T04:58:33.882 INFO:teuthology.orchestra.run.smithi006.stdout: "map_full": 0, 2024-05-09T04:58:33.882 INFO:teuthology.orchestra.run.smithi006.stdout: "map_inc": 0, 2024-05-09T04:58:33.882 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_sessions": 6, 2024-05-09T04:58:33.882 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_session_open": 6, 2024-05-09T04:58:33.882 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_session_close": 0, 2024-05-09T04:58:33.882 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_laggy": 0, 2024-05-09T04:58:33.882 INFO:teuthology.orchestra.run.smithi006.stdout: "omap_wr": 0, 2024-05-09T04:58:33.882 INFO:teuthology.orchestra.run.smithi006.stdout: "omap_rd": 2, 2024-05-09T04:58:33.882 INFO:teuthology.orchestra.run.smithi006.stdout: "omap_del": 0 2024-05-09T04:58:33.882 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.882 INFO:teuthology.orchestra.run.smithi006.stdout: "oft": { 2024-05-09T04:58:33.882 INFO:teuthology.orchestra.run.smithi006.stdout: "omap_total_objs": 0, 2024-05-09T04:58:33.883 INFO:teuthology.orchestra.run.smithi006.stdout: "omap_total_kv_pairs": 0, 2024-05-09T04:58:33.883 INFO:teuthology.orchestra.run.smithi006.stdout: "omap_total_updates": 0, 2024-05-09T04:58:33.883 INFO:teuthology.orchestra.run.smithi006.stdout: "omap_total_removes": 0 2024-05-09T04:58:33.883 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.883 INFO:teuthology.orchestra.run.smithi006.stdout: "osd-slow-ops": { 2024-05-09T04:58:33.883 INFO:teuthology.orchestra.run.smithi006.stdout: "slow_ops_count": 0 2024-05-09T04:58:33.883 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.883 INFO:teuthology.orchestra.run.smithi006.stdout: "purge_queue": { 2024-05-09T04:58:33.883 INFO:teuthology.orchestra.run.smithi006.stdout: "pq_executing_ops": 0, 2024-05-09T04:58:33.883 INFO:teuthology.orchestra.run.smithi006.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T04:58:33.883 INFO:teuthology.orchestra.run.smithi006.stdout: "pq_executing": 0, 2024-05-09T04:58:33.883 INFO:teuthology.orchestra.run.smithi006.stdout: "pq_executing_high_water": 0, 2024-05-09T04:58:33.883 INFO:teuthology.orchestra.run.smithi006.stdout: "pq_executed_ops": 0, 2024-05-09T04:58:33.883 INFO:teuthology.orchestra.run.smithi006.stdout: "pq_executed": 0, 2024-05-09T04:58:33.884 INFO:teuthology.orchestra.run.smithi006.stdout: "pq_item_in_journal": 0 2024-05-09T04:58:33.884 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.884 INFO:teuthology.orchestra.run.smithi006.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T04:58:33.884 INFO:teuthology.orchestra.run.smithi006.stdout: "val": 0, 2024-05-09T04:58:33.884 INFO:teuthology.orchestra.run.smithi006.stdout: "max": 104857600, 2024-05-09T04:58:33.884 INFO:teuthology.orchestra.run.smithi006.stdout: "get_started": 0, 2024-05-09T04:58:33.884 INFO:teuthology.orchestra.run.smithi006.stdout: "get": 1416, 2024-05-09T04:58:33.884 INFO:teuthology.orchestra.run.smithi006.stdout: "get_sum": 1046784, 2024-05-09T04:58:33.884 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:33.884 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_success": 1416, 2024-05-09T04:58:33.884 INFO:teuthology.orchestra.run.smithi006.stdout: "take": 0, 2024-05-09T04:58:33.884 INFO:teuthology.orchestra.run.smithi006.stdout: "take_sum": 0, 2024-05-09T04:58:33.884 INFO:teuthology.orchestra.run.smithi006.stdout: "put": 1416, 2024-05-09T04:58:33.884 INFO:teuthology.orchestra.run.smithi006.stdout: "put_sum": 1046784, 2024-05-09T04:58:33.885 INFO:teuthology.orchestra.run.smithi006.stdout: "wait": { 2024-05-09T04:58:33.885 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.885 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.885 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.885 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:33.885 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.885 INFO:teuthology.orchestra.run.smithi006.stdout: "throttle-objecter_bytes": { 2024-05-09T04:58:33.885 INFO:teuthology.orchestra.run.smithi006.stdout: "val": 0, 2024-05-09T04:58:33.885 INFO:teuthology.orchestra.run.smithi006.stdout: "max": 104857600, 2024-05-09T04:58:33.885 INFO:teuthology.orchestra.run.smithi006.stdout: "get_started": 0, 2024-05-09T04:58:33.885 INFO:teuthology.orchestra.run.smithi006.stdout: "get": 0, 2024-05-09T04:58:33.885 INFO:teuthology.orchestra.run.smithi006.stdout: "get_sum": 0, 2024-05-09T04:58:33.885 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:33.885 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_success": 0, 2024-05-09T04:58:33.886 INFO:teuthology.orchestra.run.smithi006.stdout: "take": 1244, 2024-05-09T04:58:33.886 INFO:teuthology.orchestra.run.smithi006.stdout: "take_sum": 1435, 2024-05-09T04:58:33.886 INFO:teuthology.orchestra.run.smithi006.stdout: "put": 1244, 2024-05-09T04:58:33.886 INFO:teuthology.orchestra.run.smithi006.stdout: "put_sum": 1435, 2024-05-09T04:58:33.886 INFO:teuthology.orchestra.run.smithi006.stdout: "wait": { 2024-05-09T04:58:33.886 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.886 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.886 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.886 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:33.886 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.886 INFO:teuthology.orchestra.run.smithi006.stdout: "throttle-objecter_ops": { 2024-05-09T04:58:33.886 INFO:teuthology.orchestra.run.smithi006.stdout: "val": 0, 2024-05-09T04:58:33.886 INFO:teuthology.orchestra.run.smithi006.stdout: "max": 1024, 2024-05-09T04:58:33.886 INFO:teuthology.orchestra.run.smithi006.stdout: "get_started": 0, 2024-05-09T04:58:33.887 INFO:teuthology.orchestra.run.smithi006.stdout: "get": 0, 2024-05-09T04:58:33.887 INFO:teuthology.orchestra.run.smithi006.stdout: "get_sum": 0, 2024-05-09T04:58:33.887 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:33.887 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_success": 0, 2024-05-09T04:58:33.887 INFO:teuthology.orchestra.run.smithi006.stdout: "take": 1244, 2024-05-09T04:58:33.887 INFO:teuthology.orchestra.run.smithi006.stdout: "take_sum": 1244, 2024-05-09T04:58:33.887 INFO:teuthology.orchestra.run.smithi006.stdout: "put": 1244, 2024-05-09T04:58:33.887 INFO:teuthology.orchestra.run.smithi006.stdout: "put_sum": 1244, 2024-05-09T04:58:33.887 INFO:teuthology.orchestra.run.smithi006.stdout: "wait": { 2024-05-09T04:58:33.887 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.887 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.887 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.887 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:33.888 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.888 INFO:teuthology.orchestra.run.smithi006.stdout: "throttle-write_buf_throttle": { 2024-05-09T04:58:33.888 INFO:teuthology.orchestra.run.smithi006.stdout: "val": 0, 2024-05-09T04:58:33.888 INFO:teuthology.orchestra.run.smithi006.stdout: "max": 3758096384, 2024-05-09T04:58:33.888 INFO:teuthology.orchestra.run.smithi006.stdout: "get_started": 0, 2024-05-09T04:58:33.888 INFO:teuthology.orchestra.run.smithi006.stdout: "get": 0, 2024-05-09T04:58:33.888 INFO:teuthology.orchestra.run.smithi006.stdout: "get_sum": 0, 2024-05-09T04:58:33.888 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:33.888 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_success": 0, 2024-05-09T04:58:33.888 INFO:teuthology.orchestra.run.smithi006.stdout: "take": 0, 2024-05-09T04:58:33.888 INFO:teuthology.orchestra.run.smithi006.stdout: "take_sum": 0, 2024-05-09T04:58:33.888 INFO:teuthology.orchestra.run.smithi006.stdout: "put": 0, 2024-05-09T04:58:33.888 INFO:teuthology.orchestra.run.smithi006.stdout: "put_sum": 0, 2024-05-09T04:58:33.888 INFO:teuthology.orchestra.run.smithi006.stdout: "wait": { 2024-05-09T04:58:33.888 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.889 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.889 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.889 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:33.889 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:33.889 INFO:teuthology.orchestra.run.smithi006.stdout: "throttle-write_buf_throttle-0x55cf071b4410": { 2024-05-09T04:58:33.889 INFO:teuthology.orchestra.run.smithi006.stdout: "val": 0, 2024-05-09T04:58:33.889 INFO:teuthology.orchestra.run.smithi006.stdout: "max": 3758096384, 2024-05-09T04:58:33.889 INFO:teuthology.orchestra.run.smithi006.stdout: "get_started": 0, 2024-05-09T04:58:33.889 INFO:teuthology.orchestra.run.smithi006.stdout: "get": 0, 2024-05-09T04:58:33.889 INFO:teuthology.orchestra.run.smithi006.stdout: "get_sum": 0, 2024-05-09T04:58:33.889 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:33.889 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_success": 0, 2024-05-09T04:58:33.889 INFO:teuthology.orchestra.run.smithi006.stdout: "take": 0, 2024-05-09T04:58:33.889 INFO:teuthology.orchestra.run.smithi006.stdout: "take_sum": 0, 2024-05-09T04:58:33.890 INFO:teuthology.orchestra.run.smithi006.stdout: "put": 0, 2024-05-09T04:58:33.890 INFO:teuthology.orchestra.run.smithi006.stdout: "put_sum": 0, 2024-05-09T04:58:33.890 INFO:teuthology.orchestra.run.smithi006.stdout: "wait": { 2024-05-09T04:58:33.890 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:33.890 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:33.890 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:33.890 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:33.890 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:33.890 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-09T04:58:34.887 INFO:tasks.check_counter:Daemon mds.d mds.exported=0 2024-05-09T04:58:34.887 INFO:tasks.check_counter:Daemon mds.d mds.imported=0 2024-05-09T04:58:34.887 INFO:tasks.check_counter:Daemon mds.d mds_cache.dir_update=0 2024-05-09T04:58:34.887 INFO:tasks.check_counter:Daemon mds.d mds_cache.dir_update_receipt=0 2024-05-09T04:58:34.888 DEBUG:tasks.check_counter:Getting stats from g 2024-05-09T04:58:34.888 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-09T04:58:35.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:34 smithi006 ceph-mon[29835]: pgmap v608: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-09T04:58:35.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:34 smithi186 ceph-mon[34668]: pgmap v608: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-09T04:58:35.352 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:58:35.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:34 smithi169 ceph-mon[34075]: pgmap v608: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-09T04:58:37.070 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:36 smithi006 ceph-mon[29835]: pgmap v609: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-09T04:58:37.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:36 smithi186 ceph-mon[34668]: pgmap v609: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-09T04:58:37.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:36 smithi169 ceph-mon[34075]: pgmap v609: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-09T04:58:37.804 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-09T04:58:37.804 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T04:58:37.804 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 17, 2024-05-09T04:58:37.804 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 19, 2024-05-09T04:58:37.804 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 4357, 2024-05-09T04:58:37.804 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 36645, 2024-05-09T04:58:37.804 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 10, 2024-05-09T04:58:37.805 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 3, 2024-05-09T04:58:37.805 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.014374114, 2024-05-09T04:58:37.805 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.000675625, 2024-05-09T04:58:37.805 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.007907541, 2024-05-09T04:58:37.805 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.000826609, 2024-05-09T04:58:37.805 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:58:37.805 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 17, 2024-05-09T04:58:37.805 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.038041398, 2024-05-09T04:58:37.805 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.002237729 2024-05-09T04:58:37.805 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.805 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:58:37.805 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 2, 2024-05-09T04:58:37.805 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000294, 2024-05-09T04:58:37.805 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000147 2024-05-09T04:58:37.806 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.806 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T04:58:37.806 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T04:58:37.806 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.806 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T04:58:37.806 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 244, 2024-05-09T04:58:37.806 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 231, 2024-05-09T04:58:37.806 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 865585, 2024-05-09T04:58:37.806 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 161962, 2024-05-09T04:58:37.806 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 12, 2024-05-09T04:58:37.806 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 6, 2024-05-09T04:58:37.806 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.083051088, 2024-05-09T04:58:37.806 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.019897935, 2024-05-09T04:58:37.807 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.075586009, 2024-05-09T04:58:37.807 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.008415490, 2024-05-09T04:58:37.807 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:58:37.807 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 227, 2024-05-09T04:58:37.807 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.039311645, 2024-05-09T04:58:37.807 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000173179 2024-05-09T04:58:37.807 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.807 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:58:37.807 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 24, 2024-05-09T04:58:37.807 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000006539, 2024-05-09T04:58:37.807 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000272 2024-05-09T04:58:37.807 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.807 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 860080, 2024-05-09T04:58:37.807 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 103824 2024-05-09T04:58:37.808 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.808 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T04:58:37.808 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 332, 2024-05-09T04:58:37.808 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 491, 2024-05-09T04:58:37.808 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 94347, 2024-05-09T04:58:37.808 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 362353, 2024-05-09T04:58:37.808 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 7, 2024-05-09T04:58:37.808 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 4, 2024-05-09T04:58:37.808 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.098821969, 2024-05-09T04:58:37.808 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.051889533, 2024-05-09T04:58:37.808 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.040536480, 2024-05-09T04:58:37.808 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.004953072, 2024-05-09T04:58:37.808 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:58:37.808 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 493, 2024-05-09T04:58:37.809 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.056210171, 2024-05-09T04:58:37.809 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000114016 2024-05-09T04:58:37.809 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.809 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:58:37.809 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 673, 2024-05-09T04:58:37.809 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000443184, 2024-05-09T04:58:37.809 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000658 2024-05-09T04:58:37.809 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.809 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T04:58:37.809 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 257360 2024-05-09T04:58:37.809 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.809 INFO:teuthology.orchestra.run.smithi006.stdout: "cct": { 2024-05-09T04:58:37.809 INFO:teuthology.orchestra.run.smithi006.stdout: "total_workers": 1, 2024-05-09T04:58:37.809 INFO:teuthology.orchestra.run.smithi006.stdout: "unhealthy_workers": 0 2024-05-09T04:58:37.810 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.810 INFO:teuthology.orchestra.run.smithi006.stdout: "finisher-MDSRank": { 2024-05-09T04:58:37.810 INFO:teuthology.orchestra.run.smithi006.stdout: "queue_len": 0, 2024-05-09T04:58:37.810 INFO:teuthology.orchestra.run.smithi006.stdout: "complete_latency": { 2024-05-09T04:58:37.810 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 37, 2024-05-09T04:58:37.810 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.003152861, 2024-05-09T04:58:37.810 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000085212 2024-05-09T04:58:37.810 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:37.810 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.810 INFO:teuthology.orchestra.run.smithi006.stdout: "finisher-PurgeQueue": { 2024-05-09T04:58:37.810 INFO:teuthology.orchestra.run.smithi006.stdout: "queue_len": 0, 2024-05-09T04:58:37.810 INFO:teuthology.orchestra.run.smithi006.stdout: "complete_latency": { 2024-05-09T04:58:37.810 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 2, 2024-05-09T04:58:37.810 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000011906, 2024-05-09T04:58:37.811 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000005953 2024-05-09T04:58:37.811 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:37.811 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.811 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": { 2024-05-09T04:58:37.811 INFO:teuthology.orchestra.run.smithi006.stdout: "request": 2, 2024-05-09T04:58:37.811 INFO:teuthology.orchestra.run.smithi006.stdout: "reply": 1, 2024-05-09T04:58:37.811 INFO:teuthology.orchestra.run.smithi006.stdout: "reply_latency": { 2024-05-09T04:58:37.811 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 1, 2024-05-09T04:58:37.811 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000405312, 2024-05-09T04:58:37.811 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000405312 2024-05-09T04:58:37.811 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.811 INFO:teuthology.orchestra.run.smithi006.stdout: "slow_reply": 0, 2024-05-09T04:58:37.811 INFO:teuthology.orchestra.run.smithi006.stdout: "forward": 1, 2024-05-09T04:58:37.811 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_fetch_complete": 0, 2024-05-09T04:58:37.812 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_fetch_keys": 0, 2024-05-09T04:58:37.812 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_commit": 11, 2024-05-09T04:58:37.812 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_split": 0, 2024-05-09T04:58:37.812 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_merge": 0, 2024-05-09T04:58:37.812 INFO:teuthology.orchestra.run.smithi006.stdout: "inodes": 16, 2024-05-09T04:58:37.812 INFO:teuthology.orchestra.run.smithi006.stdout: "inodes_top": 0, 2024-05-09T04:58:37.812 INFO:teuthology.orchestra.run.smithi006.stdout: "inodes_bottom": 16, 2024-05-09T04:58:37.812 INFO:teuthology.orchestra.run.smithi006.stdout: "inodes_pin_tail": 0, 2024-05-09T04:58:37.812 INFO:teuthology.orchestra.run.smithi006.stdout: "inodes_pinned": 13, 2024-05-09T04:58:37.812 INFO:teuthology.orchestra.run.smithi006.stdout: "inodes_expired": 4, 2024-05-09T04:58:37.812 INFO:teuthology.orchestra.run.smithi006.stdout: "inodes_with_caps": 0, 2024-05-09T04:58:37.812 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": 0, 2024-05-09T04:58:37.812 INFO:teuthology.orchestra.run.smithi006.stdout: "subtrees": 3, 2024-05-09T04:58:37.812 INFO:teuthology.orchestra.run.smithi006.stdout: "traverse": 16, 2024-05-09T04:58:37.813 INFO:teuthology.orchestra.run.smithi006.stdout: "traverse_hit": 10, 2024-05-09T04:58:37.813 INFO:teuthology.orchestra.run.smithi006.stdout: "traverse_forward": 0, 2024-05-09T04:58:37.813 INFO:teuthology.orchestra.run.smithi006.stdout: "traverse_discover": 5, 2024-05-09T04:58:37.813 INFO:teuthology.orchestra.run.smithi006.stdout: "traverse_dir_fetch": 0, 2024-05-09T04:58:37.813 INFO:teuthology.orchestra.run.smithi006.stdout: "traverse_remote_ino": 0, 2024-05-09T04:58:37.813 INFO:teuthology.orchestra.run.smithi006.stdout: "traverse_lock": 0, 2024-05-09T04:58:37.813 INFO:teuthology.orchestra.run.smithi006.stdout: "load_cent": 0, 2024-05-09T04:58:37.813 INFO:teuthology.orchestra.run.smithi006.stdout: "q": 0, 2024-05-09T04:58:37.813 INFO:teuthology.orchestra.run.smithi006.stdout: "exported": 0, 2024-05-09T04:58:37.813 INFO:teuthology.orchestra.run.smithi006.stdout: "exported_inodes": 0, 2024-05-09T04:58:37.813 INFO:teuthology.orchestra.run.smithi006.stdout: "imported": 0, 2024-05-09T04:58:37.813 INFO:teuthology.orchestra.run.smithi006.stdout: "imported_inodes": 0, 2024-05-09T04:58:37.813 INFO:teuthology.orchestra.run.smithi006.stdout: "openino_dir_fetch": 0, 2024-05-09T04:58:37.813 INFO:teuthology.orchestra.run.smithi006.stdout: "openino_backtrace_fetch": 0, 2024-05-09T04:58:37.813 INFO:teuthology.orchestra.run.smithi006.stdout: "openino_peer_discover": 0, 2024-05-09T04:58:37.814 INFO:teuthology.orchestra.run.smithi006.stdout: "root_rfiles": 4, 2024-05-09T04:58:37.814 INFO:teuthology.orchestra.run.smithi006.stdout: "root_rbytes": 234, 2024-05-09T04:58:37.814 INFO:teuthology.orchestra.run.smithi006.stdout: "root_rsnaps": 0, 2024-05-09T04:58:37.814 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T04:58:37.814 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_set_tag": 0, 2024-05-09T04:58:37.814 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T04:58:37.814 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_inotable_repaired": 0, 2024-05-09T04:58:37.814 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_dir_inodes": 0, 2024-05-09T04:58:37.814 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T04:58:37.814 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T04:58:37.814 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_file_inodes": 0, 2024-05-09T04:58:37.814 INFO:teuthology.orchestra.run.smithi006.stdout: "handle_inode_file_caps": 0, 2024-05-09T04:58:37.814 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T04:58:37.814 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_cap_op_grant": 0, 2024-05-09T04:58:37.815 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T04:58:37.815 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T04:58:37.815 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T04:58:37.815 INFO:teuthology.orchestra.run.smithi006.stdout: "handle_client_caps": 0, 2024-05-09T04:58:37.815 INFO:teuthology.orchestra.run.smithi006.stdout: "handle_client_caps_dirty": 0, 2024-05-09T04:58:37.815 INFO:teuthology.orchestra.run.smithi006.stdout: "handle_client_cap_release": 0, 2024-05-09T04:58:37.815 INFO:teuthology.orchestra.run.smithi006.stdout: "process_request_cap_release": 1 2024-05-09T04:58:37.815 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.815 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_cache": { 2024-05-09T04:58:37.815 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_update": 0, 2024-05-09T04:58:37.815 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_update_receipt": 12, 2024-05-09T04:58:37.815 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_try_discover": 8, 2024-05-09T04:58:37.815 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_send_discover": 10, 2024-05-09T04:58:37.815 INFO:teuthology.orchestra.run.smithi006.stdout: "dir_handle_discover": 0, 2024-05-09T04:58:37.815 INFO:teuthology.orchestra.run.smithi006.stdout: "num_strays": 0, 2024-05-09T04:58:37.816 INFO:teuthology.orchestra.run.smithi006.stdout: "num_strays_delayed": 0, 2024-05-09T04:58:37.816 INFO:teuthology.orchestra.run.smithi006.stdout: "num_strays_enqueuing": 0, 2024-05-09T04:58:37.816 INFO:teuthology.orchestra.run.smithi006.stdout: "strays_created": 0, 2024-05-09T04:58:37.816 INFO:teuthology.orchestra.run.smithi006.stdout: "strays_enqueued": 0, 2024-05-09T04:58:37.816 INFO:teuthology.orchestra.run.smithi006.stdout: "strays_reintegrated": 0, 2024-05-09T04:58:37.816 INFO:teuthology.orchestra.run.smithi006.stdout: "strays_migrated": 0, 2024-05-09T04:58:37.816 INFO:teuthology.orchestra.run.smithi006.stdout: "num_recovering_processing": 0, 2024-05-09T04:58:37.816 INFO:teuthology.orchestra.run.smithi006.stdout: "num_recovering_enqueued": 0, 2024-05-09T04:58:37.816 INFO:teuthology.orchestra.run.smithi006.stdout: "num_recovering_prioritized": 0, 2024-05-09T04:58:37.816 INFO:teuthology.orchestra.run.smithi006.stdout: "recovery_started": 0, 2024-05-09T04:58:37.816 INFO:teuthology.orchestra.run.smithi006.stdout: "recovery_completed": 0, 2024-05-09T04:58:37.816 INFO:teuthology.orchestra.run.smithi006.stdout: "ireq_quiesce_path": 2, 2024-05-09T04:58:37.816 INFO:teuthology.orchestra.run.smithi006.stdout: "ireq_quiesce_inode": 11, 2024-05-09T04:58:37.816 INFO:teuthology.orchestra.run.smithi006.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T04:58:37.816 INFO:teuthology.orchestra.run.smithi006.stdout: "ireq_exportdir": 0, 2024-05-09T04:58:37.817 INFO:teuthology.orchestra.run.smithi006.stdout: "ireq_flush": 0, 2024-05-09T04:58:37.817 INFO:teuthology.orchestra.run.smithi006.stdout: "ireq_fragmentdir": 0, 2024-05-09T04:58:37.817 INFO:teuthology.orchestra.run.smithi006.stdout: "ireq_fragstats": 0, 2024-05-09T04:58:37.817 INFO:teuthology.orchestra.run.smithi006.stdout: "ireq_inodestats": 0 2024-05-09T04:58:37.817 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.817 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_log": { 2024-05-09T04:58:37.817 INFO:teuthology.orchestra.run.smithi006.stdout: "evlrg": 0, 2024-05-09T04:58:37.817 INFO:teuthology.orchestra.run.smithi006.stdout: "evadd": 6, 2024-05-09T04:58:37.817 INFO:teuthology.orchestra.run.smithi006.stdout: "evex": 0, 2024-05-09T04:58:37.817 INFO:teuthology.orchestra.run.smithi006.stdout: "evtrm": 0, 2024-05-09T04:58:37.817 INFO:teuthology.orchestra.run.smithi006.stdout: "ev": 6, 2024-05-09T04:58:37.817 INFO:teuthology.orchestra.run.smithi006.stdout: "evexg": 0, 2024-05-09T04:58:37.817 INFO:teuthology.orchestra.run.smithi006.stdout: "evexd": 0, 2024-05-09T04:58:37.817 INFO:teuthology.orchestra.run.smithi006.stdout: "segadd": 2, 2024-05-09T04:58:37.817 INFO:teuthology.orchestra.run.smithi006.stdout: "segex": 0, 2024-05-09T04:58:37.817 INFO:teuthology.orchestra.run.smithi006.stdout: "segtrm": 0, 2024-05-09T04:58:37.818 INFO:teuthology.orchestra.run.smithi006.stdout: "seg": 2, 2024-05-09T04:58:37.818 INFO:teuthology.orchestra.run.smithi006.stdout: "segmjr": 2, 2024-05-09T04:58:37.818 INFO:teuthology.orchestra.run.smithi006.stdout: "segexg": 0, 2024-05-09T04:58:37.818 INFO:teuthology.orchestra.run.smithi006.stdout: "segexd": 0, 2024-05-09T04:58:37.818 INFO:teuthology.orchestra.run.smithi006.stdout: "expos": 4194304, 2024-05-09T04:58:37.818 INFO:teuthology.orchestra.run.smithi006.stdout: "wrpos": 4195956, 2024-05-09T04:58:37.818 INFO:teuthology.orchestra.run.smithi006.stdout: "rdpos": 0, 2024-05-09T04:58:37.818 INFO:teuthology.orchestra.run.smithi006.stdout: "jlat": { 2024-05-09T04:58:37.818 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 5, 2024-05-09T04:58:37.818 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.012656670, 2024-05-09T04:58:37.818 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.002531334 2024-05-09T04:58:37.818 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.818 INFO:teuthology.orchestra.run.smithi006.stdout: "replayed": 0 2024-05-09T04:58:37.818 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.818 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_mem": { 2024-05-09T04:58:37.819 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 20, 2024-05-09T04:58:37.819 INFO:teuthology.orchestra.run.smithi006.stdout: "ino+": 22, 2024-05-09T04:58:37.819 INFO:teuthology.orchestra.run.smithi006.stdout: "ino-": 2, 2024-05-09T04:58:37.819 INFO:teuthology.orchestra.run.smithi006.stdout: "dir": 19, 2024-05-09T04:58:37.819 INFO:teuthology.orchestra.run.smithi006.stdout: "dir+": 22, 2024-05-09T04:58:37.819 INFO:teuthology.orchestra.run.smithi006.stdout: "dir-": 3, 2024-05-09T04:58:37.819 INFO:teuthology.orchestra.run.smithi006.stdout: "dn": 16, 2024-05-09T04:58:37.819 INFO:teuthology.orchestra.run.smithi006.stdout: "dn+": 20, 2024-05-09T04:58:37.819 INFO:teuthology.orchestra.run.smithi006.stdout: "dn-": 4, 2024-05-09T04:58:37.819 INFO:teuthology.orchestra.run.smithi006.stdout: "cap": 0, 2024-05-09T04:58:37.819 INFO:teuthology.orchestra.run.smithi006.stdout: "cap+": 1, 2024-05-09T04:58:37.819 INFO:teuthology.orchestra.run.smithi006.stdout: "cap-": 1, 2024-05-09T04:58:37.819 INFO:teuthology.orchestra.run.smithi006.stdout: "rss": 119680, 2024-05-09T04:58:37.819 INFO:teuthology.orchestra.run.smithi006.stdout: "heap": 223632 2024-05-09T04:58:37.819 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.820 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_server": { 2024-05-09T04:58:37.820 INFO:teuthology.orchestra.run.smithi006.stdout: "dispatch_client_request": 3, 2024-05-09T04:58:37.820 INFO:teuthology.orchestra.run.smithi006.stdout: "dispatch_server_request": 0, 2024-05-09T04:58:37.820 INFO:teuthology.orchestra.run.smithi006.stdout: "handle_client_request": 2, 2024-05-09T04:58:37.820 INFO:teuthology.orchestra.run.smithi006.stdout: "handle_client_session": 9, 2024-05-09T04:58:37.820 INFO:teuthology.orchestra.run.smithi006.stdout: "handle_peer_request": 11, 2024-05-09T04:58:37.820 INFO:teuthology.orchestra.run.smithi006.stdout: "req_create_latency": { 2024-05-09T04:58:37.820 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:37.820 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:37.820 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:37.820 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.820 INFO:teuthology.orchestra.run.smithi006.stdout: "req_getattr_latency": { 2024-05-09T04:58:37.820 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 1, 2024-05-09T04:58:37.820 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000405312, 2024-05-09T04:58:37.820 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000405312 2024-05-09T04:58:37.821 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.821 INFO:teuthology.orchestra.run.smithi006.stdout: "req_getfilelock_latency": { 2024-05-09T04:58:37.821 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:37.821 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:37.821 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:37.821 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.821 INFO:teuthology.orchestra.run.smithi006.stdout: "req_link_latency": { 2024-05-09T04:58:37.821 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:37.821 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:37.821 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:37.821 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.821 INFO:teuthology.orchestra.run.smithi006.stdout: "req_lookup_latency": { 2024-05-09T04:58:37.821 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:37.821 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:37.821 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:37.821 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.822 INFO:teuthology.orchestra.run.smithi006.stdout: "req_lookuphash_latency": { 2024-05-09T04:58:37.822 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:37.822 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:37.822 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:37.822 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.822 INFO:teuthology.orchestra.run.smithi006.stdout: "req_lookupino_latency": { 2024-05-09T04:58:37.822 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:37.822 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:37.822 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:37.822 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.822 INFO:teuthology.orchestra.run.smithi006.stdout: "req_lookupname_latency": { 2024-05-09T04:58:37.822 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:37.822 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:37.823 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:37.823 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.823 INFO:teuthology.orchestra.run.smithi006.stdout: "req_lookupparent_latency": { 2024-05-09T04:58:37.823 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:37.823 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:37.823 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:37.823 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.823 INFO:teuthology.orchestra.run.smithi006.stdout: "req_lookupsnap_latency": { 2024-05-09T04:58:37.823 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:37.823 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:37.823 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:37.823 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.823 INFO:teuthology.orchestra.run.smithi006.stdout: "req_lssnap_latency": { 2024-05-09T04:58:37.823 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:37.823 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:37.824 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:37.824 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.824 INFO:teuthology.orchestra.run.smithi006.stdout: "req_mkdir_latency": { 2024-05-09T04:58:37.824 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:37.824 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:37.824 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:37.824 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.825 INFO:teuthology.orchestra.run.smithi006.stdout: "req_mknod_latency": { 2024-05-09T04:58:37.825 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:37.825 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:37.825 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:37.825 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.825 INFO:teuthology.orchestra.run.smithi006.stdout: "req_mksnap_latency": { 2024-05-09T04:58:37.825 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:37.825 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:37.825 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:37.825 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.825 INFO:teuthology.orchestra.run.smithi006.stdout: "req_open_latency": { 2024-05-09T04:58:37.825 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:37.825 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:37.826 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:37.826 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.826 INFO:teuthology.orchestra.run.smithi006.stdout: "req_readdir_latency": { 2024-05-09T04:58:37.826 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:37.826 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:37.826 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:37.826 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.826 INFO:teuthology.orchestra.run.smithi006.stdout: "req_rename_latency": { 2024-05-09T04:58:37.826 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:37.826 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:37.826 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:37.826 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.826 INFO:teuthology.orchestra.run.smithi006.stdout: "req_renamesnap_latency": { 2024-05-09T04:58:37.826 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:37.827 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:37.827 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:37.827 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.827 INFO:teuthology.orchestra.run.smithi006.stdout: "req_snapdiff_latency": { 2024-05-09T04:58:37.827 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:37.827 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:37.827 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:37.827 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.827 INFO:teuthology.orchestra.run.smithi006.stdout: "req_rmdir_latency": { 2024-05-09T04:58:37.827 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:37.827 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:37.827 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:37.827 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.827 INFO:teuthology.orchestra.run.smithi006.stdout: "req_rmsnap_latency": { 2024-05-09T04:58:37.828 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:37.828 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:37.828 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:37.828 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.828 INFO:teuthology.orchestra.run.smithi006.stdout: "req_rmxattr_latency": { 2024-05-09T04:58:37.828 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:37.828 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:37.828 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:37.828 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.828 INFO:teuthology.orchestra.run.smithi006.stdout: "req_setattr_latency": { 2024-05-09T04:58:37.828 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:37.828 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:37.828 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:37.828 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.828 INFO:teuthology.orchestra.run.smithi006.stdout: "req_setdirlayout_latency": { 2024-05-09T04:58:37.829 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:37.829 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:37.829 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:37.829 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.829 INFO:teuthology.orchestra.run.smithi006.stdout: "req_setfilelock_latency": { 2024-05-09T04:58:37.829 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:37.829 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:37.829 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:37.829 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.829 INFO:teuthology.orchestra.run.smithi006.stdout: "req_setlayout_latency": { 2024-05-09T04:58:37.829 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:37.829 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:37.829 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:37.829 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.829 INFO:teuthology.orchestra.run.smithi006.stdout: "req_setxattr_latency": { 2024-05-09T04:58:37.830 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:37.830 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:37.830 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:37.830 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.830 INFO:teuthology.orchestra.run.smithi006.stdout: "req_symlink_latency": { 2024-05-09T04:58:37.830 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:37.830 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:37.830 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:37.830 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.830 INFO:teuthology.orchestra.run.smithi006.stdout: "req_unlink_latency": { 2024-05-09T04:58:37.830 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:37.830 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:37.830 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:37.830 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.831 INFO:teuthology.orchestra.run.smithi006.stdout: "cap_revoke_eviction": 0, 2024-05-09T04:58:37.831 INFO:teuthology.orchestra.run.smithi006.stdout: "cap_acquisition_throttle": 0, 2024-05-09T04:58:37.831 INFO:teuthology.orchestra.run.smithi006.stdout: "req_getvxattr_latency": { 2024-05-09T04:58:37.831 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:37.831 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:37.831 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:37.831 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:37.831 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.831 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_sessions": { 2024-05-09T04:58:37.831 INFO:teuthology.orchestra.run.smithi006.stdout: "session_count": 0, 2024-05-09T04:58:37.831 INFO:teuthology.orchestra.run.smithi006.stdout: "session_add": 2, 2024-05-09T04:58:37.831 INFO:teuthology.orchestra.run.smithi006.stdout: "session_remove": 2, 2024-05-09T04:58:37.831 INFO:teuthology.orchestra.run.smithi006.stdout: "sessions_open": 0, 2024-05-09T04:58:37.831 INFO:teuthology.orchestra.run.smithi006.stdout: "sessions_stale": 0, 2024-05-09T04:58:37.832 INFO:teuthology.orchestra.run.smithi006.stdout: "total_load": 1, 2024-05-09T04:58:37.832 INFO:teuthology.orchestra.run.smithi006.stdout: "average_load": 1, 2024-05-09T04:58:37.832 INFO:teuthology.orchestra.run.smithi006.stdout: "avg_session_uptime": 84, 2024-05-09T04:58:37.832 INFO:teuthology.orchestra.run.smithi006.stdout: "mdthresh_evicted": 0 2024-05-09T04:58:37.832 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.832 INFO:teuthology.orchestra.run.smithi006.stdout: "mempool": { 2024-05-09T04:58:37.832 INFO:teuthology.orchestra.run.smithi006.stdout: "bloom_filter_bytes": 0, 2024-05-09T04:58:37.832 INFO:teuthology.orchestra.run.smithi006.stdout: "bloom_filter_items": 0, 2024-05-09T04:58:37.832 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T04:58:37.832 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_alloc_items": 0, 2024-05-09T04:58:37.832 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T04:58:37.832 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_data_items": 0, 2024-05-09T04:58:37.832 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T04:58:37.832 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T04:58:37.833 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T04:58:37.833 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T04:58:37.833 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T04:58:37.833 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_other_items": 0, 2024-05-09T04:58:37.833 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T04:58:37.833 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T04:58:37.833 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_bytes": 0, 2024-05-09T04:58:37.833 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_items": 0, 2024-05-09T04:58:37.833 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_blob_bytes": 0, 2024-05-09T04:58:37.833 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_blob_items": 0, 2024-05-09T04:58:37.833 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T04:58:37.834 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T04:58:37.834 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T04:58:37.834 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T04:58:37.834 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T04:58:37.834 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_items": 0, 2024-05-09T04:58:37.834 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_txc_bytes": 0, 2024-05-09T04:58:37.834 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_txc_items": 0, 2024-05-09T04:58:37.834 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T04:58:37.834 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T04:58:37.834 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_bytes": 0, 2024-05-09T04:58:37.834 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_items": 0, 2024-05-09T04:58:37.834 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_bytes": 0, 2024-05-09T04:58:37.834 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_items": 0, 2024-05-09T04:58:37.834 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T04:58:37.835 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_reader_items": 0, 2024-05-09T04:58:37.835 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T04:58:37.835 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_writer_items": 0, 2024-05-09T04:58:37.835 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_anon_bytes": 246599, 2024-05-09T04:58:37.835 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_anon_items": 79, 2024-05-09T04:58:37.835 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_meta_bytes": 0, 2024-05-09T04:58:37.835 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_meta_items": 0, 2024-05-09T04:58:37.835 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_bytes": 0, 2024-05-09T04:58:37.835 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_items": 0, 2024-05-09T04:58:37.835 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_mapbl_bytes": 0, 2024-05-09T04:58:37.835 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_mapbl_items": 0, 2024-05-09T04:58:37.835 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_pglog_bytes": 0, 2024-05-09T04:58:37.835 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_pglog_items": 0, 2024-05-09T04:58:37.835 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_bytes": 8800, 2024-05-09T04:58:37.836 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_items": 169, 2024-05-09T04:58:37.836 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T04:58:37.836 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_mapping_items": 0, 2024-05-09T04:58:37.836 INFO:teuthology.orchestra.run.smithi006.stdout: "pgmap_bytes": 0, 2024-05-09T04:58:37.836 INFO:teuthology.orchestra.run.smithi006.stdout: "pgmap_items": 0, 2024-05-09T04:58:37.836 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_co_bytes": 81037, 2024-05-09T04:58:37.836 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_co_items": 336, 2024-05-09T04:58:37.836 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_1_bytes": 0, 2024-05-09T04:58:37.836 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_1_items": 0, 2024-05-09T04:58:37.836 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_2_bytes": 0, 2024-05-09T04:58:37.836 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_2_items": 0 2024-05-09T04:58:37.836 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.836 INFO:teuthology.orchestra.run.smithi006.stdout: "objecter": { 2024-05-09T04:58:37.836 INFO:teuthology.orchestra.run.smithi006.stdout: "op_active": 0, 2024-05-09T04:58:37.836 INFO:teuthology.orchestra.run.smithi006.stdout: "op_laggy": 0, 2024-05-09T04:58:37.837 INFO:teuthology.orchestra.run.smithi006.stdout: "op_send": 42, 2024-05-09T04:58:37.837 INFO:teuthology.orchestra.run.smithi006.stdout: "op_send_bytes": 12044, 2024-05-09T04:58:37.837 INFO:teuthology.orchestra.run.smithi006.stdout: "op_resend": 33, 2024-05-09T04:58:37.837 INFO:teuthology.orchestra.run.smithi006.stdout: "op_reply": 42, 2024-05-09T04:58:37.837 INFO:teuthology.orchestra.run.smithi006.stdout: "op_latency": { 2024-05-09T04:58:37.837 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 42, 2024-05-09T04:58:37.837 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.327996238, 2024-05-09T04:58:37.837 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.007809434 2024-05-09T04:58:37.837 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.837 INFO:teuthology.orchestra.run.smithi006.stdout: "op_inflight": 0, 2024-05-09T04:58:37.837 INFO:teuthology.orchestra.run.smithi006.stdout: "oplen_avg": { 2024-05-09T04:58:37.837 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 42, 2024-05-09T04:58:37.837 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 64 2024-05-09T04:58:37.837 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.838 INFO:teuthology.orchestra.run.smithi006.stdout: "op": 42, 2024-05-09T04:58:37.838 INFO:teuthology.orchestra.run.smithi006.stdout: "op_r": 0, 2024-05-09T04:58:37.838 INFO:teuthology.orchestra.run.smithi006.stdout: "op_w": 42, 2024-05-09T04:58:37.838 INFO:teuthology.orchestra.run.smithi006.stdout: "op_rmw": 0, 2024-05-09T04:58:37.838 INFO:teuthology.orchestra.run.smithi006.stdout: "op_pg": 0, 2024-05-09T04:58:37.838 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_stat": 0, 2024-05-09T04:58:37.838 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_create": 10, 2024-05-09T04:58:37.838 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_read": 0, 2024-05-09T04:58:37.838 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_write": 5, 2024-05-09T04:58:37.838 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_writefull": 8, 2024-05-09T04:58:37.838 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_writesame": 0, 2024-05-09T04:58:37.838 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_append": 0, 2024-05-09T04:58:37.838 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_zero": 0, 2024-05-09T04:58:37.838 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_truncate": 0, 2024-05-09T04:58:37.838 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_delete": 6, 2024-05-09T04:58:37.839 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_mapext": 0, 2024-05-09T04:58:37.839 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_sparse_read": 0, 2024-05-09T04:58:37.839 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_clonerange": 0, 2024-05-09T04:58:37.839 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_getxattr": 0, 2024-05-09T04:58:37.839 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_setxattr": 20, 2024-05-09T04:58:37.839 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_cmpxattr": 0, 2024-05-09T04:58:37.839 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_rmxattr": 0, 2024-05-09T04:58:37.839 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_resetxattrs": 0, 2024-05-09T04:58:37.839 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_call": 0, 2024-05-09T04:58:37.839 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_watch": 0, 2024-05-09T04:58:37.839 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_notify": 0, 2024-05-09T04:58:37.839 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T04:58:37.839 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_pgls": 0, 2024-05-09T04:58:37.839 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_pgls_filter": 0, 2024-05-09T04:58:37.839 INFO:teuthology.orchestra.run.smithi006.stdout: "osdop_other": 0, 2024-05-09T04:58:37.840 INFO:teuthology.orchestra.run.smithi006.stdout: "linger_active": 0, 2024-05-09T04:58:37.840 INFO:teuthology.orchestra.run.smithi006.stdout: "linger_send": 0, 2024-05-09T04:58:37.840 INFO:teuthology.orchestra.run.smithi006.stdout: "linger_resend": 0, 2024-05-09T04:58:37.840 INFO:teuthology.orchestra.run.smithi006.stdout: "linger_ping": 0, 2024-05-09T04:58:37.840 INFO:teuthology.orchestra.run.smithi006.stdout: "poolop_active": 0, 2024-05-09T04:58:37.840 INFO:teuthology.orchestra.run.smithi006.stdout: "poolop_send": 0, 2024-05-09T04:58:37.840 INFO:teuthology.orchestra.run.smithi006.stdout: "poolop_resend": 0, 2024-05-09T04:58:37.840 INFO:teuthology.orchestra.run.smithi006.stdout: "poolstat_active": 0, 2024-05-09T04:58:37.840 INFO:teuthology.orchestra.run.smithi006.stdout: "poolstat_send": 0, 2024-05-09T04:58:37.840 INFO:teuthology.orchestra.run.smithi006.stdout: "poolstat_resend": 0, 2024-05-09T04:58:37.840 INFO:teuthology.orchestra.run.smithi006.stdout: "statfs_active": 0, 2024-05-09T04:58:37.840 INFO:teuthology.orchestra.run.smithi006.stdout: "statfs_send": 0, 2024-05-09T04:58:37.840 INFO:teuthology.orchestra.run.smithi006.stdout: "statfs_resend": 0, 2024-05-09T04:58:37.840 INFO:teuthology.orchestra.run.smithi006.stdout: "command_active": 0, 2024-05-09T04:58:37.841 INFO:teuthology.orchestra.run.smithi006.stdout: "command_send": 0, 2024-05-09T04:58:37.841 INFO:teuthology.orchestra.run.smithi006.stdout: "command_resend": 0, 2024-05-09T04:58:37.841 INFO:teuthology.orchestra.run.smithi006.stdout: "map_epoch": 0, 2024-05-09T04:58:37.841 INFO:teuthology.orchestra.run.smithi006.stdout: "map_full": 0, 2024-05-09T04:58:37.841 INFO:teuthology.orchestra.run.smithi006.stdout: "map_inc": 0, 2024-05-09T04:58:37.841 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_sessions": 10, 2024-05-09T04:58:37.841 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_session_open": 10, 2024-05-09T04:58:37.841 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_session_close": 0, 2024-05-09T04:58:37.841 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_laggy": 0, 2024-05-09T04:58:37.841 INFO:teuthology.orchestra.run.smithi006.stdout: "omap_wr": 14, 2024-05-09T04:58:37.841 INFO:teuthology.orchestra.run.smithi006.stdout: "omap_rd": 0, 2024-05-09T04:58:37.841 INFO:teuthology.orchestra.run.smithi006.stdout: "omap_del": 1 2024-05-09T04:58:37.841 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.841 INFO:teuthology.orchestra.run.smithi006.stdout: "oft": { 2024-05-09T04:58:37.842 INFO:teuthology.orchestra.run.smithi006.stdout: "omap_total_objs": 1, 2024-05-09T04:58:37.842 INFO:teuthology.orchestra.run.smithi006.stdout: "omap_total_kv_pairs": 0, 2024-05-09T04:58:37.842 INFO:teuthology.orchestra.run.smithi006.stdout: "omap_total_updates": 0, 2024-05-09T04:58:37.842 INFO:teuthology.orchestra.run.smithi006.stdout: "omap_total_removes": 0 2024-05-09T04:58:37.842 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.842 INFO:teuthology.orchestra.run.smithi006.stdout: "osd-slow-ops": { 2024-05-09T04:58:37.842 INFO:teuthology.orchestra.run.smithi006.stdout: "slow_ops_count": 12 2024-05-09T04:58:37.842 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.842 INFO:teuthology.orchestra.run.smithi006.stdout: "purge_queue": { 2024-05-09T04:58:37.842 INFO:teuthology.orchestra.run.smithi006.stdout: "pq_executing_ops": 0, 2024-05-09T04:58:37.842 INFO:teuthology.orchestra.run.smithi006.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T04:58:37.842 INFO:teuthology.orchestra.run.smithi006.stdout: "pq_executing": 0, 2024-05-09T04:58:37.842 INFO:teuthology.orchestra.run.smithi006.stdout: "pq_executing_high_water": 0, 2024-05-09T04:58:37.842 INFO:teuthology.orchestra.run.smithi006.stdout: "pq_executed_ops": 0, 2024-05-09T04:58:37.843 INFO:teuthology.orchestra.run.smithi006.stdout: "pq_executed": 0, 2024-05-09T04:58:37.843 INFO:teuthology.orchestra.run.smithi006.stdout: "pq_item_in_journal": 0 2024-05-09T04:58:37.843 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.843 INFO:teuthology.orchestra.run.smithi006.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T04:58:37.843 INFO:teuthology.orchestra.run.smithi006.stdout: "val": 0, 2024-05-09T04:58:37.843 INFO:teuthology.orchestra.run.smithi006.stdout: "max": 104857600, 2024-05-09T04:58:37.843 INFO:teuthology.orchestra.run.smithi006.stdout: "get_started": 0, 2024-05-09T04:58:37.843 INFO:teuthology.orchestra.run.smithi006.stdout: "get": 589, 2024-05-09T04:58:37.843 INFO:teuthology.orchestra.run.smithi006.stdout: "get_sum": 897395, 2024-05-09T04:58:37.843 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:37.843 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_success": 589, 2024-05-09T04:58:37.843 INFO:teuthology.orchestra.run.smithi006.stdout: "take": 0, 2024-05-09T04:58:37.843 INFO:teuthology.orchestra.run.smithi006.stdout: "take_sum": 0, 2024-05-09T04:58:37.843 INFO:teuthology.orchestra.run.smithi006.stdout: "put": 589, 2024-05-09T04:58:37.843 INFO:teuthology.orchestra.run.smithi006.stdout: "put_sum": 897395, 2024-05-09T04:58:37.844 INFO:teuthology.orchestra.run.smithi006.stdout: "wait": { 2024-05-09T04:58:37.844 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:37.844 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:37.844 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:37.844 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:37.844 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.844 INFO:teuthology.orchestra.run.smithi006.stdout: "throttle-objecter_bytes": { 2024-05-09T04:58:37.844 INFO:teuthology.orchestra.run.smithi006.stdout: "val": 0, 2024-05-09T04:58:37.844 INFO:teuthology.orchestra.run.smithi006.stdout: "max": 104857600, 2024-05-09T04:58:37.844 INFO:teuthology.orchestra.run.smithi006.stdout: "get_started": 0, 2024-05-09T04:58:37.844 INFO:teuthology.orchestra.run.smithi006.stdout: "get": 0, 2024-05-09T04:58:37.844 INFO:teuthology.orchestra.run.smithi006.stdout: "get_sum": 0, 2024-05-09T04:58:37.844 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:37.844 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_success": 0, 2024-05-09T04:58:37.845 INFO:teuthology.orchestra.run.smithi006.stdout: "take": 42, 2024-05-09T04:58:37.845 INFO:teuthology.orchestra.run.smithi006.stdout: "take_sum": 12044, 2024-05-09T04:58:37.845 INFO:teuthology.orchestra.run.smithi006.stdout: "put": 42, 2024-05-09T04:58:37.845 INFO:teuthology.orchestra.run.smithi006.stdout: "put_sum": 12044, 2024-05-09T04:58:37.845 INFO:teuthology.orchestra.run.smithi006.stdout: "wait": { 2024-05-09T04:58:37.845 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:37.845 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:37.845 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:37.845 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:37.845 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.845 INFO:teuthology.orchestra.run.smithi006.stdout: "throttle-objecter_ops": { 2024-05-09T04:58:37.845 INFO:teuthology.orchestra.run.smithi006.stdout: "val": 0, 2024-05-09T04:58:37.845 INFO:teuthology.orchestra.run.smithi006.stdout: "max": 1024, 2024-05-09T04:58:37.845 INFO:teuthology.orchestra.run.smithi006.stdout: "get_started": 0, 2024-05-09T04:58:37.846 INFO:teuthology.orchestra.run.smithi006.stdout: "get": 0, 2024-05-09T04:58:37.846 INFO:teuthology.orchestra.run.smithi006.stdout: "get_sum": 0, 2024-05-09T04:58:37.846 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:37.846 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_success": 0, 2024-05-09T04:58:37.846 INFO:teuthology.orchestra.run.smithi006.stdout: "take": 42, 2024-05-09T04:58:37.846 INFO:teuthology.orchestra.run.smithi006.stdout: "take_sum": 42, 2024-05-09T04:58:37.846 INFO:teuthology.orchestra.run.smithi006.stdout: "put": 42, 2024-05-09T04:58:37.846 INFO:teuthology.orchestra.run.smithi006.stdout: "put_sum": 42, 2024-05-09T04:58:37.846 INFO:teuthology.orchestra.run.smithi006.stdout: "wait": { 2024-05-09T04:58:37.846 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:37.846 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:37.846 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:37.846 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:37.846 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.847 INFO:teuthology.orchestra.run.smithi006.stdout: "throttle-write_buf_throttle": { 2024-05-09T04:58:37.847 INFO:teuthology.orchestra.run.smithi006.stdout: "val": 0, 2024-05-09T04:58:37.847 INFO:teuthology.orchestra.run.smithi006.stdout: "max": 3758096384, 2024-05-09T04:58:37.847 INFO:teuthology.orchestra.run.smithi006.stdout: "get_started": 0, 2024-05-09T04:58:37.847 INFO:teuthology.orchestra.run.smithi006.stdout: "get": 0, 2024-05-09T04:58:37.847 INFO:teuthology.orchestra.run.smithi006.stdout: "get_sum": 0, 2024-05-09T04:58:37.847 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:37.847 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_success": 0, 2024-05-09T04:58:37.847 INFO:teuthology.orchestra.run.smithi006.stdout: "take": 0, 2024-05-09T04:58:37.847 INFO:teuthology.orchestra.run.smithi006.stdout: "take_sum": 0, 2024-05-09T04:58:37.847 INFO:teuthology.orchestra.run.smithi006.stdout: "put": 0, 2024-05-09T04:58:37.847 INFO:teuthology.orchestra.run.smithi006.stdout: "put_sum": 0, 2024-05-09T04:58:37.847 INFO:teuthology.orchestra.run.smithi006.stdout: "wait": { 2024-05-09T04:58:37.847 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:37.847 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:37.848 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:37.848 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:37.848 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:37.848 INFO:teuthology.orchestra.run.smithi006.stdout: "throttle-write_buf_throttle-0x558fb0bde410": { 2024-05-09T04:58:37.848 INFO:teuthology.orchestra.run.smithi006.stdout: "val": 0, 2024-05-09T04:58:37.848 INFO:teuthology.orchestra.run.smithi006.stdout: "max": 3758096384, 2024-05-09T04:58:37.848 INFO:teuthology.orchestra.run.smithi006.stdout: "get_started": 0, 2024-05-09T04:58:37.848 INFO:teuthology.orchestra.run.smithi006.stdout: "get": 6, 2024-05-09T04:58:37.848 INFO:teuthology.orchestra.run.smithi006.stdout: "get_sum": 1652, 2024-05-09T04:58:37.848 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:37.848 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_success": 6, 2024-05-09T04:58:37.848 INFO:teuthology.orchestra.run.smithi006.stdout: "take": 0, 2024-05-09T04:58:37.848 INFO:teuthology.orchestra.run.smithi006.stdout: "take_sum": 0, 2024-05-09T04:58:37.848 INFO:teuthology.orchestra.run.smithi006.stdout: "put": 5, 2024-05-09T04:58:37.848 INFO:teuthology.orchestra.run.smithi006.stdout: "put_sum": 1652, 2024-05-09T04:58:37.849 INFO:teuthology.orchestra.run.smithi006.stdout: "wait": { 2024-05-09T04:58:37.849 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:37.849 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:37.849 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:37.849 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:37.849 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:37.849 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-09T04:58:38.935 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-05-09T04:58:38.935 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-05-09T04:58:38.935 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-05-09T04:58:38.935 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=12 2024-05-09T04:58:38.935 DEBUG:tasks.check_counter:Getting stats from j 2024-05-09T04:58:38.936 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-09T04:58:39.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:38 smithi006 ceph-mon[29835]: pgmap v610: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:58:39.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:38 smithi186 ceph-mon[34668]: pgmap v610: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:58:39.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:38 smithi169 ceph-mon[34075]: pgmap v610: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:58:39.524 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:58:41.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:40 smithi186 ceph-mon[34668]: pgmap v611: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:58:41.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:40 smithi169 ceph-mon[34075]: pgmap v611: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:58:41.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:40 smithi006 ceph-mon[29835]: pgmap v611: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:58:42.296 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-09T04:58:42.296 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T04:58:42.296 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 172, 2024-05-09T04:58:42.296 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 165, 2024-05-09T04:58:42.296 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 801264, 2024-05-09T04:58:42.296 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 100336, 2024-05-09T04:58:42.296 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 4, 2024-05-09T04:58:42.296 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 0, 2024-05-09T04:58:42.296 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.074091890, 2024-05-09T04:58:42.296 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.025213562, 2024-05-09T04:58:42.297 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.051153966, 2024-05-09T04:58:42.297 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.007162864, 2024-05-09T04:58:42.297 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:58:42.297 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 165, 2024-05-09T04:58:42.297 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.017622528, 2024-05-09T04:58:42.297 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000106803 2024-05-09T04:58:42.297 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:42.297 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:58:42.297 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:42.297 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:42.297 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:42.297 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:42.297 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 801264, 2024-05-09T04:58:42.297 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 100336 2024-05-09T04:58:42.297 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:42.298 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T04:58:42.298 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 1, 2024-05-09T04:58:42.298 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 123, 2024-05-09T04:58:42.298 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 160, 2024-05-09T04:58:42.298 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 100288, 2024-05-09T04:58:42.298 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 3, 2024-05-09T04:58:42.298 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 2, 2024-05-09T04:58:42.298 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.023534493, 2024-05-09T04:58:42.298 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.017507720, 2024-05-09T04:58:42.298 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.003231385, 2024-05-09T04:58:42.298 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T04:58:42.298 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:58:42.298 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 121, 2024-05-09T04:58:42.298 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.010711619, 2024-05-09T04:58:42.299 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000088525 2024-05-09T04:58:42.299 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:42.299 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:58:42.299 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:42.299 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:42.299 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:42.299 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:42.299 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T04:58:42.299 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 100288 2024-05-09T04:58:42.299 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:42.299 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T04:58:42.299 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 0, 2024-05-09T04:58:42.299 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 2, 2024-05-09T04:58:42.299 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 0, 2024-05-09T04:58:42.300 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 0, 2024-05-09T04:58:42.300 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 2, 2024-05-09T04:58:42.300 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 1, 2024-05-09T04:58:42.300 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.001189239, 2024-05-09T04:58:42.300 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.000000000, 2024-05-09T04:58:42.300 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.001028214, 2024-05-09T04:58:42.300 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T04:58:42.300 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:58:42.300 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:42.300 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:42.300 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:42.300 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:42.300 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:58:42.300 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:42.301 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:42.301 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:42.301 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:42.301 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T04:58:42.301 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T04:58:42.301 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:42.301 INFO:teuthology.orchestra.run.smithi006.stdout: "cct": { 2024-05-09T04:58:42.301 INFO:teuthology.orchestra.run.smithi006.stdout: "total_workers": 0, 2024-05-09T04:58:42.301 INFO:teuthology.orchestra.run.smithi006.stdout: "unhealthy_workers": 0 2024-05-09T04:58:42.301 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:42.301 INFO:teuthology.orchestra.run.smithi006.stdout: "mempool": { 2024-05-09T04:58:42.301 INFO:teuthology.orchestra.run.smithi006.stdout: "bloom_filter_bytes": 0, 2024-05-09T04:58:42.301 INFO:teuthology.orchestra.run.smithi006.stdout: "bloom_filter_items": 0, 2024-05-09T04:58:42.301 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T04:58:42.301 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_alloc_items": 0, 2024-05-09T04:58:42.302 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T04:58:42.302 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_data_items": 0, 2024-05-09T04:58:42.302 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T04:58:42.302 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T04:58:42.302 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T04:58:42.302 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T04:58:42.302 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T04:58:42.302 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_other_items": 0, 2024-05-09T04:58:42.302 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T04:58:42.302 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T04:58:42.302 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_bytes": 0, 2024-05-09T04:58:42.302 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_items": 0, 2024-05-09T04:58:42.302 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_blob_bytes": 0, 2024-05-09T04:58:42.302 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_blob_items": 0, 2024-05-09T04:58:42.303 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T04:58:42.303 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T04:58:42.303 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T04:58:42.303 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T04:58:42.303 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T04:58:42.303 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_items": 0, 2024-05-09T04:58:42.303 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_txc_bytes": 0, 2024-05-09T04:58:42.303 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_txc_items": 0, 2024-05-09T04:58:42.303 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T04:58:42.303 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T04:58:42.303 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_bytes": 0, 2024-05-09T04:58:42.303 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_items": 0, 2024-05-09T04:58:42.303 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_bytes": 0, 2024-05-09T04:58:42.303 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_items": 0, 2024-05-09T04:58:42.303 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T04:58:42.304 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_reader_items": 0, 2024-05-09T04:58:42.304 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T04:58:42.304 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_writer_items": 0, 2024-05-09T04:58:42.304 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_anon_bytes": 148967, 2024-05-09T04:58:42.304 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_anon_items": 29, 2024-05-09T04:58:42.304 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_meta_bytes": 0, 2024-05-09T04:58:42.304 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_meta_items": 0, 2024-05-09T04:58:42.304 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_bytes": 0, 2024-05-09T04:58:42.304 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_items": 0, 2024-05-09T04:58:42.304 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_mapbl_bytes": 0, 2024-05-09T04:58:42.305 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_mapbl_items": 0, 2024-05-09T04:58:42.305 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_pglog_bytes": 0, 2024-05-09T04:58:42.305 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_pglog_items": 0, 2024-05-09T04:58:42.305 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_bytes": 0, 2024-05-09T04:58:42.305 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_items": 0, 2024-05-09T04:58:42.305 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T04:58:42.305 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_mapping_items": 0, 2024-05-09T04:58:42.305 INFO:teuthology.orchestra.run.smithi006.stdout: "pgmap_bytes": 0, 2024-05-09T04:58:42.305 INFO:teuthology.orchestra.run.smithi006.stdout: "pgmap_items": 0, 2024-05-09T04:58:42.305 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_co_bytes": 928, 2024-05-09T04:58:42.305 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_co_items": 2, 2024-05-09T04:58:42.305 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_1_bytes": 0, 2024-05-09T04:58:42.305 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_1_items": 0, 2024-05-09T04:58:42.306 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_2_bytes": 0, 2024-05-09T04:58:42.306 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_2_items": 0 2024-05-09T04:58:42.306 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-09T04:58:42.306 INFO:teuthology.orchestra.run.smithi006.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T04:58:42.306 INFO:teuthology.orchestra.run.smithi006.stdout: "val": 0, 2024-05-09T04:58:42.306 INFO:teuthology.orchestra.run.smithi006.stdout: "max": 104857600, 2024-05-09T04:58:42.306 INFO:teuthology.orchestra.run.smithi006.stdout: "get_started": 0, 2024-05-09T04:58:42.306 INFO:teuthology.orchestra.run.smithi006.stdout: "get": 169, 2024-05-09T04:58:42.306 INFO:teuthology.orchestra.run.smithi006.stdout: "get_sum": 774491, 2024-05-09T04:58:42.306 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:42.306 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_success": 169, 2024-05-09T04:58:42.306 INFO:teuthology.orchestra.run.smithi006.stdout: "take": 0, 2024-05-09T04:58:42.306 INFO:teuthology.orchestra.run.smithi006.stdout: "take_sum": 0, 2024-05-09T04:58:42.306 INFO:teuthology.orchestra.run.smithi006.stdout: "put": 169, 2024-05-09T04:58:42.306 INFO:teuthology.orchestra.run.smithi006.stdout: "put_sum": 774491, 2024-05-09T04:58:42.307 INFO:teuthology.orchestra.run.smithi006.stdout: "wait": { 2024-05-09T04:58:42.307 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2024-05-09T04:58:42.307 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2024-05-09T04:58:42.307 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2024-05-09T04:58:42.307 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:42.307 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-09T04:58:42.307 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-09T04:58:43.170 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:42 smithi006 ceph-mon[29835]: pgmap v612: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 7.0 MiB/s wr, 12 op/s 2024-05-09T04:58:43.228 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-09T04:58:43.228 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-09T04:58:43.228 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-05-09T04:58:43.228 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-05-09T04:58:43.228 DEBUG:tasks.check_counter:Getting stats from b 2024-05-09T04:58:43.229 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-09T04:58:43.255 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:42 smithi169 ceph-mon[34075]: pgmap v612: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 7.0 MiB/s wr, 12 op/s 2024-05-09T04:58:43.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:42 smithi186 ceph-mon[34668]: pgmap v612: 129 pgs: 129 active+clean; 257 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 7.0 MiB/s wr, 12 op/s 2024-05-09T04:58:43.481 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T04:58:45.190 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:44 smithi169 ceph-mon[34075]: pgmap v613: 129 pgs: 129 active+clean; 85 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 21 MiB/s wr, 18 op/s 2024-05-09T04:58:45.339 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-05-09T04:58:45.339 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T04:58:45.339 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 1285, 2024-05-09T04:58:45.339 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 935, 2024-05-09T04:58:45.339 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 421325, 2024-05-09T04:58:45.339 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 1084863, 2024-05-09T04:58:45.339 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 21, 2024-05-09T04:58:45.340 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 3, 2024-05-09T04:58:45.340 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.148673696, 2024-05-09T04:58:45.340 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.044926183, 2024-05-09T04:58:45.340 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.071563851, 2024-05-09T04:58:45.340 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.033503055, 2024-05-09T04:58:45.340 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:58:45.340 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 933, 2024-05-09T04:58:45.340 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.614647091, 2024-05-09T04:58:45.340 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000658785 2024-05-09T04:58:45.340 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.340 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:58:45.340 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1031, 2024-05-09T04:58:45.340 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000376373, 2024-05-09T04:58:45.340 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000365 2024-05-09T04:58:45.341 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.341 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 64352, 2024-05-09T04:58:45.341 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 30016 2024-05-09T04:58:45.341 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.341 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T04:58:45.341 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 648, 2024-05-09T04:58:45.341 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 713, 2024-05-09T04:58:45.341 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 191963, 2024-05-09T04:58:45.341 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 473067, 2024-05-09T04:58:45.341 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 10, 2024-05-09T04:58:45.341 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 6, 2024-05-09T04:58:45.341 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.056435715, 2024-05-09T04:58:45.341 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.021912712, 2024-05-09T04:58:45.342 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.032445648, 2024-05-09T04:58:45.342 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.006506855, 2024-05-09T04:58:45.342 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:58:45.342 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 712, 2024-05-09T04:58:45.342 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.109019097, 2024-05-09T04:58:45.342 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000153116 2024-05-09T04:58:45.342 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.342 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:58:45.342 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 657, 2024-05-09T04:58:45.342 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000195729, 2024-05-09T04:58:45.342 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000297 2024-05-09T04:58:45.342 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.342 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-05-09T04:58:45.343 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 286864 2024-05-09T04:58:45.343 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.343 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T04:58:45.343 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 1246, 2024-05-09T04:58:45.343 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 1042, 2024-05-09T04:58:45.343 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 1137814, 2024-05-09T04:58:45.343 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 477988, 2024-05-09T04:58:45.343 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 19, 2024-05-09T04:58:45.343 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 7, 2024-05-09T04:58:45.343 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.130731411, 2024-05-09T04:58:45.343 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.044206401, 2024-05-09T04:58:45.343 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.088032032, 2024-05-09T04:58:45.343 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.015886668, 2024-05-09T04:58:45.343 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:58:45.344 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1041, 2024-05-09T04:58:45.344 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.092126229, 2024-05-09T04:58:45.344 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000088497 2024-05-09T04:58:45.344 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.344 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:58:45.344 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1460, 2024-05-09T04:58:45.344 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000372990, 2024-05-09T04:58:45.344 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000255 2024-05-09T04:58:45.344 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.344 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 911072, 2024-05-09T04:58:45.344 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 105296 2024-05-09T04:58:45.344 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.344 INFO:teuthology.orchestra.run.smithi169.stdout: "cct": { 2024-05-09T04:58:45.344 INFO:teuthology.orchestra.run.smithi169.stdout: "total_workers": 1, 2024-05-09T04:58:45.345 INFO:teuthology.orchestra.run.smithi169.stdout: "unhealthy_workers": 0 2024-05-09T04:58:45.345 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.345 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-MDSRank": { 2024-05-09T04:58:45.345 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-05-09T04:58:45.345 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-05-09T04:58:45.345 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 386, 2024-05-09T04:58:45.345 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.102457017, 2024-05-09T04:58:45.345 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000265432 2024-05-09T04:58:45.345 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:45.345 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.345 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-PurgeQueue": { 2024-05-09T04:58:45.345 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-05-09T04:58:45.345 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-05-09T04:58:45.345 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 100, 2024-05-09T04:58:45.346 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.004764475, 2024-05-09T04:58:45.346 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000047644 2024-05-09T04:58:45.346 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:45.346 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.346 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": { 2024-05-09T04:58:45.346 INFO:teuthology.orchestra.run.smithi169.stdout: "request": 67, 2024-05-09T04:58:45.346 INFO:teuthology.orchestra.run.smithi169.stdout: "reply": 67, 2024-05-09T04:58:45.346 INFO:teuthology.orchestra.run.smithi169.stdout: "reply_latency": { 2024-05-09T04:58:45.346 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 67, 2024-05-09T04:58:45.346 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 7.250070233, 2024-05-09T04:58:45.346 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.108210003 2024-05-09T04:58:45.346 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.346 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_reply": 0, 2024-05-09T04:58:45.347 INFO:teuthology.orchestra.run.smithi169.stdout: "forward": 0, 2024-05-09T04:58:45.347 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_complete": 0, 2024-05-09T04:58:45.347 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_keys": 0, 2024-05-09T04:58:45.347 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_commit": 12, 2024-05-09T04:58:45.347 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_split": 0, 2024-05-09T04:58:45.347 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_merge": 0, 2024-05-09T04:58:45.347 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes": 25, 2024-05-09T04:58:45.347 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_top": 0, 2024-05-09T04:58:45.347 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_bottom": 25, 2024-05-09T04:58:45.347 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pin_tail": 0, 2024-05-09T04:58:45.347 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pinned": 24, 2024-05-09T04:58:45.347 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_expired": 0, 2024-05-09T04:58:45.347 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_with_caps": 1, 2024-05-09T04:58:45.347 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": 1, 2024-05-09T04:58:45.348 INFO:teuthology.orchestra.run.smithi169.stdout: "subtrees": 2, 2024-05-09T04:58:45.348 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse": 102, 2024-05-09T04:58:45.348 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_hit": 91, 2024-05-09T04:58:45.348 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_forward": 0, 2024-05-09T04:58:45.348 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_discover": 0, 2024-05-09T04:58:45.348 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_dir_fetch": 0, 2024-05-09T04:58:45.348 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_remote_ino": 0, 2024-05-09T04:58:45.348 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_lock": 0, 2024-05-09T04:58:45.348 INFO:teuthology.orchestra.run.smithi169.stdout: "load_cent": 0, 2024-05-09T04:58:45.348 INFO:teuthology.orchestra.run.smithi169.stdout: "q": 0, 2024-05-09T04:58:45.348 INFO:teuthology.orchestra.run.smithi169.stdout: "exported": 0, 2024-05-09T04:58:45.348 INFO:teuthology.orchestra.run.smithi169.stdout: "exported_inodes": 0, 2024-05-09T04:58:45.348 INFO:teuthology.orchestra.run.smithi169.stdout: "imported": 0, 2024-05-09T04:58:45.348 INFO:teuthology.orchestra.run.smithi169.stdout: "imported_inodes": 0, 2024-05-09T04:58:45.348 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_dir_fetch": 0, 2024-05-09T04:58:45.349 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_backtrace_fetch": 0, 2024-05-09T04:58:45.349 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_peer_discover": 0, 2024-05-09T04:58:45.349 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rfiles": 4, 2024-05-09T04:58:45.349 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rbytes": 234, 2024-05-09T04:58:45.349 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rsnaps": 0, 2024-05-09T04:58:45.349 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T04:58:45.349 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_set_tag": 0, 2024-05-09T04:58:45.349 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T04:58:45.349 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_inotable_repaired": 0, 2024-05-09T04:58:45.349 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_inodes": 0, 2024-05-09T04:58:45.349 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T04:58:45.349 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T04:58:45.349 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_file_inodes": 0, 2024-05-09T04:58:45.349 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_inode_file_caps": 0, 2024-05-09T04:58:45.350 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_revoke": 18, 2024-05-09T04:58:45.350 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_grant": 33, 2024-05-09T04:58:45.350 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T04:58:45.350 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T04:58:45.350 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T04:58:45.350 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps": 127, 2024-05-09T04:58:45.350 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps_dirty": 103, 2024-05-09T04:58:45.350 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_cap_release": 4, 2024-05-09T04:58:45.350 INFO:teuthology.orchestra.run.smithi169.stdout: "process_request_cap_release": 28 2024-05-09T04:58:45.350 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.350 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_cache": { 2024-05-09T04:58:45.350 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update": 19, 2024-05-09T04:58:45.350 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update_receipt": 0, 2024-05-09T04:58:45.351 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_try_discover": 0, 2024-05-09T04:58:45.351 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_send_discover": 0, 2024-05-09T04:58:45.351 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_handle_discover": 40, 2024-05-09T04:58:45.351 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays": 0, 2024-05-09T04:58:45.351 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_delayed": 0, 2024-05-09T04:58:45.351 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_enqueuing": 0, 2024-05-09T04:58:45.351 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_created": 6, 2024-05-09T04:58:45.351 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_enqueued": 6, 2024-05-09T04:58:45.351 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_reintegrated": 0, 2024-05-09T04:58:45.351 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_migrated": 0, 2024-05-09T04:58:45.351 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_processing": 0, 2024-05-09T04:58:45.351 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_enqueued": 0, 2024-05-09T04:58:45.351 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_prioritized": 0, 2024-05-09T04:58:45.351 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_started": 0, 2024-05-09T04:58:45.352 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_completed": 0, 2024-05-09T04:58:45.352 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_path": 2, 2024-05-09T04:58:45.352 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_inode": 24, 2024-05-09T04:58:45.352 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T04:58:45.352 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_exportdir": 0, 2024-05-09T04:58:45.352 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_flush": 0, 2024-05-09T04:58:45.352 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragmentdir": 0, 2024-05-09T04:58:45.352 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragstats": 0, 2024-05-09T04:58:45.352 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_inodestats": 0 2024-05-09T04:58:45.352 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.352 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_log": { 2024-05-09T04:58:45.352 INFO:teuthology.orchestra.run.smithi169.stdout: "evlrg": 0, 2024-05-09T04:58:45.352 INFO:teuthology.orchestra.run.smithi169.stdout: "evadd": 187, 2024-05-09T04:58:45.352 INFO:teuthology.orchestra.run.smithi169.stdout: "evex": 0, 2024-05-09T04:58:45.353 INFO:teuthology.orchestra.run.smithi169.stdout: "evtrm": 0, 2024-05-09T04:58:45.353 INFO:teuthology.orchestra.run.smithi169.stdout: "ev": 187, 2024-05-09T04:58:45.353 INFO:teuthology.orchestra.run.smithi169.stdout: "evexg": 0, 2024-05-09T04:58:45.353 INFO:teuthology.orchestra.run.smithi169.stdout: "evexd": 0, 2024-05-09T04:58:45.353 INFO:teuthology.orchestra.run.smithi169.stdout: "segadd": 2, 2024-05-09T04:58:45.353 INFO:teuthology.orchestra.run.smithi169.stdout: "segex": 0, 2024-05-09T04:58:45.353 INFO:teuthology.orchestra.run.smithi169.stdout: "segtrm": 0, 2024-05-09T04:58:45.353 INFO:teuthology.orchestra.run.smithi169.stdout: "seg": 2, 2024-05-09T04:58:45.353 INFO:teuthology.orchestra.run.smithi169.stdout: "segmjr": 2, 2024-05-09T04:58:45.354 INFO:teuthology.orchestra.run.smithi169.stdout: "segexg": 0, 2024-05-09T04:58:45.354 INFO:teuthology.orchestra.run.smithi169.stdout: "segexd": 0, 2024-05-09T04:58:45.354 INFO:teuthology.orchestra.run.smithi169.stdout: "expos": 4194304, 2024-05-09T04:58:45.354 INFO:teuthology.orchestra.run.smithi169.stdout: "wrpos": 4866193, 2024-05-09T04:58:45.354 INFO:teuthology.orchestra.run.smithi169.stdout: "rdpos": 0, 2024-05-09T04:58:45.354 INFO:teuthology.orchestra.run.smithi169.stdout: "jlat": { 2024-05-09T04:58:45.354 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 155, 2024-05-09T04:58:45.354 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.348691879, 2024-05-09T04:58:45.354 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.002249625 2024-05-09T04:58:45.354 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.354 INFO:teuthology.orchestra.run.smithi169.stdout: "replayed": 0 2024-05-09T04:58:45.354 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.354 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_mem": { 2024-05-09T04:58:45.354 INFO:teuthology.orchestra.run.smithi169.stdout: "ino": 24, 2024-05-09T04:58:45.355 INFO:teuthology.orchestra.run.smithi169.stdout: "ino+": 29, 2024-05-09T04:58:45.355 INFO:teuthology.orchestra.run.smithi169.stdout: "ino-": 5, 2024-05-09T04:58:45.355 INFO:teuthology.orchestra.run.smithi169.stdout: "dir": 18, 2024-05-09T04:58:45.355 INFO:teuthology.orchestra.run.smithi169.stdout: "dir+": 20, 2024-05-09T04:58:45.355 INFO:teuthology.orchestra.run.smithi169.stdout: "dir-": 2, 2024-05-09T04:58:45.355 INFO:teuthology.orchestra.run.smithi169.stdout: "dn": 25, 2024-05-09T04:58:45.355 INFO:teuthology.orchestra.run.smithi169.stdout: "dn+": 35, 2024-05-09T04:58:45.355 INFO:teuthology.orchestra.run.smithi169.stdout: "dn-": 10, 2024-05-09T04:58:45.355 INFO:teuthology.orchestra.run.smithi169.stdout: "cap": 1, 2024-05-09T04:58:45.355 INFO:teuthology.orchestra.run.smithi169.stdout: "cap+": 21, 2024-05-09T04:58:45.355 INFO:teuthology.orchestra.run.smithi169.stdout: "cap-": 20, 2024-05-09T04:58:45.355 INFO:teuthology.orchestra.run.smithi169.stdout: "rss": 124356, 2024-05-09T04:58:45.355 INFO:teuthology.orchestra.run.smithi169.stdout: "heap": 231824 2024-05-09T04:58:45.355 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.356 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_server": { 2024-05-09T04:58:45.356 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_client_request": 93, 2024-05-09T04:58:45.356 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_server_request": 45, 2024-05-09T04:58:45.356 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_request": 67, 2024-05-09T04:58:45.356 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_session": 19, 2024-05-09T04:58:45.356 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_peer_request": 90, 2024-05-09T04:58:45.356 INFO:teuthology.orchestra.run.smithi169.stdout: "req_create_latency": { 2024-05-09T04:58:45.356 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 8, 2024-05-09T04:58:45.356 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.008851836, 2024-05-09T04:58:45.356 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.001106479 2024-05-09T04:58:45.356 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.356 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getattr_latency": { 2024-05-09T04:58:45.356 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 28, 2024-05-09T04:58:45.356 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.007806259, 2024-05-09T04:58:45.357 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000278794 2024-05-09T04:58:45.357 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.357 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getfilelock_latency": { 2024-05-09T04:58:45.357 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:45.357 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:45.357 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:45.357 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.357 INFO:teuthology.orchestra.run.smithi169.stdout: "req_link_latency": { 2024-05-09T04:58:45.357 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:45.357 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:45.357 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:45.357 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.357 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookup_latency": { 2024-05-09T04:58:45.357 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1, 2024-05-09T04:58:45.357 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000553005, 2024-05-09T04:58:45.358 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000553005 2024-05-09T04:58:45.358 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.358 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookuphash_latency": { 2024-05-09T04:58:45.358 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:45.358 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:45.358 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:45.358 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.358 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupino_latency": { 2024-05-09T04:58:45.358 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:45.358 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:45.358 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:45.358 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.358 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupname_latency": { 2024-05-09T04:58:45.358 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:45.359 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:45.359 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:45.359 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.359 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupparent_latency": { 2024-05-09T04:58:45.359 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:45.359 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:45.359 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:45.359 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.359 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupsnap_latency": { 2024-05-09T04:58:45.359 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:45.359 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:45.359 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:45.359 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.359 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lssnap_latency": { 2024-05-09T04:58:45.360 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:45.360 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:45.360 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:45.360 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.361 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:44 smithi186 ceph-mon[34668]: pgmap v613: 129 pgs: 129 active+clean; 85 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 21 MiB/s wr, 18 op/s 2024-05-09T04:58:45.361 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mkdir_latency": { 2024-05-09T04:58:45.361 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 8, 2024-05-09T04:58:45.361 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 7.170939351, 2024-05-09T04:58:45.361 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.896367418 2024-05-09T04:58:45.361 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.361 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mknod_latency": { 2024-05-09T04:58:45.361 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:45.361 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:45.361 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:45.361 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.362 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mksnap_latency": { 2024-05-09T04:58:45.362 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:45.362 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:45.362 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:45.362 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.362 INFO:teuthology.orchestra.run.smithi169.stdout: "req_open_latency": { 2024-05-09T04:58:45.362 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1, 2024-05-09T04:58:45.362 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000249383, 2024-05-09T04:58:45.362 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000249383 2024-05-09T04:58:45.362 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.362 INFO:teuthology.orchestra.run.smithi169.stdout: "req_readdir_latency": { 2024-05-09T04:58:45.362 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2, 2024-05-09T04:58:45.362 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.005459431, 2024-05-09T04:58:45.362 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.002729715 2024-05-09T04:58:45.363 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.363 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rename_latency": { 2024-05-09T04:58:45.363 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2, 2024-05-09T04:58:45.363 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.003322642, 2024-05-09T04:58:45.363 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.001661321 2024-05-09T04:58:45.363 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.363 INFO:teuthology.orchestra.run.smithi169.stdout: "req_renamesnap_latency": { 2024-05-09T04:58:45.363 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:45.363 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:45.363 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:45.363 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.363 INFO:teuthology.orchestra.run.smithi169.stdout: "req_snapdiff_latency": { 2024-05-09T04:58:45.363 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:45.363 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:45.364 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:45.364 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.364 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmdir_latency": { 2024-05-09T04:58:45.364 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2, 2024-05-09T04:58:45.364 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.018863649, 2024-05-09T04:58:45.364 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.009431824 2024-05-09T04:58:45.364 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.364 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmsnap_latency": { 2024-05-09T04:58:45.364 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:45.364 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:45.364 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:45.364 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.364 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmxattr_latency": { 2024-05-09T04:58:45.364 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:45.365 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:45.365 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:45.365 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.365 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setattr_latency": { 2024-05-09T04:58:45.365 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2, 2024-05-09T04:58:45.365 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.001407467, 2024-05-09T04:58:45.365 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000703733 2024-05-09T04:58:45.365 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.365 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setdirlayout_latency": { 2024-05-09T04:58:45.365 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:45.365 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:45.365 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:45.365 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.365 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setfilelock_latency": { 2024-05-09T04:58:45.365 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:45.366 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:45.366 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:45.366 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.366 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setlayout_latency": { 2024-05-09T04:58:45.366 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:45.366 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:45.366 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:45.366 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.366 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setxattr_latency": { 2024-05-09T04:58:45.366 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 9, 2024-05-09T04:58:45.366 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.027138863, 2024-05-09T04:58:45.366 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.003015429 2024-05-09T04:58:45.366 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.366 INFO:teuthology.orchestra.run.smithi169.stdout: "req_symlink_latency": { 2024-05-09T04:58:45.367 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:45.367 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:45.367 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:45.367 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.367 INFO:teuthology.orchestra.run.smithi169.stdout: "req_unlink_latency": { 2024-05-09T04:58:45.367 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 4, 2024-05-09T04:58:45.367 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.005478347, 2024-05-09T04:58:45.367 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.001369586 2024-05-09T04:58:45.367 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.367 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_revoke_eviction": 0, 2024-05-09T04:58:45.367 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_acquisition_throttle": 0, 2024-05-09T04:58:45.367 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getvxattr_latency": { 2024-05-09T04:58:45.367 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:45.368 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:45.368 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:45.368 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:45.368 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.368 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_sessions": { 2024-05-09T04:58:45.368 INFO:teuthology.orchestra.run.smithi169.stdout: "session_count": 1, 2024-05-09T04:58:45.368 INFO:teuthology.orchestra.run.smithi169.stdout: "session_add": 4, 2024-05-09T04:58:45.368 INFO:teuthology.orchestra.run.smithi169.stdout: "session_remove": 3, 2024-05-09T04:58:45.368 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_open": 1, 2024-05-09T04:58:45.368 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_stale": 0, 2024-05-09T04:58:45.368 INFO:teuthology.orchestra.run.smithi169.stdout: "total_load": 12, 2024-05-09T04:58:45.368 INFO:teuthology.orchestra.run.smithi169.stdout: "average_load": 12, 2024-05-09T04:58:45.368 INFO:teuthology.orchestra.run.smithi169.stdout: "avg_session_uptime": 225, 2024-05-09T04:58:45.368 INFO:teuthology.orchestra.run.smithi169.stdout: "mdthresh_evicted": 0 2024-05-09T04:58:45.368 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.369 INFO:teuthology.orchestra.run.smithi169.stdout: "mempool": { 2024-05-09T04:58:45.369 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_bytes": 0, 2024-05-09T04:58:45.369 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_items": 0, 2024-05-09T04:58:45.369 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T04:58:45.369 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_items": 0, 2024-05-09T04:58:45.369 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T04:58:45.369 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_items": 0, 2024-05-09T04:58:45.369 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T04:58:45.369 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T04:58:45.369 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T04:58:45.369 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T04:58:45.369 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T04:58:45.369 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_items": 0, 2024-05-09T04:58:45.369 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T04:58:45.370 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T04:58:45.370 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_bytes": 0, 2024-05-09T04:58:45.370 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_items": 0, 2024-05-09T04:58:45.370 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_bytes": 0, 2024-05-09T04:58:45.370 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_items": 0, 2024-05-09T04:58:45.370 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T04:58:45.370 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T04:58:45.370 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T04:58:45.370 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T04:58:45.370 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T04:58:45.370 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_items": 0, 2024-05-09T04:58:45.370 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_bytes": 0, 2024-05-09T04:58:45.370 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_items": 0, 2024-05-09T04:58:45.370 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T04:58:45.371 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T04:58:45.371 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_bytes": 0, 2024-05-09T04:58:45.371 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_items": 0, 2024-05-09T04:58:45.371 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_bytes": 0, 2024-05-09T04:58:45.371 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_items": 0, 2024-05-09T04:58:45.371 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T04:58:45.371 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_items": 0, 2024-05-09T04:58:45.371 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T04:58:45.371 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_items": 0, 2024-05-09T04:58:45.371 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_bytes": 339602, 2024-05-09T04:58:45.371 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_items": 116, 2024-05-09T04:58:45.371 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_bytes": 0, 2024-05-09T04:58:45.371 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_items": 0, 2024-05-09T04:58:45.371 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_bytes": 0, 2024-05-09T04:58:45.372 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_items": 0, 2024-05-09T04:58:45.372 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_bytes": 0, 2024-05-09T04:58:45.372 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_items": 0, 2024-05-09T04:58:45.372 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_bytes": 0, 2024-05-09T04:58:45.372 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_items": 0, 2024-05-09T04:58:45.372 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_bytes": 8800, 2024-05-09T04:58:45.372 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_items": 169, 2024-05-09T04:58:45.372 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T04:58:45.372 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_items": 0, 2024-05-09T04:58:45.372 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_bytes": 0, 2024-05-09T04:58:45.372 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_items": 0, 2024-05-09T04:58:45.372 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_bytes": 103133, 2024-05-09T04:58:45.372 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_items": 914, 2024-05-09T04:58:45.372 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_bytes": 0, 2024-05-09T04:58:45.372 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_items": 0, 2024-05-09T04:58:45.373 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_bytes": 0, 2024-05-09T04:58:45.373 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_items": 0 2024-05-09T04:58:45.373 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.373 INFO:teuthology.orchestra.run.smithi169.stdout: "objecter": { 2024-05-09T04:58:45.373 INFO:teuthology.orchestra.run.smithi169.stdout: "op_active": 0, 2024-05-09T04:58:45.373 INFO:teuthology.orchestra.run.smithi169.stdout: "op_laggy": 0, 2024-05-09T04:58:45.373 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send": 824, 2024-05-09T04:58:45.373 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send_bytes": 688857, 2024-05-09T04:58:45.373 INFO:teuthology.orchestra.run.smithi169.stdout: "op_resend": 37, 2024-05-09T04:58:45.373 INFO:teuthology.orchestra.run.smithi169.stdout: "op_reply": 824, 2024-05-09T04:58:45.373 INFO:teuthology.orchestra.run.smithi169.stdout: "op_latency": { 2024-05-09T04:58:45.373 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 824, 2024-05-09T04:58:45.373 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 26.262433292, 2024-05-09T04:58:45.373 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.031871885 2024-05-09T04:58:45.374 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.374 INFO:teuthology.orchestra.run.smithi169.stdout: "op_inflight": 0, 2024-05-09T04:58:45.374 INFO:teuthology.orchestra.run.smithi169.stdout: "oplen_avg": { 2024-05-09T04:58:45.374 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 824, 2024-05-09T04:58:45.374 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 875 2024-05-09T04:58:45.374 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.374 INFO:teuthology.orchestra.run.smithi169.stdout: "op": 824, 2024-05-09T04:58:45.374 INFO:teuthology.orchestra.run.smithi169.stdout: "op_r": 4, 2024-05-09T04:58:45.374 INFO:teuthology.orchestra.run.smithi169.stdout: "op_w": 820, 2024-05-09T04:58:45.374 INFO:teuthology.orchestra.run.smithi169.stdout: "op_rmw": 0, 2024-05-09T04:58:45.374 INFO:teuthology.orchestra.run.smithi169.stdout: "op_pg": 0, 2024-05-09T04:58:45.374 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_stat": 0, 2024-05-09T04:58:45.374 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_create": 11, 2024-05-09T04:58:45.374 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_read": 4, 2024-05-09T04:58:45.375 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_write": 159, 2024-05-09T04:58:45.375 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writefull": 25, 2024-05-09T04:58:45.375 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writesame": 0, 2024-05-09T04:58:45.375 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_append": 0, 2024-05-09T04:58:45.375 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_zero": 0, 2024-05-09T04:58:45.375 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_truncate": 0, 2024-05-09T04:58:45.375 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_delete": 585, 2024-05-09T04:58:45.375 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_mapext": 0, 2024-05-09T04:58:45.375 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_sparse_read": 0, 2024-05-09T04:58:45.375 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_clonerange": 0, 2024-05-09T04:58:45.375 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_getxattr": 0, 2024-05-09T04:58:45.375 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_setxattr": 22, 2024-05-09T04:58:45.375 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_cmpxattr": 0, 2024-05-09T04:58:45.375 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_rmxattr": 0, 2024-05-09T04:58:45.375 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_resetxattrs": 0, 2024-05-09T04:58:45.376 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_call": 0, 2024-05-09T04:58:45.376 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_watch": 0, 2024-05-09T04:58:45.376 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_notify": 0, 2024-05-09T04:58:45.376 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T04:58:45.376 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls": 0, 2024-05-09T04:58:45.376 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls_filter": 0, 2024-05-09T04:58:45.376 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_other": 0, 2024-05-09T04:58:45.376 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_active": 0, 2024-05-09T04:58:45.376 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_send": 0, 2024-05-09T04:58:45.376 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_resend": 0, 2024-05-09T04:58:45.376 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_ping": 0, 2024-05-09T04:58:45.376 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_active": 0, 2024-05-09T04:58:45.376 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_send": 0, 2024-05-09T04:58:45.376 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_resend": 0, 2024-05-09T04:58:45.377 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_active": 0, 2024-05-09T04:58:45.377 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_send": 0, 2024-05-09T04:58:45.377 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_resend": 0, 2024-05-09T04:58:45.377 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_active": 0, 2024-05-09T04:58:45.377 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_send": 0, 2024-05-09T04:58:45.377 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_resend": 0, 2024-05-09T04:58:45.377 INFO:teuthology.orchestra.run.smithi169.stdout: "command_active": 0, 2024-05-09T04:58:45.377 INFO:teuthology.orchestra.run.smithi169.stdout: "command_send": 0, 2024-05-09T04:58:45.377 INFO:teuthology.orchestra.run.smithi169.stdout: "command_resend": 0, 2024-05-09T04:58:45.377 INFO:teuthology.orchestra.run.smithi169.stdout: "map_epoch": 89, 2024-05-09T04:58:45.377 INFO:teuthology.orchestra.run.smithi169.stdout: "map_full": 0, 2024-05-09T04:58:45.377 INFO:teuthology.orchestra.run.smithi169.stdout: "map_inc": 10, 2024-05-09T04:58:45.377 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_sessions": 12, 2024-05-09T04:58:45.377 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_open": 12, 2024-05-09T04:58:45.377 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_close": 0, 2024-05-09T04:58:45.378 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_laggy": 0, 2024-05-09T04:58:45.378 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_wr": 57, 2024-05-09T04:58:45.378 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_rd": 0, 2024-05-09T04:58:45.378 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_del": 12 2024-05-09T04:58:45.378 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.378 INFO:teuthology.orchestra.run.smithi169.stdout: "oft": { 2024-05-09T04:58:45.378 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_objs": 1, 2024-05-09T04:58:45.378 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_kv_pairs": 0, 2024-05-09T04:58:45.378 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_updates": 0, 2024-05-09T04:58:45.378 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_removes": 0 2024-05-09T04:58:45.378 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.378 INFO:teuthology.orchestra.run.smithi169.stdout: "osd-slow-ops": { 2024-05-09T04:58:45.378 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_ops_count": 66 2024-05-09T04:58:45.378 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.379 INFO:teuthology.orchestra.run.smithi169.stdout: "purge_queue": { 2024-05-09T04:58:45.379 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops": 0, 2024-05-09T04:58:45.379 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops_high_water": 65, 2024-05-09T04:58:45.379 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing": 0, 2024-05-09T04:58:45.379 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_high_water": 3, 2024-05-09T04:58:45.379 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed_ops": 73, 2024-05-09T04:58:45.379 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed": 6, 2024-05-09T04:58:45.379 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_item_in_journal": 0 2024-05-09T04:58:45.379 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.379 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T04:58:45.379 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T04:58:45.379 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-05-09T04:58:45.379 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T04:58:45.379 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 3175, 2024-05-09T04:58:45.379 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 1445707, 2024-05-09T04:58:45.380 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:45.380 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 3175, 2024-05-09T04:58:45.380 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-05-09T04:58:45.380 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-05-09T04:58:45.380 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 3175, 2024-05-09T04:58:45.380 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 1445707, 2024-05-09T04:58:45.380 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T04:58:45.380 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:45.380 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:45.380 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:45.380 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:45.380 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.380 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_bytes": { 2024-05-09T04:58:45.380 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T04:58:45.380 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-05-09T04:58:45.381 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T04:58:45.381 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-05-09T04:58:45.381 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-05-09T04:58:45.381 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:45.381 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-05-09T04:58:45.381 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 824, 2024-05-09T04:58:45.381 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 689495, 2024-05-09T04:58:45.381 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 824, 2024-05-09T04:58:45.381 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 689495, 2024-05-09T04:58:45.381 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T04:58:45.381 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:45.381 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:45.381 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:45.381 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:45.382 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.382 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_ops": { 2024-05-09T04:58:45.382 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T04:58:45.382 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 1024, 2024-05-09T04:58:45.382 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T04:58:45.382 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-05-09T04:58:45.382 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-05-09T04:58:45.382 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:45.382 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-05-09T04:58:45.382 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 824, 2024-05-09T04:58:45.382 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 824, 2024-05-09T04:58:45.382 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 824, 2024-05-09T04:58:45.382 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 824, 2024-05-09T04:58:45.383 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T04:58:45.383 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:45.383 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:45.383 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:45.383 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:45.383 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.383 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle": { 2024-05-09T04:58:45.383 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T04:58:45.383 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-05-09T04:58:45.383 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T04:58:45.383 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 6, 2024-05-09T04:58:45.383 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 638, 2024-05-09T04:58:45.383 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:45.383 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 6, 2024-05-09T04:58:45.384 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-05-09T04:58:45.384 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-05-09T04:58:45.384 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 4, 2024-05-09T04:58:45.384 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 638, 2024-05-09T04:58:45.384 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T04:58:45.384 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:45.384 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:45.384 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:45.384 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:45.384 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:45.384 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle-0x56463c64c410": { 2024-05-09T04:58:45.384 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 599, 2024-05-09T04:58:45.384 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-05-09T04:58:45.384 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T04:58:45.384 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 187, 2024-05-09T04:58:45.385 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 671889, 2024-05-09T04:58:45.385 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:45.385 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 187, 2024-05-09T04:58:45.385 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-05-09T04:58:45.385 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-05-09T04:58:45.385 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 155, 2024-05-09T04:58:45.385 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 671290, 2024-05-09T04:58:45.385 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T04:58:45.385 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:45.385 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:45.385 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:45.385 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:45.385 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:45.385 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-05-09T04:58:45.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:44 smithi006 ceph-mon[29835]: pgmap v613: 129 pgs: 129 active+clean; 85 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 21 MiB/s wr, 18 op/s 2024-05-09T04:58:46.053 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-05-09T04:58:46.053 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-05-09T04:58:46.053 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=19 2024-05-09T04:58:46.053 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-05-09T04:58:46.053 DEBUG:tasks.check_counter:Getting stats from e 2024-05-09T04:58:46.054 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-09T04:58:46.341 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T04:58:47.101 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:46 smithi169 ceph-mon[34075]: pgmap v614: 129 pgs: 129 active+clean; 45 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 21 MiB/s wr, 21 op/s 2024-05-09T04:58:47.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:46 smithi186 ceph-mon[34668]: pgmap v614: 129 pgs: 129 active+clean; 45 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 21 MiB/s wr, 21 op/s 2024-05-09T04:58:47.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:46 smithi006 ceph-mon[29835]: pgmap v614: 129 pgs: 129 active+clean; 45 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 21 MiB/s wr, 21 op/s 2024-05-09T04:58:47.960 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-05-09T04:58:47.960 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T04:58:47.960 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 1020, 2024-05-09T04:58:47.960 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 1020, 2024-05-09T04:58:47.960 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 974670, 2024-05-09T04:58:47.961 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 323546, 2024-05-09T04:58:47.961 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 6, 2024-05-09T04:58:47.961 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 2, 2024-05-09T04:58:47.961 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.100464505, 2024-05-09T04:58:47.961 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.038919893, 2024-05-09T04:58:47.961 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.024252114, 2024-05-09T04:58:47.961 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.027487956, 2024-05-09T04:58:47.961 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:58:47.961 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1019, 2024-05-09T04:58:47.961 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.040509707, 2024-05-09T04:58:47.961 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000039754 2024-05-09T04:58:47.961 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.961 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:58:47.961 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.962 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.962 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.962 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.962 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-05-09T04:58:47.962 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T04:58:47.962 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.962 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T04:58:47.962 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 179, 2024-05-09T04:58:47.962 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 172, 2024-05-09T04:58:47.962 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 829720, 2024-05-09T04:58:47.962 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 101578, 2024-05-09T04:58:47.962 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 4, 2024-05-09T04:58:47.962 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 3, 2024-05-09T04:58:47.963 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.033352278, 2024-05-09T04:58:47.963 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.011337219, 2024-05-09T04:58:47.963 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.021433515, 2024-05-09T04:58:47.963 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.004247408, 2024-05-09T04:58:47.963 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:58:47.963 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 171, 2024-05-09T04:58:47.963 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.013365551, 2024-05-09T04:58:47.963 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000078161 2024-05-09T04:58:47.963 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.963 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:58:47.963 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.963 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.963 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.963 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.963 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 828064, 2024-05-09T04:58:47.964 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 100240 2024-05-09T04:58:47.964 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.964 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T04:58:47.964 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 466, 2024-05-09T04:58:47.964 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 588, 2024-05-09T04:58:47.964 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 114621, 2024-05-09T04:58:47.964 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 407727, 2024-05-09T04:58:47.964 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 4, 2024-05-09T04:58:47.964 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 3, 2024-05-09T04:58:47.964 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.050253250, 2024-05-09T04:58:47.964 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.017857975, 2024-05-09T04:58:47.964 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.010564956, 2024-05-09T04:58:47.964 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.016657902, 2024-05-09T04:58:47.964 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:58:47.965 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 586, 2024-05-09T04:58:47.965 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.021288061, 2024-05-09T04:58:47.965 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000036327 2024-05-09T04:58:47.965 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.965 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:58:47.965 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.965 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.965 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.965 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.965 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T04:58:47.965 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 259856 2024-05-09T04:58:47.965 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.965 INFO:teuthology.orchestra.run.smithi169.stdout: "cct": { 2024-05-09T04:58:47.965 INFO:teuthology.orchestra.run.smithi169.stdout: "total_workers": 1, 2024-05-09T04:58:47.966 INFO:teuthology.orchestra.run.smithi169.stdout: "unhealthy_workers": 0 2024-05-09T04:58:47.966 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.966 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-MDSRank": { 2024-05-09T04:58:47.966 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-05-09T04:58:47.966 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-05-09T04:58:47.966 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2492, 2024-05-09T04:58:47.966 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.070426928, 2024-05-09T04:58:47.966 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000028261 2024-05-09T04:58:47.966 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:47.966 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.966 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-PurgeQueue": { 2024-05-09T04:58:47.966 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-05-09T04:58:47.966 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-05-09T04:58:47.966 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.967 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.967 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.967 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:47.967 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.967 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": { 2024-05-09T04:58:47.967 INFO:teuthology.orchestra.run.smithi169.stdout: "request": 0, 2024-05-09T04:58:47.967 INFO:teuthology.orchestra.run.smithi169.stdout: "reply": 0, 2024-05-09T04:58:47.967 INFO:teuthology.orchestra.run.smithi169.stdout: "reply_latency": { 2024-05-09T04:58:47.967 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.967 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.967 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.967 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.967 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_reply": 0, 2024-05-09T04:58:47.967 INFO:teuthology.orchestra.run.smithi169.stdout: "forward": 0, 2024-05-09T04:58:47.968 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_complete": 0, 2024-05-09T04:58:47.968 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_keys": 0, 2024-05-09T04:58:47.968 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_commit": 0, 2024-05-09T04:58:47.968 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_split": 0, 2024-05-09T04:58:47.968 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_merge": 0, 2024-05-09T04:58:47.968 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes": 20, 2024-05-09T04:58:47.968 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_top": 0, 2024-05-09T04:58:47.968 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_bottom": 20, 2024-05-09T04:58:47.968 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pin_tail": 0, 2024-05-09T04:58:47.968 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pinned": 20, 2024-05-09T04:58:47.968 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_expired": 0, 2024-05-09T04:58:47.968 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_with_caps": 0, 2024-05-09T04:58:47.968 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": 0, 2024-05-09T04:58:47.968 INFO:teuthology.orchestra.run.smithi169.stdout: "subtrees": 2, 2024-05-09T04:58:47.968 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse": 0, 2024-05-09T04:58:47.969 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_hit": 0, 2024-05-09T04:58:47.969 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_forward": 0, 2024-05-09T04:58:47.969 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_discover": 0, 2024-05-09T04:58:47.969 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_dir_fetch": 0, 2024-05-09T04:58:47.969 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_remote_ino": 0, 2024-05-09T04:58:47.969 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_lock": 0, 2024-05-09T04:58:47.969 INFO:teuthology.orchestra.run.smithi169.stdout: "load_cent": 0, 2024-05-09T04:58:47.969 INFO:teuthology.orchestra.run.smithi169.stdout: "q": 0, 2024-05-09T04:58:47.969 INFO:teuthology.orchestra.run.smithi169.stdout: "exported": 0, 2024-05-09T04:58:47.969 INFO:teuthology.orchestra.run.smithi169.stdout: "exported_inodes": 0, 2024-05-09T04:58:47.969 INFO:teuthology.orchestra.run.smithi169.stdout: "imported": 0, 2024-05-09T04:58:47.969 INFO:teuthology.orchestra.run.smithi169.stdout: "imported_inodes": 0, 2024-05-09T04:58:47.969 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_dir_fetch": 0, 2024-05-09T04:58:47.969 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_backtrace_fetch": 0, 2024-05-09T04:58:47.969 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_peer_discover": 0, 2024-05-09T04:58:47.970 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rfiles": 4, 2024-05-09T04:58:47.970 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rbytes": 234, 2024-05-09T04:58:47.970 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rsnaps": 0, 2024-05-09T04:58:47.970 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T04:58:47.970 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_set_tag": 0, 2024-05-09T04:58:47.970 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T04:58:47.970 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_inotable_repaired": 0, 2024-05-09T04:58:47.970 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_inodes": 0, 2024-05-09T04:58:47.970 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T04:58:47.970 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T04:58:47.970 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_file_inodes": 0, 2024-05-09T04:58:47.970 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_inode_file_caps": 0, 2024-05-09T04:58:47.970 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T04:58:47.971 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_grant": 0, 2024-05-09T04:58:47.971 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T04:58:47.971 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T04:58:47.971 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T04:58:47.971 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps": 0, 2024-05-09T04:58:47.971 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps_dirty": 0, 2024-05-09T04:58:47.971 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_cap_release": 0, 2024-05-09T04:58:47.971 INFO:teuthology.orchestra.run.smithi169.stdout: "process_request_cap_release": 0 2024-05-09T04:58:47.971 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.971 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_cache": { 2024-05-09T04:58:47.971 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update": 0, 2024-05-09T04:58:47.971 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update_receipt": 0, 2024-05-09T04:58:47.971 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_try_discover": 0, 2024-05-09T04:58:47.971 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_send_discover": 0, 2024-05-09T04:58:47.971 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_handle_discover": 0, 2024-05-09T04:58:47.972 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays": 0, 2024-05-09T04:58:47.972 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_delayed": 0, 2024-05-09T04:58:47.972 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_enqueuing": 0, 2024-05-09T04:58:47.972 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_created": 0, 2024-05-09T04:58:47.972 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_enqueued": 0, 2024-05-09T04:58:47.972 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_reintegrated": 0, 2024-05-09T04:58:47.972 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_migrated": 0, 2024-05-09T04:58:47.972 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_processing": 0, 2024-05-09T04:58:47.972 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_enqueued": 0, 2024-05-09T04:58:47.972 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_prioritized": 0, 2024-05-09T04:58:47.972 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_started": 0, 2024-05-09T04:58:47.972 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_completed": 0, 2024-05-09T04:58:47.972 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_path": 0, 2024-05-09T04:58:47.972 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_inode": 0, 2024-05-09T04:58:47.972 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T04:58:47.973 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_exportdir": 0, 2024-05-09T04:58:47.973 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_flush": 0, 2024-05-09T04:58:47.973 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragmentdir": 0, 2024-05-09T04:58:47.973 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragstats": 0, 2024-05-09T04:58:47.973 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_inodestats": 0 2024-05-09T04:58:47.973 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.973 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_log": { 2024-05-09T04:58:47.973 INFO:teuthology.orchestra.run.smithi169.stdout: "evlrg": 0, 2024-05-09T04:58:47.973 INFO:teuthology.orchestra.run.smithi169.stdout: "evadd": 0, 2024-05-09T04:58:47.973 INFO:teuthology.orchestra.run.smithi169.stdout: "evex": 0, 2024-05-09T04:58:47.973 INFO:teuthology.orchestra.run.smithi169.stdout: "evtrm": 0, 2024-05-09T04:58:47.973 INFO:teuthology.orchestra.run.smithi169.stdout: "ev": 187, 2024-05-09T04:58:47.973 INFO:teuthology.orchestra.run.smithi169.stdout: "evexg": 0, 2024-05-09T04:58:47.973 INFO:teuthology.orchestra.run.smithi169.stdout: "evexd": 0, 2024-05-09T04:58:47.974 INFO:teuthology.orchestra.run.smithi169.stdout: "segadd": 0, 2024-05-09T04:58:47.974 INFO:teuthology.orchestra.run.smithi169.stdout: "segex": 0, 2024-05-09T04:58:47.974 INFO:teuthology.orchestra.run.smithi169.stdout: "segtrm": 0, 2024-05-09T04:58:47.974 INFO:teuthology.orchestra.run.smithi169.stdout: "seg": 2, 2024-05-09T04:58:47.974 INFO:teuthology.orchestra.run.smithi169.stdout: "segmjr": 2, 2024-05-09T04:58:47.974 INFO:teuthology.orchestra.run.smithi169.stdout: "segexg": 0, 2024-05-09T04:58:47.974 INFO:teuthology.orchestra.run.smithi169.stdout: "segexd": 0, 2024-05-09T04:58:47.974 INFO:teuthology.orchestra.run.smithi169.stdout: "expos": 4194304, 2024-05-09T04:58:47.974 INFO:teuthology.orchestra.run.smithi169.stdout: "wrpos": 4866193, 2024-05-09T04:58:47.974 INFO:teuthology.orchestra.run.smithi169.stdout: "rdpos": 4865594, 2024-05-09T04:58:47.974 INFO:teuthology.orchestra.run.smithi169.stdout: "jlat": { 2024-05-09T04:58:47.974 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.974 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.974 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.975 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.975 INFO:teuthology.orchestra.run.smithi169.stdout: "replayed": 187 2024-05-09T04:58:47.975 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.975 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_mem": { 2024-05-09T04:58:47.975 INFO:teuthology.orchestra.run.smithi169.stdout: "ino": 14, 2024-05-09T04:58:47.975 INFO:teuthology.orchestra.run.smithi169.stdout: "ino+": 20, 2024-05-09T04:58:47.975 INFO:teuthology.orchestra.run.smithi169.stdout: "ino-": 6, 2024-05-09T04:58:47.975 INFO:teuthology.orchestra.run.smithi169.stdout: "dir": 9, 2024-05-09T04:58:47.975 INFO:teuthology.orchestra.run.smithi169.stdout: "dir+": 11, 2024-05-09T04:58:47.975 INFO:teuthology.orchestra.run.smithi169.stdout: "dir-": 2, 2024-05-09T04:58:47.975 INFO:teuthology.orchestra.run.smithi169.stdout: "dn": 20, 2024-05-09T04:58:47.975 INFO:teuthology.orchestra.run.smithi169.stdout: "dn+": 25, 2024-05-09T04:58:47.975 INFO:teuthology.orchestra.run.smithi169.stdout: "dn-": 5, 2024-05-09T04:58:47.975 INFO:teuthology.orchestra.run.smithi169.stdout: "cap": 0, 2024-05-09T04:58:47.975 INFO:teuthology.orchestra.run.smithi169.stdout: "cap+": 0, 2024-05-09T04:58:47.976 INFO:teuthology.orchestra.run.smithi169.stdout: "cap-": 0, 2024-05-09T04:58:47.976 INFO:teuthology.orchestra.run.smithi169.stdout: "rss": 121664, 2024-05-09T04:58:47.976 INFO:teuthology.orchestra.run.smithi169.stdout: "heap": 215440 2024-05-09T04:58:47.976 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.976 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_server": { 2024-05-09T04:58:47.976 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_client_request": 0, 2024-05-09T04:58:47.976 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_server_request": 0, 2024-05-09T04:58:47.976 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_request": 0, 2024-05-09T04:58:47.976 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_session": 0, 2024-05-09T04:58:47.976 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_peer_request": 0, 2024-05-09T04:58:47.976 INFO:teuthology.orchestra.run.smithi169.stdout: "req_create_latency": { 2024-05-09T04:58:47.976 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.976 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.976 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.976 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.977 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getattr_latency": { 2024-05-09T04:58:47.977 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.977 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.977 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.977 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.977 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getfilelock_latency": { 2024-05-09T04:58:47.977 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.977 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.977 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.977 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.977 INFO:teuthology.orchestra.run.smithi169.stdout: "req_link_latency": { 2024-05-09T04:58:47.977 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.977 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.977 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.977 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.978 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookup_latency": { 2024-05-09T04:58:47.978 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.978 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.978 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.978 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.978 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookuphash_latency": { 2024-05-09T04:58:47.978 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.978 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.978 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.978 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.978 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupino_latency": { 2024-05-09T04:58:47.978 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.978 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.978 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.979 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.979 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupname_latency": { 2024-05-09T04:58:47.979 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.979 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.979 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.979 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.979 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupparent_latency": { 2024-05-09T04:58:47.979 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.979 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.979 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.979 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.979 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupsnap_latency": { 2024-05-09T04:58:47.979 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.979 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.979 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.980 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.980 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lssnap_latency": { 2024-05-09T04:58:47.980 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.980 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.980 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.980 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.980 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mkdir_latency": { 2024-05-09T04:58:47.980 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.981 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.981 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.981 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.981 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mknod_latency": { 2024-05-09T04:58:47.981 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.981 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.981 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.981 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.981 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mksnap_latency": { 2024-05-09T04:58:47.981 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.981 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.981 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.982 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.982 INFO:teuthology.orchestra.run.smithi169.stdout: "req_open_latency": { 2024-05-09T04:58:47.982 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.982 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.982 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.982 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.982 INFO:teuthology.orchestra.run.smithi169.stdout: "req_readdir_latency": { 2024-05-09T04:58:47.982 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.982 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.982 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.982 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.982 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rename_latency": { 2024-05-09T04:58:47.982 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.982 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.983 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.983 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.983 INFO:teuthology.orchestra.run.smithi169.stdout: "req_renamesnap_latency": { 2024-05-09T04:58:47.983 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.983 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.983 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.983 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.983 INFO:teuthology.orchestra.run.smithi169.stdout: "req_snapdiff_latency": { 2024-05-09T04:58:47.983 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.983 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.983 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.983 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.983 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmdir_latency": { 2024-05-09T04:58:47.983 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.984 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.984 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.984 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.984 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmsnap_latency": { 2024-05-09T04:58:47.984 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.984 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.984 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.984 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.984 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmxattr_latency": { 2024-05-09T04:58:47.984 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.984 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.984 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.984 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.984 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setattr_latency": { 2024-05-09T04:58:47.985 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.985 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.985 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.985 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.985 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setdirlayout_latency": { 2024-05-09T04:58:47.985 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.985 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.985 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.985 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.985 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setfilelock_latency": { 2024-05-09T04:58:47.985 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.985 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.985 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.985 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.985 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setlayout_latency": { 2024-05-09T04:58:47.986 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.986 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.986 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.986 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.986 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setxattr_latency": { 2024-05-09T04:58:47.986 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.986 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.986 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.986 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.986 INFO:teuthology.orchestra.run.smithi169.stdout: "req_symlink_latency": { 2024-05-09T04:58:47.986 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.986 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.986 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.986 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.987 INFO:teuthology.orchestra.run.smithi169.stdout: "req_unlink_latency": { 2024-05-09T04:58:47.987 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.987 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.987 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.987 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.987 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_revoke_eviction": 0, 2024-05-09T04:58:47.987 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_acquisition_throttle": 0, 2024-05-09T04:58:47.987 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getvxattr_latency": { 2024-05-09T04:58:47.987 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:47.987 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:47.987 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:47.987 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:47.987 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.988 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_sessions": { 2024-05-09T04:58:47.988 INFO:teuthology.orchestra.run.smithi169.stdout: "session_count": 1, 2024-05-09T04:58:47.988 INFO:teuthology.orchestra.run.smithi169.stdout: "session_add": 4, 2024-05-09T04:58:47.988 INFO:teuthology.orchestra.run.smithi169.stdout: "session_remove": 3, 2024-05-09T04:58:47.988 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_open": 2, 2024-05-09T04:58:47.988 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_stale": 0, 2024-05-09T04:58:47.988 INFO:teuthology.orchestra.run.smithi169.stdout: "total_load": 0, 2024-05-09T04:58:47.988 INFO:teuthology.orchestra.run.smithi169.stdout: "average_load": 0, 2024-05-09T04:58:47.988 INFO:teuthology.orchestra.run.smithi169.stdout: "avg_session_uptime": 3789, 2024-05-09T04:58:47.988 INFO:teuthology.orchestra.run.smithi169.stdout: "mdthresh_evicted": 0 2024-05-09T04:58:47.988 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.988 INFO:teuthology.orchestra.run.smithi169.stdout: "mempool": { 2024-05-09T04:58:47.988 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_bytes": 0, 2024-05-09T04:58:47.988 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_items": 0, 2024-05-09T04:58:47.989 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T04:58:47.989 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_items": 0, 2024-05-09T04:58:47.989 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T04:58:47.989 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_items": 0, 2024-05-09T04:58:47.989 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T04:58:47.989 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T04:58:47.989 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T04:58:47.989 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T04:58:47.989 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T04:58:47.989 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_items": 0, 2024-05-09T04:58:47.989 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T04:58:47.989 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T04:58:47.989 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_bytes": 0, 2024-05-09T04:58:47.989 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_items": 0, 2024-05-09T04:58:47.989 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_bytes": 0, 2024-05-09T04:58:47.990 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_items": 0, 2024-05-09T04:58:47.990 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T04:58:47.990 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T04:58:47.990 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T04:58:47.990 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T04:58:47.990 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T04:58:47.990 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_items": 0, 2024-05-09T04:58:47.990 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_bytes": 0, 2024-05-09T04:58:47.990 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_items": 0, 2024-05-09T04:58:47.990 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T04:58:47.990 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T04:58:47.990 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_bytes": 0, 2024-05-09T04:58:47.990 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_items": 0, 2024-05-09T04:58:47.990 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_bytes": 0, 2024-05-09T04:58:47.990 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_items": 0, 2024-05-09T04:58:47.991 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T04:58:47.991 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_items": 0, 2024-05-09T04:58:47.991 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T04:58:47.991 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_items": 0, 2024-05-09T04:58:47.991 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_bytes": 189415, 2024-05-09T04:58:47.991 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_items": 52, 2024-05-09T04:58:47.991 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_bytes": 0, 2024-05-09T04:58:47.991 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_items": 0, 2024-05-09T04:58:47.991 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_bytes": 0, 2024-05-09T04:58:47.991 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_items": 0, 2024-05-09T04:58:47.991 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_bytes": 0, 2024-05-09T04:58:47.991 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_items": 0, 2024-05-09T04:58:47.991 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_bytes": 0, 2024-05-09T04:58:47.991 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_items": 0, 2024-05-09T04:58:47.992 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_bytes": 8800, 2024-05-09T04:58:47.992 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_items": 169, 2024-05-09T04:58:47.992 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T04:58:47.992 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_items": 0, 2024-05-09T04:58:47.992 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_bytes": 0, 2024-05-09T04:58:47.992 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_items": 0, 2024-05-09T04:58:47.992 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_bytes": 59492, 2024-05-09T04:58:47.992 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_items": 385, 2024-05-09T04:58:47.992 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_bytes": 0, 2024-05-09T04:58:47.992 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_items": 0, 2024-05-09T04:58:47.992 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_bytes": 0, 2024-05-09T04:58:47.992 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_items": 0 2024-05-09T04:58:47.992 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.992 INFO:teuthology.orchestra.run.smithi169.stdout: "objecter": { 2024-05-09T04:58:47.992 INFO:teuthology.orchestra.run.smithi169.stdout: "op_active": 0, 2024-05-09T04:58:47.993 INFO:teuthology.orchestra.run.smithi169.stdout: "op_laggy": 0, 2024-05-09T04:58:47.993 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send": 1485, 2024-05-09T04:58:47.993 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send_bytes": 26, 2024-05-09T04:58:47.993 INFO:teuthology.orchestra.run.smithi169.stdout: "op_resend": 4, 2024-05-09T04:58:47.993 INFO:teuthology.orchestra.run.smithi169.stdout: "op_reply": 1485, 2024-05-09T04:58:47.993 INFO:teuthology.orchestra.run.smithi169.stdout: "op_latency": { 2024-05-09T04:58:47.993 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1485, 2024-05-09T04:58:47.993 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.660985766, 2024-05-09T04:58:47.993 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000445108 2024-05-09T04:58:47.993 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.993 INFO:teuthology.orchestra.run.smithi169.stdout: "op_inflight": 0, 2024-05-09T04:58:47.993 INFO:teuthology.orchestra.run.smithi169.stdout: "oplen_avg": { 2024-05-09T04:58:47.993 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1485, 2024-05-09T04:58:47.993 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 1486 2024-05-09T04:58:47.993 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.994 INFO:teuthology.orchestra.run.smithi169.stdout: "op": 1485, 2024-05-09T04:58:47.994 INFO:teuthology.orchestra.run.smithi169.stdout: "op_r": 1485, 2024-05-09T04:58:47.994 INFO:teuthology.orchestra.run.smithi169.stdout: "op_w": 0, 2024-05-09T04:58:47.994 INFO:teuthology.orchestra.run.smithi169.stdout: "op_rmw": 0, 2024-05-09T04:58:47.994 INFO:teuthology.orchestra.run.smithi169.stdout: "op_pg": 0, 2024-05-09T04:58:47.994 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_stat": 928, 2024-05-09T04:58:47.994 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_create": 0, 2024-05-09T04:58:47.994 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_read": 554, 2024-05-09T04:58:47.994 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_write": 0, 2024-05-09T04:58:47.994 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writefull": 0, 2024-05-09T04:58:47.994 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writesame": 0, 2024-05-09T04:58:47.994 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_append": 0, 2024-05-09T04:58:47.994 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_zero": 0, 2024-05-09T04:58:47.995 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_truncate": 0, 2024-05-09T04:58:47.995 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_delete": 0, 2024-05-09T04:58:47.995 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_mapext": 0, 2024-05-09T04:58:47.995 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_sparse_read": 0, 2024-05-09T04:58:47.995 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_clonerange": 0, 2024-05-09T04:58:47.995 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_getxattr": 2, 2024-05-09T04:58:47.995 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_setxattr": 0, 2024-05-09T04:58:47.995 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_cmpxattr": 0, 2024-05-09T04:58:47.995 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_rmxattr": 0, 2024-05-09T04:58:47.995 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_resetxattrs": 0, 2024-05-09T04:58:47.995 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_call": 0, 2024-05-09T04:58:47.995 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_watch": 0, 2024-05-09T04:58:47.995 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_notify": 0, 2024-05-09T04:58:47.995 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T04:58:47.995 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls": 0, 2024-05-09T04:58:47.995 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls_filter": 0, 2024-05-09T04:58:47.996 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_other": 0, 2024-05-09T04:58:47.996 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_active": 0, 2024-05-09T04:58:47.996 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_send": 0, 2024-05-09T04:58:47.996 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_resend": 0, 2024-05-09T04:58:47.996 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_ping": 0, 2024-05-09T04:58:47.996 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_active": 0, 2024-05-09T04:58:47.996 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_send": 0, 2024-05-09T04:58:47.996 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_resend": 0, 2024-05-09T04:58:47.996 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_active": 0, 2024-05-09T04:58:47.996 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_send": 0, 2024-05-09T04:58:47.996 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_resend": 0, 2024-05-09T04:58:47.996 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_active": 0, 2024-05-09T04:58:47.996 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_send": 0, 2024-05-09T04:58:47.996 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_resend": 0, 2024-05-09T04:58:47.996 INFO:teuthology.orchestra.run.smithi169.stdout: "command_active": 0, 2024-05-09T04:58:47.997 INFO:teuthology.orchestra.run.smithi169.stdout: "command_send": 0, 2024-05-09T04:58:47.997 INFO:teuthology.orchestra.run.smithi169.stdout: "command_resend": 0, 2024-05-09T04:58:47.997 INFO:teuthology.orchestra.run.smithi169.stdout: "map_epoch": 0, 2024-05-09T04:58:47.997 INFO:teuthology.orchestra.run.smithi169.stdout: "map_full": 0, 2024-05-09T04:58:47.997 INFO:teuthology.orchestra.run.smithi169.stdout: "map_inc": 0, 2024-05-09T04:58:47.997 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_sessions": 5, 2024-05-09T04:58:47.997 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_open": 5, 2024-05-09T04:58:47.997 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_close": 0, 2024-05-09T04:58:47.997 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_laggy": 0, 2024-05-09T04:58:47.997 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_wr": 0, 2024-05-09T04:58:47.997 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_rd": 2, 2024-05-09T04:58:47.997 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_del": 0 2024-05-09T04:58:47.997 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.997 INFO:teuthology.orchestra.run.smithi169.stdout: "oft": { 2024-05-09T04:58:47.997 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_objs": 0, 2024-05-09T04:58:47.998 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_kv_pairs": 0, 2024-05-09T04:58:47.998 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_updates": 0, 2024-05-09T04:58:47.998 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_removes": 0 2024-05-09T04:58:47.998 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.998 INFO:teuthology.orchestra.run.smithi169.stdout: "osd-slow-ops": { 2024-05-09T04:58:47.998 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_ops_count": 0 2024-05-09T04:58:47.998 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.998 INFO:teuthology.orchestra.run.smithi169.stdout: "purge_queue": { 2024-05-09T04:58:47.998 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops": 0, 2024-05-09T04:58:47.998 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T04:58:47.998 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing": 0, 2024-05-09T04:58:47.998 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_high_water": 0, 2024-05-09T04:58:47.998 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed_ops": 0, 2024-05-09T04:58:47.998 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed": 0, 2024-05-09T04:58:47.999 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_item_in_journal": 0 2024-05-09T04:58:47.999 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:47.999 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T04:58:47.999 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T04:58:47.999 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-05-09T04:58:47.999 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T04:58:47.999 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 1661, 2024-05-09T04:58:47.999 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 1757498, 2024-05-09T04:58:47.999 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:47.999 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 1661, 2024-05-09T04:58:47.999 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-05-09T04:58:47.999 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-05-09T04:58:47.999 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 1661, 2024-05-09T04:58:48.000 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 1757498, 2024-05-09T04:58:48.000 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T04:58:48.000 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:48.000 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:48.000 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:48.000 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:48.000 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:48.000 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_bytes": { 2024-05-09T04:58:48.000 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T04:58:48.000 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-05-09T04:58:48.000 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T04:58:48.000 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-05-09T04:58:48.000 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-05-09T04:58:48.000 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:48.000 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-05-09T04:58:48.001 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 1485, 2024-05-09T04:58:48.001 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 671899, 2024-05-09T04:58:48.001 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 1485, 2024-05-09T04:58:48.001 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 671899, 2024-05-09T04:58:48.001 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T04:58:48.001 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:48.001 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:48.001 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:48.001 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:48.001 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:48.001 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_ops": { 2024-05-09T04:58:48.001 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T04:58:48.001 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 1024, 2024-05-09T04:58:48.002 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T04:58:48.002 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-05-09T04:58:48.002 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-05-09T04:58:48.002 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:48.002 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-05-09T04:58:48.002 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 1485, 2024-05-09T04:58:48.002 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 1485, 2024-05-09T04:58:48.002 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 1485, 2024-05-09T04:58:48.002 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 1485, 2024-05-09T04:58:48.002 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T04:58:48.002 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:48.002 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:48.002 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:48.002 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:48.003 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:48.003 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle": { 2024-05-09T04:58:48.003 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T04:58:48.003 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-05-09T04:58:48.003 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T04:58:48.003 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-05-09T04:58:48.003 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-05-09T04:58:48.003 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:48.003 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-05-09T04:58:48.003 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-05-09T04:58:48.003 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-05-09T04:58:48.003 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 0, 2024-05-09T04:58:48.004 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 0, 2024-05-09T04:58:48.004 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T04:58:48.004 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:48.004 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:48.004 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:48.004 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:48.004 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:48.004 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle-0x55d80d1b2410": { 2024-05-09T04:58:48.004 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T04:58:48.004 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-05-09T04:58:48.004 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T04:58:48.004 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-05-09T04:58:48.004 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-05-09T04:58:48.005 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:48.005 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-05-09T04:58:48.005 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-05-09T04:58:48.005 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-05-09T04:58:48.005 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 0, 2024-05-09T04:58:48.005 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 0, 2024-05-09T04:58:48.005 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T04:58:48.005 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:48.005 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:48.005 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:48.005 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:48.005 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:48.005 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-05-09T04:58:48.763 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-09T04:58:48.763 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-09T04:58:48.763 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-05-09T04:58:48.764 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-05-09T04:58:48.764 DEBUG:tasks.check_counter:Getting stats from h 2024-05-09T04:58:48.764 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-09T04:58:49.005 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T04:58:49.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:48 smithi169 ceph-mon[34075]: pgmap v615: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 21 MiB/s wr, 20 op/s 2024-05-09T04:58:49.250 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:48 smithi006 ceph-mon[29835]: pgmap v615: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 21 MiB/s wr, 20 op/s 2024-05-09T04:58:49.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:48 smithi186 ceph-mon[34668]: pgmap v615: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 21 MiB/s wr, 20 op/s 2024-05-09T04:58:50.818 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-05-09T04:58:50.818 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T04:58:50.818 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 522, 2024-05-09T04:58:50.818 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 572, 2024-05-09T04:58:50.818 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 942739, 2024-05-09T04:58:50.818 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 231809, 2024-05-09T04:58:50.818 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 7, 2024-05-09T04:58:50.819 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 2, 2024-05-09T04:58:50.819 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.077275470, 2024-05-09T04:58:50.819 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.030739642, 2024-05-09T04:58:50.819 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.045316691, 2024-05-09T04:58:50.819 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.006835370, 2024-05-09T04:58:50.819 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:58:50.819 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 574, 2024-05-09T04:58:50.819 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.037516242, 2024-05-09T04:58:50.819 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000065359 2024-05-09T04:58:50.819 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.819 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:58:50.819 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 648, 2024-05-09T04:58:50.819 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000218957, 2024-05-09T04:58:50.819 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000337 2024-05-09T04:58:50.820 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.820 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 850944, 2024-05-09T04:58:50.820 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 103440 2024-05-09T04:58:50.820 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.820 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T04:58:50.820 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 34, 2024-05-09T04:58:50.820 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 156, 2024-05-09T04:58:50.820 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 10508, 2024-05-09T04:58:50.820 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 324708, 2024-05-09T04:58:50.820 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 13, 2024-05-09T04:58:50.820 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 6, 2024-05-09T04:58:50.820 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.014357517, 2024-05-09T04:58:50.820 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.006579533, 2024-05-09T04:58:50.820 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.006092969, 2024-05-09T04:58:50.821 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.000347074, 2024-05-09T04:58:50.821 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:58:50.821 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 152, 2024-05-09T04:58:50.821 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.022408369, 2024-05-09T04:58:50.821 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000147423 2024-05-09T04:58:50.821 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.821 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:58:50.821 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 19, 2024-05-09T04:58:50.821 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000008495, 2024-05-09T04:58:50.821 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000447 2024-05-09T04:58:50.821 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.821 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 4288, 2024-05-09T04:58:50.821 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 262640 2024-05-09T04:58:50.821 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.821 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T04:58:50.822 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 15, 2024-05-09T04:58:50.822 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 17, 2024-05-09T04:58:50.822 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 3820, 2024-05-09T04:58:50.822 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 30396, 2024-05-09T04:58:50.822 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 8, 2024-05-09T04:58:50.822 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 4, 2024-05-09T04:58:50.822 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.003407647, 2024-05-09T04:58:50.822 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.000233338, 2024-05-09T04:58:50.822 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.002565946, 2024-05-09T04:58:50.822 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.000420555, 2024-05-09T04:58:50.822 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:58:50.822 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 15, 2024-05-09T04:58:50.822 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.018499299, 2024-05-09T04:58:50.822 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.001233286 2024-05-09T04:58:50.823 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.823 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:58:50.823 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2, 2024-05-09T04:58:50.823 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000225, 2024-05-09T04:58:50.823 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000112 2024-05-09T04:58:50.823 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.823 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T04:58:50.823 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T04:58:50.823 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.823 INFO:teuthology.orchestra.run.smithi169.stdout: "cct": { 2024-05-09T04:58:50.823 INFO:teuthology.orchestra.run.smithi169.stdout: "total_workers": 1, 2024-05-09T04:58:50.823 INFO:teuthology.orchestra.run.smithi169.stdout: "unhealthy_workers": 0 2024-05-09T04:58:50.823 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.823 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-MDSRank": { 2024-05-09T04:58:50.824 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-05-09T04:58:50.824 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-05-09T04:58:50.824 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 54, 2024-05-09T04:58:50.824 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.001813208, 2024-05-09T04:58:50.824 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000033577 2024-05-09T04:58:50.824 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:50.824 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.824 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-PurgeQueue": { 2024-05-09T04:58:50.824 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-05-09T04:58:50.824 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-05-09T04:58:50.824 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2, 2024-05-09T04:58:50.824 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000009652, 2024-05-09T04:58:50.824 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000004826 2024-05-09T04:58:50.824 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:50.824 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.825 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": { 2024-05-09T04:58:50.825 INFO:teuthology.orchestra.run.smithi169.stdout: "request": 1, 2024-05-09T04:58:50.825 INFO:teuthology.orchestra.run.smithi169.stdout: "reply": 1, 2024-05-09T04:58:50.825 INFO:teuthology.orchestra.run.smithi169.stdout: "reply_latency": { 2024-05-09T04:58:50.825 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1, 2024-05-09T04:58:50.825 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000172995, 2024-05-09T04:58:50.825 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000172995 2024-05-09T04:58:50.825 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.825 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_reply": 0, 2024-05-09T04:58:50.825 INFO:teuthology.orchestra.run.smithi169.stdout: "forward": 0, 2024-05-09T04:58:50.825 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_complete": 0, 2024-05-09T04:58:50.825 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_keys": 0, 2024-05-09T04:58:50.825 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_commit": 11, 2024-05-09T04:58:50.825 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_split": 0, 2024-05-09T04:58:50.826 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_merge": 0, 2024-05-09T04:58:50.826 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes": 16, 2024-05-09T04:58:50.826 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_top": 0, 2024-05-09T04:58:50.826 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_bottom": 16, 2024-05-09T04:58:50.826 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pin_tail": 0, 2024-05-09T04:58:50.826 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pinned": 13, 2024-05-09T04:58:50.826 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_expired": 4, 2024-05-09T04:58:50.826 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_with_caps": 1, 2024-05-09T04:58:50.826 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": 1, 2024-05-09T04:58:50.826 INFO:teuthology.orchestra.run.smithi169.stdout: "subtrees": 3, 2024-05-09T04:58:50.826 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse": 13, 2024-05-09T04:58:50.826 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_hit": 9, 2024-05-09T04:58:50.826 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_forward": 0, 2024-05-09T04:58:50.826 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_discover": 4, 2024-05-09T04:58:50.827 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_dir_fetch": 0, 2024-05-09T04:58:50.827 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_remote_ino": 0, 2024-05-09T04:58:50.827 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_lock": 0, 2024-05-09T04:58:50.827 INFO:teuthology.orchestra.run.smithi169.stdout: "load_cent": 0, 2024-05-09T04:58:50.827 INFO:teuthology.orchestra.run.smithi169.stdout: "q": 0, 2024-05-09T04:58:50.827 INFO:teuthology.orchestra.run.smithi169.stdout: "exported": 0, 2024-05-09T04:58:50.827 INFO:teuthology.orchestra.run.smithi169.stdout: "exported_inodes": 0, 2024-05-09T04:58:50.827 INFO:teuthology.orchestra.run.smithi169.stdout: "imported": 0, 2024-05-09T04:58:50.827 INFO:teuthology.orchestra.run.smithi169.stdout: "imported_inodes": 0, 2024-05-09T04:58:50.827 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_dir_fetch": 0, 2024-05-09T04:58:50.827 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_backtrace_fetch": 0, 2024-05-09T04:58:50.827 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_peer_discover": 0, 2024-05-09T04:58:50.827 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rfiles": 4, 2024-05-09T04:58:50.827 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rbytes": 234, 2024-05-09T04:58:50.827 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rsnaps": 0, 2024-05-09T04:58:50.828 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T04:58:50.828 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_set_tag": 0, 2024-05-09T04:58:50.829 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T04:58:50.829 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_inotable_repaired": 0, 2024-05-09T04:58:50.829 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_inodes": 0, 2024-05-09T04:58:50.829 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T04:58:50.829 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T04:58:50.829 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_file_inodes": 0, 2024-05-09T04:58:50.829 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_inode_file_caps": 0, 2024-05-09T04:58:50.829 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_revoke": 1, 2024-05-09T04:58:50.830 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_grant": 0, 2024-05-09T04:58:50.830 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T04:58:50.830 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T04:58:50.830 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T04:58:50.830 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps": 1, 2024-05-09T04:58:50.830 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps_dirty": 0, 2024-05-09T04:58:50.830 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_cap_release": 0, 2024-05-09T04:58:50.830 INFO:teuthology.orchestra.run.smithi169.stdout: "process_request_cap_release": 0 2024-05-09T04:58:50.830 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.830 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_cache": { 2024-05-09T04:58:50.830 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update": 0, 2024-05-09T04:58:50.830 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update_receipt": 14, 2024-05-09T04:58:50.830 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_try_discover": 9, 2024-05-09T04:58:50.830 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_send_discover": 10, 2024-05-09T04:58:50.831 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_handle_discover": 0, 2024-05-09T04:58:50.831 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays": 0, 2024-05-09T04:58:50.831 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_delayed": 0, 2024-05-09T04:58:50.831 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_enqueuing": 0, 2024-05-09T04:58:50.831 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_created": 0, 2024-05-09T04:58:50.831 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_enqueued": 0, 2024-05-09T04:58:50.831 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_reintegrated": 0, 2024-05-09T04:58:50.831 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_migrated": 0, 2024-05-09T04:58:50.831 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_processing": 0, 2024-05-09T04:58:50.831 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_enqueued": 0, 2024-05-09T04:58:50.831 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_prioritized": 0, 2024-05-09T04:58:50.831 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_started": 0, 2024-05-09T04:58:50.831 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_completed": 0, 2024-05-09T04:58:50.831 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_path": 2, 2024-05-09T04:58:50.831 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_inode": 11, 2024-05-09T04:58:50.832 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T04:58:50.832 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_exportdir": 0, 2024-05-09T04:58:50.832 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_flush": 0, 2024-05-09T04:58:50.832 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragmentdir": 0, 2024-05-09T04:58:50.832 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragstats": 0, 2024-05-09T04:58:50.832 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_inodestats": 0 2024-05-09T04:58:50.832 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.832 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_log": { 2024-05-09T04:58:50.832 INFO:teuthology.orchestra.run.smithi169.stdout: "evlrg": 0, 2024-05-09T04:58:50.832 INFO:teuthology.orchestra.run.smithi169.stdout: "evadd": 5, 2024-05-09T04:58:50.832 INFO:teuthology.orchestra.run.smithi169.stdout: "evex": 0, 2024-05-09T04:58:50.832 INFO:teuthology.orchestra.run.smithi169.stdout: "evtrm": 0, 2024-05-09T04:58:50.832 INFO:teuthology.orchestra.run.smithi169.stdout: "ev": 5, 2024-05-09T04:58:50.832 INFO:teuthology.orchestra.run.smithi169.stdout: "evexg": 0, 2024-05-09T04:58:50.833 INFO:teuthology.orchestra.run.smithi169.stdout: "evexd": 0, 2024-05-09T04:58:50.833 INFO:teuthology.orchestra.run.smithi169.stdout: "segadd": 2, 2024-05-09T04:58:50.833 INFO:teuthology.orchestra.run.smithi169.stdout: "segex": 0, 2024-05-09T04:58:50.833 INFO:teuthology.orchestra.run.smithi169.stdout: "segtrm": 0, 2024-05-09T04:58:50.833 INFO:teuthology.orchestra.run.smithi169.stdout: "seg": 2, 2024-05-09T04:58:50.833 INFO:teuthology.orchestra.run.smithi169.stdout: "segmjr": 2, 2024-05-09T04:58:50.833 INFO:teuthology.orchestra.run.smithi169.stdout: "segexg": 0, 2024-05-09T04:58:50.833 INFO:teuthology.orchestra.run.smithi169.stdout: "segexd": 0, 2024-05-09T04:58:50.833 INFO:teuthology.orchestra.run.smithi169.stdout: "expos": 4194304, 2024-05-09T04:58:50.833 INFO:teuthology.orchestra.run.smithi169.stdout: "wrpos": 4195734, 2024-05-09T04:58:50.833 INFO:teuthology.orchestra.run.smithi169.stdout: "rdpos": 0, 2024-05-09T04:58:50.833 INFO:teuthology.orchestra.run.smithi169.stdout: "jlat": { 2024-05-09T04:58:50.833 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 4, 2024-05-09T04:58:50.833 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.008190936, 2024-05-09T04:58:50.833 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.002047734 2024-05-09T04:58:50.834 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.834 INFO:teuthology.orchestra.run.smithi169.stdout: "replayed": 0 2024-05-09T04:58:50.834 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.834 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_mem": { 2024-05-09T04:58:50.834 INFO:teuthology.orchestra.run.smithi169.stdout: "ino": 20, 2024-05-09T04:58:50.834 INFO:teuthology.orchestra.run.smithi169.stdout: "ino+": 22, 2024-05-09T04:58:50.834 INFO:teuthology.orchestra.run.smithi169.stdout: "ino-": 2, 2024-05-09T04:58:50.834 INFO:teuthology.orchestra.run.smithi169.stdout: "dir": 19, 2024-05-09T04:58:50.834 INFO:teuthology.orchestra.run.smithi169.stdout: "dir+": 23, 2024-05-09T04:58:50.834 INFO:teuthology.orchestra.run.smithi169.stdout: "dir-": 4, 2024-05-09T04:58:50.834 INFO:teuthology.orchestra.run.smithi169.stdout: "dn": 16, 2024-05-09T04:58:50.834 INFO:teuthology.orchestra.run.smithi169.stdout: "dn+": 20, 2024-05-09T04:58:50.834 INFO:teuthology.orchestra.run.smithi169.stdout: "dn-": 4, 2024-05-09T04:58:50.834 INFO:teuthology.orchestra.run.smithi169.stdout: "cap": 1, 2024-05-09T04:58:50.834 INFO:teuthology.orchestra.run.smithi169.stdout: "cap+": 1, 2024-05-09T04:58:50.835 INFO:teuthology.orchestra.run.smithi169.stdout: "cap-": 0, 2024-05-09T04:58:50.835 INFO:teuthology.orchestra.run.smithi169.stdout: "rss": 121180, 2024-05-09T04:58:50.835 INFO:teuthology.orchestra.run.smithi169.stdout: "heap": 223632 2024-05-09T04:58:50.835 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.835 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_server": { 2024-05-09T04:58:50.835 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_client_request": 1, 2024-05-09T04:58:50.835 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_server_request": 0, 2024-05-09T04:58:50.835 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_request": 1, 2024-05-09T04:58:50.835 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_session": 9, 2024-05-09T04:58:50.835 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_peer_request": 11, 2024-05-09T04:58:50.835 INFO:teuthology.orchestra.run.smithi169.stdout: "req_create_latency": { 2024-05-09T04:58:50.835 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:50.835 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:50.835 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:50.835 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.836 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getattr_latency": { 2024-05-09T04:58:50.836 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:50.836 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:50.836 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:50.836 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.836 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getfilelock_latency": { 2024-05-09T04:58:50.836 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:50.836 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:50.836 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:50.836 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.836 INFO:teuthology.orchestra.run.smithi169.stdout: "req_link_latency": { 2024-05-09T04:58:50.836 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:50.836 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:50.836 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:50.836 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.837 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookup_latency": { 2024-05-09T04:58:50.837 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:50.837 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:50.837 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:50.837 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.837 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookuphash_latency": { 2024-05-09T04:58:50.837 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:50.837 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:50.837 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:50.837 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.837 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupino_latency": { 2024-05-09T04:58:50.837 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1, 2024-05-09T04:58:50.837 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000172995, 2024-05-09T04:58:50.837 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000172995 2024-05-09T04:58:50.838 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.838 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupname_latency": { 2024-05-09T04:58:50.838 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:50.838 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:50.838 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:50.838 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.838 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupparent_latency": { 2024-05-09T04:58:50.838 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:50.838 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:50.838 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:50.838 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.838 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupsnap_latency": { 2024-05-09T04:58:50.838 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:50.838 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:50.838 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:50.839 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.839 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lssnap_latency": { 2024-05-09T04:58:50.839 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:50.839 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:50.839 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:50.839 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.839 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mkdir_latency": { 2024-05-09T04:58:50.839 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:50.840 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:50.840 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:50.840 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.840 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mknod_latency": { 2024-05-09T04:58:50.840 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:50.840 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:50.840 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:50.840 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.840 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mksnap_latency": { 2024-05-09T04:58:50.840 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:50.840 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:50.840 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:50.841 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.841 INFO:teuthology.orchestra.run.smithi169.stdout: "req_open_latency": { 2024-05-09T04:58:50.841 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:50.841 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:50.841 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:50.841 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.841 INFO:teuthology.orchestra.run.smithi169.stdout: "req_readdir_latency": { 2024-05-09T04:58:50.841 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:50.841 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:50.841 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:50.841 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.841 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rename_latency": { 2024-05-09T04:58:50.841 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:50.841 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:50.842 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:50.842 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.842 INFO:teuthology.orchestra.run.smithi169.stdout: "req_renamesnap_latency": { 2024-05-09T04:58:50.842 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:50.842 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:50.842 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:50.842 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.842 INFO:teuthology.orchestra.run.smithi169.stdout: "req_snapdiff_latency": { 2024-05-09T04:58:50.842 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:50.842 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:50.842 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:50.842 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.842 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmdir_latency": { 2024-05-09T04:58:50.842 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:50.843 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:50.843 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:50.843 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.843 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmsnap_latency": { 2024-05-09T04:58:50.843 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:50.843 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:50.843 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:50.843 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.843 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmxattr_latency": { 2024-05-09T04:58:50.843 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:50.843 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:50.843 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:50.843 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.843 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setattr_latency": { 2024-05-09T04:58:50.844 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:50.844 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:50.844 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:50.844 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.844 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setdirlayout_latency": { 2024-05-09T04:58:50.844 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:50.844 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:50.844 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:50.844 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.844 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setfilelock_latency": { 2024-05-09T04:58:50.844 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:50.844 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:50.844 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:50.844 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.844 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setlayout_latency": { 2024-05-09T04:58:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setxattr_latency": { 2024-05-09T04:58:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: "req_symlink_latency": { 2024-05-09T04:58:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.846 INFO:teuthology.orchestra.run.smithi169.stdout: "req_unlink_latency": { 2024-05-09T04:58:50.846 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:50.846 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:50.846 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:50.846 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.846 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_revoke_eviction": 0, 2024-05-09T04:58:50.846 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_acquisition_throttle": 0, 2024-05-09T04:58:50.846 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getvxattr_latency": { 2024-05-09T04:58:50.846 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:50.846 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:50.846 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:50.846 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:50.846 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.847 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_sessions": { 2024-05-09T04:58:50.847 INFO:teuthology.orchestra.run.smithi169.stdout: "session_count": 1, 2024-05-09T04:58:50.847 INFO:teuthology.orchestra.run.smithi169.stdout: "session_add": 2, 2024-05-09T04:58:50.847 INFO:teuthology.orchestra.run.smithi169.stdout: "session_remove": 1, 2024-05-09T04:58:50.847 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_open": 1, 2024-05-09T04:58:50.847 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_stale": 0, 2024-05-09T04:58:50.847 INFO:teuthology.orchestra.run.smithi169.stdout: "total_load": 1, 2024-05-09T04:58:50.847 INFO:teuthology.orchestra.run.smithi169.stdout: "average_load": 1, 2024-05-09T04:58:50.847 INFO:teuthology.orchestra.run.smithi169.stdout: "avg_session_uptime": 202, 2024-05-09T04:58:50.847 INFO:teuthology.orchestra.run.smithi169.stdout: "mdthresh_evicted": 0 2024-05-09T04:58:50.847 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.847 INFO:teuthology.orchestra.run.smithi169.stdout: "mempool": { 2024-05-09T04:58:50.847 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_bytes": 0, 2024-05-09T04:58:50.847 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_items": 0, 2024-05-09T04:58:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T04:58:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_items": 0, 2024-05-09T04:58:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T04:58:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_items": 0, 2024-05-09T04:58:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T04:58:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T04:58:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T04:58:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T04:58:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T04:58:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_items": 0, 2024-05-09T04:58:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T04:58:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T04:58:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_bytes": 0, 2024-05-09T04:58:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_items": 0, 2024-05-09T04:58:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_bytes": 0, 2024-05-09T04:58:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_items": 0, 2024-05-09T04:58:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T04:58:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T04:58:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T04:58:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T04:58:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T04:58:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_items": 0, 2024-05-09T04:58:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_bytes": 0, 2024-05-09T04:58:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_items": 0, 2024-05-09T04:58:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T04:58:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T04:58:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_bytes": 0, 2024-05-09T04:58:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_items": 0, 2024-05-09T04:58:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_bytes": 0, 2024-05-09T04:58:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_items": 0, 2024-05-09T04:58:50.850 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T04:58:50.850 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_items": 0, 2024-05-09T04:58:50.850 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T04:58:50.850 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_items": 0, 2024-05-09T04:58:50.850 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_bytes": 246197, 2024-05-09T04:58:50.850 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_items": 75, 2024-05-09T04:58:50.850 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_bytes": 0, 2024-05-09T04:58:50.850 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_items": 0, 2024-05-09T04:58:50.850 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_bytes": 0, 2024-05-09T04:58:50.850 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_items": 0, 2024-05-09T04:58:50.850 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_bytes": 0, 2024-05-09T04:58:50.850 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_items": 0, 2024-05-09T04:58:50.850 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_bytes": 0, 2024-05-09T04:58:50.851 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_items": 0, 2024-05-09T04:58:50.851 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_bytes": 8800, 2024-05-09T04:58:50.851 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_items": 169, 2024-05-09T04:58:50.851 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T04:58:50.851 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_items": 0, 2024-05-09T04:58:50.851 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_bytes": 0, 2024-05-09T04:58:50.851 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_items": 0, 2024-05-09T04:58:50.851 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_bytes": 81373, 2024-05-09T04:58:50.851 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_items": 334, 2024-05-09T04:58:50.851 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_bytes": 0, 2024-05-09T04:58:50.851 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_items": 0, 2024-05-09T04:58:50.851 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_bytes": 0, 2024-05-09T04:58:50.851 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_items": 0 2024-05-09T04:58:50.851 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.851 INFO:teuthology.orchestra.run.smithi169.stdout: "objecter": { 2024-05-09T04:58:50.852 INFO:teuthology.orchestra.run.smithi169.stdout: "op_active": 0, 2024-05-09T04:58:50.852 INFO:teuthology.orchestra.run.smithi169.stdout: "op_laggy": 0, 2024-05-09T04:58:50.852 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send": 41, 2024-05-09T04:58:50.852 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send_bytes": 11943, 2024-05-09T04:58:50.852 INFO:teuthology.orchestra.run.smithi169.stdout: "op_resend": 28, 2024-05-09T04:58:50.852 INFO:teuthology.orchestra.run.smithi169.stdout: "op_reply": 41, 2024-05-09T04:58:50.852 INFO:teuthology.orchestra.run.smithi169.stdout: "op_latency": { 2024-05-09T04:58:50.852 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 41, 2024-05-09T04:58:50.852 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.346992533, 2024-05-09T04:58:50.852 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.008463232 2024-05-09T04:58:50.852 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.852 INFO:teuthology.orchestra.run.smithi169.stdout: "op_inflight": 0, 2024-05-09T04:58:50.852 INFO:teuthology.orchestra.run.smithi169.stdout: "oplen_avg": { 2024-05-09T04:58:50.852 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 41, 2024-05-09T04:58:50.853 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 64 2024-05-09T04:58:50.853 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.853 INFO:teuthology.orchestra.run.smithi169.stdout: "op": 41, 2024-05-09T04:58:50.853 INFO:teuthology.orchestra.run.smithi169.stdout: "op_r": 0, 2024-05-09T04:58:50.853 INFO:teuthology.orchestra.run.smithi169.stdout: "op_w": 41, 2024-05-09T04:58:50.853 INFO:teuthology.orchestra.run.smithi169.stdout: "op_rmw": 0, 2024-05-09T04:58:50.853 INFO:teuthology.orchestra.run.smithi169.stdout: "op_pg": 0, 2024-05-09T04:58:50.853 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_stat": 0, 2024-05-09T04:58:50.853 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_create": 10, 2024-05-09T04:58:50.853 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_read": 0, 2024-05-09T04:58:50.853 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_write": 4, 2024-05-09T04:58:50.853 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writefull": 7, 2024-05-09T04:58:50.853 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writesame": 0, 2024-05-09T04:58:50.853 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_append": 0, 2024-05-09T04:58:50.853 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_zero": 0, 2024-05-09T04:58:50.854 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_truncate": 0, 2024-05-09T04:58:50.854 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_delete": 6, 2024-05-09T04:58:50.854 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_mapext": 0, 2024-05-09T04:58:50.854 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_sparse_read": 0, 2024-05-09T04:58:50.854 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_clonerange": 0, 2024-05-09T04:58:50.854 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_getxattr": 0, 2024-05-09T04:58:50.854 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_setxattr": 20, 2024-05-09T04:58:50.854 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_cmpxattr": 0, 2024-05-09T04:58:50.854 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_rmxattr": 0, 2024-05-09T04:58:50.854 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_resetxattrs": 0, 2024-05-09T04:58:50.854 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_call": 0, 2024-05-09T04:58:50.854 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_watch": 0, 2024-05-09T04:58:50.854 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_notify": 0, 2024-05-09T04:58:50.854 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T04:58:50.855 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls": 0, 2024-05-09T04:58:50.855 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls_filter": 0, 2024-05-09T04:58:50.855 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_other": 0, 2024-05-09T04:58:50.855 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_active": 0, 2024-05-09T04:58:50.855 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_send": 0, 2024-05-09T04:58:50.855 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_resend": 0, 2024-05-09T04:58:50.855 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_ping": 0, 2024-05-09T04:58:50.855 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_active": 0, 2024-05-09T04:58:50.855 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_send": 0, 2024-05-09T04:58:50.855 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_resend": 0, 2024-05-09T04:58:50.855 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_active": 0, 2024-05-09T04:58:50.855 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_send": 0, 2024-05-09T04:58:50.855 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_resend": 0, 2024-05-09T04:58:50.855 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_active": 0, 2024-05-09T04:58:50.855 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_send": 0, 2024-05-09T04:58:50.856 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_resend": 0, 2024-05-09T04:58:50.856 INFO:teuthology.orchestra.run.smithi169.stdout: "command_active": 0, 2024-05-09T04:58:50.856 INFO:teuthology.orchestra.run.smithi169.stdout: "command_send": 0, 2024-05-09T04:58:50.856 INFO:teuthology.orchestra.run.smithi169.stdout: "command_resend": 0, 2024-05-09T04:58:50.856 INFO:teuthology.orchestra.run.smithi169.stdout: "map_epoch": 0, 2024-05-09T04:58:50.856 INFO:teuthology.orchestra.run.smithi169.stdout: "map_full": 0, 2024-05-09T04:58:50.856 INFO:teuthology.orchestra.run.smithi169.stdout: "map_inc": 0, 2024-05-09T04:58:50.856 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_sessions": 9, 2024-05-09T04:58:50.856 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_open": 9, 2024-05-09T04:58:50.856 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_close": 0, 2024-05-09T04:58:50.856 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_laggy": 0, 2024-05-09T04:58:50.856 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_wr": 16, 2024-05-09T04:58:50.856 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_rd": 0, 2024-05-09T04:58:50.856 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_del": 1 2024-05-09T04:58:50.857 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.857 INFO:teuthology.orchestra.run.smithi169.stdout: "oft": { 2024-05-09T04:58:50.857 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_objs": 1, 2024-05-09T04:58:50.857 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_kv_pairs": 0, 2024-05-09T04:58:50.857 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_updates": 0, 2024-05-09T04:58:50.857 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_removes": 0 2024-05-09T04:58:50.857 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.857 INFO:teuthology.orchestra.run.smithi169.stdout: "osd-slow-ops": { 2024-05-09T04:58:50.857 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_ops_count": 12 2024-05-09T04:58:50.857 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.857 INFO:teuthology.orchestra.run.smithi169.stdout: "purge_queue": { 2024-05-09T04:58:50.857 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops": 0, 2024-05-09T04:58:50.857 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T04:58:50.857 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing": 0, 2024-05-09T04:58:50.857 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_high_water": 0, 2024-05-09T04:58:50.858 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed_ops": 0, 2024-05-09T04:58:50.858 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed": 0, 2024-05-09T04:58:50.858 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_item_in_journal": 0 2024-05-09T04:58:50.858 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.858 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T04:58:50.858 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T04:58:50.858 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-05-09T04:58:50.858 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T04:58:50.858 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 567, 2024-05-09T04:58:50.858 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 892570, 2024-05-09T04:58:50.858 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:50.858 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 567, 2024-05-09T04:58:50.858 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-05-09T04:58:50.858 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-05-09T04:58:50.859 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 567, 2024-05-09T04:58:50.859 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 892570, 2024-05-09T04:58:50.859 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T04:58:50.859 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:50.859 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:50.859 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:50.859 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:50.859 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.859 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_bytes": { 2024-05-09T04:58:50.859 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T04:58:50.859 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-05-09T04:58:50.859 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T04:58:50.859 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-05-09T04:58:50.859 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-05-09T04:58:50.859 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:50.860 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-05-09T04:58:50.860 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 41, 2024-05-09T04:58:50.860 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 11943, 2024-05-09T04:58:50.860 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 41, 2024-05-09T04:58:50.860 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 11943, 2024-05-09T04:58:50.860 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T04:58:50.860 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:50.860 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:50.860 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:50.860 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:50.860 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.860 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_ops": { 2024-05-09T04:58:50.860 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T04:58:50.860 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 1024, 2024-05-09T04:58:50.861 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T04:58:50.861 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-05-09T04:58:50.861 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-05-09T04:58:50.861 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:50.861 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-05-09T04:58:50.861 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 41, 2024-05-09T04:58:50.861 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 41, 2024-05-09T04:58:50.861 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 41, 2024-05-09T04:58:50.861 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 41, 2024-05-09T04:58:50.861 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T04:58:50.861 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:50.861 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:50.861 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:50.861 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:50.862 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.862 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle": { 2024-05-09T04:58:50.862 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T04:58:50.862 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-05-09T04:58:50.862 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T04:58:50.862 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-05-09T04:58:50.862 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-05-09T04:58:50.862 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:50.862 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-05-09T04:58:50.862 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-05-09T04:58:50.862 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-05-09T04:58:50.862 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 0, 2024-05-09T04:58:50.862 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 0, 2024-05-09T04:58:50.862 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T04:58:50.862 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:50.863 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:50.863 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:50.863 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:50.863 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:50.863 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle-0x5625c0afe410": { 2024-05-09T04:58:50.863 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T04:58:50.863 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-05-09T04:58:50.863 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T04:58:50.863 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 5, 2024-05-09T04:58:50.863 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 1430, 2024-05-09T04:58:50.863 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:50.863 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 5, 2024-05-09T04:58:50.863 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-05-09T04:58:50.863 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-05-09T04:58:50.863 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 4, 2024-05-09T04:58:50.864 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 1430, 2024-05-09T04:58:50.864 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T04:58:50.864 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:50.864 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:50.864 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:50.864 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:50.864 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:50.864 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-05-09T04:58:51.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:50 smithi169 ceph-mon[34075]: pgmap v616: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 21 MiB/s wr, 20 op/s 2024-05-09T04:58:51.117 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:50 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:58:51.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:50 smithi186 ceph-mon[34668]: pgmap v616: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 21 MiB/s wr, 20 op/s 2024-05-09T04:58:51.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:50 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:58:51.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:50 smithi006 ceph-mon[29835]: pgmap v616: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 21 MiB/s wr, 20 op/s 2024-05-09T04:58:51.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:50 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:58:51.651 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-05-09T04:58:51.651 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-05-09T04:58:51.651 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-05-09T04:58:51.651 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=14 2024-05-09T04:58:51.651 DEBUG:tasks.check_counter:Getting stats from k 2024-05-09T04:58:51.651 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-09T04:58:51.902 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T04:58:52.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:58:52.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:58:52.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:58:52.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:58:52.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:58:52.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:58:52.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:58:52.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:52 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:58:52.271 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:58:52.271 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:58:52.271 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:58:52.271 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:58:52.271 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:58:52.272 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:58:52.272 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:58:52.272 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:52 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:58:52.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:58:52.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:58:52.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:58:52.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:58:52.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:58:52.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:58:52.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:58:52.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:52 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:58:53.297 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:53 smithi169 ceph-mon[34075]: Adjusting osd_memory_target on smithi186 to 176.2M 2024-05-09T04:58:53.298 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:53 smithi169 ceph-mon[34075]: Unable to set osd_memory_target on smithi186 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-05-09T04:58:53.298 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:53 smithi169 ceph-mon[34075]: pgmap v617: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 21 MiB/s wr, 22 op/s 2024-05-09T04:58:53.298 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:53 smithi169 ceph-mon[34075]: Adjusting osd_memory_target on smithi169 to 176.2M 2024-05-09T04:58:53.298 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:53 smithi169 ceph-mon[34075]: Unable to set osd_memory_target on smithi169 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-05-09T04:58:53.298 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:53 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:58:53.298 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:53 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:58:53.298 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:53 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:58:53.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:53 smithi186 ceph-mon[34668]: Adjusting osd_memory_target on smithi186 to 176.2M 2024-05-09T04:58:53.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:53 smithi186 ceph-mon[34668]: Unable to set osd_memory_target on smithi186 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-05-09T04:58:53.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:53 smithi186 ceph-mon[34668]: pgmap v617: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 21 MiB/s wr, 22 op/s 2024-05-09T04:58:53.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:53 smithi186 ceph-mon[34668]: Adjusting osd_memory_target on smithi169 to 176.2M 2024-05-09T04:58:53.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:53 smithi186 ceph-mon[34668]: Unable to set osd_memory_target on smithi169 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-05-09T04:58:53.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:53 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:58:53.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:53 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:58:53.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:53 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:58:53.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:53 smithi006 ceph-mon[29835]: Adjusting osd_memory_target on smithi186 to 176.2M 2024-05-09T04:58:53.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:53 smithi006 ceph-mon[29835]: Unable to set osd_memory_target on smithi186 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-05-09T04:58:53.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:53 smithi006 ceph-mon[29835]: pgmap v617: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 21 MiB/s wr, 22 op/s 2024-05-09T04:58:53.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:53 smithi006 ceph-mon[29835]: Adjusting osd_memory_target on smithi169 to 176.2M 2024-05-09T04:58:53.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:53 smithi006 ceph-mon[29835]: Unable to set osd_memory_target on smithi169 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-05-09T04:58:53.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:53 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:58:53.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:53 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:58:53.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:53 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' 2024-05-09T04:58:53.651 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-05-09T04:58:53.651 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T04:58:53.651 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 653, 2024-05-09T04:58:53.651 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 644, 2024-05-09T04:58:53.651 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 990224, 2024-05-09T04:58:53.652 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 253188, 2024-05-09T04:58:53.652 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 5, 2024-05-09T04:58:53.652 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 1, 2024-05-09T04:58:53.652 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.090102336, 2024-05-09T04:58:53.652 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.033224670, 2024-05-09T04:58:53.652 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.034915510, 2024-05-09T04:58:53.652 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.020174951, 2024-05-09T04:58:53.652 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:58:53.652 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 644, 2024-05-09T04:58:53.652 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.034585944, 2024-05-09T04:58:53.652 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000053704 2024-05-09T04:58:53.652 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.652 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:58:53.652 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.653 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.653 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.653 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.653 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 832304, 2024-05-09T04:58:53.653 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 103728 2024-05-09T04:58:53.653 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.653 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T04:58:53.653 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 4, 2024-05-09T04:58:53.653 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 130, 2024-05-09T04:58:53.653 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 1550, 2024-05-09T04:58:53.653 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 263198, 2024-05-09T04:58:53.653 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 5, 2024-05-09T04:58:53.653 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 4, 2024-05-09T04:58:53.653 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.007945223, 2024-05-09T04:58:53.653 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.005255923, 2024-05-09T04:58:53.654 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.001238673, 2024-05-09T04:58:53.654 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.000072049, 2024-05-09T04:58:53.654 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:58:53.654 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 128, 2024-05-09T04:58:53.654 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.010472579, 2024-05-09T04:58:53.654 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000081817 2024-05-09T04:58:53.654 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.654 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:58:53.654 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.654 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.654 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.654 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.654 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T04:58:53.654 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 262176 2024-05-09T04:58:53.655 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.655 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T04:58:53.655 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 715, 2024-05-09T04:58:53.655 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 717, 2024-05-09T04:58:53.655 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 184897, 2024-05-09T04:58:53.655 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 227371, 2024-05-09T04:58:53.655 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 5, 2024-05-09T04:58:53.655 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 4, 2024-05-09T04:58:53.655 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.054208641, 2024-05-09T04:58:53.655 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.015663367, 2024-05-09T04:58:53.655 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.014840899, 2024-05-09T04:58:53.655 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.019235137, 2024-05-09T04:58:53.655 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:58:53.655 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 715, 2024-05-09T04:58:53.655 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.023660217, 2024-05-09T04:58:53.655 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000033091 2024-05-09T04:58:53.656 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.656 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:58:53.656 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.656 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.656 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.656 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.656 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T04:58:53.656 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T04:58:53.656 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.656 INFO:teuthology.orchestra.run.smithi169.stdout: "cct": { 2024-05-09T04:58:53.656 INFO:teuthology.orchestra.run.smithi169.stdout: "total_workers": 1, 2024-05-09T04:58:53.656 INFO:teuthology.orchestra.run.smithi169.stdout: "unhealthy_workers": 0 2024-05-09T04:58:53.656 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.657 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-MDSRank": { 2024-05-09T04:58:53.657 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-05-09T04:58:53.657 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-05-09T04:58:53.657 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2127, 2024-05-09T04:58:53.657 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.061766554, 2024-05-09T04:58:53.657 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000029039 2024-05-09T04:58:53.657 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:53.657 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.657 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-PurgeQueue": { 2024-05-09T04:58:53.657 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-05-09T04:58:53.657 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-05-09T04:58:53.657 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.657 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.657 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.658 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:53.658 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.658 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": { 2024-05-09T04:58:53.658 INFO:teuthology.orchestra.run.smithi169.stdout: "request": 0, 2024-05-09T04:58:53.658 INFO:teuthology.orchestra.run.smithi169.stdout: "reply": 0, 2024-05-09T04:58:53.658 INFO:teuthology.orchestra.run.smithi169.stdout: "reply_latency": { 2024-05-09T04:58:53.658 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.658 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.658 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.658 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.658 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_reply": 0, 2024-05-09T04:58:53.658 INFO:teuthology.orchestra.run.smithi169.stdout: "forward": 0, 2024-05-09T04:58:53.658 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_complete": 0, 2024-05-09T04:58:53.658 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_keys": 0, 2024-05-09T04:58:53.659 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_commit": 0, 2024-05-09T04:58:53.659 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_split": 0, 2024-05-09T04:58:53.659 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_merge": 0, 2024-05-09T04:58:53.659 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes": 0, 2024-05-09T04:58:53.659 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_top": 0, 2024-05-09T04:58:53.659 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_bottom": 0, 2024-05-09T04:58:53.659 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pin_tail": 0, 2024-05-09T04:58:53.659 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pinned": 0, 2024-05-09T04:58:53.659 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_expired": 0, 2024-05-09T04:58:53.659 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_with_caps": 0, 2024-05-09T04:58:53.659 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": 0, 2024-05-09T04:58:53.659 INFO:teuthology.orchestra.run.smithi169.stdout: "subtrees": 1, 2024-05-09T04:58:53.659 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse": 0, 2024-05-09T04:58:53.659 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_hit": 0, 2024-05-09T04:58:53.659 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_forward": 0, 2024-05-09T04:58:53.660 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_discover": 0, 2024-05-09T04:58:53.660 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_dir_fetch": 0, 2024-05-09T04:58:53.660 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_remote_ino": 0, 2024-05-09T04:58:53.660 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_lock": 0, 2024-05-09T04:58:53.660 INFO:teuthology.orchestra.run.smithi169.stdout: "load_cent": 0, 2024-05-09T04:58:53.660 INFO:teuthology.orchestra.run.smithi169.stdout: "q": 0, 2024-05-09T04:58:53.660 INFO:teuthology.orchestra.run.smithi169.stdout: "exported": 0, 2024-05-09T04:58:53.660 INFO:teuthology.orchestra.run.smithi169.stdout: "exported_inodes": 0, 2024-05-09T04:58:53.660 INFO:teuthology.orchestra.run.smithi169.stdout: "imported": 0, 2024-05-09T04:58:53.660 INFO:teuthology.orchestra.run.smithi169.stdout: "imported_inodes": 0, 2024-05-09T04:58:53.660 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_dir_fetch": 0, 2024-05-09T04:58:53.660 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_backtrace_fetch": 0, 2024-05-09T04:58:53.660 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_peer_discover": 0, 2024-05-09T04:58:53.660 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rfiles": 0, 2024-05-09T04:58:53.661 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rbytes": 0, 2024-05-09T04:58:53.661 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rsnaps": 0, 2024-05-09T04:58:53.661 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T04:58:53.661 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_set_tag": 0, 2024-05-09T04:58:53.661 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T04:58:53.661 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_inotable_repaired": 0, 2024-05-09T04:58:53.661 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_inodes": 0, 2024-05-09T04:58:53.661 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T04:58:53.661 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T04:58:53.661 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_file_inodes": 0, 2024-05-09T04:58:53.661 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_inode_file_caps": 0, 2024-05-09T04:58:53.661 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T04:58:53.661 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_grant": 0, 2024-05-09T04:58:53.661 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T04:58:53.662 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T04:58:53.662 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T04:58:53.662 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps": 0, 2024-05-09T04:58:53.662 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps_dirty": 0, 2024-05-09T04:58:53.662 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_cap_release": 0, 2024-05-09T04:58:53.662 INFO:teuthology.orchestra.run.smithi169.stdout: "process_request_cap_release": 0 2024-05-09T04:58:53.662 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.662 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_cache": { 2024-05-09T04:58:53.662 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update": 0, 2024-05-09T04:58:53.662 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update_receipt": 0, 2024-05-09T04:58:53.662 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_try_discover": 0, 2024-05-09T04:58:53.662 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_send_discover": 0, 2024-05-09T04:58:53.662 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_handle_discover": 0, 2024-05-09T04:58:53.662 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays": 0, 2024-05-09T04:58:53.663 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_delayed": 0, 2024-05-09T04:58:53.663 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_enqueuing": 0, 2024-05-09T04:58:53.663 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_created": 0, 2024-05-09T04:58:53.663 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_enqueued": 0, 2024-05-09T04:58:53.663 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_reintegrated": 0, 2024-05-09T04:58:53.663 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_migrated": 0, 2024-05-09T04:58:53.663 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_processing": 0, 2024-05-09T04:58:53.663 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_enqueued": 0, 2024-05-09T04:58:53.663 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_prioritized": 0, 2024-05-09T04:58:53.663 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_started": 0, 2024-05-09T04:58:53.663 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_completed": 0, 2024-05-09T04:58:53.663 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_path": 0, 2024-05-09T04:58:53.663 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_inode": 0, 2024-05-09T04:58:53.663 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T04:58:53.663 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_exportdir": 0, 2024-05-09T04:58:53.663 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_flush": 0, 2024-05-09T04:58:53.664 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragmentdir": 0, 2024-05-09T04:58:53.664 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragstats": 0, 2024-05-09T04:58:53.664 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_inodestats": 0 2024-05-09T04:58:53.664 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.664 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_log": { 2024-05-09T04:58:53.664 INFO:teuthology.orchestra.run.smithi169.stdout: "evlrg": 0, 2024-05-09T04:58:53.664 INFO:teuthology.orchestra.run.smithi169.stdout: "evadd": 0, 2024-05-09T04:58:53.664 INFO:teuthology.orchestra.run.smithi169.stdout: "evex": 0, 2024-05-09T04:58:53.664 INFO:teuthology.orchestra.run.smithi169.stdout: "evtrm": 0, 2024-05-09T04:58:53.664 INFO:teuthology.orchestra.run.smithi169.stdout: "ev": 5, 2024-05-09T04:58:53.664 INFO:teuthology.orchestra.run.smithi169.stdout: "evexg": 0, 2024-05-09T04:58:53.664 INFO:teuthology.orchestra.run.smithi169.stdout: "evexd": 0, 2024-05-09T04:58:53.664 INFO:teuthology.orchestra.run.smithi169.stdout: "segadd": 0, 2024-05-09T04:58:53.664 INFO:teuthology.orchestra.run.smithi169.stdout: "segex": 0, 2024-05-09T04:58:53.664 INFO:teuthology.orchestra.run.smithi169.stdout: "segtrm": 0, 2024-05-09T04:58:53.665 INFO:teuthology.orchestra.run.smithi169.stdout: "seg": 2, 2024-05-09T04:58:53.665 INFO:teuthology.orchestra.run.smithi169.stdout: "segmjr": 2, 2024-05-09T04:58:53.665 INFO:teuthology.orchestra.run.smithi169.stdout: "segexg": 0, 2024-05-09T04:58:53.665 INFO:teuthology.orchestra.run.smithi169.stdout: "segexd": 0, 2024-05-09T04:58:53.665 INFO:teuthology.orchestra.run.smithi169.stdout: "expos": 4194304, 2024-05-09T04:58:53.665 INFO:teuthology.orchestra.run.smithi169.stdout: "wrpos": 4195734, 2024-05-09T04:58:53.665 INFO:teuthology.orchestra.run.smithi169.stdout: "rdpos": 4195593, 2024-05-09T04:58:53.665 INFO:teuthology.orchestra.run.smithi169.stdout: "jlat": { 2024-05-09T04:58:53.665 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.665 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.665 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.665 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.665 INFO:teuthology.orchestra.run.smithi169.stdout: "replayed": 5 2024-05-09T04:58:53.665 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.666 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_mem": { 2024-05-09T04:58:53.666 INFO:teuthology.orchestra.run.smithi169.stdout: "ino": 3, 2024-05-09T04:58:53.666 INFO:teuthology.orchestra.run.smithi169.stdout: "ino+": 3, 2024-05-09T04:58:53.666 INFO:teuthology.orchestra.run.smithi169.stdout: "ino-": 0, 2024-05-09T04:58:53.666 INFO:teuthology.orchestra.run.smithi169.stdout: "dir": 1, 2024-05-09T04:58:53.666 INFO:teuthology.orchestra.run.smithi169.stdout: "dir+": 1, 2024-05-09T04:58:53.666 INFO:teuthology.orchestra.run.smithi169.stdout: "dir-": 0, 2024-05-09T04:58:53.666 INFO:teuthology.orchestra.run.smithi169.stdout: "dn": 0, 2024-05-09T04:58:53.666 INFO:teuthology.orchestra.run.smithi169.stdout: "dn+": 0, 2024-05-09T04:58:53.666 INFO:teuthology.orchestra.run.smithi169.stdout: "dn-": 0, 2024-05-09T04:58:53.666 INFO:teuthology.orchestra.run.smithi169.stdout: "cap": 0, 2024-05-09T04:58:53.666 INFO:teuthology.orchestra.run.smithi169.stdout: "cap+": 0, 2024-05-09T04:58:53.666 INFO:teuthology.orchestra.run.smithi169.stdout: "cap-": 0, 2024-05-09T04:58:53.666 INFO:teuthology.orchestra.run.smithi169.stdout: "rss": 118308, 2024-05-09T04:58:53.666 INFO:teuthology.orchestra.run.smithi169.stdout: "heap": 215440 2024-05-09T04:58:53.666 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.667 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_server": { 2024-05-09T04:58:53.667 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_client_request": 0, 2024-05-09T04:58:53.667 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_server_request": 0, 2024-05-09T04:58:53.667 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_request": 0, 2024-05-09T04:58:53.667 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_session": 0, 2024-05-09T04:58:53.667 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_peer_request": 0, 2024-05-09T04:58:53.667 INFO:teuthology.orchestra.run.smithi169.stdout: "req_create_latency": { 2024-05-09T04:58:53.667 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.667 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.667 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.667 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.667 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getattr_latency": { 2024-05-09T04:58:53.667 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.667 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.667 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.668 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.668 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getfilelock_latency": { 2024-05-09T04:58:53.668 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.668 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.668 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.668 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.668 INFO:teuthology.orchestra.run.smithi169.stdout: "req_link_latency": { 2024-05-09T04:58:53.668 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.668 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.668 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.668 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.668 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookup_latency": { 2024-05-09T04:58:53.668 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.668 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.668 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.668 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.669 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookuphash_latency": { 2024-05-09T04:58:53.669 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.669 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.669 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.669 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.669 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupino_latency": { 2024-05-09T04:58:53.669 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.669 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.669 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.669 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.669 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupname_latency": { 2024-05-09T04:58:53.669 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.669 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.669 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.669 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.669 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupparent_latency": { 2024-05-09T04:58:53.670 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.670 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.670 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.670 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.670 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupsnap_latency": { 2024-05-09T04:58:53.670 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.670 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.670 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.670 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.670 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lssnap_latency": { 2024-05-09T04:58:53.670 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.670 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.670 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.670 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.670 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mkdir_latency": { 2024-05-09T04:58:53.670 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.671 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.671 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.671 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.671 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mknod_latency": { 2024-05-09T04:58:53.671 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.671 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.671 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.671 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.672 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mksnap_latency": { 2024-05-09T04:58:53.672 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.672 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.672 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.672 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.672 INFO:teuthology.orchestra.run.smithi169.stdout: "req_open_latency": { 2024-05-09T04:58:53.672 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.672 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.672 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.672 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.672 INFO:teuthology.orchestra.run.smithi169.stdout: "req_readdir_latency": { 2024-05-09T04:58:53.672 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.672 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.672 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.672 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.672 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rename_latency": { 2024-05-09T04:58:53.673 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.673 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.673 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.673 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.673 INFO:teuthology.orchestra.run.smithi169.stdout: "req_renamesnap_latency": { 2024-05-09T04:58:53.673 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.673 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.673 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.673 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.673 INFO:teuthology.orchestra.run.smithi169.stdout: "req_snapdiff_latency": { 2024-05-09T04:58:53.673 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.673 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.673 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.673 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.673 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmdir_latency": { 2024-05-09T04:58:53.673 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.673 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.674 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.674 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.674 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmsnap_latency": { 2024-05-09T04:58:53.674 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.674 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.674 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.674 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.674 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmxattr_latency": { 2024-05-09T04:58:53.674 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.674 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.674 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.674 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.674 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setattr_latency": { 2024-05-09T04:58:53.674 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.675 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.675 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.675 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.675 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setdirlayout_latency": { 2024-05-09T04:58:53.675 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.675 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.675 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.675 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.675 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setfilelock_latency": { 2024-05-09T04:58:53.675 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.675 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.675 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.675 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.675 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setlayout_latency": { 2024-05-09T04:58:53.675 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.675 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.675 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.676 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.676 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setxattr_latency": { 2024-05-09T04:58:53.676 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.676 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.676 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.676 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.676 INFO:teuthology.orchestra.run.smithi169.stdout: "req_symlink_latency": { 2024-05-09T04:58:53.676 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.676 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.676 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.676 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.676 INFO:teuthology.orchestra.run.smithi169.stdout: "req_unlink_latency": { 2024-05-09T04:58:53.676 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.676 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.676 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.676 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.677 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_revoke_eviction": 0, 2024-05-09T04:58:53.677 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_acquisition_throttle": 0, 2024-05-09T04:58:53.677 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getvxattr_latency": { 2024-05-09T04:58:53.677 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.677 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.677 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.677 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:53.677 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.677 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_sessions": { 2024-05-09T04:58:53.677 INFO:teuthology.orchestra.run.smithi169.stdout: "session_count": 1, 2024-05-09T04:58:53.677 INFO:teuthology.orchestra.run.smithi169.stdout: "session_add": 2, 2024-05-09T04:58:53.677 INFO:teuthology.orchestra.run.smithi169.stdout: "session_remove": 1, 2024-05-09T04:58:53.677 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_open": 2, 2024-05-09T04:58:53.677 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_stale": 0, 2024-05-09T04:58:53.678 INFO:teuthology.orchestra.run.smithi169.stdout: "total_load": 0, 2024-05-09T04:58:53.678 INFO:teuthology.orchestra.run.smithi169.stdout: "average_load": 0, 2024-05-09T04:58:53.678 INFO:teuthology.orchestra.run.smithi169.stdout: "avg_session_uptime": 3796, 2024-05-09T04:58:53.678 INFO:teuthology.orchestra.run.smithi169.stdout: "mdthresh_evicted": 0 2024-05-09T04:58:53.678 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.678 INFO:teuthology.orchestra.run.smithi169.stdout: "mempool": { 2024-05-09T04:58:53.678 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_bytes": 0, 2024-05-09T04:58:53.678 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_items": 0, 2024-05-09T04:58:53.678 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T04:58:53.678 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_items": 0, 2024-05-09T04:58:53.678 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T04:58:53.678 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_items": 0, 2024-05-09T04:58:53.678 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T04:58:53.678 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T04:58:53.678 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T04:58:53.678 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T04:58:53.679 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T04:58:53.679 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_items": 0, 2024-05-09T04:58:53.679 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T04:58:53.679 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T04:58:53.679 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_bytes": 0, 2024-05-09T04:58:53.679 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_items": 0, 2024-05-09T04:58:53.679 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_bytes": 0, 2024-05-09T04:58:53.679 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_items": 0, 2024-05-09T04:58:53.679 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T04:58:53.679 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T04:58:53.679 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T04:58:53.679 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T04:58:53.679 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T04:58:53.679 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_items": 0, 2024-05-09T04:58:53.679 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_bytes": 0, 2024-05-09T04:58:53.679 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_items": 0, 2024-05-09T04:58:53.679 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T04:58:53.680 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T04:58:53.680 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_bytes": 0, 2024-05-09T04:58:53.680 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_items": 0, 2024-05-09T04:58:53.680 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_bytes": 0, 2024-05-09T04:58:53.680 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_items": 0, 2024-05-09T04:58:53.680 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T04:58:53.680 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_items": 0, 2024-05-09T04:58:53.680 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T04:58:53.680 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_items": 0, 2024-05-09T04:58:53.680 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_bytes": 198287, 2024-05-09T04:58:53.680 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_items": 57, 2024-05-09T04:58:53.680 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_bytes": 0, 2024-05-09T04:58:53.680 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_items": 0, 2024-05-09T04:58:53.680 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_bytes": 0, 2024-05-09T04:58:53.680 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_items": 0, 2024-05-09T04:58:53.680 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_bytes": 0, 2024-05-09T04:58:53.680 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_items": 0, 2024-05-09T04:58:53.681 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_bytes": 0, 2024-05-09T04:58:53.681 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_items": 0, 2024-05-09T04:58:53.681 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_bytes": 8800, 2024-05-09T04:58:53.681 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_items": 169, 2024-05-09T04:58:53.681 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T04:58:53.681 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_items": 0, 2024-05-09T04:58:53.681 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_bytes": 0, 2024-05-09T04:58:53.681 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_items": 0, 2024-05-09T04:58:53.681 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_bytes": 8512, 2024-05-09T04:58:53.681 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_items": 13, 2024-05-09T04:58:53.681 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_bytes": 0, 2024-05-09T04:58:53.681 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_items": 0, 2024-05-09T04:58:53.681 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_bytes": 0, 2024-05-09T04:58:53.681 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_items": 0 2024-05-09T04:58:53.681 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.682 INFO:teuthology.orchestra.run.smithi169.stdout: "objecter": { 2024-05-09T04:58:53.682 INFO:teuthology.orchestra.run.smithi169.stdout: "op_active": 0, 2024-05-09T04:58:53.682 INFO:teuthology.orchestra.run.smithi169.stdout: "op_laggy": 0, 2024-05-09T04:58:53.682 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send": 1188, 2024-05-09T04:58:53.682 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send_bytes": 21, 2024-05-09T04:58:53.682 INFO:teuthology.orchestra.run.smithi169.stdout: "op_resend": 3, 2024-05-09T04:58:53.682 INFO:teuthology.orchestra.run.smithi169.stdout: "op_reply": 1188, 2024-05-09T04:58:53.682 INFO:teuthology.orchestra.run.smithi169.stdout: "op_latency": { 2024-05-09T04:58:53.682 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1188, 2024-05-09T04:58:53.682 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.637986265, 2024-05-09T04:58:53.682 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000537025 2024-05-09T04:58:53.682 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.682 INFO:teuthology.orchestra.run.smithi169.stdout: "op_inflight": 0, 2024-05-09T04:58:53.682 INFO:teuthology.orchestra.run.smithi169.stdout: "oplen_avg": { 2024-05-09T04:58:53.682 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1188, 2024-05-09T04:58:53.683 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 1189 2024-05-09T04:58:53.683 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.683 INFO:teuthology.orchestra.run.smithi169.stdout: "op": 1188, 2024-05-09T04:58:53.683 INFO:teuthology.orchestra.run.smithi169.stdout: "op_r": 1188, 2024-05-09T04:58:53.683 INFO:teuthology.orchestra.run.smithi169.stdout: "op_w": 0, 2024-05-09T04:58:53.683 INFO:teuthology.orchestra.run.smithi169.stdout: "op_rmw": 0, 2024-05-09T04:58:53.683 INFO:teuthology.orchestra.run.smithi169.stdout: "op_pg": 0, 2024-05-09T04:58:53.683 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_stat": 709, 2024-05-09T04:58:53.683 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_create": 0, 2024-05-09T04:58:53.683 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_read": 477, 2024-05-09T04:58:53.683 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_write": 0, 2024-05-09T04:58:53.683 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writefull": 0, 2024-05-09T04:58:53.683 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writesame": 0, 2024-05-09T04:58:53.683 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_append": 0, 2024-05-09T04:58:53.683 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_zero": 0, 2024-05-09T04:58:53.683 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_truncate": 0, 2024-05-09T04:58:53.684 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_delete": 0, 2024-05-09T04:58:53.684 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_mapext": 0, 2024-05-09T04:58:53.684 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_sparse_read": 0, 2024-05-09T04:58:53.684 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_clonerange": 0, 2024-05-09T04:58:53.684 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_getxattr": 1, 2024-05-09T04:58:53.684 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_setxattr": 0, 2024-05-09T04:58:53.684 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_cmpxattr": 0, 2024-05-09T04:58:53.684 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_rmxattr": 0, 2024-05-09T04:58:53.684 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_resetxattrs": 0, 2024-05-09T04:58:53.684 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_call": 0, 2024-05-09T04:58:53.684 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_watch": 0, 2024-05-09T04:58:53.684 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_notify": 0, 2024-05-09T04:58:53.684 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T04:58:53.684 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls": 0, 2024-05-09T04:58:53.685 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls_filter": 0, 2024-05-09T04:58:53.685 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_other": 0, 2024-05-09T04:58:53.685 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_active": 0, 2024-05-09T04:58:53.685 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_send": 0, 2024-05-09T04:58:53.685 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_resend": 0, 2024-05-09T04:58:53.685 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_ping": 0, 2024-05-09T04:58:53.685 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_active": 0, 2024-05-09T04:58:53.685 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_send": 0, 2024-05-09T04:58:53.685 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_resend": 0, 2024-05-09T04:58:53.685 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_active": 0, 2024-05-09T04:58:53.685 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_send": 0, 2024-05-09T04:58:53.685 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_resend": 0, 2024-05-09T04:58:53.685 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_active": 0, 2024-05-09T04:58:53.685 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_send": 0, 2024-05-09T04:58:53.685 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_resend": 0, 2024-05-09T04:58:53.686 INFO:teuthology.orchestra.run.smithi169.stdout: "command_active": 0, 2024-05-09T04:58:53.686 INFO:teuthology.orchestra.run.smithi169.stdout: "command_send": 0, 2024-05-09T04:58:53.686 INFO:teuthology.orchestra.run.smithi169.stdout: "command_resend": 0, 2024-05-09T04:58:53.686 INFO:teuthology.orchestra.run.smithi169.stdout: "map_epoch": 0, 2024-05-09T04:58:53.686 INFO:teuthology.orchestra.run.smithi169.stdout: "map_full": 0, 2024-05-09T04:58:53.686 INFO:teuthology.orchestra.run.smithi169.stdout: "map_inc": 0, 2024-05-09T04:58:53.686 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_sessions": 6, 2024-05-09T04:58:53.686 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_open": 6, 2024-05-09T04:58:53.686 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_close": 0, 2024-05-09T04:58:53.686 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_laggy": 0, 2024-05-09T04:58:53.686 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_wr": 0, 2024-05-09T04:58:53.686 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_rd": 2, 2024-05-09T04:58:53.686 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_del": 0 2024-05-09T04:58:53.686 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.686 INFO:teuthology.orchestra.run.smithi169.stdout: "oft": { 2024-05-09T04:58:53.687 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_objs": 0, 2024-05-09T04:58:53.687 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_kv_pairs": 0, 2024-05-09T04:58:53.687 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_updates": 0, 2024-05-09T04:58:53.687 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_removes": 0 2024-05-09T04:58:53.687 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.687 INFO:teuthology.orchestra.run.smithi169.stdout: "osd-slow-ops": { 2024-05-09T04:58:53.687 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_ops_count": 0 2024-05-09T04:58:53.687 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.687 INFO:teuthology.orchestra.run.smithi169.stdout: "purge_queue": { 2024-05-09T04:58:53.687 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops": 0, 2024-05-09T04:58:53.687 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T04:58:53.687 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing": 0, 2024-05-09T04:58:53.687 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_high_water": 0, 2024-05-09T04:58:53.687 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed_ops": 0, 2024-05-09T04:58:53.687 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed": 0, 2024-05-09T04:58:53.688 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_item_in_journal": 0 2024-05-09T04:58:53.688 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.688 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T04:58:53.688 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T04:58:53.688 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-05-09T04:58:53.688 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T04:58:53.688 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 1368, 2024-05-09T04:58:53.688 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 1041314, 2024-05-09T04:58:53.688 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:53.688 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 1368, 2024-05-09T04:58:53.688 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-05-09T04:58:53.688 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-05-09T04:58:53.688 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 1368, 2024-05-09T04:58:53.688 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 1041314, 2024-05-09T04:58:53.688 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T04:58:53.688 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.689 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.689 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.689 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:53.689 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.689 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_bytes": { 2024-05-09T04:58:53.689 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T04:58:53.689 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-05-09T04:58:53.689 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T04:58:53.689 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-05-09T04:58:53.689 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-05-09T04:58:53.689 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:53.689 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-05-09T04:58:53.689 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 1188, 2024-05-09T04:58:53.689 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 1435, 2024-05-09T04:58:53.689 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 1188, 2024-05-09T04:58:53.690 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 1435, 2024-05-09T04:58:53.690 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T04:58:53.690 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.690 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.690 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.690 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:53.690 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.690 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_ops": { 2024-05-09T04:58:53.690 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T04:58:53.690 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 1024, 2024-05-09T04:58:53.690 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T04:58:53.690 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-05-09T04:58:53.690 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-05-09T04:58:53.690 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:53.691 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-05-09T04:58:53.691 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 1188, 2024-05-09T04:58:53.691 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 1188, 2024-05-09T04:58:53.691 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 1188, 2024-05-09T04:58:53.691 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 1188, 2024-05-09T04:58:53.691 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T04:58:53.691 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.691 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.691 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.691 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:53.691 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.691 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle": { 2024-05-09T04:58:53.691 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T04:58:53.691 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-05-09T04:58:53.691 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T04:58:53.692 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-05-09T04:58:53.692 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-05-09T04:58:53.692 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:53.692 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-05-09T04:58:53.692 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-05-09T04:58:53.692 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-05-09T04:58:53.692 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 0, 2024-05-09T04:58:53.692 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 0, 2024-05-09T04:58:53.692 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T04:58:53.692 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.692 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.692 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.692 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:53.692 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T04:58:53.692 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle-0x55eb1583e410": { 2024-05-09T04:58:53.692 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T04:58:53.693 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-05-09T04:58:53.693 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T04:58:53.693 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-05-09T04:58:53.693 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-05-09T04:58:53.693 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:53.693 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-05-09T04:58:53.693 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-05-09T04:58:53.693 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-05-09T04:58:53.693 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 0, 2024-05-09T04:58:53.693 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 0, 2024-05-09T04:58:53.693 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T04:58:53.693 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T04:58:53.693 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T04:58:53.693 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T04:58:53.693 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:53.693 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T04:58:53.693 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-05-09T04:58:54.349 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-05-09T04:58:54.349 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-05-09T04:58:54.349 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-05-09T04:58:54.349 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-05-09T04:58:54.350 DEBUG:tasks.check_counter:Getting stats from c 2024-05-09T04:58:54.350 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-09T04:58:54.599 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T04:58:55.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:55 smithi186 ceph-mon[34668]: pgmap v618: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 14 MiB/s wr, 19 op/s 2024-05-09T04:58:55.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:55 smithi169 ceph-mon[34075]: pgmap v618: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 14 MiB/s wr, 19 op/s 2024-05-09T04:58:55.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:55 smithi006 ceph-mon[29835]: pgmap v618: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 14 MiB/s wr, 19 op/s 2024-05-09T04:58:56.431 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2024-05-09T04:58:56.431 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T04:58:56.431 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 12, 2024-05-09T04:58:56.431 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 14, 2024-05-09T04:58:56.431 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 3204, 2024-05-09T04:58:56.431 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 5856, 2024-05-09T04:58:56.431 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 8, 2024-05-09T04:58:56.431 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 3, 2024-05-09T04:58:56.431 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.003708522, 2024-05-09T04:58:56.431 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.000127514, 2024-05-09T04:58:56.431 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.001711863, 2024-05-09T04:58:56.431 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000355380, 2024-05-09T04:58:56.432 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:58:56.432 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 12, 2024-05-09T04:58:56.432 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.012944019, 2024-05-09T04:58:56.432 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.001078668 2024-05-09T04:58:56.432 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.432 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:58:56.432 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.432 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.432 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.432 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.432 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T04:58:56.432 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T04:58:56.432 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.433 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T04:58:56.433 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 20, 2024-05-09T04:58:56.433 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 145, 2024-05-09T04:58:56.433 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 6567, 2024-05-09T04:58:56.433 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 423393, 2024-05-09T04:58:56.433 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 14, 2024-05-09T04:58:56.433 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 9, 2024-05-09T04:58:56.433 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.012248210, 2024-05-09T04:58:56.433 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.005586542, 2024-05-09T04:58:56.433 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.004977430, 2024-05-09T04:58:56.433 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000212720, 2024-05-09T04:58:56.433 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:58:56.433 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 140, 2024-05-09T04:58:56.433 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.021959329, 2024-05-09T04:58:56.433 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000156852 2024-05-09T04:58:56.434 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.434 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:58:56.434 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 6, 2024-05-09T04:58:56.434 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000817, 2024-05-09T04:58:56.434 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000136 2024-05-09T04:58:56.434 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.434 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-05-09T04:58:56.434 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 341088 2024-05-09T04:58:56.434 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.434 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T04:58:56.434 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 539, 2024-05-09T04:58:56.434 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 570, 2024-05-09T04:58:56.434 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 937151, 2024-05-09T04:58:56.435 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 185975, 2024-05-09T04:58:56.435 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 6, 2024-05-09T04:58:56.435 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 4, 2024-05-09T04:58:56.435 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.074133012, 2024-05-09T04:58:56.435 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.028783078, 2024-05-09T04:58:56.435 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.044024981, 2024-05-09T04:58:56.435 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.007399317, 2024-05-09T04:58:56.435 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:58:56.435 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 573, 2024-05-09T04:58:56.435 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.046533108, 2024-05-09T04:58:56.435 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000081209 2024-05-09T04:58:56.435 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.435 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:58:56.435 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 661, 2024-05-09T04:58:56.435 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000213136, 2024-05-09T04:58:56.436 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000322 2024-05-09T04:58:56.436 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.436 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 840640, 2024-05-09T04:58:56.436 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 105520 2024-05-09T04:58:56.436 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.436 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2024-05-09T04:58:56.436 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 1, 2024-05-09T04:58:56.436 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2024-05-09T04:58:56.436 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.436 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-MDSRank": { 2024-05-09T04:58:56.436 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-05-09T04:58:56.436 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-05-09T04:58:56.436 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 38, 2024-05-09T04:58:56.436 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.001518384, 2024-05-09T04:58:56.437 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000039957 2024-05-09T04:58:56.437 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:58:56.437 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.437 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-PurgeQueue": { 2024-05-09T04:58:56.437 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-05-09T04:58:56.437 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-05-09T04:58:56.437 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2, 2024-05-09T04:58:56.437 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000008944, 2024-05-09T04:58:56.437 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000004472 2024-05-09T04:58:56.437 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:58:56.437 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.437 INFO:teuthology.orchestra.run.smithi186.stdout: "mds": { 2024-05-09T04:58:56.437 INFO:teuthology.orchestra.run.smithi186.stdout: "request": 0, 2024-05-09T04:58:56.437 INFO:teuthology.orchestra.run.smithi186.stdout: "reply": 0, 2024-05-09T04:58:56.438 INFO:teuthology.orchestra.run.smithi186.stdout: "reply_latency": { 2024-05-09T04:58:56.438 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.438 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.438 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.438 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.438 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_reply": 0, 2024-05-09T04:58:56.438 INFO:teuthology.orchestra.run.smithi186.stdout: "forward": 0, 2024-05-09T04:58:56.438 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_complete": 0, 2024-05-09T04:58:56.438 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_keys": 0, 2024-05-09T04:58:56.438 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_commit": 11, 2024-05-09T04:58:56.438 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_split": 0, 2024-05-09T04:58:56.438 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_merge": 0, 2024-05-09T04:58:56.438 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes": 16, 2024-05-09T04:58:56.438 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_top": 0, 2024-05-09T04:58:56.439 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_bottom": 16, 2024-05-09T04:58:56.439 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pin_tail": 0, 2024-05-09T04:58:56.439 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pinned": 13, 2024-05-09T04:58:56.439 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_expired": 4, 2024-05-09T04:58:56.439 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_with_caps": 0, 2024-05-09T04:58:56.439 INFO:teuthology.orchestra.run.smithi186.stdout: "caps": 0, 2024-05-09T04:58:56.439 INFO:teuthology.orchestra.run.smithi186.stdout: "subtrees": 3, 2024-05-09T04:58:56.439 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse": 13, 2024-05-09T04:58:56.439 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_hit": 9, 2024-05-09T04:58:56.439 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_forward": 0, 2024-05-09T04:58:56.439 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_discover": 4, 2024-05-09T04:58:56.439 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_dir_fetch": 0, 2024-05-09T04:58:56.439 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_remote_ino": 0, 2024-05-09T04:58:56.439 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_lock": 0, 2024-05-09T04:58:56.439 INFO:teuthology.orchestra.run.smithi186.stdout: "load_cent": 0, 2024-05-09T04:58:56.440 INFO:teuthology.orchestra.run.smithi186.stdout: "q": 0, 2024-05-09T04:58:56.440 INFO:teuthology.orchestra.run.smithi186.stdout: "exported": 0, 2024-05-09T04:58:56.440 INFO:teuthology.orchestra.run.smithi186.stdout: "exported_inodes": 0, 2024-05-09T04:58:56.440 INFO:teuthology.orchestra.run.smithi186.stdout: "imported": 0, 2024-05-09T04:58:56.440 INFO:teuthology.orchestra.run.smithi186.stdout: "imported_inodes": 0, 2024-05-09T04:58:56.440 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_dir_fetch": 0, 2024-05-09T04:58:56.440 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_backtrace_fetch": 0, 2024-05-09T04:58:56.440 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_peer_discover": 0, 2024-05-09T04:58:56.440 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rfiles": 4, 2024-05-09T04:58:56.440 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rbytes": 234, 2024-05-09T04:58:56.440 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rsnaps": 0, 2024-05-09T04:58:56.440 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T04:58:56.440 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_set_tag": 0, 2024-05-09T04:58:56.440 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T04:58:56.441 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_inotable_repaired": 0, 2024-05-09T04:58:56.441 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_inodes": 0, 2024-05-09T04:58:56.441 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T04:58:56.441 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T04:58:56.441 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_file_inodes": 0, 2024-05-09T04:58:56.441 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_inode_file_caps": 0, 2024-05-09T04:58:56.441 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T04:58:56.441 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_grant": 0, 2024-05-09T04:58:56.441 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T04:58:56.441 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T04:58:56.441 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T04:58:56.441 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps": 0, 2024-05-09T04:58:56.441 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps_dirty": 0, 2024-05-09T04:58:56.441 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_cap_release": 0, 2024-05-09T04:58:56.441 INFO:teuthology.orchestra.run.smithi186.stdout: "process_request_cap_release": 0 2024-05-09T04:58:56.442 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.442 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_cache": { 2024-05-09T04:58:56.442 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update": 0, 2024-05-09T04:58:56.442 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update_receipt": 14, 2024-05-09T04:58:56.442 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_try_discover": 9, 2024-05-09T04:58:56.442 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_send_discover": 10, 2024-05-09T04:58:56.442 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_handle_discover": 0, 2024-05-09T04:58:56.442 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays": 0, 2024-05-09T04:58:56.442 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_delayed": 0, 2024-05-09T04:58:56.442 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_enqueuing": 0, 2024-05-09T04:58:56.442 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_created": 0, 2024-05-09T04:58:56.442 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_enqueued": 0, 2024-05-09T04:58:56.442 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_reintegrated": 0, 2024-05-09T04:58:56.442 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_migrated": 0, 2024-05-09T04:58:56.442 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_processing": 0, 2024-05-09T04:58:56.443 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_enqueued": 0, 2024-05-09T04:58:56.443 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_prioritized": 0, 2024-05-09T04:58:56.443 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_started": 0, 2024-05-09T04:58:56.443 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_completed": 0, 2024-05-09T04:58:56.443 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_quiesce_path": 2, 2024-05-09T04:58:56.443 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_quiesce_inode": 11, 2024-05-09T04:58:56.443 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T04:58:56.443 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_exportdir": 0, 2024-05-09T04:58:56.443 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_flush": 0, 2024-05-09T04:58:56.443 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragmentdir": 0, 2024-05-09T04:58:56.443 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragstats": 0, 2024-05-09T04:58:56.443 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_inodestats": 0 2024-05-09T04:58:56.443 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.443 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_log": { 2024-05-09T04:58:56.444 INFO:teuthology.orchestra.run.smithi186.stdout: "evlrg": 0, 2024-05-09T04:58:56.444 INFO:teuthology.orchestra.run.smithi186.stdout: "evadd": 2, 2024-05-09T04:58:56.444 INFO:teuthology.orchestra.run.smithi186.stdout: "evex": 0, 2024-05-09T04:58:56.444 INFO:teuthology.orchestra.run.smithi186.stdout: "evtrm": 0, 2024-05-09T04:58:56.444 INFO:teuthology.orchestra.run.smithi186.stdout: "ev": 2, 2024-05-09T04:58:56.444 INFO:teuthology.orchestra.run.smithi186.stdout: "evexg": 0, 2024-05-09T04:58:56.444 INFO:teuthology.orchestra.run.smithi186.stdout: "evexd": 0, 2024-05-09T04:58:56.444 INFO:teuthology.orchestra.run.smithi186.stdout: "segadd": 2, 2024-05-09T04:58:56.444 INFO:teuthology.orchestra.run.smithi186.stdout: "segex": 0, 2024-05-09T04:58:56.444 INFO:teuthology.orchestra.run.smithi186.stdout: "segtrm": 0, 2024-05-09T04:58:56.444 INFO:teuthology.orchestra.run.smithi186.stdout: "seg": 2, 2024-05-09T04:58:56.444 INFO:teuthology.orchestra.run.smithi186.stdout: "segmjr": 2, 2024-05-09T04:58:56.444 INFO:teuthology.orchestra.run.smithi186.stdout: "segexg": 0, 2024-05-09T04:58:56.444 INFO:teuthology.orchestra.run.smithi186.stdout: "segexd": 0, 2024-05-09T04:58:56.444 INFO:teuthology.orchestra.run.smithi186.stdout: "expos": 4194304, 2024-05-09T04:58:56.445 INFO:teuthology.orchestra.run.smithi186.stdout: "wrpos": 4194892, 2024-05-09T04:58:56.445 INFO:teuthology.orchestra.run.smithi186.stdout: "rdpos": 0, 2024-05-09T04:58:56.445 INFO:teuthology.orchestra.run.smithi186.stdout: "jlat": { 2024-05-09T04:58:56.445 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1, 2024-05-09T04:58:56.445 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.002043599, 2024-05-09T04:58:56.445 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.002043599 2024-05-09T04:58:56.445 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.445 INFO:teuthology.orchestra.run.smithi186.stdout: "replayed": 0 2024-05-09T04:58:56.445 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.445 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_mem": { 2024-05-09T04:58:56.445 INFO:teuthology.orchestra.run.smithi186.stdout: "ino": 20, 2024-05-09T04:58:56.445 INFO:teuthology.orchestra.run.smithi186.stdout: "ino+": 22, 2024-05-09T04:58:56.445 INFO:teuthology.orchestra.run.smithi186.stdout: "ino-": 2, 2024-05-09T04:58:56.445 INFO:teuthology.orchestra.run.smithi186.stdout: "dir": 19, 2024-05-09T04:58:56.445 INFO:teuthology.orchestra.run.smithi186.stdout: "dir+": 23, 2024-05-09T04:58:56.445 INFO:teuthology.orchestra.run.smithi186.stdout: "dir-": 4, 2024-05-09T04:58:56.446 INFO:teuthology.orchestra.run.smithi186.stdout: "dn": 16, 2024-05-09T04:58:56.446 INFO:teuthology.orchestra.run.smithi186.stdout: "dn+": 20, 2024-05-09T04:58:56.446 INFO:teuthology.orchestra.run.smithi186.stdout: "dn-": 4, 2024-05-09T04:58:56.446 INFO:teuthology.orchestra.run.smithi186.stdout: "cap": 0, 2024-05-09T04:58:56.446 INFO:teuthology.orchestra.run.smithi186.stdout: "cap+": 0, 2024-05-09T04:58:56.446 INFO:teuthology.orchestra.run.smithi186.stdout: "cap-": 0, 2024-05-09T04:58:56.446 INFO:teuthology.orchestra.run.smithi186.stdout: "rss": 120612, 2024-05-09T04:58:56.446 INFO:teuthology.orchestra.run.smithi186.stdout: "heap": 223632 2024-05-09T04:58:56.446 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.446 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_server": { 2024-05-09T04:58:56.446 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_client_request": 0, 2024-05-09T04:58:56.446 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_server_request": 0, 2024-05-09T04:58:56.446 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_request": 0, 2024-05-09T04:58:56.446 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_session": 0, 2024-05-09T04:58:56.446 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_peer_request": 11, 2024-05-09T04:58:56.446 INFO:teuthology.orchestra.run.smithi186.stdout: "req_create_latency": { 2024-05-09T04:58:56.447 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.447 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.447 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.447 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.447 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getattr_latency": { 2024-05-09T04:58:56.447 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.447 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.447 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.447 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.447 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getfilelock_latency": { 2024-05-09T04:58:56.447 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.447 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.447 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.447 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.447 INFO:teuthology.orchestra.run.smithi186.stdout: "req_link_latency": { 2024-05-09T04:58:56.448 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.448 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.448 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.448 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.448 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookup_latency": { 2024-05-09T04:58:56.448 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.448 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.448 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.448 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.448 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookuphash_latency": { 2024-05-09T04:58:56.448 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.448 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.448 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.448 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.448 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupino_latency": { 2024-05-09T04:58:56.449 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.449 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.449 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.449 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.449 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupname_latency": { 2024-05-09T04:58:56.449 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.449 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.449 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.449 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.449 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupparent_latency": { 2024-05-09T04:58:56.449 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.449 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.449 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.449 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.449 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupsnap_latency": { 2024-05-09T04:58:56.449 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.450 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.450 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.450 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.450 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lssnap_latency": { 2024-05-09T04:58:56.450 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.450 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.450 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.450 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.450 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mkdir_latency": { 2024-05-09T04:58:56.450 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.451 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.451 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.451 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.451 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mknod_latency": { 2024-05-09T04:58:56.451 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.451 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.451 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.451 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.451 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mksnap_latency": { 2024-05-09T04:58:56.451 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.451 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.451 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.452 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.452 INFO:teuthology.orchestra.run.smithi186.stdout: "req_open_latency": { 2024-05-09T04:58:56.452 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.452 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.452 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.452 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.452 INFO:teuthology.orchestra.run.smithi186.stdout: "req_readdir_latency": { 2024-05-09T04:58:56.452 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.452 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.452 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.452 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.452 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rename_latency": { 2024-05-09T04:58:56.452 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.452 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.452 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.452 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.453 INFO:teuthology.orchestra.run.smithi186.stdout: "req_renamesnap_latency": { 2024-05-09T04:58:56.453 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.453 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.453 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.453 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.453 INFO:teuthology.orchestra.run.smithi186.stdout: "req_snapdiff_latency": { 2024-05-09T04:58:56.453 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.453 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.453 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.453 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.453 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmdir_latency": { 2024-05-09T04:58:56.453 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.453 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.453 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.453 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.454 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmsnap_latency": { 2024-05-09T04:58:56.454 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.454 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.454 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.454 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.454 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmxattr_latency": { 2024-05-09T04:58:56.454 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.454 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.454 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.454 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.454 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setattr_latency": { 2024-05-09T04:58:56.454 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.454 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.454 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.454 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.455 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setdirlayout_latency": { 2024-05-09T04:58:56.455 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.455 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.455 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.455 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.455 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setfilelock_latency": { 2024-05-09T04:58:56.455 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.455 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.455 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.455 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.455 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setlayout_latency": { 2024-05-09T04:58:56.455 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.455 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.455 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.455 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.455 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setxattr_latency": { 2024-05-09T04:58:56.456 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.456 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.456 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.456 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.456 INFO:teuthology.orchestra.run.smithi186.stdout: "req_symlink_latency": { 2024-05-09T04:58:56.456 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.456 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.456 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.456 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.456 INFO:teuthology.orchestra.run.smithi186.stdout: "req_unlink_latency": { 2024-05-09T04:58:56.456 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.456 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.456 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.456 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.456 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_revoke_eviction": 0, 2024-05-09T04:58:56.457 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_acquisition_throttle": 0, 2024-05-09T04:58:56.457 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getvxattr_latency": { 2024-05-09T04:58:56.457 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.457 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.457 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.457 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:58:56.457 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.457 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_sessions": { 2024-05-09T04:58:56.457 INFO:teuthology.orchestra.run.smithi186.stdout: "session_count": 0, 2024-05-09T04:58:56.457 INFO:teuthology.orchestra.run.smithi186.stdout: "session_add": 0, 2024-05-09T04:58:56.457 INFO:teuthology.orchestra.run.smithi186.stdout: "session_remove": 0, 2024-05-09T04:58:56.457 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_open": 0, 2024-05-09T04:58:56.457 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_stale": 0, 2024-05-09T04:58:56.457 INFO:teuthology.orchestra.run.smithi186.stdout: "total_load": 0, 2024-05-09T04:58:56.457 INFO:teuthology.orchestra.run.smithi186.stdout: "average_load": 0, 2024-05-09T04:58:56.457 INFO:teuthology.orchestra.run.smithi186.stdout: "avg_session_uptime": 0, 2024-05-09T04:58:56.458 INFO:teuthology.orchestra.run.smithi186.stdout: "mdthresh_evicted": 0 2024-05-09T04:58:56.458 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.458 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2024-05-09T04:58:56.458 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2024-05-09T04:58:56.458 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2024-05-09T04:58:56.458 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T04:58:56.458 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2024-05-09T04:58:56.458 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T04:58:56.458 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2024-05-09T04:58:56.458 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T04:58:56.458 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T04:58:56.458 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T04:58:56.458 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T04:58:56.458 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T04:58:56.458 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2024-05-09T04:58:56.459 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T04:58:56.459 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T04:58:56.459 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2024-05-09T04:58:56.459 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2024-05-09T04:58:56.459 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2024-05-09T04:58:56.459 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2024-05-09T04:58:56.459 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T04:58:56.459 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T04:58:56.459 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T04:58:56.459 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T04:58:56.459 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T04:58:56.459 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2024-05-09T04:58:56.459 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2024-05-09T04:58:56.459 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2024-05-09T04:58:56.459 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T04:58:56.459 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T04:58:56.460 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2024-05-09T04:58:56.460 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2024-05-09T04:58:56.460 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2024-05-09T04:58:56.460 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2024-05-09T04:58:56.460 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T04:58:56.460 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2024-05-09T04:58:56.460 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T04:58:56.460 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2024-05-09T04:58:56.460 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 238343, 2024-05-09T04:58:56.460 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 79, 2024-05-09T04:58:56.460 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 0, 2024-05-09T04:58:56.460 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 0, 2024-05-09T04:58:56.460 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2024-05-09T04:58:56.460 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2024-05-09T04:58:56.460 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2024-05-09T04:58:56.461 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2024-05-09T04:58:56.461 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2024-05-09T04:58:56.461 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2024-05-09T04:58:56.461 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 8800, 2024-05-09T04:58:56.461 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 169, 2024-05-09T04:58:56.461 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T04:58:56.461 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2024-05-09T04:58:56.461 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2024-05-09T04:58:56.461 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2024-05-09T04:58:56.461 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 81013, 2024-05-09T04:58:56.461 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 333, 2024-05-09T04:58:56.461 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2024-05-09T04:58:56.461 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2024-05-09T04:58:56.461 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2024-05-09T04:58:56.461 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2024-05-09T04:58:56.461 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.462 INFO:teuthology.orchestra.run.smithi186.stdout: "objecter": { 2024-05-09T04:58:56.462 INFO:teuthology.orchestra.run.smithi186.stdout: "op_active": 0, 2024-05-09T04:58:56.462 INFO:teuthology.orchestra.run.smithi186.stdout: "op_laggy": 0, 2024-05-09T04:58:56.462 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send": 35, 2024-05-09T04:58:56.462 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send_bytes": 10710, 2024-05-09T04:58:56.462 INFO:teuthology.orchestra.run.smithi186.stdout: "op_resend": 33, 2024-05-09T04:58:56.462 INFO:teuthology.orchestra.run.smithi186.stdout: "op_reply": 35, 2024-05-09T04:58:56.462 INFO:teuthology.orchestra.run.smithi186.stdout: "op_latency": { 2024-05-09T04:58:56.462 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 35, 2024-05-09T04:58:56.462 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.262994836, 2024-05-09T04:58:56.462 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.007514138 2024-05-09T04:58:56.462 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.462 INFO:teuthology.orchestra.run.smithi186.stdout: "op_inflight": 0, 2024-05-09T04:58:56.462 INFO:teuthology.orchestra.run.smithi186.stdout: "oplen_avg": { 2024-05-09T04:58:56.463 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 35, 2024-05-09T04:58:56.463 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 57 2024-05-09T04:58:56.463 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.463 INFO:teuthology.orchestra.run.smithi186.stdout: "op": 35, 2024-05-09T04:58:56.463 INFO:teuthology.orchestra.run.smithi186.stdout: "op_r": 0, 2024-05-09T04:58:56.463 INFO:teuthology.orchestra.run.smithi186.stdout: "op_w": 35, 2024-05-09T04:58:56.463 INFO:teuthology.orchestra.run.smithi186.stdout: "op_rmw": 0, 2024-05-09T04:58:56.463 INFO:teuthology.orchestra.run.smithi186.stdout: "op_pg": 0, 2024-05-09T04:58:56.463 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_stat": 0, 2024-05-09T04:58:56.463 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_create": 10, 2024-05-09T04:58:56.463 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_read": 0, 2024-05-09T04:58:56.463 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_write": 1, 2024-05-09T04:58:56.463 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writefull": 5, 2024-05-09T04:58:56.463 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writesame": 0, 2024-05-09T04:58:56.463 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_append": 0, 2024-05-09T04:58:56.463 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_zero": 0, 2024-05-09T04:58:56.463 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_truncate": 0, 2024-05-09T04:58:56.464 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_delete": 6, 2024-05-09T04:58:56.464 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_mapext": 0, 2024-05-09T04:58:56.464 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_sparse_read": 0, 2024-05-09T04:58:56.464 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_clonerange": 0, 2024-05-09T04:58:56.464 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_getxattr": 0, 2024-05-09T04:58:56.464 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_setxattr": 20, 2024-05-09T04:58:56.464 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_cmpxattr": 0, 2024-05-09T04:58:56.464 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_rmxattr": 0, 2024-05-09T04:58:56.464 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_resetxattrs": 0, 2024-05-09T04:58:56.464 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_call": 0, 2024-05-09T04:58:56.464 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_watch": 0, 2024-05-09T04:58:56.464 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_notify": 0, 2024-05-09T04:58:56.464 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T04:58:56.464 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls": 0, 2024-05-09T04:58:56.464 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls_filter": 0, 2024-05-09T04:58:56.465 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_other": 0, 2024-05-09T04:58:56.465 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_active": 0, 2024-05-09T04:58:56.465 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_send": 0, 2024-05-09T04:58:56.465 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_resend": 0, 2024-05-09T04:58:56.465 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_ping": 0, 2024-05-09T04:58:56.465 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_active": 0, 2024-05-09T04:58:56.465 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_send": 0, 2024-05-09T04:58:56.465 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_resend": 0, 2024-05-09T04:58:56.465 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_active": 0, 2024-05-09T04:58:56.465 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_send": 0, 2024-05-09T04:58:56.465 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_resend": 0, 2024-05-09T04:58:56.465 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_active": 0, 2024-05-09T04:58:56.465 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_send": 0, 2024-05-09T04:58:56.465 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_resend": 0, 2024-05-09T04:58:56.465 INFO:teuthology.orchestra.run.smithi186.stdout: "command_active": 0, 2024-05-09T04:58:56.466 INFO:teuthology.orchestra.run.smithi186.stdout: "command_send": 0, 2024-05-09T04:58:56.466 INFO:teuthology.orchestra.run.smithi186.stdout: "command_resend": 0, 2024-05-09T04:58:56.466 INFO:teuthology.orchestra.run.smithi186.stdout: "map_epoch": 0, 2024-05-09T04:58:56.466 INFO:teuthology.orchestra.run.smithi186.stdout: "map_full": 0, 2024-05-09T04:58:56.466 INFO:teuthology.orchestra.run.smithi186.stdout: "map_inc": 0, 2024-05-09T04:58:56.466 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_sessions": 10, 2024-05-09T04:58:56.466 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_open": 10, 2024-05-09T04:58:56.466 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_close": 0, 2024-05-09T04:58:56.466 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_laggy": 0, 2024-05-09T04:58:56.466 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_wr": 14, 2024-05-09T04:58:56.466 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_rd": 0, 2024-05-09T04:58:56.466 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_del": 1 2024-05-09T04:58:56.466 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.466 INFO:teuthology.orchestra.run.smithi186.stdout: "oft": { 2024-05-09T04:58:56.466 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_objs": 1, 2024-05-09T04:58:56.467 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_kv_pairs": 0, 2024-05-09T04:58:56.467 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_updates": 0, 2024-05-09T04:58:56.467 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_removes": 0 2024-05-09T04:58:56.467 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.467 INFO:teuthology.orchestra.run.smithi186.stdout: "osd-slow-ops": { 2024-05-09T04:58:56.467 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_ops_count": 12 2024-05-09T04:58:56.467 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.467 INFO:teuthology.orchestra.run.smithi186.stdout: "purge_queue": { 2024-05-09T04:58:56.467 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops": 0, 2024-05-09T04:58:56.467 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T04:58:56.467 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing": 0, 2024-05-09T04:58:56.467 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_high_water": 0, 2024-05-09T04:58:56.467 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed_ops": 0, 2024-05-09T04:58:56.467 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed": 0, 2024-05-09T04:58:56.467 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_item_in_journal": 0 2024-05-09T04:58:56.467 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.468 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T04:58:56.468 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-09T04:58:56.468 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-05-09T04:58:56.468 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-09T04:58:56.468 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 567, 2024-05-09T04:58:56.468 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 882391, 2024-05-09T04:58:56.468 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:56.468 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 567, 2024-05-09T04:58:56.468 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-05-09T04:58:56.468 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-05-09T04:58:56.468 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 567, 2024-05-09T04:58:56.468 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 882391, 2024-05-09T04:58:56.468 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-09T04:58:56.468 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.468 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.469 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.469 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:58:56.469 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.469 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_bytes": { 2024-05-09T04:58:56.469 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-09T04:58:56.469 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-05-09T04:58:56.469 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-09T04:58:56.469 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-05-09T04:58:56.469 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-05-09T04:58:56.469 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:56.469 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-05-09T04:58:56.469 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 35, 2024-05-09T04:58:56.469 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 10710, 2024-05-09T04:58:56.469 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 35, 2024-05-09T04:58:56.469 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 10710, 2024-05-09T04:58:56.470 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-09T04:58:56.470 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.470 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.470 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.470 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:58:56.470 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.470 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_ops": { 2024-05-09T04:58:56.470 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-09T04:58:56.470 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 1024, 2024-05-09T04:58:56.470 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-09T04:58:56.470 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-05-09T04:58:56.470 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-05-09T04:58:56.470 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:56.470 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-05-09T04:58:56.471 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 35, 2024-05-09T04:58:56.471 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 35, 2024-05-09T04:58:56.471 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 35, 2024-05-09T04:58:56.471 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 35, 2024-05-09T04:58:56.471 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-09T04:58:56.471 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.471 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.471 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.471 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:58:56.471 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.471 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle": { 2024-05-09T04:58:56.471 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-09T04:58:56.471 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-05-09T04:58:56.471 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-09T04:58:56.471 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-05-09T04:58:56.472 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-05-09T04:58:56.472 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:56.472 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-05-09T04:58:56.472 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-05-09T04:58:56.472 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-05-09T04:58:56.472 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2024-05-09T04:58:56.472 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2024-05-09T04:58:56.472 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-09T04:58:56.472 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.472 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.472 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.472 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:58:56.472 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:56.472 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle-0x56397be584e0": { 2024-05-09T04:58:56.472 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-09T04:58:56.473 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-05-09T04:58:56.473 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-09T04:58:56.473 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 2, 2024-05-09T04:58:56.473 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 588, 2024-05-09T04:58:56.473 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:56.473 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 2, 2024-05-09T04:58:56.473 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-05-09T04:58:56.473 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-05-09T04:58:56.473 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 1, 2024-05-09T04:58:56.473 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 588, 2024-05-09T04:58:56.473 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-09T04:58:56.473 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:56.473 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:56.473 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:56.473 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:58:56.474 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:58:56.474 INFO:teuthology.orchestra.run.smithi186.stdout:} 2024-05-09T04:58:57.156 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-05-09T04:58:57.156 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-05-09T04:58:57.156 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-05-09T04:58:57.156 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=14 2024-05-09T04:58:57.156 DEBUG:tasks.check_counter:Getting stats from f 2024-05-09T04:58:57.156 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-09T04:58:57.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:57 smithi186 ceph-mon[34668]: pgmap v619: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 12 op/s 2024-05-09T04:58:57.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:57 smithi169 ceph-mon[34075]: pgmap v619: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 12 op/s 2024-05-09T04:58:57.399 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T04:58:57.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:57 smithi006 ceph-mon[29835]: pgmap v619: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 12 op/s 2024-05-09T04:58:59.154 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:58:59 smithi186 ceph-mon[34668]: pgmap v620: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 10 op/s 2024-05-09T04:58:59.155 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2024-05-09T04:58:59.155 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T04:58:59.155 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 1320, 2024-05-09T04:58:59.155 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 1322, 2024-05-09T04:58:59.155 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 377345, 2024-05-09T04:58:59.155 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 419767, 2024-05-09T04:58:59.155 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 6, 2024-05-09T04:58:59.155 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 2, 2024-05-09T04:58:59.155 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.134607751, 2024-05-09T04:58:59.155 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.043981286, 2024-05-09T04:58:59.155 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.033289323, 2024-05-09T04:58:59.155 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.046513823, 2024-05-09T04:58:59.155 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:58:59.155 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1320, 2024-05-09T04:58:59.156 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.059467966, 2024-05-09T04:58:59.156 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000045051 2024-05-09T04:58:59.156 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.156 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:58:59.156 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.156 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.156 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.156 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.156 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T04:58:59.156 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T04:58:59.156 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.156 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T04:58:59.156 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 3, 2024-05-09T04:58:59.156 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 127, 2024-05-09T04:58:59.156 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 716, 2024-05-09T04:58:59.157 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 264138, 2024-05-09T04:58:59.157 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 4, 2024-05-09T04:58:59.157 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 3, 2024-05-09T04:58:59.157 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.007640263, 2024-05-09T04:58:59.157 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.005236589, 2024-05-09T04:58:59.157 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.000989698, 2024-05-09T04:58:59.157 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000065524, 2024-05-09T04:58:59.157 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:58:59.157 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 125, 2024-05-09T04:58:59.157 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.010223632, 2024-05-09T04:58:59.157 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000081789 2024-05-09T04:58:59.157 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.157 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:58:59.158 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.158 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.158 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.158 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.158 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T04:58:59.158 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 263440 2024-05-09T04:58:59.158 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.158 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T04:58:59.158 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 181, 2024-05-09T04:58:59.158 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 172, 2024-05-09T04:58:59.158 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 831756, 2024-05-09T04:58:59.158 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 102126, 2024-05-09T04:58:59.158 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 3, 2024-05-09T04:58:59.158 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 2, 2024-05-09T04:58:59.159 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.032778938, 2024-05-09T04:58:59.159 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.011032641, 2024-05-09T04:58:59.159 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.020733500, 2024-05-09T04:58:59.159 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.004110996, 2024-05-09T04:58:59.159 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:58:59.159 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 172, 2024-05-09T04:58:59.159 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.011970404, 2024-05-09T04:58:59.159 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000069595 2024-05-09T04:58:59.159 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.159 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:58:59.159 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.159 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.159 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.159 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.159 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 831488, 2024-05-09T04:58:59.160 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 101808 2024-05-09T04:58:59.160 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.160 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2024-05-09T04:58:59.160 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 1, 2024-05-09T04:58:59.160 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2024-05-09T04:58:59.160 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.160 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-MDSRank": { 2024-05-09T04:58:59.160 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-05-09T04:58:59.160 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-05-09T04:58:59.160 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2276, 2024-05-09T04:58:59.160 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.076281856, 2024-05-09T04:58:59.160 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000033515 2024-05-09T04:58:59.160 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:58:59.160 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.160 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-PurgeQueue": { 2024-05-09T04:58:59.161 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-05-09T04:58:59.161 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-05-09T04:58:59.161 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.161 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.161 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.161 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:58:59.161 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.161 INFO:teuthology.orchestra.run.smithi186.stdout: "mds": { 2024-05-09T04:58:59.161 INFO:teuthology.orchestra.run.smithi186.stdout: "request": 0, 2024-05-09T04:58:59.161 INFO:teuthology.orchestra.run.smithi186.stdout: "reply": 0, 2024-05-09T04:58:59.161 INFO:teuthology.orchestra.run.smithi186.stdout: "reply_latency": { 2024-05-09T04:58:59.161 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.161 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.161 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.161 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.161 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_reply": 0, 2024-05-09T04:58:59.162 INFO:teuthology.orchestra.run.smithi186.stdout: "forward": 0, 2024-05-09T04:58:59.162 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_complete": 0, 2024-05-09T04:58:59.162 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_keys": 0, 2024-05-09T04:58:59.162 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_commit": 0, 2024-05-09T04:58:59.162 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_split": 0, 2024-05-09T04:58:59.162 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_merge": 0, 2024-05-09T04:58:59.162 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes": 0, 2024-05-09T04:58:59.162 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_top": 0, 2024-05-09T04:58:59.162 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_bottom": 0, 2024-05-09T04:58:59.162 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pin_tail": 0, 2024-05-09T04:58:59.162 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pinned": 0, 2024-05-09T04:58:59.162 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_expired": 0, 2024-05-09T04:58:59.162 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_with_caps": 0, 2024-05-09T04:58:59.162 INFO:teuthology.orchestra.run.smithi186.stdout: "caps": 0, 2024-05-09T04:58:59.163 INFO:teuthology.orchestra.run.smithi186.stdout: "subtrees": 1, 2024-05-09T04:58:59.163 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse": 0, 2024-05-09T04:58:59.163 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_hit": 0, 2024-05-09T04:58:59.163 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_forward": 0, 2024-05-09T04:58:59.163 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_discover": 0, 2024-05-09T04:58:59.163 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_dir_fetch": 0, 2024-05-09T04:58:59.163 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_remote_ino": 0, 2024-05-09T04:58:59.163 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_lock": 0, 2024-05-09T04:58:59.163 INFO:teuthology.orchestra.run.smithi186.stdout: "load_cent": 0, 2024-05-09T04:58:59.163 INFO:teuthology.orchestra.run.smithi186.stdout: "q": 0, 2024-05-09T04:58:59.163 INFO:teuthology.orchestra.run.smithi186.stdout: "exported": 0, 2024-05-09T04:58:59.163 INFO:teuthology.orchestra.run.smithi186.stdout: "exported_inodes": 0, 2024-05-09T04:58:59.163 INFO:teuthology.orchestra.run.smithi186.stdout: "imported": 0, 2024-05-09T04:58:59.163 INFO:teuthology.orchestra.run.smithi186.stdout: "imported_inodes": 0, 2024-05-09T04:58:59.163 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_dir_fetch": 0, 2024-05-09T04:58:59.163 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_backtrace_fetch": 0, 2024-05-09T04:58:59.164 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_peer_discover": 0, 2024-05-09T04:58:59.164 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rfiles": 0, 2024-05-09T04:58:59.164 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rbytes": 0, 2024-05-09T04:58:59.164 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rsnaps": 0, 2024-05-09T04:58:59.164 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T04:58:59.164 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_set_tag": 0, 2024-05-09T04:58:59.164 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T04:58:59.164 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_inotable_repaired": 0, 2024-05-09T04:58:59.164 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_inodes": 0, 2024-05-09T04:58:59.164 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T04:58:59.164 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T04:58:59.164 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_file_inodes": 0, 2024-05-09T04:58:59.164 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_inode_file_caps": 0, 2024-05-09T04:58:59.164 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T04:58:59.164 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_grant": 0, 2024-05-09T04:58:59.164 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T04:58:59.164 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T04:58:59.165 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T04:58:59.165 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps": 0, 2024-05-09T04:58:59.165 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps_dirty": 0, 2024-05-09T04:58:59.165 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_cap_release": 0, 2024-05-09T04:58:59.165 INFO:teuthology.orchestra.run.smithi186.stdout: "process_request_cap_release": 0 2024-05-09T04:58:59.165 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.165 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_cache": { 2024-05-09T04:58:59.165 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update": 0, 2024-05-09T04:58:59.165 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update_receipt": 0, 2024-05-09T04:58:59.165 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_try_discover": 0, 2024-05-09T04:58:59.165 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_send_discover": 0, 2024-05-09T04:58:59.165 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_handle_discover": 0, 2024-05-09T04:58:59.165 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays": 0, 2024-05-09T04:58:59.165 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_delayed": 0, 2024-05-09T04:58:59.165 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_enqueuing": 0, 2024-05-09T04:58:59.165 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_created": 0, 2024-05-09T04:58:59.166 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_enqueued": 0, 2024-05-09T04:58:59.166 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_reintegrated": 0, 2024-05-09T04:58:59.166 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_migrated": 0, 2024-05-09T04:58:59.166 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_processing": 0, 2024-05-09T04:58:59.166 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_enqueued": 0, 2024-05-09T04:58:59.166 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_prioritized": 0, 2024-05-09T04:58:59.166 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_started": 0, 2024-05-09T04:58:59.166 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_completed": 0, 2024-05-09T04:58:59.166 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_quiesce_path": 0, 2024-05-09T04:58:59.166 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_quiesce_inode": 0, 2024-05-09T04:58:59.166 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T04:58:59.166 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_exportdir": 0, 2024-05-09T04:58:59.166 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_flush": 0, 2024-05-09T04:58:59.166 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragmentdir": 0, 2024-05-09T04:58:59.167 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragstats": 0, 2024-05-09T04:58:59.167 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_inodestats": 0 2024-05-09T04:58:59.167 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.167 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_log": { 2024-05-09T04:58:59.167 INFO:teuthology.orchestra.run.smithi186.stdout: "evlrg": 0, 2024-05-09T04:58:59.167 INFO:teuthology.orchestra.run.smithi186.stdout: "evadd": 0, 2024-05-09T04:58:59.167 INFO:teuthology.orchestra.run.smithi186.stdout: "evex": 0, 2024-05-09T04:58:59.167 INFO:teuthology.orchestra.run.smithi186.stdout: "evtrm": 0, 2024-05-09T04:58:59.167 INFO:teuthology.orchestra.run.smithi186.stdout: "ev": 6, 2024-05-09T04:58:59.167 INFO:teuthology.orchestra.run.smithi186.stdout: "evexg": 0, 2024-05-09T04:58:59.167 INFO:teuthology.orchestra.run.smithi186.stdout: "evexd": 0, 2024-05-09T04:58:59.167 INFO:teuthology.orchestra.run.smithi186.stdout: "segadd": 0, 2024-05-09T04:58:59.167 INFO:teuthology.orchestra.run.smithi186.stdout: "segex": 0, 2024-05-09T04:58:59.167 INFO:teuthology.orchestra.run.smithi186.stdout: "segtrm": 0, 2024-05-09T04:58:59.167 INFO:teuthology.orchestra.run.smithi186.stdout: "seg": 2, 2024-05-09T04:58:59.167 INFO:teuthology.orchestra.run.smithi186.stdout: "segmjr": 2, 2024-05-09T04:58:59.168 INFO:teuthology.orchestra.run.smithi186.stdout: "segexg": 0, 2024-05-09T04:58:59.168 INFO:teuthology.orchestra.run.smithi186.stdout: "segexd": 0, 2024-05-09T04:58:59.168 INFO:teuthology.orchestra.run.smithi186.stdout: "expos": 4194304, 2024-05-09T04:58:59.168 INFO:teuthology.orchestra.run.smithi186.stdout: "wrpos": 4195956, 2024-05-09T04:58:59.168 INFO:teuthology.orchestra.run.smithi186.stdout: "rdpos": 4195815, 2024-05-09T04:58:59.168 INFO:teuthology.orchestra.run.smithi186.stdout: "jlat": { 2024-05-09T04:58:59.168 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.168 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.168 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.168 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.168 INFO:teuthology.orchestra.run.smithi186.stdout: "replayed": 6 2024-05-09T04:58:59.168 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.168 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_mem": { 2024-05-09T04:58:59.168 INFO:teuthology.orchestra.run.smithi186.stdout: "ino": 3, 2024-05-09T04:58:59.168 INFO:teuthology.orchestra.run.smithi186.stdout: "ino+": 3, 2024-05-09T04:58:59.168 INFO:teuthology.orchestra.run.smithi186.stdout: "ino-": 0, 2024-05-09T04:58:59.169 INFO:teuthology.orchestra.run.smithi186.stdout: "dir": 1, 2024-05-09T04:58:59.169 INFO:teuthology.orchestra.run.smithi186.stdout: "dir+": 1, 2024-05-09T04:58:59.169 INFO:teuthology.orchestra.run.smithi186.stdout: "dir-": 0, 2024-05-09T04:58:59.169 INFO:teuthology.orchestra.run.smithi186.stdout: "dn": 0, 2024-05-09T04:58:59.169 INFO:teuthology.orchestra.run.smithi186.stdout: "dn+": 0, 2024-05-09T04:58:59.169 INFO:teuthology.orchestra.run.smithi186.stdout: "dn-": 0, 2024-05-09T04:58:59.169 INFO:teuthology.orchestra.run.smithi186.stdout: "cap": 0, 2024-05-09T04:58:59.169 INFO:teuthology.orchestra.run.smithi186.stdout: "cap+": 0, 2024-05-09T04:58:59.169 INFO:teuthology.orchestra.run.smithi186.stdout: "cap-": 0, 2024-05-09T04:58:59.169 INFO:teuthology.orchestra.run.smithi186.stdout: "rss": 118004, 2024-05-09T04:58:59.169 INFO:teuthology.orchestra.run.smithi186.stdout: "heap": 215440 2024-05-09T04:58:59.169 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.169 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_server": { 2024-05-09T04:58:59.169 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_client_request": 0, 2024-05-09T04:58:59.170 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_server_request": 0, 2024-05-09T04:58:59.170 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_request": 0, 2024-05-09T04:58:59.170 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_session": 0, 2024-05-09T04:58:59.170 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_peer_request": 0, 2024-05-09T04:58:59.170 INFO:teuthology.orchestra.run.smithi186.stdout: "req_create_latency": { 2024-05-09T04:58:59.170 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.170 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.170 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.170 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.170 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getattr_latency": { 2024-05-09T04:58:59.170 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.170 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.170 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.170 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.170 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getfilelock_latency": { 2024-05-09T04:58:59.171 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.171 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.171 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.171 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.171 INFO:teuthology.orchestra.run.smithi186.stdout: "req_link_latency": { 2024-05-09T04:58:59.171 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.171 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.171 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.171 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.171 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookup_latency": { 2024-05-09T04:58:59.171 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.171 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.171 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.171 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.171 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookuphash_latency": { 2024-05-09T04:58:59.172 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.172 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.172 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.172 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.172 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupino_latency": { 2024-05-09T04:58:59.172 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.172 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.172 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.172 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.172 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupname_latency": { 2024-05-09T04:58:59.172 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.172 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.172 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.172 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.172 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupparent_latency": { 2024-05-09T04:58:59.172 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.173 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.173 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.173 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.173 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupsnap_latency": { 2024-05-09T04:58:59.173 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.173 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.173 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.173 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.173 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lssnap_latency": { 2024-05-09T04:58:59.173 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.173 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.173 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.173 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.173 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mkdir_latency": { 2024-05-09T04:58:59.173 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.174 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.174 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.174 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.174 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mknod_latency": { 2024-05-09T04:58:59.174 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.174 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.174 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.175 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.175 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mksnap_latency": { 2024-05-09T04:58:59.175 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.175 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.175 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.175 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.175 INFO:teuthology.orchestra.run.smithi186.stdout: "req_open_latency": { 2024-05-09T04:58:59.175 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.175 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.175 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.175 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.175 INFO:teuthology.orchestra.run.smithi186.stdout: "req_readdir_latency": { 2024-05-09T04:58:59.175 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.175 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.175 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.175 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.176 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rename_latency": { 2024-05-09T04:58:59.176 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.176 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.176 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.176 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.176 INFO:teuthology.orchestra.run.smithi186.stdout: "req_renamesnap_latency": { 2024-05-09T04:58:59.176 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.176 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.176 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.176 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.176 INFO:teuthology.orchestra.run.smithi186.stdout: "req_snapdiff_latency": { 2024-05-09T04:58:59.176 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.176 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.176 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.177 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.177 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmdir_latency": { 2024-05-09T04:58:59.177 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.177 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.177 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.177 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.177 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmsnap_latency": { 2024-05-09T04:58:59.177 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.177 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.177 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.177 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.177 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmxattr_latency": { 2024-05-09T04:58:59.177 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.177 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.177 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.178 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.178 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setattr_latency": { 2024-05-09T04:58:59.178 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.178 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.178 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.178 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.178 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setdirlayout_latency": { 2024-05-09T04:58:59.178 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.178 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.178 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.178 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.178 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setfilelock_latency": { 2024-05-09T04:58:59.178 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.179 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.179 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.179 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.179 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setlayout_latency": { 2024-05-09T04:58:59.179 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.179 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.179 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.179 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.179 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setxattr_latency": { 2024-05-09T04:58:59.179 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.179 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.179 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.179 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.179 INFO:teuthology.orchestra.run.smithi186.stdout: "req_symlink_latency": { 2024-05-09T04:58:59.179 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.179 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.180 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.180 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.180 INFO:teuthology.orchestra.run.smithi186.stdout: "req_unlink_latency": { 2024-05-09T04:58:59.180 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.180 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.180 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.180 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.180 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_revoke_eviction": 0, 2024-05-09T04:58:59.180 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_acquisition_throttle": 0, 2024-05-09T04:58:59.180 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getvxattr_latency": { 2024-05-09T04:58:59.180 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.180 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.180 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.180 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:58:59.180 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.181 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_sessions": { 2024-05-09T04:58:59.181 INFO:teuthology.orchestra.run.smithi186.stdout: "session_count": 0, 2024-05-09T04:58:59.181 INFO:teuthology.orchestra.run.smithi186.stdout: "session_add": 2, 2024-05-09T04:58:59.181 INFO:teuthology.orchestra.run.smithi186.stdout: "session_remove": 2, 2024-05-09T04:58:59.181 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_open": 1, 2024-05-09T04:58:59.181 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_stale": 0, 2024-05-09T04:58:59.181 INFO:teuthology.orchestra.run.smithi186.stdout: "total_load": 0, 2024-05-09T04:58:59.181 INFO:teuthology.orchestra.run.smithi186.stdout: "average_load": 0, 2024-05-09T04:58:59.181 INFO:teuthology.orchestra.run.smithi186.stdout: "avg_session_uptime": 1878, 2024-05-09T04:58:59.181 INFO:teuthology.orchestra.run.smithi186.stdout: "mdthresh_evicted": 0 2024-05-09T04:58:59.181 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.181 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2024-05-09T04:58:59.181 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2024-05-09T04:58:59.181 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2024-05-09T04:58:59.181 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T04:58:59.181 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2024-05-09T04:58:59.182 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T04:58:59.182 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2024-05-09T04:58:59.182 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T04:58:59.182 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T04:58:59.182 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T04:58:59.182 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T04:58:59.182 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T04:58:59.182 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2024-05-09T04:58:59.182 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T04:58:59.182 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T04:58:59.182 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2024-05-09T04:58:59.182 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2024-05-09T04:58:59.182 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2024-05-09T04:58:59.182 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2024-05-09T04:58:59.182 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T04:58:59.182 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T04:58:59.183 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T04:58:59.183 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T04:58:59.183 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T04:58:59.183 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2024-05-09T04:58:59.183 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2024-05-09T04:58:59.183 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2024-05-09T04:58:59.183 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T04:58:59.183 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T04:58:59.183 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2024-05-09T04:58:59.183 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2024-05-09T04:58:59.183 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2024-05-09T04:58:59.183 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2024-05-09T04:58:59.183 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T04:58:59.183 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2024-05-09T04:58:59.183 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T04:58:59.183 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2024-05-09T04:58:59.183 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 181298, 2024-05-09T04:58:59.184 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 47, 2024-05-09T04:58:59.184 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 0, 2024-05-09T04:58:59.184 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 0, 2024-05-09T04:58:59.184 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2024-05-09T04:58:59.184 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2024-05-09T04:58:59.184 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2024-05-09T04:58:59.184 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2024-05-09T04:58:59.184 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2024-05-09T04:58:59.184 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2024-05-09T04:58:59.184 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 8800, 2024-05-09T04:58:59.184 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 169, 2024-05-09T04:58:59.184 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T04:58:59.184 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2024-05-09T04:58:59.184 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2024-05-09T04:58:59.184 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2024-05-09T04:58:59.184 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 8512, 2024-05-09T04:58:59.185 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 13, 2024-05-09T04:58:59.185 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2024-05-09T04:58:59.185 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2024-05-09T04:58:59.185 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2024-05-09T04:58:59.185 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2024-05-09T04:58:59.185 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.185 INFO:teuthology.orchestra.run.smithi186.stdout: "objecter": { 2024-05-09T04:58:59.185 INFO:teuthology.orchestra.run.smithi186.stdout: "op_active": 0, 2024-05-09T04:58:59.185 INFO:teuthology.orchestra.run.smithi186.stdout: "op_laggy": 0, 2024-05-09T04:58:59.185 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send": 1323, 2024-05-09T04:58:59.185 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send_bytes": 21, 2024-05-09T04:58:59.185 INFO:teuthology.orchestra.run.smithi186.stdout: "op_resend": 3, 2024-05-09T04:58:59.185 INFO:teuthology.orchestra.run.smithi186.stdout: "op_reply": 1323, 2024-05-09T04:58:59.185 INFO:teuthology.orchestra.run.smithi186.stdout: "op_latency": { 2024-05-09T04:58:59.185 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1323, 2024-05-09T04:58:59.185 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.626987920, 2024-05-09T04:58:59.186 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000473913 2024-05-09T04:58:59.186 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.186 INFO:teuthology.orchestra.run.smithi186.stdout: "op_inflight": 0, 2024-05-09T04:58:59.186 INFO:teuthology.orchestra.run.smithi186.stdout: "oplen_avg": { 2024-05-09T04:58:59.186 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1323, 2024-05-09T04:58:59.186 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 1324 2024-05-09T04:58:59.186 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.186 INFO:teuthology.orchestra.run.smithi186.stdout: "op": 1323, 2024-05-09T04:58:59.186 INFO:teuthology.orchestra.run.smithi186.stdout: "op_r": 1323, 2024-05-09T04:58:59.186 INFO:teuthology.orchestra.run.smithi186.stdout: "op_w": 0, 2024-05-09T04:58:59.186 INFO:teuthology.orchestra.run.smithi186.stdout: "op_rmw": 0, 2024-05-09T04:58:59.186 INFO:teuthology.orchestra.run.smithi186.stdout: "op_pg": 0, 2024-05-09T04:58:59.186 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_stat": 838, 2024-05-09T04:58:59.186 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_create": 0, 2024-05-09T04:58:59.186 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_read": 483, 2024-05-09T04:58:59.187 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_write": 0, 2024-05-09T04:58:59.187 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writefull": 0, 2024-05-09T04:58:59.187 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writesame": 0, 2024-05-09T04:58:59.187 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_append": 0, 2024-05-09T04:58:59.187 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_zero": 0, 2024-05-09T04:58:59.187 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_truncate": 0, 2024-05-09T04:58:59.187 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_delete": 0, 2024-05-09T04:58:59.187 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_mapext": 0, 2024-05-09T04:58:59.187 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_sparse_read": 0, 2024-05-09T04:58:59.187 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_clonerange": 0, 2024-05-09T04:58:59.187 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_getxattr": 1, 2024-05-09T04:58:59.187 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_setxattr": 0, 2024-05-09T04:58:59.187 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_cmpxattr": 0, 2024-05-09T04:58:59.187 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_rmxattr": 0, 2024-05-09T04:58:59.187 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_resetxattrs": 0, 2024-05-09T04:58:59.187 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_call": 0, 2024-05-09T04:58:59.187 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_watch": 0, 2024-05-09T04:58:59.188 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_notify": 0, 2024-05-09T04:58:59.188 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T04:58:59.188 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls": 0, 2024-05-09T04:58:59.188 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls_filter": 0, 2024-05-09T04:58:59.188 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_other": 0, 2024-05-09T04:58:59.188 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_active": 0, 2024-05-09T04:58:59.188 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_send": 0, 2024-05-09T04:58:59.188 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_resend": 0, 2024-05-09T04:58:59.188 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_ping": 0, 2024-05-09T04:58:59.188 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_active": 0, 2024-05-09T04:58:59.188 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_send": 0, 2024-05-09T04:58:59.188 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_resend": 0, 2024-05-09T04:58:59.188 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_active": 0, 2024-05-09T04:58:59.188 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_send": 0, 2024-05-09T04:58:59.188 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_resend": 0, 2024-05-09T04:58:59.188 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_active": 0, 2024-05-09T04:58:59.188 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_send": 0, 2024-05-09T04:58:59.189 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_resend": 0, 2024-05-09T04:58:59.189 INFO:teuthology.orchestra.run.smithi186.stdout: "command_active": 0, 2024-05-09T04:58:59.189 INFO:teuthology.orchestra.run.smithi186.stdout: "command_send": 0, 2024-05-09T04:58:59.189 INFO:teuthology.orchestra.run.smithi186.stdout: "command_resend": 0, 2024-05-09T04:58:59.189 INFO:teuthology.orchestra.run.smithi186.stdout: "map_epoch": 0, 2024-05-09T04:58:59.189 INFO:teuthology.orchestra.run.smithi186.stdout: "map_full": 0, 2024-05-09T04:58:59.189 INFO:teuthology.orchestra.run.smithi186.stdout: "map_inc": 0, 2024-05-09T04:58:59.189 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_sessions": 4, 2024-05-09T04:58:59.189 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_open": 4, 2024-05-09T04:58:59.189 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_close": 0, 2024-05-09T04:58:59.189 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_laggy": 0, 2024-05-09T04:58:59.189 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_wr": 0, 2024-05-09T04:58:59.189 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_rd": 2, 2024-05-09T04:58:59.189 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_del": 0 2024-05-09T04:58:59.189 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.189 INFO:teuthology.orchestra.run.smithi186.stdout: "oft": { 2024-05-09T04:58:59.189 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_objs": 0, 2024-05-09T04:58:59.190 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_kv_pairs": 0, 2024-05-09T04:58:59.190 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_updates": 0, 2024-05-09T04:58:59.190 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_removes": 0 2024-05-09T04:58:59.190 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.190 INFO:teuthology.orchestra.run.smithi186.stdout: "osd-slow-ops": { 2024-05-09T04:58:59.190 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_ops_count": 0 2024-05-09T04:58:59.190 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.190 INFO:teuthology.orchestra.run.smithi186.stdout: "purge_queue": { 2024-05-09T04:58:59.190 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops": 0, 2024-05-09T04:58:59.190 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T04:58:59.190 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing": 0, 2024-05-09T04:58:59.190 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_high_water": 0, 2024-05-09T04:58:59.190 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed_ops": 0, 2024-05-09T04:58:59.190 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed": 0, 2024-05-09T04:58:59.190 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_item_in_journal": 0 2024-05-09T04:58:59.190 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.190 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T04:58:59.191 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-09T04:58:59.191 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-05-09T04:58:59.191 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-09T04:58:59.191 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 1500, 2024-05-09T04:58:59.191 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 1062736, 2024-05-09T04:58:59.191 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:59.191 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 1500, 2024-05-09T04:58:59.191 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-05-09T04:58:59.191 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-05-09T04:58:59.191 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 1500, 2024-05-09T04:58:59.191 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 1062736, 2024-05-09T04:58:59.191 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-09T04:58:59.191 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.191 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.191 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.191 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:58:59.192 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.192 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_bytes": { 2024-05-09T04:58:59.192 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-09T04:58:59.192 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-05-09T04:58:59.192 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-09T04:58:59.192 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-05-09T04:58:59.192 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-05-09T04:58:59.192 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:59.192 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-05-09T04:58:59.192 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 1323, 2024-05-09T04:58:59.192 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 1657, 2024-05-09T04:58:59.192 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 1323, 2024-05-09T04:58:59.192 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 1657, 2024-05-09T04:58:59.192 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-09T04:58:59.192 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.192 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.192 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.193 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:58:59.193 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.193 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_ops": { 2024-05-09T04:58:59.193 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-09T04:58:59.193 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 1024, 2024-05-09T04:58:59.193 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-09T04:58:59.193 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-05-09T04:58:59.193 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-05-09T04:58:59.193 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:59.193 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-05-09T04:58:59.193 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 1323, 2024-05-09T04:58:59.193 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 1323, 2024-05-09T04:58:59.193 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 1323, 2024-05-09T04:58:59.193 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 1323, 2024-05-09T04:58:59.193 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-09T04:58:59.193 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.194 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.194 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.194 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:58:59.194 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.194 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle": { 2024-05-09T04:58:59.194 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-09T04:58:59.194 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-05-09T04:58:59.194 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-09T04:58:59.194 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-05-09T04:58:59.194 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-05-09T04:58:59.194 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:59.194 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-05-09T04:58:59.194 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-05-09T04:58:59.194 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-05-09T04:58:59.194 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2024-05-09T04:58:59.195 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2024-05-09T04:58:59.195 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-09T04:58:59.195 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.195 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.195 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.195 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:58:59.195 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:58:59.195 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle-0x5645f88c08f0": { 2024-05-09T04:58:59.195 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-09T04:58:59.195 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-05-09T04:58:59.195 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-09T04:58:59.195 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-05-09T04:58:59.195 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-05-09T04:58:59.195 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-09T04:58:59.195 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-05-09T04:58:59.196 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-05-09T04:58:59.196 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-05-09T04:58:59.196 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2024-05-09T04:58:59.196 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2024-05-09T04:58:59.196 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-09T04:58:59.196 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:58:59.196 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:58:59.196 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:58:59.196 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:58:59.196 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:58:59.196 INFO:teuthology.orchestra.run.smithi186.stdout:} 2024-05-09T04:58:59.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:58:59 smithi169 ceph-mon[34075]: pgmap v620: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 10 op/s 2024-05-09T04:58:59.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:58:59 smithi006 ceph-mon[29835]: pgmap v620: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 10 op/s 2024-05-09T04:58:59.924 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-09T04:58:59.924 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-09T04:58:59.924 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-05-09T04:58:59.924 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-05-09T04:58:59.924 DEBUG:tasks.check_counter:Getting stats from i 2024-05-09T04:58:59.924 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-09T04:59:00.162 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T04:59:01.284 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:01 smithi186 ceph-mon[34668]: pgmap v621: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-09T04:59:01.347 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:01 smithi006 ceph-mon[29835]: pgmap v621: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-09T04:59:01.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:01 smithi169 ceph-mon[34075]: pgmap v621: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-09T04:59:01.775 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2024-05-09T04:59:01.775 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T04:59:01.775 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 536, 2024-05-09T04:59:01.775 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 579, 2024-05-09T04:59:01.775 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 941603, 2024-05-09T04:59:01.775 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 256047, 2024-05-09T04:59:01.775 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 9, 2024-05-09T04:59:01.775 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 5, 2024-05-09T04:59:01.775 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.075827075, 2024-05-09T04:59:01.775 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.027540260, 2024-05-09T04:59:01.775 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.043068975, 2024-05-09T04:59:01.776 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.007797574, 2024-05-09T04:59:01.776 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:59:01.776 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 578, 2024-05-09T04:59:01.776 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.037115384, 2024-05-09T04:59:01.776 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000064213 2024-05-09T04:59:01.776 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.776 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:59:01.776 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 680, 2024-05-09T04:59:01.776 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000232266, 2024-05-09T04:59:01.776 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000341 2024-05-09T04:59:01.776 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.776 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 846272, 2024-05-09T04:59:01.776 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 104720 2024-05-09T04:59:01.776 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.777 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T04:59:01.777 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 27, 2024-05-09T04:59:01.777 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 150, 2024-05-09T04:59:01.777 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 7013, 2024-05-09T04:59:01.777 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 321650, 2024-05-09T04:59:01.777 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 11, 2024-05-09T04:59:01.777 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 6, 2024-05-09T04:59:01.777 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.014204692, 2024-05-09T04:59:01.777 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.006564712, 2024-05-09T04:59:01.777 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.005697054, 2024-05-09T04:59:01.777 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000465705, 2024-05-09T04:59:01.777 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:59:01.777 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 148, 2024-05-09T04:59:01.777 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.025238891, 2024-05-09T04:59:01.778 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000170533 2024-05-09T04:59:01.778 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.778 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:59:01.778 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 11, 2024-05-09T04:59:01.778 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000006349, 2024-05-09T04:59:01.778 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000577 2024-05-09T04:59:01.778 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.778 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T04:59:01.778 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 264880 2024-05-09T04:59:01.778 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.778 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T04:59:01.778 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 29, 2024-05-09T04:59:01.778 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 29, 2024-05-09T04:59:01.779 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 10160, 2024-05-09T04:59:01.779 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 17244, 2024-05-09T04:59:01.779 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 7, 2024-05-09T04:59:01.779 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 5, 2024-05-09T04:59:01.779 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.004168078, 2024-05-09T04:59:01.779 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.000642384, 2024-05-09T04:59:01.779 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.003213677, 2024-05-09T04:59:01.779 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000385397, 2024-05-09T04:59:01.779 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:59:01.779 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 28, 2024-05-09T04:59:01.779 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.013846832, 2024-05-09T04:59:01.779 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000494529 2024-05-09T04:59:01.779 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.779 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:59:01.780 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 21, 2024-05-09T04:59:01.780 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000003919, 2024-05-09T04:59:01.780 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000186 2024-05-09T04:59:01.780 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.780 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 6416, 2024-05-09T04:59:01.780 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 4976 2024-05-09T04:59:01.780 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.780 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2024-05-09T04:59:01.780 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 1, 2024-05-09T04:59:01.780 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2024-05-09T04:59:01.780 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.780 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-MDSRank": { 2024-05-09T04:59:01.780 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-05-09T04:59:01.780 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-05-09T04:59:01.781 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 50, 2024-05-09T04:59:01.781 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.001845091, 2024-05-09T04:59:01.781 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000036901 2024-05-09T04:59:01.781 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:59:01.781 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.781 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-PurgeQueue": { 2024-05-09T04:59:01.781 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-05-09T04:59:01.781 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-05-09T04:59:01.781 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2, 2024-05-09T04:59:01.781 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000008555, 2024-05-09T04:59:01.781 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000004277 2024-05-09T04:59:01.781 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:59:01.781 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.782 INFO:teuthology.orchestra.run.smithi186.stdout: "mds": { 2024-05-09T04:59:01.782 INFO:teuthology.orchestra.run.smithi186.stdout: "request": 6, 2024-05-09T04:59:01.782 INFO:teuthology.orchestra.run.smithi186.stdout: "reply": 4, 2024-05-09T04:59:01.782 INFO:teuthology.orchestra.run.smithi186.stdout: "reply_latency": { 2024-05-09T04:59:01.782 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 4, 2024-05-09T04:59:01.782 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.003489714, 2024-05-09T04:59:01.782 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000872428 2024-05-09T04:59:01.782 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.782 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_reply": 0, 2024-05-09T04:59:01.782 INFO:teuthology.orchestra.run.smithi186.stdout: "forward": 2, 2024-05-09T04:59:01.782 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_complete": 0, 2024-05-09T04:59:01.782 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_keys": 0, 2024-05-09T04:59:01.782 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_commit": 11, 2024-05-09T04:59:01.783 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_split": 0, 2024-05-09T04:59:01.783 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_merge": 0, 2024-05-09T04:59:01.783 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes": 16, 2024-05-09T04:59:01.783 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_top": 0, 2024-05-09T04:59:01.783 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_bottom": 16, 2024-05-09T04:59:01.783 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pin_tail": 0, 2024-05-09T04:59:01.783 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pinned": 13, 2024-05-09T04:59:01.783 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_expired": 4, 2024-05-09T04:59:01.783 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_with_caps": 1, 2024-05-09T04:59:01.783 INFO:teuthology.orchestra.run.smithi186.stdout: "caps": 1, 2024-05-09T04:59:01.783 INFO:teuthology.orchestra.run.smithi186.stdout: "subtrees": 3, 2024-05-09T04:59:01.783 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse": 20, 2024-05-09T04:59:01.783 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_hit": 15, 2024-05-09T04:59:01.783 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_forward": 1, 2024-05-09T04:59:01.783 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_discover": 4, 2024-05-09T04:59:01.784 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_dir_fetch": 0, 2024-05-09T04:59:01.784 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_remote_ino": 0, 2024-05-09T04:59:01.784 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_lock": 0, 2024-05-09T04:59:01.784 INFO:teuthology.orchestra.run.smithi186.stdout: "load_cent": 0, 2024-05-09T04:59:01.784 INFO:teuthology.orchestra.run.smithi186.stdout: "q": 0, 2024-05-09T04:59:01.784 INFO:teuthology.orchestra.run.smithi186.stdout: "exported": 0, 2024-05-09T04:59:01.784 INFO:teuthology.orchestra.run.smithi186.stdout: "exported_inodes": 0, 2024-05-09T04:59:01.784 INFO:teuthology.orchestra.run.smithi186.stdout: "imported": 0, 2024-05-09T04:59:01.784 INFO:teuthology.orchestra.run.smithi186.stdout: "imported_inodes": 0, 2024-05-09T04:59:01.784 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_dir_fetch": 0, 2024-05-09T04:59:01.784 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_backtrace_fetch": 0, 2024-05-09T04:59:01.784 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_peer_discover": 0, 2024-05-09T04:59:01.784 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rfiles": 4, 2024-05-09T04:59:01.784 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rbytes": 234, 2024-05-09T04:59:01.784 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rsnaps": 0, 2024-05-09T04:59:01.785 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T04:59:01.785 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_set_tag": 0, 2024-05-09T04:59:01.785 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T04:59:01.785 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_inotable_repaired": 0, 2024-05-09T04:59:01.785 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_inodes": 0, 2024-05-09T04:59:01.785 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T04:59:01.785 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T04:59:01.785 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_file_inodes": 0, 2024-05-09T04:59:01.785 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_inode_file_caps": 0, 2024-05-09T04:59:01.785 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_revoke": 4, 2024-05-09T04:59:01.785 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_grant": 0, 2024-05-09T04:59:01.785 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T04:59:01.785 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T04:59:01.785 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T04:59:01.786 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps": 4, 2024-05-09T04:59:01.786 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps_dirty": 0, 2024-05-09T04:59:01.786 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_cap_release": 0, 2024-05-09T04:59:01.786 INFO:teuthology.orchestra.run.smithi186.stdout: "process_request_cap_release": 0 2024-05-09T04:59:01.786 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.786 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_cache": { 2024-05-09T04:59:01.786 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update": 0, 2024-05-09T04:59:01.786 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update_receipt": 14, 2024-05-09T04:59:01.786 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_try_discover": 9, 2024-05-09T04:59:01.786 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_send_discover": 10, 2024-05-09T04:59:01.786 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_handle_discover": 0, 2024-05-09T04:59:01.786 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays": 0, 2024-05-09T04:59:01.786 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_delayed": 0, 2024-05-09T04:59:01.787 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_enqueuing": 0, 2024-05-09T04:59:01.787 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_created": 0, 2024-05-09T04:59:01.787 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_enqueued": 0, 2024-05-09T04:59:01.787 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_reintegrated": 0, 2024-05-09T04:59:01.787 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_migrated": 0, 2024-05-09T04:59:01.787 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_processing": 0, 2024-05-09T04:59:01.787 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_enqueued": 0, 2024-05-09T04:59:01.787 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_prioritized": 0, 2024-05-09T04:59:01.787 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_started": 0, 2024-05-09T04:59:01.787 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_completed": 0, 2024-05-09T04:59:01.787 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_quiesce_path": 2, 2024-05-09T04:59:01.787 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_quiesce_inode": 11, 2024-05-09T04:59:01.787 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T04:59:01.787 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_exportdir": 0, 2024-05-09T04:59:01.788 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_flush": 0, 2024-05-09T04:59:01.788 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragmentdir": 0, 2024-05-09T04:59:01.788 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragstats": 0, 2024-05-09T04:59:01.788 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_inodestats": 0 2024-05-09T04:59:01.788 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.788 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_log": { 2024-05-09T04:59:01.788 INFO:teuthology.orchestra.run.smithi186.stdout: "evlrg": 0, 2024-05-09T04:59:01.788 INFO:teuthology.orchestra.run.smithi186.stdout: "evadd": 5, 2024-05-09T04:59:01.788 INFO:teuthology.orchestra.run.smithi186.stdout: "evex": 0, 2024-05-09T04:59:01.788 INFO:teuthology.orchestra.run.smithi186.stdout: "evtrm": 0, 2024-05-09T04:59:01.788 INFO:teuthology.orchestra.run.smithi186.stdout: "ev": 5, 2024-05-09T04:59:01.788 INFO:teuthology.orchestra.run.smithi186.stdout: "evexg": 0, 2024-05-09T04:59:01.788 INFO:teuthology.orchestra.run.smithi186.stdout: "evexd": 0, 2024-05-09T04:59:01.788 INFO:teuthology.orchestra.run.smithi186.stdout: "segadd": 2, 2024-05-09T04:59:01.788 INFO:teuthology.orchestra.run.smithi186.stdout: "segex": 0, 2024-05-09T04:59:01.788 INFO:teuthology.orchestra.run.smithi186.stdout: "segtrm": 0, 2024-05-09T04:59:01.789 INFO:teuthology.orchestra.run.smithi186.stdout: "seg": 2, 2024-05-09T04:59:01.789 INFO:teuthology.orchestra.run.smithi186.stdout: "segmjr": 2, 2024-05-09T04:59:01.789 INFO:teuthology.orchestra.run.smithi186.stdout: "segexg": 0, 2024-05-09T04:59:01.789 INFO:teuthology.orchestra.run.smithi186.stdout: "segexd": 0, 2024-05-09T04:59:01.789 INFO:teuthology.orchestra.run.smithi186.stdout: "expos": 4194304, 2024-05-09T04:59:01.789 INFO:teuthology.orchestra.run.smithi186.stdout: "wrpos": 4195734, 2024-05-09T04:59:01.789 INFO:teuthology.orchestra.run.smithi186.stdout: "rdpos": 0, 2024-05-09T04:59:01.789 INFO:teuthology.orchestra.run.smithi186.stdout: "jlat": { 2024-05-09T04:59:01.789 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 4, 2024-05-09T04:59:01.789 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.008766938, 2024-05-09T04:59:01.789 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.002191734 2024-05-09T04:59:01.789 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.789 INFO:teuthology.orchestra.run.smithi186.stdout: "replayed": 0 2024-05-09T04:59:01.790 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.790 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_mem": { 2024-05-09T04:59:01.790 INFO:teuthology.orchestra.run.smithi186.stdout: "ino": 20, 2024-05-09T04:59:01.790 INFO:teuthology.orchestra.run.smithi186.stdout: "ino+": 22, 2024-05-09T04:59:01.790 INFO:teuthology.orchestra.run.smithi186.stdout: "ino-": 2, 2024-05-09T04:59:01.790 INFO:teuthology.orchestra.run.smithi186.stdout: "dir": 19, 2024-05-09T04:59:01.790 INFO:teuthology.orchestra.run.smithi186.stdout: "dir+": 23, 2024-05-09T04:59:01.790 INFO:teuthology.orchestra.run.smithi186.stdout: "dir-": 4, 2024-05-09T04:59:01.790 INFO:teuthology.orchestra.run.smithi186.stdout: "dn": 16, 2024-05-09T04:59:01.790 INFO:teuthology.orchestra.run.smithi186.stdout: "dn+": 20, 2024-05-09T04:59:01.790 INFO:teuthology.orchestra.run.smithi186.stdout: "dn-": 4, 2024-05-09T04:59:01.790 INFO:teuthology.orchestra.run.smithi186.stdout: "cap": 1, 2024-05-09T04:59:01.790 INFO:teuthology.orchestra.run.smithi186.stdout: "cap+": 4, 2024-05-09T04:59:01.790 INFO:teuthology.orchestra.run.smithi186.stdout: "cap-": 3, 2024-05-09T04:59:01.791 INFO:teuthology.orchestra.run.smithi186.stdout: "rss": 120564, 2024-05-09T04:59:01.791 INFO:teuthology.orchestra.run.smithi186.stdout: "heap": 223632 2024-05-09T04:59:01.791 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.791 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_server": { 2024-05-09T04:59:01.791 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_client_request": 9, 2024-05-09T04:59:01.791 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_server_request": 0, 2024-05-09T04:59:01.791 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_request": 6, 2024-05-09T04:59:01.791 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_session": 8, 2024-05-09T04:59:01.791 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_peer_request": 12, 2024-05-09T04:59:01.791 INFO:teuthology.orchestra.run.smithi186.stdout: "req_create_latency": { 2024-05-09T04:59:01.791 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:01.791 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:01.791 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:01.791 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.791 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getattr_latency": { 2024-05-09T04:59:01.792 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1, 2024-05-09T04:59:01.792 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.002483267, 2024-05-09T04:59:01.792 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.002483267 2024-05-09T04:59:01.792 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.792 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getfilelock_latency": { 2024-05-09T04:59:01.792 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:01.792 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:01.792 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:01.792 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.792 INFO:teuthology.orchestra.run.smithi186.stdout: "req_link_latency": { 2024-05-09T04:59:01.792 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:01.792 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:01.792 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:01.792 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.792 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookup_latency": { 2024-05-09T04:59:01.792 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2, 2024-05-09T04:59:01.793 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000719475, 2024-05-09T04:59:01.793 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000359737 2024-05-09T04:59:01.793 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.793 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookuphash_latency": { 2024-05-09T04:59:01.793 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:01.793 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:01.793 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:01.793 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.793 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupino_latency": { 2024-05-09T04:59:01.793 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1, 2024-05-09T04:59:01.793 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000286972, 2024-05-09T04:59:01.793 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000286972 2024-05-09T04:59:01.793 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.793 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupname_latency": { 2024-05-09T04:59:01.794 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:01.794 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:01.794 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:01.794 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.794 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupparent_latency": { 2024-05-09T04:59:01.794 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:01.794 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:01.794 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:01.794 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.794 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupsnap_latency": { 2024-05-09T04:59:01.794 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:01.794 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:01.794 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:01.794 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.794 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lssnap_latency": { 2024-05-09T04:59:01.795 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:01.795 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:01.795 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:01.795 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.795 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mkdir_latency": { 2024-05-09T04:59:01.795 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:01.795 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:01.795 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:01.796 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.796 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mknod_latency": { 2024-05-09T04:59:01.796 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:01.796 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:01.796 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:01.796 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.796 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mksnap_latency": { 2024-05-09T04:59:01.796 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:01.796 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:01.796 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:01.796 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.796 INFO:teuthology.orchestra.run.smithi186.stdout: "req_open_latency": { 2024-05-09T04:59:01.796 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:01.796 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:01.797 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:01.797 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.797 INFO:teuthology.orchestra.run.smithi186.stdout: "req_readdir_latency": { 2024-05-09T04:59:01.797 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:01.797 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:01.797 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:01.797 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.797 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rename_latency": { 2024-05-09T04:59:01.797 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:01.797 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:01.797 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:01.797 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.797 INFO:teuthology.orchestra.run.smithi186.stdout: "req_renamesnap_latency": { 2024-05-09T04:59:01.797 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:01.797 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:01.797 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:01.798 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.798 INFO:teuthology.orchestra.run.smithi186.stdout: "req_snapdiff_latency": { 2024-05-09T04:59:01.798 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:01.798 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:01.798 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:01.798 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.798 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmdir_latency": { 2024-05-09T04:59:01.798 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:01.798 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:01.798 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:01.798 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.798 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmsnap_latency": { 2024-05-09T04:59:01.798 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:01.798 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:01.798 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:01.799 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.799 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmxattr_latency": { 2024-05-09T04:59:01.799 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:01.799 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:01.799 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:01.799 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.799 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setattr_latency": { 2024-05-09T04:59:01.799 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:01.799 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:01.799 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:01.799 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.799 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setdirlayout_latency": { 2024-05-09T04:59:01.799 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:01.799 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:01.799 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:01.800 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.800 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setfilelock_latency": { 2024-05-09T04:59:01.800 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:01.800 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:01.800 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:01.800 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.800 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setlayout_latency": { 2024-05-09T04:59:01.800 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:01.800 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:01.800 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:01.800 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.800 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setxattr_latency": { 2024-05-09T04:59:01.800 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:01.800 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:01.800 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:01.800 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.801 INFO:teuthology.orchestra.run.smithi186.stdout: "req_symlink_latency": { 2024-05-09T04:59:01.801 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:01.801 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:01.801 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:01.801 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.801 INFO:teuthology.orchestra.run.smithi186.stdout: "req_unlink_latency": { 2024-05-09T04:59:01.801 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:01.801 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:01.801 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:01.801 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.801 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_revoke_eviction": 0, 2024-05-09T04:59:01.801 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_acquisition_throttle": 0, 2024-05-09T04:59:01.801 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getvxattr_latency": { 2024-05-09T04:59:01.801 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:01.802 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:01.802 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:01.802 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:59:01.802 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.802 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_sessions": { 2024-05-09T04:59:01.802 INFO:teuthology.orchestra.run.smithi186.stdout: "session_count": 1, 2024-05-09T04:59:01.802 INFO:teuthology.orchestra.run.smithi186.stdout: "session_add": 2, 2024-05-09T04:59:01.802 INFO:teuthology.orchestra.run.smithi186.stdout: "session_remove": 1, 2024-05-09T04:59:01.802 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_open": 1, 2024-05-09T04:59:01.802 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_stale": 0, 2024-05-09T04:59:01.802 INFO:teuthology.orchestra.run.smithi186.stdout: "total_load": 3, 2024-05-09T04:59:01.802 INFO:teuthology.orchestra.run.smithi186.stdout: "average_load": 1, 2024-05-09T04:59:01.802 INFO:teuthology.orchestra.run.smithi186.stdout: "avg_session_uptime": 246, 2024-05-09T04:59:01.802 INFO:teuthology.orchestra.run.smithi186.stdout: "mdthresh_evicted": 0 2024-05-09T04:59:01.803 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.803 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2024-05-09T04:59:01.803 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2024-05-09T04:59:01.803 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2024-05-09T04:59:01.803 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T04:59:01.803 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2024-05-09T04:59:01.803 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T04:59:01.803 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2024-05-09T04:59:01.803 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T04:59:01.803 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T04:59:01.803 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T04:59:01.803 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T04:59:01.804 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T04:59:01.804 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2024-05-09T04:59:01.804 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T04:59:01.804 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T04:59:01.804 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2024-05-09T04:59:01.804 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2024-05-09T04:59:01.804 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2024-05-09T04:59:01.804 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2024-05-09T04:59:01.804 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T04:59:01.804 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T04:59:01.804 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T04:59:01.804 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T04:59:01.804 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T04:59:01.804 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2024-05-09T04:59:01.804 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2024-05-09T04:59:01.805 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2024-05-09T04:59:01.805 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T04:59:01.805 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T04:59:01.805 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2024-05-09T04:59:01.805 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2024-05-09T04:59:01.805 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2024-05-09T04:59:01.805 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2024-05-09T04:59:01.805 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T04:59:01.805 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2024-05-09T04:59:01.805 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T04:59:01.805 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2024-05-09T04:59:01.805 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 239913, 2024-05-09T04:59:01.805 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 89, 2024-05-09T04:59:01.805 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 88, 2024-05-09T04:59:01.805 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 1, 2024-05-09T04:59:01.805 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2024-05-09T04:59:01.806 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2024-05-09T04:59:01.806 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2024-05-09T04:59:01.806 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2024-05-09T04:59:01.806 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2024-05-09T04:59:01.806 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2024-05-09T04:59:01.806 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 8800, 2024-05-09T04:59:01.806 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 169, 2024-05-09T04:59:01.806 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T04:59:01.806 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2024-05-09T04:59:01.806 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2024-05-09T04:59:01.806 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2024-05-09T04:59:01.806 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 81461, 2024-05-09T04:59:01.806 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 345, 2024-05-09T04:59:01.806 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2024-05-09T04:59:01.806 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2024-05-09T04:59:01.806 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2024-05-09T04:59:01.807 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2024-05-09T04:59:01.807 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.807 INFO:teuthology.orchestra.run.smithi186.stdout: "objecter": { 2024-05-09T04:59:01.807 INFO:teuthology.orchestra.run.smithi186.stdout: "op_active": 0, 2024-05-09T04:59:01.807 INFO:teuthology.orchestra.run.smithi186.stdout: "op_laggy": 0, 2024-05-09T04:59:01.807 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send": 42, 2024-05-09T04:59:01.807 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send_bytes": 11957, 2024-05-09T04:59:01.807 INFO:teuthology.orchestra.run.smithi186.stdout: "op_resend": 28, 2024-05-09T04:59:01.807 INFO:teuthology.orchestra.run.smithi186.stdout: "op_reply": 42, 2024-05-09T04:59:01.807 INFO:teuthology.orchestra.run.smithi186.stdout: "op_latency": { 2024-05-09T04:59:01.807 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 42, 2024-05-09T04:59:01.807 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.260994884, 2024-05-09T04:59:01.807 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.006214163 2024-05-09T04:59:01.807 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.807 INFO:teuthology.orchestra.run.smithi186.stdout: "op_inflight": 0, 2024-05-09T04:59:01.807 INFO:teuthology.orchestra.run.smithi186.stdout: "oplen_avg": { 2024-05-09T04:59:01.808 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 42, 2024-05-09T04:59:01.808 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 66 2024-05-09T04:59:01.808 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.808 INFO:teuthology.orchestra.run.smithi186.stdout: "op": 42, 2024-05-09T04:59:01.808 INFO:teuthology.orchestra.run.smithi186.stdout: "op_r": 0, 2024-05-09T04:59:01.808 INFO:teuthology.orchestra.run.smithi186.stdout: "op_w": 42, 2024-05-09T04:59:01.808 INFO:teuthology.orchestra.run.smithi186.stdout: "op_rmw": 0, 2024-05-09T04:59:01.808 INFO:teuthology.orchestra.run.smithi186.stdout: "op_pg": 0, 2024-05-09T04:59:01.808 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_stat": 0, 2024-05-09T04:59:01.808 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_create": 10, 2024-05-09T04:59:01.808 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_read": 0, 2024-05-09T04:59:01.808 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_write": 4, 2024-05-09T04:59:01.808 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writefull": 7, 2024-05-09T04:59:01.808 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writesame": 0, 2024-05-09T04:59:01.808 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_append": 0, 2024-05-09T04:59:01.808 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_zero": 0, 2024-05-09T04:59:01.808 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_truncate": 0, 2024-05-09T04:59:01.809 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_delete": 6, 2024-05-09T04:59:01.809 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_mapext": 0, 2024-05-09T04:59:01.809 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_sparse_read": 0, 2024-05-09T04:59:01.809 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_clonerange": 0, 2024-05-09T04:59:01.809 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_getxattr": 0, 2024-05-09T04:59:01.809 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_setxattr": 20, 2024-05-09T04:59:01.809 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_cmpxattr": 0, 2024-05-09T04:59:01.809 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_rmxattr": 0, 2024-05-09T04:59:01.809 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_resetxattrs": 0, 2024-05-09T04:59:01.809 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_call": 0, 2024-05-09T04:59:01.809 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_watch": 0, 2024-05-09T04:59:01.809 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_notify": 0, 2024-05-09T04:59:01.809 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T04:59:01.809 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls": 0, 2024-05-09T04:59:01.809 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls_filter": 0, 2024-05-09T04:59:01.809 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_other": 0, 2024-05-09T04:59:01.810 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_active": 0, 2024-05-09T04:59:01.810 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_send": 0, 2024-05-09T04:59:01.810 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_resend": 0, 2024-05-09T04:59:01.810 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_ping": 0, 2024-05-09T04:59:01.810 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_active": 0, 2024-05-09T04:59:01.810 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_send": 0, 2024-05-09T04:59:01.810 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_resend": 0, 2024-05-09T04:59:01.810 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_active": 0, 2024-05-09T04:59:01.810 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_send": 0, 2024-05-09T04:59:01.810 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_resend": 0, 2024-05-09T04:59:01.810 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_active": 0, 2024-05-09T04:59:01.810 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_send": 0, 2024-05-09T04:59:01.810 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_resend": 0, 2024-05-09T04:59:01.810 INFO:teuthology.orchestra.run.smithi186.stdout: "command_active": 0, 2024-05-09T04:59:01.810 INFO:teuthology.orchestra.run.smithi186.stdout: "command_send": 0, 2024-05-09T04:59:01.811 INFO:teuthology.orchestra.run.smithi186.stdout: "command_resend": 0, 2024-05-09T04:59:01.811 INFO:teuthology.orchestra.run.smithi186.stdout: "map_epoch": 0, 2024-05-09T04:59:01.811 INFO:teuthology.orchestra.run.smithi186.stdout: "map_full": 0, 2024-05-09T04:59:01.811 INFO:teuthology.orchestra.run.smithi186.stdout: "map_inc": 0, 2024-05-09T04:59:01.811 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_sessions": 9, 2024-05-09T04:59:01.811 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_open": 9, 2024-05-09T04:59:01.811 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_close": 0, 2024-05-09T04:59:01.811 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_laggy": 0, 2024-05-09T04:59:01.811 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_wr": 17, 2024-05-09T04:59:01.811 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_rd": 0, 2024-05-09T04:59:01.811 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_del": 2 2024-05-09T04:59:01.811 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.811 INFO:teuthology.orchestra.run.smithi186.stdout: "oft": { 2024-05-09T04:59:01.812 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_objs": 1, 2024-05-09T04:59:01.812 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_kv_pairs": 0, 2024-05-09T04:59:01.812 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_updates": 0, 2024-05-09T04:59:01.812 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_removes": 0 2024-05-09T04:59:01.812 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.812 INFO:teuthology.orchestra.run.smithi186.stdout: "osd-slow-ops": { 2024-05-09T04:59:01.812 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_ops_count": 12 2024-05-09T04:59:01.812 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.812 INFO:teuthology.orchestra.run.smithi186.stdout: "purge_queue": { 2024-05-09T04:59:01.812 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops": 0, 2024-05-09T04:59:01.812 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T04:59:01.812 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing": 0, 2024-05-09T04:59:01.812 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_high_water": 0, 2024-05-09T04:59:01.812 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed_ops": 0, 2024-05-09T04:59:01.812 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed": 0, 2024-05-09T04:59:01.812 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_item_in_journal": 0 2024-05-09T04:59:01.813 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.813 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T04:59:01.813 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-09T04:59:01.813 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-05-09T04:59:01.813 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-09T04:59:01.813 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 588, 2024-05-09T04:59:01.813 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 892104, 2024-05-09T04:59:01.813 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-09T04:59:01.813 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 588, 2024-05-09T04:59:01.813 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-05-09T04:59:01.813 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-05-09T04:59:01.813 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 588, 2024-05-09T04:59:01.813 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 892104, 2024-05-09T04:59:01.813 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-09T04:59:01.813 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:01.814 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:01.814 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:01.814 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:59:01.814 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.814 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_bytes": { 2024-05-09T04:59:01.814 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-09T04:59:01.814 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-05-09T04:59:01.814 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-09T04:59:01.814 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-05-09T04:59:01.814 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-05-09T04:59:01.814 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-09T04:59:01.814 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-05-09T04:59:01.814 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 42, 2024-05-09T04:59:01.814 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 11957, 2024-05-09T04:59:01.814 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 42, 2024-05-09T04:59:01.814 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 11957, 2024-05-09T04:59:01.815 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-09T04:59:01.815 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:01.815 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:01.815 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:01.815 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:59:01.815 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.815 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_ops": { 2024-05-09T04:59:01.815 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-09T04:59:01.815 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 1024, 2024-05-09T04:59:01.815 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-09T04:59:01.815 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-05-09T04:59:01.815 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-05-09T04:59:01.815 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-09T04:59:01.815 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-05-09T04:59:01.815 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 42, 2024-05-09T04:59:01.815 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 42, 2024-05-09T04:59:01.816 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 42, 2024-05-09T04:59:01.816 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 42, 2024-05-09T04:59:01.816 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-09T04:59:01.816 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:01.816 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:01.816 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:01.816 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:59:01.816 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.816 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle": { 2024-05-09T04:59:01.816 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-09T04:59:01.816 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-05-09T04:59:01.816 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-09T04:59:01.816 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-05-09T04:59:01.816 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-05-09T04:59:01.816 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-09T04:59:01.816 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-05-09T04:59:01.817 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-05-09T04:59:01.817 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-05-09T04:59:01.817 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2024-05-09T04:59:01.817 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2024-05-09T04:59:01.817 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-09T04:59:01.817 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:01.817 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:01.817 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:01.817 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:59:01.817 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:01.817 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle-0x5600058e0410": { 2024-05-09T04:59:01.817 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-09T04:59:01.817 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-05-09T04:59:01.817 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-09T04:59:01.817 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 5, 2024-05-09T04:59:01.818 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 1430, 2024-05-09T04:59:01.818 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-09T04:59:01.818 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 5, 2024-05-09T04:59:01.818 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-05-09T04:59:01.818 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-05-09T04:59:01.818 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 4, 2024-05-09T04:59:01.818 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 1430, 2024-05-09T04:59:01.818 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-09T04:59:01.818 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:01.818 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:01.818 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:01.818 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:59:01.818 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:59:01.818 INFO:teuthology.orchestra.run.smithi186.stdout:} 2024-05-09T04:59:02.656 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-05-09T04:59:02.657 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-05-09T04:59:02.657 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-05-09T04:59:02.657 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=14 2024-05-09T04:59:02.657 DEBUG:tasks.check_counter:Getting stats from l 2024-05-09T04:59:02.657 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-09T04:59:02.897 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T04:59:03.315 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:03 smithi186 ceph-mon[34668]: pgmap v622: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-05-09T04:59:03.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:03 smithi169 ceph-mon[34075]: pgmap v622: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-05-09T04:59:03.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:03 smithi006 ceph-mon[29835]: pgmap v622: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-05-09T04:59:04.623 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2024-05-09T04:59:04.623 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T04:59:04.624 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 662, 2024-05-09T04:59:04.624 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 655, 2024-05-09T04:59:04.624 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 991360, 2024-05-09T04:59:04.624 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 257840, 2024-05-09T04:59:04.624 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 5, 2024-05-09T04:59:04.624 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 1, 2024-05-09T04:59:04.624 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.096705078, 2024-05-09T04:59:04.624 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.029691653, 2024-05-09T04:59:04.624 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.039651059, 2024-05-09T04:59:04.624 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.025183245, 2024-05-09T04:59:04.624 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:59:04.624 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 654, 2024-05-09T04:59:04.624 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.040614646, 2024-05-09T04:59:04.625 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000062101 2024-05-09T04:59:04.625 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.625 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:59:04.625 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.625 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.625 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.625 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.625 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 830080, 2024-05-09T04:59:04.625 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 105200 2024-05-09T04:59:04.625 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.625 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T04:59:04.625 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 487, 2024-05-09T04:59:04.625 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 615, 2024-05-09T04:59:04.626 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 130104, 2024-05-09T04:59:04.626 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 500501, 2024-05-09T04:59:04.626 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 6, 2024-05-09T04:59:04.626 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 4, 2024-05-09T04:59:04.626 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.063750502, 2024-05-09T04:59:04.626 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.019929853, 2024-05-09T04:59:04.626 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.018391297, 2024-05-09T04:59:04.626 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.019392137, 2024-05-09T04:59:04.626 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:59:04.626 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 614, 2024-05-09T04:59:04.626 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.031378862, 2024-05-09T04:59:04.626 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000051105 2024-05-09T04:59:04.627 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.627 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:59:04.627 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.627 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.627 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.627 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.627 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-05-09T04:59:04.627 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 347168 2024-05-09T04:59:04.627 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.627 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T04:59:04.627 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 4, 2024-05-09T04:59:04.627 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 6, 2024-05-09T04:59:04.627 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 1629, 2024-05-09T04:59:04.627 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 1284, 2024-05-09T04:59:04.628 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 5, 2024-05-09T04:59:04.628 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 4, 2024-05-09T04:59:04.628 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.001653724, 2024-05-09T04:59:04.628 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.000100179, 2024-05-09T04:59:04.628 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.001325403, 2024-05-09T04:59:04.628 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000109714, 2024-05-09T04:59:04.628 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T04:59:04.628 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 4, 2024-05-09T04:59:04.628 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.003834420, 2024-05-09T04:59:04.628 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000958605 2024-05-09T04:59:04.628 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.628 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-05-09T04:59:04.628 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.628 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.628 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.629 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.629 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T04:59:04.629 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T04:59:04.629 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.629 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2024-05-09T04:59:04.629 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 1, 2024-05-09T04:59:04.629 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2024-05-09T04:59:04.629 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.629 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-MDSRank": { 2024-05-09T04:59:04.629 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-05-09T04:59:04.629 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-05-09T04:59:04.629 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1925, 2024-05-09T04:59:04.629 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.070217745, 2024-05-09T04:59:04.629 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000036476 2024-05-09T04:59:04.629 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:59:04.630 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.630 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-PurgeQueue": { 2024-05-09T04:59:04.630 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-05-09T04:59:04.630 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-05-09T04:59:04.630 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.630 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.630 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.630 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:59:04.630 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.630 INFO:teuthology.orchestra.run.smithi186.stdout: "mds": { 2024-05-09T04:59:04.630 INFO:teuthology.orchestra.run.smithi186.stdout: "request": 0, 2024-05-09T04:59:04.630 INFO:teuthology.orchestra.run.smithi186.stdout: "reply": 0, 2024-05-09T04:59:04.630 INFO:teuthology.orchestra.run.smithi186.stdout: "reply_latency": { 2024-05-09T04:59:04.630 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.631 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.631 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.631 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.631 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_reply": 0, 2024-05-09T04:59:04.631 INFO:teuthology.orchestra.run.smithi186.stdout: "forward": 0, 2024-05-09T04:59:04.631 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_complete": 0, 2024-05-09T04:59:04.631 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_keys": 0, 2024-05-09T04:59:04.631 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_commit": 0, 2024-05-09T04:59:04.631 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_split": 0, 2024-05-09T04:59:04.631 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_merge": 0, 2024-05-09T04:59:04.631 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes": 0, 2024-05-09T04:59:04.631 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_top": 0, 2024-05-09T04:59:04.631 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_bottom": 0, 2024-05-09T04:59:04.632 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pin_tail": 0, 2024-05-09T04:59:04.632 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pinned": 0, 2024-05-09T04:59:04.632 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_expired": 0, 2024-05-09T04:59:04.632 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_with_caps": 0, 2024-05-09T04:59:04.632 INFO:teuthology.orchestra.run.smithi186.stdout: "caps": 0, 2024-05-09T04:59:04.632 INFO:teuthology.orchestra.run.smithi186.stdout: "subtrees": 1, 2024-05-09T04:59:04.632 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse": 0, 2024-05-09T04:59:04.632 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_hit": 0, 2024-05-09T04:59:04.632 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_forward": 0, 2024-05-09T04:59:04.632 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_discover": 0, 2024-05-09T04:59:04.632 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_dir_fetch": 0, 2024-05-09T04:59:04.632 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_remote_ino": 0, 2024-05-09T04:59:04.632 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_lock": 0, 2024-05-09T04:59:04.632 INFO:teuthology.orchestra.run.smithi186.stdout: "load_cent": 0, 2024-05-09T04:59:04.632 INFO:teuthology.orchestra.run.smithi186.stdout: "q": 0, 2024-05-09T04:59:04.632 INFO:teuthology.orchestra.run.smithi186.stdout: "exported": 0, 2024-05-09T04:59:04.633 INFO:teuthology.orchestra.run.smithi186.stdout: "exported_inodes": 0, 2024-05-09T04:59:04.633 INFO:teuthology.orchestra.run.smithi186.stdout: "imported": 0, 2024-05-09T04:59:04.633 INFO:teuthology.orchestra.run.smithi186.stdout: "imported_inodes": 0, 2024-05-09T04:59:04.633 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_dir_fetch": 0, 2024-05-09T04:59:04.633 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_backtrace_fetch": 0, 2024-05-09T04:59:04.633 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_peer_discover": 0, 2024-05-09T04:59:04.633 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rfiles": 0, 2024-05-09T04:59:04.633 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rbytes": 0, 2024-05-09T04:59:04.633 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rsnaps": 0, 2024-05-09T04:59:04.633 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T04:59:04.633 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_set_tag": 0, 2024-05-09T04:59:04.633 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T04:59:04.633 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_inotable_repaired": 0, 2024-05-09T04:59:04.633 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_inodes": 0, 2024-05-09T04:59:04.633 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T04:59:04.633 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T04:59:04.634 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_file_inodes": 0, 2024-05-09T04:59:04.634 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_inode_file_caps": 0, 2024-05-09T04:59:04.634 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T04:59:04.634 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_grant": 0, 2024-05-09T04:59:04.634 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T04:59:04.634 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T04:59:04.634 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T04:59:04.634 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps": 0, 2024-05-09T04:59:04.634 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps_dirty": 0, 2024-05-09T04:59:04.634 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_cap_release": 0, 2024-05-09T04:59:04.634 INFO:teuthology.orchestra.run.smithi186.stdout: "process_request_cap_release": 0 2024-05-09T04:59:04.634 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.634 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_cache": { 2024-05-09T04:59:04.634 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update": 0, 2024-05-09T04:59:04.635 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update_receipt": 0, 2024-05-09T04:59:04.635 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_try_discover": 0, 2024-05-09T04:59:04.635 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_send_discover": 0, 2024-05-09T04:59:04.635 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_handle_discover": 0, 2024-05-09T04:59:04.635 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays": 0, 2024-05-09T04:59:04.635 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_delayed": 0, 2024-05-09T04:59:04.635 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_enqueuing": 0, 2024-05-09T04:59:04.635 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_created": 0, 2024-05-09T04:59:04.635 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_enqueued": 0, 2024-05-09T04:59:04.635 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_reintegrated": 0, 2024-05-09T04:59:04.635 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_migrated": 0, 2024-05-09T04:59:04.635 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_processing": 0, 2024-05-09T04:59:04.635 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_enqueued": 0, 2024-05-09T04:59:04.635 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_prioritized": 0, 2024-05-09T04:59:04.635 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_started": 0, 2024-05-09T04:59:04.636 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_completed": 0, 2024-05-09T04:59:04.636 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_quiesce_path": 0, 2024-05-09T04:59:04.636 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_quiesce_inode": 0, 2024-05-09T04:59:04.636 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T04:59:04.636 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_exportdir": 0, 2024-05-09T04:59:04.636 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_flush": 0, 2024-05-09T04:59:04.636 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragmentdir": 0, 2024-05-09T04:59:04.636 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragstats": 0, 2024-05-09T04:59:04.636 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_inodestats": 0 2024-05-09T04:59:04.636 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.636 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_log": { 2024-05-09T04:59:04.636 INFO:teuthology.orchestra.run.smithi186.stdout: "evlrg": 0, 2024-05-09T04:59:04.636 INFO:teuthology.orchestra.run.smithi186.stdout: "evadd": 0, 2024-05-09T04:59:04.636 INFO:teuthology.orchestra.run.smithi186.stdout: "evex": 0, 2024-05-09T04:59:04.636 INFO:teuthology.orchestra.run.smithi186.stdout: "evtrm": 0, 2024-05-09T04:59:04.636 INFO:teuthology.orchestra.run.smithi186.stdout: "ev": 2, 2024-05-09T04:59:04.637 INFO:teuthology.orchestra.run.smithi186.stdout: "evexg": 0, 2024-05-09T04:59:04.637 INFO:teuthology.orchestra.run.smithi186.stdout: "evexd": 0, 2024-05-09T04:59:04.637 INFO:teuthology.orchestra.run.smithi186.stdout: "segadd": 0, 2024-05-09T04:59:04.637 INFO:teuthology.orchestra.run.smithi186.stdout: "segex": 0, 2024-05-09T04:59:04.637 INFO:teuthology.orchestra.run.smithi186.stdout: "segtrm": 0, 2024-05-09T04:59:04.637 INFO:teuthology.orchestra.run.smithi186.stdout: "seg": 2, 2024-05-09T04:59:04.637 INFO:teuthology.orchestra.run.smithi186.stdout: "segmjr": 2, 2024-05-09T04:59:04.637 INFO:teuthology.orchestra.run.smithi186.stdout: "segexg": 0, 2024-05-09T04:59:04.637 INFO:teuthology.orchestra.run.smithi186.stdout: "segexd": 0, 2024-05-09T04:59:04.637 INFO:teuthology.orchestra.run.smithi186.stdout: "expos": 4194304, 2024-05-09T04:59:04.637 INFO:teuthology.orchestra.run.smithi186.stdout: "wrpos": 4194892, 2024-05-09T04:59:04.637 INFO:teuthology.orchestra.run.smithi186.stdout: "rdpos": 4194352, 2024-05-09T04:59:04.637 INFO:teuthology.orchestra.run.smithi186.stdout: "jlat": { 2024-05-09T04:59:04.637 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.638 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.638 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.638 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.638 INFO:teuthology.orchestra.run.smithi186.stdout: "replayed": 2 2024-05-09T04:59:04.638 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.638 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_mem": { 2024-05-09T04:59:04.638 INFO:teuthology.orchestra.run.smithi186.stdout: "ino": 3, 2024-05-09T04:59:04.638 INFO:teuthology.orchestra.run.smithi186.stdout: "ino+": 3, 2024-05-09T04:59:04.638 INFO:teuthology.orchestra.run.smithi186.stdout: "ino-": 0, 2024-05-09T04:59:04.638 INFO:teuthology.orchestra.run.smithi186.stdout: "dir": 1, 2024-05-09T04:59:04.638 INFO:teuthology.orchestra.run.smithi186.stdout: "dir+": 1, 2024-05-09T04:59:04.638 INFO:teuthology.orchestra.run.smithi186.stdout: "dir-": 0, 2024-05-09T04:59:04.638 INFO:teuthology.orchestra.run.smithi186.stdout: "dn": 0, 2024-05-09T04:59:04.639 INFO:teuthology.orchestra.run.smithi186.stdout: "dn+": 0, 2024-05-09T04:59:04.639 INFO:teuthology.orchestra.run.smithi186.stdout: "dn-": 0, 2024-05-09T04:59:04.639 INFO:teuthology.orchestra.run.smithi186.stdout: "cap": 0, 2024-05-09T04:59:04.639 INFO:teuthology.orchestra.run.smithi186.stdout: "cap+": 0, 2024-05-09T04:59:04.639 INFO:teuthology.orchestra.run.smithi186.stdout: "cap-": 0, 2024-05-09T04:59:04.639 INFO:teuthology.orchestra.run.smithi186.stdout: "rss": 121432, 2024-05-09T04:59:04.639 INFO:teuthology.orchestra.run.smithi186.stdout: "heap": 215440 2024-05-09T04:59:04.639 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.639 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_server": { 2024-05-09T04:59:04.639 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_client_request": 0, 2024-05-09T04:59:04.639 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_server_request": 0, 2024-05-09T04:59:04.639 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_request": 0, 2024-05-09T04:59:04.639 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_session": 0, 2024-05-09T04:59:04.639 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_peer_request": 0, 2024-05-09T04:59:04.639 INFO:teuthology.orchestra.run.smithi186.stdout: "req_create_latency": { 2024-05-09T04:59:04.640 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.640 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.640 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.640 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.640 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getattr_latency": { 2024-05-09T04:59:04.640 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.640 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.640 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.640 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.640 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getfilelock_latency": { 2024-05-09T04:59:04.640 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.640 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.640 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.640 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.640 INFO:teuthology.orchestra.run.smithi186.stdout: "req_link_latency": { 2024-05-09T04:59:04.641 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.641 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.641 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.641 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.641 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookup_latency": { 2024-05-09T04:59:04.641 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.641 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.641 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.641 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.641 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookuphash_latency": { 2024-05-09T04:59:04.641 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.641 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.641 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.641 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.641 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupino_latency": { 2024-05-09T04:59:04.642 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.642 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.642 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.642 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.642 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupname_latency": { 2024-05-09T04:59:04.642 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.642 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.642 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.642 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.642 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupparent_latency": { 2024-05-09T04:59:04.642 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.642 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.642 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.642 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.643 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupsnap_latency": { 2024-05-09T04:59:04.643 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.643 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.643 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.643 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.643 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lssnap_latency": { 2024-05-09T04:59:04.643 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.643 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.643 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.643 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.643 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mkdir_latency": { 2024-05-09T04:59:04.643 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.643 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.644 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.644 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.644 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mknod_latency": { 2024-05-09T04:59:04.644 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.644 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.644 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.644 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.644 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mksnap_latency": { 2024-05-09T04:59:04.645 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.645 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.645 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.645 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.645 INFO:teuthology.orchestra.run.smithi186.stdout: "req_open_latency": { 2024-05-09T04:59:04.645 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.645 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.645 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.645 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.645 INFO:teuthology.orchestra.run.smithi186.stdout: "req_readdir_latency": { 2024-05-09T04:59:04.645 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.645 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.645 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.645 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.645 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rename_latency": { 2024-05-09T04:59:04.646 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.646 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.646 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.646 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.646 INFO:teuthology.orchestra.run.smithi186.stdout: "req_renamesnap_latency": { 2024-05-09T04:59:04.646 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.646 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.646 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.646 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.646 INFO:teuthology.orchestra.run.smithi186.stdout: "req_snapdiff_latency": { 2024-05-09T04:59:04.646 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.646 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.646 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.646 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.647 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmdir_latency": { 2024-05-09T04:59:04.647 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.647 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.647 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.647 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.647 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmsnap_latency": { 2024-05-09T04:59:04.647 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.647 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.647 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.647 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.647 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmxattr_latency": { 2024-05-09T04:59:04.647 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.647 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.647 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.647 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.648 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setattr_latency": { 2024-05-09T04:59:04.648 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.648 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.648 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.648 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.648 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setdirlayout_latency": { 2024-05-09T04:59:04.648 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.648 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.648 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.648 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.648 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setfilelock_latency": { 2024-05-09T04:59:04.648 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.648 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.648 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.648 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.648 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setlayout_latency": { 2024-05-09T04:59:04.649 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.649 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.649 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.649 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.649 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setxattr_latency": { 2024-05-09T04:59:04.649 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.649 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.649 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.649 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.649 INFO:teuthology.orchestra.run.smithi186.stdout: "req_symlink_latency": { 2024-05-09T04:59:04.649 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.649 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.649 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.649 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.649 INFO:teuthology.orchestra.run.smithi186.stdout: "req_unlink_latency": { 2024-05-09T04:59:04.650 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.650 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.650 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.650 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.650 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_revoke_eviction": 0, 2024-05-09T04:59:04.650 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_acquisition_throttle": 0, 2024-05-09T04:59:04.650 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getvxattr_latency": { 2024-05-09T04:59:04.650 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.650 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.650 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.650 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:59:04.650 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.650 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_sessions": { 2024-05-09T04:59:04.651 INFO:teuthology.orchestra.run.smithi186.stdout: "session_count": 0, 2024-05-09T04:59:04.651 INFO:teuthology.orchestra.run.smithi186.stdout: "session_add": 0, 2024-05-09T04:59:04.651 INFO:teuthology.orchestra.run.smithi186.stdout: "session_remove": 0, 2024-05-09T04:59:04.651 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_open": 0, 2024-05-09T04:59:04.651 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_stale": 0, 2024-05-09T04:59:04.651 INFO:teuthology.orchestra.run.smithi186.stdout: "total_load": 0, 2024-05-09T04:59:04.651 INFO:teuthology.orchestra.run.smithi186.stdout: "average_load": 0, 2024-05-09T04:59:04.651 INFO:teuthology.orchestra.run.smithi186.stdout: "avg_session_uptime": 0, 2024-05-09T04:59:04.651 INFO:teuthology.orchestra.run.smithi186.stdout: "mdthresh_evicted": 0 2024-05-09T04:59:04.651 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.651 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2024-05-09T04:59:04.651 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2024-05-09T04:59:04.651 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2024-05-09T04:59:04.652 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T04:59:04.652 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2024-05-09T04:59:04.652 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T04:59:04.652 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2024-05-09T04:59:04.652 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T04:59:04.652 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T04:59:04.652 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T04:59:04.652 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T04:59:04.652 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T04:59:04.652 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2024-05-09T04:59:04.652 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T04:59:04.652 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T04:59:04.652 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2024-05-09T04:59:04.652 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2024-05-09T04:59:04.653 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2024-05-09T04:59:04.653 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2024-05-09T04:59:04.653 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T04:59:04.653 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T04:59:04.653 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T04:59:04.653 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T04:59:04.653 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T04:59:04.653 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2024-05-09T04:59:04.653 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2024-05-09T04:59:04.653 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2024-05-09T04:59:04.653 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T04:59:04.653 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T04:59:04.653 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2024-05-09T04:59:04.654 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2024-05-09T04:59:04.654 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2024-05-09T04:59:04.654 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2024-05-09T04:59:04.654 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T04:59:04.654 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2024-05-09T04:59:04.654 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T04:59:04.654 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2024-05-09T04:59:04.654 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 197458, 2024-05-09T04:59:04.654 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 55, 2024-05-09T04:59:04.654 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 0, 2024-05-09T04:59:04.654 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 0, 2024-05-09T04:59:04.654 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2024-05-09T04:59:04.654 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2024-05-09T04:59:04.654 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2024-05-09T04:59:04.655 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2024-05-09T04:59:04.655 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2024-05-09T04:59:04.655 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2024-05-09T04:59:04.655 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 8800, 2024-05-09T04:59:04.655 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 169, 2024-05-09T04:59:04.655 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T04:59:04.655 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2024-05-09T04:59:04.655 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2024-05-09T04:59:04.655 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2024-05-09T04:59:04.655 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 8512, 2024-05-09T04:59:04.655 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 13, 2024-05-09T04:59:04.655 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2024-05-09T04:59:04.655 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2024-05-09T04:59:04.655 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2024-05-09T04:59:04.655 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2024-05-09T04:59:04.655 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.656 INFO:teuthology.orchestra.run.smithi186.stdout: "objecter": { 2024-05-09T04:59:04.656 INFO:teuthology.orchestra.run.smithi186.stdout: "op_active": 0, 2024-05-09T04:59:04.656 INFO:teuthology.orchestra.run.smithi186.stdout: "op_laggy": 0, 2024-05-09T04:59:04.656 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send": 966, 2024-05-09T04:59:04.656 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send_bytes": 21, 2024-05-09T04:59:04.656 INFO:teuthology.orchestra.run.smithi186.stdout: "op_resend": 3, 2024-05-09T04:59:04.656 INFO:teuthology.orchestra.run.smithi186.stdout: "op_reply": 966, 2024-05-09T04:59:04.656 INFO:teuthology.orchestra.run.smithi186.stdout: "op_latency": { 2024-05-09T04:59:04.656 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 966, 2024-05-09T04:59:04.656 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.842983784, 2024-05-09T04:59:04.656 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000872654 2024-05-09T04:59:04.656 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.656 INFO:teuthology.orchestra.run.smithi186.stdout: "op_inflight": 0, 2024-05-09T04:59:04.656 INFO:teuthology.orchestra.run.smithi186.stdout: "oplen_avg": { 2024-05-09T04:59:04.656 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 966, 2024-05-09T04:59:04.657 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 967 2024-05-09T04:59:04.657 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.657 INFO:teuthology.orchestra.run.smithi186.stdout: "op": 966, 2024-05-09T04:59:04.657 INFO:teuthology.orchestra.run.smithi186.stdout: "op_r": 966, 2024-05-09T04:59:04.657 INFO:teuthology.orchestra.run.smithi186.stdout: "op_w": 0, 2024-05-09T04:59:04.657 INFO:teuthology.orchestra.run.smithi186.stdout: "op_rmw": 0, 2024-05-09T04:59:04.657 INFO:teuthology.orchestra.run.smithi186.stdout: "op_pg": 0, 2024-05-09T04:59:04.657 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_stat": 480, 2024-05-09T04:59:04.657 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_create": 0, 2024-05-09T04:59:04.657 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_read": 484, 2024-05-09T04:59:04.657 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_write": 0, 2024-05-09T04:59:04.657 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writefull": 0, 2024-05-09T04:59:04.657 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writesame": 0, 2024-05-09T04:59:04.657 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_append": 0, 2024-05-09T04:59:04.658 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_zero": 0, 2024-05-09T04:59:04.658 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_truncate": 0, 2024-05-09T04:59:04.658 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_delete": 0, 2024-05-09T04:59:04.658 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_mapext": 0, 2024-05-09T04:59:04.658 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_sparse_read": 0, 2024-05-09T04:59:04.658 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_clonerange": 0, 2024-05-09T04:59:04.658 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_getxattr": 1, 2024-05-09T04:59:04.658 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_setxattr": 0, 2024-05-09T04:59:04.658 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_cmpxattr": 0, 2024-05-09T04:59:04.658 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_rmxattr": 0, 2024-05-09T04:59:04.658 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_resetxattrs": 0, 2024-05-09T04:59:04.658 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_call": 0, 2024-05-09T04:59:04.658 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_watch": 0, 2024-05-09T04:59:04.658 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_notify": 0, 2024-05-09T04:59:04.658 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T04:59:04.659 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls": 0, 2024-05-09T04:59:04.659 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls_filter": 0, 2024-05-09T04:59:04.659 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_other": 0, 2024-05-09T04:59:04.659 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_active": 0, 2024-05-09T04:59:04.659 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_send": 0, 2024-05-09T04:59:04.659 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_resend": 0, 2024-05-09T04:59:04.659 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_ping": 0, 2024-05-09T04:59:04.659 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_active": 0, 2024-05-09T04:59:04.659 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_send": 0, 2024-05-09T04:59:04.659 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_resend": 0, 2024-05-09T04:59:04.659 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_active": 0, 2024-05-09T04:59:04.659 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_send": 0, 2024-05-09T04:59:04.659 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_resend": 0, 2024-05-09T04:59:04.659 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_active": 0, 2024-05-09T04:59:04.659 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_send": 0, 2024-05-09T04:59:04.659 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_resend": 0, 2024-05-09T04:59:04.660 INFO:teuthology.orchestra.run.smithi186.stdout: "command_active": 0, 2024-05-09T04:59:04.660 INFO:teuthology.orchestra.run.smithi186.stdout: "command_send": 0, 2024-05-09T04:59:04.660 INFO:teuthology.orchestra.run.smithi186.stdout: "command_resend": 0, 2024-05-09T04:59:04.660 INFO:teuthology.orchestra.run.smithi186.stdout: "map_epoch": 0, 2024-05-09T04:59:04.660 INFO:teuthology.orchestra.run.smithi186.stdout: "map_full": 0, 2024-05-09T04:59:04.660 INFO:teuthology.orchestra.run.smithi186.stdout: "map_inc": 0, 2024-05-09T04:59:04.660 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_sessions": 6, 2024-05-09T04:59:04.660 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_open": 6, 2024-05-09T04:59:04.660 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_close": 0, 2024-05-09T04:59:04.660 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_laggy": 0, 2024-05-09T04:59:04.660 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_wr": 0, 2024-05-09T04:59:04.660 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_rd": 2, 2024-05-09T04:59:04.660 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_del": 0 2024-05-09T04:59:04.661 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.661 INFO:teuthology.orchestra.run.smithi186.stdout: "oft": { 2024-05-09T04:59:04.661 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_objs": 0, 2024-05-09T04:59:04.661 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_kv_pairs": 0, 2024-05-09T04:59:04.661 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_updates": 0, 2024-05-09T04:59:04.661 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_removes": 0 2024-05-09T04:59:04.661 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.661 INFO:teuthology.orchestra.run.smithi186.stdout: "osd-slow-ops": { 2024-05-09T04:59:04.661 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_ops_count": 0 2024-05-09T04:59:04.661 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.661 INFO:teuthology.orchestra.run.smithi186.stdout: "purge_queue": { 2024-05-09T04:59:04.661 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops": 0, 2024-05-09T04:59:04.661 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T04:59:04.662 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing": 0, 2024-05-09T04:59:04.662 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_high_water": 0, 2024-05-09T04:59:04.662 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed_ops": 0, 2024-05-09T04:59:04.662 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed": 0, 2024-05-09T04:59:04.662 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_item_in_journal": 0 2024-05-09T04:59:04.662 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.662 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T04:59:04.662 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-09T04:59:04.662 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-05-09T04:59:04.662 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-09T04:59:04.662 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 1149, 2024-05-09T04:59:04.662 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 1007290, 2024-05-09T04:59:04.662 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-09T04:59:04.662 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 1149, 2024-05-09T04:59:04.663 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-05-09T04:59:04.663 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-05-09T04:59:04.663 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 1149, 2024-05-09T04:59:04.663 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 1007290, 2024-05-09T04:59:04.663 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-09T04:59:04.663 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.663 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.663 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.663 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:59:04.663 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.663 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_bytes": { 2024-05-09T04:59:04.663 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-09T04:59:04.663 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-05-09T04:59:04.663 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-09T04:59:04.664 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-05-09T04:59:04.664 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-05-09T04:59:04.664 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-09T04:59:04.664 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-05-09T04:59:04.664 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 966, 2024-05-09T04:59:04.664 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 593, 2024-05-09T04:59:04.664 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 966, 2024-05-09T04:59:04.664 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 593, 2024-05-09T04:59:04.664 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-09T04:59:04.664 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.664 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.664 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.664 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:59:04.664 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.665 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_ops": { 2024-05-09T04:59:04.665 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-09T04:59:04.665 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 1024, 2024-05-09T04:59:04.665 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-09T04:59:04.665 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-05-09T04:59:04.665 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-05-09T04:59:04.665 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-09T04:59:04.665 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-05-09T04:59:04.665 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 966, 2024-05-09T04:59:04.665 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 966, 2024-05-09T04:59:04.665 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 966, 2024-05-09T04:59:04.665 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 966, 2024-05-09T04:59:04.665 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-09T04:59:04.665 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.665 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.666 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.666 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:59:04.666 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.666 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle": { 2024-05-09T04:59:04.666 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-09T04:59:04.666 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-05-09T04:59:04.666 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-09T04:59:04.666 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-05-09T04:59:04.666 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-05-09T04:59:04.666 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-09T04:59:04.666 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-05-09T04:59:04.666 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-05-09T04:59:04.666 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-05-09T04:59:04.666 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2024-05-09T04:59:04.667 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2024-05-09T04:59:04.667 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-09T04:59:04.667 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.667 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.667 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.667 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:59:04.667 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-05-09T04:59:04.667 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle-0x556ada3e28f0": { 2024-05-09T04:59:04.667 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-05-09T04:59:04.667 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-05-09T04:59:04.667 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-05-09T04:59:04.667 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-05-09T04:59:04.667 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-05-09T04:59:04.667 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-05-09T04:59:04.667 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-05-09T04:59:04.668 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-05-09T04:59:04.668 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-05-09T04:59:04.668 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2024-05-09T04:59:04.668 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2024-05-09T04:59:04.668 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-05-09T04:59:04.668 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-05-09T04:59:04.668 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-05-09T04:59:04.668 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-05-09T04:59:04.668 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:59:04.668 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-05-09T04:59:04.668 INFO:teuthology.orchestra.run.smithi186.stdout:} 2024-05-09T04:59:05.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:05 smithi186 ceph-mon[34668]: pgmap v623: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:59:05.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:05 smithi169 ceph-mon[34075]: pgmap v623: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:59:05.395 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-05-09T04:59:05.395 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-05-09T04:59:05.395 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-05-09T04:59:05.395 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-05-09T04:59:05.395 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-09T04:59:05.405 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-09T04:59:05.405 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T04:59:05.405 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:59:05.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:05 smithi006 ceph-mon[29835]: pgmap v623: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:59:05.471 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-09T04:59:05.471 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:59:05.572 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-09T04:59:06.018 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:59:07.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:07 smithi186 ceph-mon[34668]: pgmap v624: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:59:07.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:07 smithi169 ceph-mon[34075]: pgmap v624: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:59:07.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:07 smithi006 ceph-mon[29835]: pgmap v624: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T04:59:08.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.417+0000 7f9c93200700 1 -- 172.21.15.6:0/155022209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c94105290 msgr2=0x7f9c941056a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:08.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.417+0000 7f9c93200700 1 --2- 172.21.15.6:0/155022209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c94105290 0x7f9c941056a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9c84009990 tx=0x7f9c84009ca0 comp rx=0 tx=0).stop 2024-05-09T04:59:08.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.417+0000 7f9c93200700 1 -- 172.21.15.6:0/155022209 shutdown_connections 2024-05-09T04:59:08.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.417+0000 7f9c93200700 1 --2- 172.21.15.6:0/155022209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c9410e700 0x7f9c94110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:08.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.417+0000 7f9c93200700 1 --2- 172.21.15.6:0/155022209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c94105290 0x7f9c941056a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:08.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.417+0000 7f9c93200700 1 --2- 172.21.15.6:0/155022209 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c94105be0 0x7f9c9410e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:08.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.417+0000 7f9c93200700 1 -- 172.21.15.6:0/155022209 >> 172.21.15.6:0/155022209 conn(0x7f9c94078ac0 msgr2=0x7f9c94078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:08.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.417+0000 7f9c93200700 1 -- 172.21.15.6:0/155022209 shutdown_connections 2024-05-09T04:59:08.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.417+0000 7f9c93200700 1 -- 172.21.15.6:0/155022209 wait complete. 2024-05-09T04:59:08.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.418+0000 7f9c93200700 1 Processor -- start 2024-05-09T04:59:08.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.419+0000 7f9c93200700 1 -- start start 2024-05-09T04:59:08.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.419+0000 7f9c93200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c94105290 0x7f9c9419ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:08.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.419+0000 7f9c93200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c94105be0 0x7f9c9419d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:08.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.419+0000 7f9c93200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c9410e700 0x7f9c941a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:08.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.419+0000 7f9c93200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9c941142a0 con 0x7f9c94105be0 2024-05-09T04:59:08.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.419+0000 7f9c93200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9c94114120 con 0x7f9c94105290 2024-05-09T04:59:08.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.419+0000 7f9c93200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9c94114420 con 0x7f9c9410e700 2024-05-09T04:59:08.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.420+0000 7f9c91400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c94105be0 0x7f9c9419d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:08.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.420+0000 7f9c91e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c94105290 0x7f9c9419ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:08.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.420+0000 7f9c91400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c94105be0 0x7f9c9419d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37348/0 (socket says 172.21.15.6:37348) 2024-05-09T04:59:08.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.420+0000 7f9c91400700 1 -- 172.21.15.6:0/812571054 learned_addr learned my addr 172.21.15.6:0/812571054 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:59:08.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.420+0000 7f9c91e00700 1 -- 172.21.15.6:0/812571054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c9410e700 msgr2=0x7f9c941a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:08.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.420+0000 7f9c92800700 1 --2- 172.21.15.6:0/812571054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c9410e700 0x7f9c941a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:08.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.420+0000 7f9c91e00700 1 --2- 172.21.15.6:0/812571054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c9410e700 0x7f9c941a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:08.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.420+0000 7f9c91e00700 1 -- 172.21.15.6:0/812571054 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c94105be0 msgr2=0x7f9c9419d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:08.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.420+0000 7f9c91e00700 1 --2- 172.21.15.6:0/812571054 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c94105be0 0x7f9c9419d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:08.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.420+0000 7f9c91e00700 1 -- 172.21.15.6:0/812571054 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c941a1da0 con 0x7f9c94105290 2024-05-09T04:59:08.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.420+0000 7f9c91400700 1 --2- 172.21.15.6:0/812571054 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c94105be0 0x7f9c9419d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:59:08.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.420+0000 7f9c91e00700 1 --2- 172.21.15.6:0/812571054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c94105290 0x7f9c9419ced0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9c84000c00 tx=0x7f9c840047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:08.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.421+0000 7f9c8aa00700 1 -- 172.21.15.6:0/812571054 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8400fb10 con 0x7f9c94105290 2024-05-09T04:59:08.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.421+0000 7f9c93200700 1 -- 172.21.15.6:0/812571054 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c941a2030 con 0x7f9c94105290 2024-05-09T04:59:08.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.421+0000 7f9c8aa00700 1 -- 172.21.15.6:0/812571054 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c84004ce0 con 0x7f9c94105290 2024-05-09T04:59:08.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.421+0000 7f9c8aa00700 1 -- 172.21.15.6:0/812571054 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c84005430 con 0x7f9c94105290 2024-05-09T04:59:08.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.421+0000 7f9c93200700 1 -- 172.21.15.6:0/812571054 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c941a2500 con 0x7f9c94105290 2024-05-09T04:59:08.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.422+0000 7f9c93200700 1 -- 172.21.15.6:0/812571054 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c940512c0 con 0x7f9c94105290 2024-05-09T04:59:08.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.425+0000 7f9c8aa00700 1 -- 172.21.15.6:0/812571054 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f9c84009d10 con 0x7f9c94105290 2024-05-09T04:59:08.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.426+0000 7f9c8aa00700 1 --2- 172.21.15.6:0/812571054 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9c780b2880 0x7f9c780b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:08.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.426+0000 7f9c8aa00700 1 -- 172.21.15.6:0/812571054 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f9c840e31d0 con 0x7f9c94105290 2024-05-09T04:59:08.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.427+0000 7f9c91400700 1 --2- 172.21.15.6:0/812571054 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9c780b2880 0x7f9c780b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:08.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.427+0000 7f9c91400700 1 --2- 172.21.15.6:0/812571054 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9c780b2880 0x7f9c780b4d90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9c94103620 tx=0x7f9c80009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:08.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.431+0000 7f9c8aa00700 1 -- 172.21.15.6:0/812571054 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9c84013070 con 0x7f9c94105290 2024-05-09T04:59:08.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.714+0000 7f9c93200700 1 -- 172.21.15.6:0/812571054 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9c941a9a70 con 0x7f9c94105290 2024-05-09T04:59:08.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.715+0000 7f9c8aa00700 1 -- 172.21.15.6:0/812571054 <== mon.2 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+579 (secure 0 0 0) 0x7f9c840aaba0 con 0x7f9c94105290 2024-05-09T04:59:08.716 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/3742411767 2024-05-10T04:40:55.357057+0000 2024-05-09T04:59:08.716 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/2410771702 2024-05-10T04:40:55.357057+0000 2024-05-09T04:59:08.716 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6801/937713898 2024-05-10T04:40:55.357057+0000 2024-05-09T04:59:08.716 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6800/937713898 2024-05-10T04:40:55.357057+0000 2024-05-09T04:59:08.716 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6801/3021503778 2024-05-10T04:40:19.639151+0000 2024-05-09T04:59:08.717 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6800/3021503778 2024-05-10T04:40:19.639151+0000 2024-05-09T04:59:08.717 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/2559510850 2024-05-10T04:40:19.639151+0000 2024-05-09T04:59:08.717 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/3810474743 2024-05-10T04:40:19.639151+0000 2024-05-09T04:59:08.717 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/4133970555 2024-05-10T04:40:55.357057+0000 2024-05-09T04:59:08.717 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/664817743 2024-05-10T04:40:19.639151+0000 2024-05-09T04:59:08.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.719+0000 7f9c93200700 1 -- 172.21.15.6:0/812571054 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9c780b2880 msgr2=0x7f9c780b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:08.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.719+0000 7f9c93200700 1 --2- 172.21.15.6:0/812571054 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9c780b2880 0x7f9c780b4d90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9c94103620 tx=0x7f9c80009260 comp rx=0 tx=0).stop 2024-05-09T04:59:08.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.719+0000 7f9c93200700 1 -- 172.21.15.6:0/812571054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c94105290 msgr2=0x7f9c9419ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:08.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.719+0000 7f9c93200700 1 --2- 172.21.15.6:0/812571054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c94105290 0x7f9c9419ced0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9c84000c00 tx=0x7f9c840047b0 comp rx=0 tx=0).stop 2024-05-09T04:59:08.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.719+0000 7f9c93200700 1 -- 172.21.15.6:0/812571054 shutdown_connections 2024-05-09T04:59:08.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.719+0000 7f9c93200700 1 --2- 172.21.15.6:0/812571054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c9410e700 0x7f9c941a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:08.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.719+0000 7f9c93200700 1 --2- 172.21.15.6:0/812571054 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f9c780b2880 0x7f9c780b4d90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:08.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.719+0000 7f9c93200700 1 --2- 172.21.15.6:0/812571054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c94105290 0x7f9c9419ced0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:08.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.719+0000 7f9c93200700 1 --2- 172.21.15.6:0/812571054 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c94105be0 0x7f9c9419d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:08.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.719+0000 7f9c93200700 1 -- 172.21.15.6:0/812571054 >> 172.21.15.6:0/812571054 conn(0x7f9c94078ac0 msgr2=0x7f9c94103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:08.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.720+0000 7f9c93200700 1 -- 172.21.15.6:0/812571054 shutdown_connections 2024-05-09T04:59:08.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:08.720+0000 7f9c93200700 1 -- 172.21.15.6:0/812571054 wait complete. 2024-05-09T04:59:08.722 INFO:teuthology.orchestra.run.smithi006.stderr:listed 10 entries 2024-05-09T04:59:09.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:09 smithi186 ceph-mon[34668]: pgmap v625: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:59:09.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:09 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/812571054' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T04:59:09.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:09 smithi169 ceph-mon[34075]: pgmap v625: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:59:09.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:09 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/812571054' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T04:59:09.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:09 smithi006 ceph-mon[29835]: pgmap v625: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T04:59:09.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:09 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/812571054' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T04:59:09.625 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-09T04:59:09.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:59:09.625 DEBUG:teuthology.orchestra.run.smithi006:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-09T04:59:09.758 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2024-05-09T04:59:09.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:59:09.758 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T04:59:09.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:59:09.805 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2024-05-09T04:59:09.884 INFO:teuthology.orchestra.run.smithi006.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T04:59:09.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:59:09.885 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T04:59:10.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:59:10.010 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link delete ceph-brx 2024-05-09T04:59:10.125 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-09T04:59:10.135 INFO:tasks.cephadm:Teardown begin 2024-05-09T04:59:10.135 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-09T04:59:10.135 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:59:10.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:10 smithi186 ceph-mon[34668]: pgmap v626: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:59:10.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:10 smithi169 ceph-mon[34075]: pgmap v626: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:59:10.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:10 smithi006 ceph-mon[29835]: pgmap v626: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:59:10.567 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:59:13.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.013+0000 7f1d4ae00700 1 -- 172.21.15.6:0/1839728178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d44107e10 msgr2=0x7f1d44108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:13.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.013+0000 7f1d4ae00700 1 --2- 172.21.15.6:0/1839728178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d44107e10 0x7f1d44108220 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1d300099c0 tx=0x7f1d30009cd0 comp rx=0 tx=0).stop 2024-05-09T04:59:13.015 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:12 smithi006 ceph-mon[29835]: pgmap v627: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T04:59:13.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.013+0000 7f1d4ae00700 1 -- 172.21.15.6:0/1839728178 shutdown_connections 2024-05-09T04:59:13.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.013+0000 7f1d4ae00700 1 --2- 172.21.15.6:0/1839728178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d44108760 0x7f1d441100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:13.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.013+0000 7f1d4ae00700 1 --2- 172.21.15.6:0/1839728178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d441034e0 0x7f1d441038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:13.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.013+0000 7f1d4ae00700 1 --2- 172.21.15.6:0/1839728178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d44107e10 0x7f1d44108220 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:13.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.013+0000 7f1d4ae00700 1 -- 172.21.15.6:0/1839728178 >> 172.21.15.6:0/1839728178 conn(0x7f1d440fef30 msgr2=0x7f1d44101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:13.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.013+0000 7f1d4ae00700 1 -- 172.21.15.6:0/1839728178 shutdown_connections 2024-05-09T04:59:13.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.013+0000 7f1d4ae00700 1 -- 172.21.15.6:0/1839728178 wait complete. 2024-05-09T04:59:13.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.015+0000 7f1d4ae00700 1 Processor -- start 2024-05-09T04:59:13.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.015+0000 7f1d4ae00700 1 -- start start 2024-05-09T04:59:13.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.016+0000 7f1d4ae00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d441034e0 0x7f1d440ff410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:13.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.016+0000 7f1d4ae00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d44107e10 0x7f1d440ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:13.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.016+0000 7f1d4ae00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d44108760 0x7f1d440ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:13.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.016+0000 7f1d4ae00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1d44113370 con 0x7f1d441034e0 2024-05-09T04:59:13.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.016+0000 7f1d4ae00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1d441131f0 con 0x7f1d44107e10 2024-05-09T04:59:13.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.016+0000 7f1d4ae00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1d441134f0 con 0x7f1d44108760 2024-05-09T04:59:13.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.016+0000 7f1d43e00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d441034e0 0x7f1d440ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:13.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.016+0000 7f1d43400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d44107e10 0x7f1d440ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:13.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.016+0000 7f1d43400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d44107e10 0x7f1d440ff950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:48088/0 (socket says 172.21.15.6:48088) 2024-05-09T04:59:13.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.016+0000 7f1d43400700 1 -- 172.21.15.6:0/906571977 learned_addr learned my addr 172.21.15.6:0/906571977 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:59:13.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.017+0000 7f1d48e00700 1 --2- 172.21.15.6:0/906571977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d44108760 0x7f1d440ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:13.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.017+0000 7f1d43400700 1 -- 172.21.15.6:0/906571977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d44108760 msgr2=0x7f1d440ffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:13.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.017+0000 7f1d43400700 1 --2- 172.21.15.6:0/906571977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d44108760 0x7f1d440ffe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:13.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.017+0000 7f1d43400700 1 -- 172.21.15.6:0/906571977 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d441034e0 msgr2=0x7f1d440ff410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:13.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.017+0000 7f1d43400700 1 --2- 172.21.15.6:0/906571977 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d441034e0 0x7f1d440ff410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:13.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.017+0000 7f1d43400700 1 -- 172.21.15.6:0/906571977 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d441a9280 con 0x7f1d44107e10 2024-05-09T04:59:13.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.017+0000 7f1d43400700 1 --2- 172.21.15.6:0/906571977 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d44107e10 0x7f1d440ff950 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1d30006010 tx=0x7f1d30005280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:13.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.018+0000 7f1d40c00700 1 -- 172.21.15.6:0/906571977 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d30026070 con 0x7f1d44107e10 2024-05-09T04:59:13.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.018+0000 7f1d40c00700 1 -- 172.21.15.6:0/906571977 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d30003c70 con 0x7f1d44107e10 2024-05-09T04:59:13.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.018+0000 7f1d4ae00700 1 -- 172.21.15.6:0/906571977 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d441a9510 con 0x7f1d44107e10 2024-05-09T04:59:13.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.018+0000 7f1d40c00700 1 -- 172.21.15.6:0/906571977 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3000fb40 con 0x7f1d44107e10 2024-05-09T04:59:13.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.018+0000 7f1d4ae00700 1 -- 172.21.15.6:0/906571977 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d441a9960 con 0x7f1d44107e10 2024-05-09T04:59:13.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.020+0000 7f1d4ae00700 1 -- 172.21.15.6:0/906571977 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d440512c0 con 0x7f1d44107e10 2024-05-09T04:59:13.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.023+0000 7f1d40c00700 1 -- 172.21.15.6:0/906571977 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f1d30029070 con 0x7f1d44107e10 2024-05-09T04:59:13.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.024+0000 7f1d40c00700 1 --2- 172.21.15.6:0/906571977 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f1d280b2880 0x7f1d280b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:13.025 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.025+0000 7f1d40c00700 1 -- 172.21.15.6:0/906571977 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f1d300e3350 con 0x7f1d44107e10 2024-05-09T04:59:13.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.025+0000 7f1d43e00700 1 --2- 172.21.15.6:0/906571977 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f1d280b2880 0x7f1d280b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:13.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.025+0000 7f1d43e00700 1 --2- 172.21.15.6:0/906571977 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f1d280b2880 0x7f1d280b4d90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009c60 tx=0x7f1d34009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:13.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.026+0000 7f1d40c00700 1 -- 172.21.15.6:0/906571977 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1d30013070 con 0x7f1d44107e10 2024-05-09T04:59:13.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.316+0000 7f1d4ae00700 1 -- 172.21.15.6:0/906571977 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d4402e5f0 con 0x7f1d44107e10 2024-05-09T04:59:13.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.317+0000 7f1d40c00700 1 -- 172.21.15.6:0/906571977 <== mon.2 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11927 (secure 0 0 0) 0x7f1d300aad20 con 0x7f1d44107e10 2024-05-09T04:59:13.318 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:59:13.319 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/162094599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":162094599},{"type":"v1","addr":"172.21.15.6:6835","nonce":162094599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/589294363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":589294363},{"type":"v1","addr":"172.21.15.6:6841","nonce":589294363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:49:36.920376+0000","modified":"2024-05-09T04:54:56.247431+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24554,"mds_1":24530,"mds_2":24512,"mds_3":24511,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24442":{"gid":24442,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6833/2977145206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2977145206},{"type":"v1","addr":"172.21.15.186:6833","nonce":2977145206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6833/133345948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":133345948},{"type":"v1","addr":"172.21.15.169:6833","nonce":133345948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.6:6837/1281521088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":1281521088},{"type":"v1","addr":"172.21.15.6:6837","nonce":1281521088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2268322007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2268322007},{"type":"v1","addr":"172.21.15.186:6835","nonce":2268322007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/3811005052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3811005052},{"type":"v1","addr":"172.21.15.169:6835","nonce":3811005052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.186:6837/1647607118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1647607118},{"type":"v1","addr":"172.21.15.186:6837","nonce":1647607118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":32,"addr":"172.21.15.186:6839/3954883105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":3954883105},{"type":"v1","addr":"172.21.15.186:6839","nonce":3954883105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.169:6837/1845715294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1845715294},{"type":"v1","addr":"172.21.15.169:6837","nonce":1845715294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.6:6839/2780306492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":2780306492},{"type":"v1","addr":"172.21.15.6:6839","nonce":2780306492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.169:6839/614388771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":614388771},{"type":"v1","addr":"172.21.15.169:6839","nonce":614388771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24512,24554,24530,24493,24511]},"id":1}]} 2024-05-09T04:59:13.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.320+0000 7f1d4ae00700 1 -- 172.21.15.6:0/906571977 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f1d280b2880 msgr2=0x7f1d280b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:13.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.320+0000 7f1d4ae00700 1 --2- 172.21.15.6:0/906571977 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f1d280b2880 0x7f1d280b4d90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009c60 tx=0x7f1d34009260 comp rx=0 tx=0).stop 2024-05-09T04:59:13.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.320+0000 7f1d4ae00700 1 -- 172.21.15.6:0/906571977 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d44107e10 msgr2=0x7f1d440ff950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:13.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.321+0000 7f1d4ae00700 1 --2- 172.21.15.6:0/906571977 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d44107e10 0x7f1d440ff950 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1d30006010 tx=0x7f1d30005280 comp rx=0 tx=0).stop 2024-05-09T04:59:13.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.321+0000 7f1d4ae00700 1 -- 172.21.15.6:0/906571977 shutdown_connections 2024-05-09T04:59:13.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.321+0000 7f1d4ae00700 1 --2- 172.21.15.6:0/906571977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d44108760 0x7f1d440ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:13.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.321+0000 7f1d4ae00700 1 --2- 172.21.15.6:0/906571977 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d441034e0 0x7f1d440ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:13.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.321+0000 7f1d4ae00700 1 --2- 172.21.15.6:0/906571977 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f1d280b2880 0x7f1d280b4d90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:13.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.321+0000 7f1d4ae00700 1 --2- 172.21.15.6:0/906571977 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d44107e10 0x7f1d440ff950 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:13.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.321+0000 7f1d4ae00700 1 -- 172.21.15.6:0/906571977 >> 172.21.15.6:0/906571977 conn(0x7f1d440fef30 msgr2=0x7f1d4406c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:13.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.321+0000 7f1d4ae00700 1 -- 172.21.15.6:0/906571977 shutdown_connections 2024-05-09T04:59:13.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:13.321+0000 7f1d4ae00700 1 -- 172.21.15.6:0/906571977 wait complete. 2024-05-09T04:59:13.324 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 30 2024-05-09T04:59:13.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:12 smithi186 ceph-mon[34668]: pgmap v627: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T04:59:13.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:12 smithi169 ceph-mon[34075]: pgmap v627: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T04:59:14.187 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:13 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/906571977' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:59:14.246 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:59:14.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:13 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/906571977' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:59:14.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:13 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/906571977' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:59:14.737 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:59:15.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:14 smithi006 ceph-mon[29835]: pgmap v628: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 596 B/s wr, 10 op/s 2024-05-09T04:59:15.251 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:14 smithi186 ceph-mon[34668]: pgmap v628: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 596 B/s wr, 10 op/s 2024-05-09T04:59:15.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:14 smithi169 ceph-mon[34075]: pgmap v628: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 596 B/s wr, 10 op/s 2024-05-09T04:59:17.157 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:16 smithi006 ceph-mon[29835]: pgmap v629: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 596 B/s wr, 11 op/s 2024-05-09T04:59:17.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.286+0000 7fd5e3200700 1 -- 172.21.15.6:0/3712997390 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5e410a7c0 msgr2=0x7fd5e410abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:17.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.286+0000 7fd5e3200700 1 --2- 172.21.15.6:0/3712997390 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5e410a7c0 0x7fd5e410abd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd5d00098f0 tx=0x7fd5d0009c00 comp rx=0 tx=0).stop 2024-05-09T04:59:17.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.287+0000 7fd5e3200700 1 -- 172.21.15.6:0/3712997390 shutdown_connections 2024-05-09T04:59:17.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.287+0000 7fd5e3200700 1 --2- 172.21.15.6:0/3712997390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5e410b110 0x7fd5e4117620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:17.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.287+0000 7fd5e3200700 1 --2- 172.21.15.6:0/3712997390 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5e410f6c0 0x7fd5e410fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:17.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.287+0000 7fd5e3200700 1 --2- 172.21.15.6:0/3712997390 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5e410a7c0 0x7fd5e410abd0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:17.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.287+0000 7fd5e3200700 1 -- 172.21.15.6:0/3712997390 >> 172.21.15.6:0/3712997390 conn(0x7fd5e4076b00 msgr2=0x7fd5e4076f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:17.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.287+0000 7fd5e3200700 1 -- 172.21.15.6:0/3712997390 shutdown_connections 2024-05-09T04:59:17.288 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.287+0000 7fd5e3200700 1 -- 172.21.15.6:0/3712997390 wait complete. 2024-05-09T04:59:17.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.288+0000 7fd5e3200700 1 Processor -- start 2024-05-09T04:59:17.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.289+0000 7fd5e3200700 1 -- start start 2024-05-09T04:59:17.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.289+0000 7fd5e3200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5e410a7c0 0x7fd5e41a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:17.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.289+0000 7fd5e3200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5e410b110 0x7fd5e41a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:17.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.289+0000 7fd5e3200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5e410f6c0 0x7fd5e41a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:17.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.289+0000 7fd5e3200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e411a8a0 con 0x7fd5e410b110 2024-05-09T04:59:17.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.289+0000 7fd5e3200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e411a720 con 0x7fd5e410a7c0 2024-05-09T04:59:17.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.289+0000 7fd5e3200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e411aa20 con 0x7fd5e410f6c0 2024-05-09T04:59:17.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.290+0000 7fd5e1e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5e410a7c0 0x7fd5e41a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:17.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.290+0000 7fd5e1400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5e410b110 0x7fd5e41a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:17.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.290+0000 7fd5e1e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5e410a7c0 0x7fd5e41a1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.6:40982/0 (socket says 172.21.15.6:40982) 2024-05-09T04:59:17.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.290+0000 7fd5e1e00700 1 -- 172.21.15.6:0/557323577 learned_addr learned my addr 172.21.15.6:0/557323577 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:59:17.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.290+0000 7fd5e2800700 1 --2- 172.21.15.6:0/557323577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5e410f6c0 0x7fd5e41a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:17.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.290+0000 7fd5e1e00700 1 -- 172.21.15.6:0/557323577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5e410f6c0 msgr2=0x7fd5e41a5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:17.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.290+0000 7fd5e1e00700 1 --2- 172.21.15.6:0/557323577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5e410f6c0 0x7fd5e41a5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:17.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.290+0000 7fd5e1e00700 1 -- 172.21.15.6:0/557323577 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5e410b110 msgr2=0x7fd5e41a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:17.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.290+0000 7fd5e1e00700 1 --2- 172.21.15.6:0/557323577 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5e410b110 0x7fd5e41a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:17.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.290+0000 7fd5e1e00700 1 -- 172.21.15.6:0/557323577 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e41a6120 con 0x7fd5e410a7c0 2024-05-09T04:59:17.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.290+0000 7fd5e1400700 1 --2- 172.21.15.6:0/557323577 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5e410b110 0x7fd5e41a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:59:17.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.291+0000 7fd5e1e00700 1 --2- 172.21.15.6:0/557323577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5e410a7c0 0x7fd5e41a1250 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400e9c0 tx=0x7fd5d400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:17.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.291+0000 7fd5daa00700 1 -- 172.21.15.6:0/557323577 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4004a30 con 0x7fd5e410a7c0 2024-05-09T04:59:17.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.291+0000 7fd5e3200700 1 -- 172.21.15.6:0/557323577 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e41a6410 con 0x7fd5e410a7c0 2024-05-09T04:59:17.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.291+0000 7fd5daa00700 1 -- 172.21.15.6:0/557323577 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5d400cc70 con 0x7fd5e410a7c0 2024-05-09T04:59:17.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.291+0000 7fd5daa00700 1 -- 172.21.15.6:0/557323577 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d40054a0 con 0x7fd5e410a7c0 2024-05-09T04:59:17.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.291+0000 7fd5e3200700 1 -- 172.21.15.6:0/557323577 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e41add30 con 0x7fd5e410a7c0 2024-05-09T04:59:17.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.292+0000 7fd5e3200700 1 -- 172.21.15.6:0/557323577 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5e40512c0 con 0x7fd5e410a7c0 2024-05-09T04:59:17.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.295+0000 7fd5daa00700 1 -- 172.21.15.6:0/557323577 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7fd5d4004bd0 con 0x7fd5e410a7c0 2024-05-09T04:59:17.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.297+0000 7fd5daa00700 1 --2- 172.21.15.6:0/557323577 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd5c80b2880 0x7fd5c80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:17.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.297+0000 7fd5daa00700 1 -- 172.21.15.6:0/557323577 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fd5d40e28a0 con 0x7fd5e410a7c0 2024-05-09T04:59:17.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.297+0000 7fd5e1400700 1 --2- 172.21.15.6:0/557323577 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd5c80b2880 0x7fd5c80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:17.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.298+0000 7fd5e1400700 1 --2- 172.21.15.6:0/557323577 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd5c80b2880 0x7fd5c80b4d90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009fd0 tx=0x7fd5d0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:17.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.299+0000 7fd5daa00700 1 -- 172.21.15.6:0/557323577 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd5d4054df0 con 0x7fd5e410a7c0 2024-05-09T04:59:17.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:16 smithi186 ceph-mon[34668]: pgmap v629: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 596 B/s wr, 11 op/s 2024-05-09T04:59:17.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:16 smithi169 ceph-mon[34075]: pgmap v629: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 596 B/s wr, 11 op/s 2024-05-09T04:59:17.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.590+0000 7fd5e3200700 1 -- 172.21.15.6:0/557323577 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5e41a25b0 con 0x7fd5e410a7c0 2024-05-09T04:59:17.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.592+0000 7fd5daa00700 1 -- 172.21.15.6:0/557323577 <== mon.2 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11927 (secure 0 0 0) 0x7fd5d40aa1e0 con 0x7fd5e410a7c0 2024-05-09T04:59:17.593 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:59:17.593 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/162094599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":162094599},{"type":"v1","addr":"172.21.15.6:6835","nonce":162094599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/589294363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":589294363},{"type":"v1","addr":"172.21.15.6:6841","nonce":589294363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:49:36.920376+0000","modified":"2024-05-09T04:54:56.247431+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24554,"mds_1":24530,"mds_2":24512,"mds_3":24511,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24442":{"gid":24442,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6833/2977145206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":2977145206},{"type":"v1","addr":"172.21.15.186:6833","nonce":2977145206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6833/133345948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":133345948},{"type":"v1","addr":"172.21.15.169:6833","nonce":133345948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.6:6837/1281521088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":1281521088},{"type":"v1","addr":"172.21.15.6:6837","nonce":1281521088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2268322007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2268322007},{"type":"v1","addr":"172.21.15.186:6835","nonce":2268322007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/3811005052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3811005052},{"type":"v1","addr":"172.21.15.169:6835","nonce":3811005052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":33,"addr":"172.21.15.186:6837/1647607118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1647607118},{"type":"v1","addr":"172.21.15.186:6837","nonce":1647607118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":32,"addr":"172.21.15.186:6839/3954883105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":3954883105},{"type":"v1","addr":"172.21.15.186:6839","nonce":3954883105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.169:6837/1845715294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1845715294},{"type":"v1","addr":"172.21.15.169:6837","nonce":1845715294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.6:6839/2780306492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":2780306492},{"type":"v1","addr":"172.21.15.6:6839","nonce":2780306492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.169:6839/614388771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":614388771},{"type":"v1","addr":"172.21.15.169:6839","nonce":614388771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24512,24554,24530,24493,24511]},"id":1}]} 2024-05-09T04:59:17.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.596+0000 7fd5e3200700 1 -- 172.21.15.6:0/557323577 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd5c80b2880 msgr2=0x7fd5c80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:17.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.596+0000 7fd5e3200700 1 --2- 172.21.15.6:0/557323577 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd5c80b2880 0x7fd5c80b4d90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009fd0 tx=0x7fd5d0019040 comp rx=0 tx=0).stop 2024-05-09T04:59:17.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.596+0000 7fd5e3200700 1 -- 172.21.15.6:0/557323577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5e410a7c0 msgr2=0x7fd5e41a1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:17.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.596+0000 7fd5e3200700 1 --2- 172.21.15.6:0/557323577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5e410a7c0 0x7fd5e41a1250 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400e9c0 tx=0x7fd5d400ecd0 comp rx=0 tx=0).stop 2024-05-09T04:59:17.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.596+0000 7fd5e3200700 1 -- 172.21.15.6:0/557323577 shutdown_connections 2024-05-09T04:59:17.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.596+0000 7fd5e3200700 1 --2- 172.21.15.6:0/557323577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5e410f6c0 0x7fd5e41a5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:17.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.596+0000 7fd5e3200700 1 --2- 172.21.15.6:0/557323577 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd5c80b2880 0x7fd5c80b4d90 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:17.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.596+0000 7fd5e3200700 1 --2- 172.21.15.6:0/557323577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5e410a7c0 0x7fd5e41a1250 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:17.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.596+0000 7fd5e3200700 1 --2- 172.21.15.6:0/557323577 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5e410b110 0x7fd5e41a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:17.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.596+0000 7fd5e3200700 1 -- 172.21.15.6:0/557323577 >> 172.21.15.6:0/557323577 conn(0x7fd5e4076b00 msgr2=0x7fd5e41088c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:17.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.596+0000 7fd5e3200700 1 -- 172.21.15.6:0/557323577 shutdown_connections 2024-05-09T04:59:17.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:17.596+0000 7fd5e3200700 1 -- 172.21.15.6:0/557323577 wait complete. 2024-05-09T04:59:17.599 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 30 2024-05-09T04:59:18.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:17 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/557323577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:59:18.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:17 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/557323577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:59:18.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:17 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/557323577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:59:18.605 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T04:59:19.080 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:59:19.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:18 smithi006 ceph-mon[29835]: pgmap v630: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-09T04:59:19.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:18 smithi186 ceph-mon[34668]: pgmap v630: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-09T04:59:19.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:18 smithi169 ceph-mon[34075]: pgmap v630: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-09T04:59:21.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:20 smithi006 ceph-mon[29835]: pgmap v631: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 596 B/s wr, 10 op/s 2024-05-09T04:59:21.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:20 smithi186 ceph-mon[34668]: pgmap v631: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 596 B/s wr, 10 op/s 2024-05-09T04:59:21.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:20 smithi169 ceph-mon[34075]: pgmap v631: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 596 B/s wr, 10 op/s 2024-05-09T04:59:21.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.566+0000 7fd856400700 1 -- 172.21.15.6:0/626911846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd850106ec0 msgr2=0x7fd850113350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:21.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.566+0000 7fd856400700 1 --2- 172.21.15.6:0/626911846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd850106ec0 0x7fd850113350 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd848009f90 tx=0x7fd84800b330 comp rx=0 tx=0).stop 2024-05-09T04:59:21.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.566+0000 7fd856400700 1 -- 172.21.15.6:0/626911846 shutdown_connections 2024-05-09T04:59:21.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.566+0000 7fd856400700 1 --2- 172.21.15.6:0/626911846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd850106ec0 0x7fd850113350 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.566+0000 7fd856400700 1 --2- 172.21.15.6:0/626911846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd85010c5b0 0x7fd85010c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.566+0000 7fd856400700 1 --2- 172.21.15.6:0/626911846 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd850106570 0x7fd850106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.567+0000 7fd856400700 1 -- 172.21.15.6:0/626911846 >> 172.21.15.6:0/626911846 conn(0x7fd850102050 msgr2=0x7fd8501044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:21.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.567+0000 7fd856400700 1 -- 172.21.15.6:0/626911846 shutdown_connections 2024-05-09T04:59:21.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.567+0000 7fd856400700 1 -- 172.21.15.6:0/626911846 wait complete. 2024-05-09T04:59:21.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.568+0000 7fd856400700 1 Processor -- start 2024-05-09T04:59:21.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.569+0000 7fd856400700 1 -- start start 2024-05-09T04:59:21.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.569+0000 7fd856400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd850106570 0x7fd85019cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:21.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.569+0000 7fd856400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd850106ec0 0x7fd85019d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:21.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.569+0000 7fd856400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd85010c5b0 0x7fd8501a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:21.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.569+0000 7fd856400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd8501166e0 con 0x7fd85010c5b0 2024-05-09T04:59:21.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.569+0000 7fd856400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd850116560 con 0x7fd850106570 2024-05-09T04:59:21.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.569+0000 7fd856400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd850116860 con 0x7fd850106ec0 2024-05-09T04:59:21.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.570+0000 7fd84fe00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd85010c5b0 0x7fd8501a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:21.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.570+0000 7fd84ea00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd850106ec0 0x7fd85019d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:21.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.570+0000 7fd84f400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd850106570 0x7fd85019cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:21.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.570+0000 7fd84ea00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd850106ec0 0x7fd85019d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:60418/0 (socket says 172.21.15.6:60418) 2024-05-09T04:59:21.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.570+0000 7fd84ea00700 1 -- 172.21.15.6:0/1354089552 learned_addr learned my addr 172.21.15.6:0/1354089552 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:59:21.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.570+0000 7fd84ea00700 1 -- 172.21.15.6:0/1354089552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd850106570 msgr2=0x7fd85019cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:21.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.570+0000 7fd84ea00700 1 --2- 172.21.15.6:0/1354089552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd850106570 0x7fd85019cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.570+0000 7fd84ea00700 1 -- 172.21.15.6:0/1354089552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd85010c5b0 msgr2=0x7fd8501a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:21.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.570+0000 7fd84ea00700 1 --2- 172.21.15.6:0/1354089552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd85010c5b0 0x7fd8501a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.570+0000 7fd84ea00700 1 -- 172.21.15.6:0/1354089552 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8501a1e80 con 0x7fd850106ec0 2024-05-09T04:59:21.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.570+0000 7fd84f400700 1 --2- 172.21.15.6:0/1354089552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd850106570 0x7fd85019cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:59:21.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.570+0000 7fd84ea00700 1 --2- 172.21.15.6:0/1354089552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd850106ec0 0x7fd85019d550 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00ebb0 tx=0x7fd83c00eec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:21.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.571+0000 7fd847e00700 1 -- 172.21.15.6:0/1354089552 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c00cc90 con 0x7fd850106ec0 2024-05-09T04:59:21.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.571+0000 7fd847e00700 1 -- 172.21.15.6:0/1354089552 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd83c00ce30 con 0x7fd850106ec0 2024-05-09T04:59:21.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.571+0000 7fd856400700 1 -- 172.21.15.6:0/1354089552 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8501a2170 con 0x7fd850106ec0 2024-05-09T04:59:21.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.571+0000 7fd847e00700 1 -- 172.21.15.6:0/1354089552 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c010430 con 0x7fd850106ec0 2024-05-09T04:59:21.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.571+0000 7fd856400700 1 -- 172.21.15.6:0/1354089552 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8501a9a90 con 0x7fd850106ec0 2024-05-09T04:59:21.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.572+0000 7fd856400700 1 -- 172.21.15.6:0/1354089552 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8500512c0 con 0x7fd850106ec0 2024-05-09T04:59:21.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.575+0000 7fd847e00700 1 -- 172.21.15.6:0/1354089552 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7fd83c003f10 con 0x7fd850106ec0 2024-05-09T04:59:21.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.577+0000 7fd847e00700 1 --2- 172.21.15.6:0/1354089552 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd8340b2880 0x7fd8340b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:21.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.577+0000 7fd847e00700 1 -- 172.21.15.6:0/1354089552 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fd83c0e2980 con 0x7fd850106ec0 2024-05-09T04:59:21.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.577+0000 7fd84f400700 1 --2- 172.21.15.6:0/1354089552 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd8340b2880 0x7fd8340b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:21.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.578+0000 7fd84f400700 1 --2- 172.21.15.6:0/1354089552 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd8340b2880 0x7fd8340b4d90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd8501079e0 tx=0x7fd840009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:21.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.579+0000 7fd847e00700 1 -- 172.21.15.6:0/1354089552 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd83c0aa350 con 0x7fd850106ec0 2024-05-09T04:59:21.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.818+0000 7fd856400700 1 -- 172.21.15.6:0/1354089552 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd85019e310 con 0x7fd850106ec0 2024-05-09T04:59:21.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.820+0000 7fd847e00700 1 -- 172.21.15.6:0/1354089552 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24659 (secure 0 0 0) 0x7fd83c0a9c50 con 0x7fd850106ec0 2024-05-09T04:59:21.822 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-09T04:59:21.823 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":89,"fsid":"f74d813c-0dbd-11ef-bc97-c7b262605968","created":"2024-05-09T04:39:35.287470+0000","modified":"2024-05-09T04:50:57.686719+0000","last_up_change":"2024-05-09T04:48:16.919842+0000","last_in_change":"2024-05-09T04:47:56.452645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:43:47.538306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:49:28.805447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T04:49:32.970287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T04:49:45.637000+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1b732fd3-e648-4b0d-aece-083bdaaa5d68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6803","nonce":1738750914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6805","nonce":1738750914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6809","nonce":1738750914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":1738750914},{"type":"v1","addr":"172.21.15.6:6807","nonce":1738750914}]},"public_addr":"172.21.15.6:6803/1738750914","cluster_addr":"172.21.15.6:6805/1738750914","heartbeat_back_addr":"172.21.15.6:6809/1738750914","heartbeat_front_addr":"172.21.15.6:6807/1738750914","state":["exists","up"]},{"osd":1,"uuid":"9770c07d-4243-43bf-a0cd-9d2ac558556c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6801","nonce":2552207737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6803","nonce":2552207737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6807","nonce":2552207737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2552207737},{"type":"v1","addr":"172.21.15.169:6805","nonce":2552207737}]},"public_addr":"172.21.15.169:6801/2552207737","cluster_addr":"172.21.15.169:6803/2552207737","heartbeat_back_addr":"172.21.15.169:6807/2552207737","heartbeat_front_addr":"172.21.15.169:6805/2552207737","state":["exists","up"]},{"osd":2,"uuid":"3f50ed89-b2a8-415e-b78d-8510d98d9368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6801","nonce":231435105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6803","nonce":231435105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6807","nonce":231435105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":231435105},{"type":"v1","addr":"172.21.15.186:6805","nonce":231435105}]},"public_addr":"172.21.15.186:6801/231435105","cluster_addr":"172.21.15.186:6803/231435105","heartbeat_back_addr":"172.21.15.186:6807/231435105","heartbeat_front_addr":"172.21.15.186:6805/231435105","state":["exists","up"]},{"osd":3,"uuid":"afcbc36b-5669-4044-8dc5-2ebfe7970774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6811","nonce":2846613703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6813","nonce":2846613703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6817","nonce":2846613703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2846613703},{"type":"v1","addr":"172.21.15.6:6815","nonce":2846613703}]},"public_addr":"172.21.15.6:6811/2846613703","cluster_addr":"172.21.15.6:6813/2846613703","heartbeat_back_addr":"172.21.15.6:6817/2846613703","heartbeat_front_addr":"172.21.15.6:6815/2846613703","state":["exists","up"]},{"osd":4,"uuid":"bb84cc5e-0297-4e23-be50-7ee2f7e59665","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6809","nonce":1928722453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6811","nonce":1928722453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6815","nonce":1928722453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1928722453},{"type":"v1","addr":"172.21.15.169:6813","nonce":1928722453}]},"public_addr":"172.21.15.169:6809/1928722453","cluster_addr":"172.21.15.169:6811/1928722453","heartbeat_back_addr":"172.21.15.169:6815/1928722453","heartbeat_front_addr":"172.21.15.169:6813/1928722453","state":["exists","up"]},{"osd":5,"uuid":"830dc7f3-352e-4b01-8d4c-a9b68dcb40d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6809","nonce":3644445439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6811","nonce":3644445439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6815","nonce":3644445439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3644445439},{"type":"v1","addr":"172.21.15.186:6813","nonce":3644445439}]},"public_addr":"172.21.15.186:6809/3644445439","cluster_addr":"172.21.15.186:6811/3644445439","heartbeat_back_addr":"172.21.15.186:6815/3644445439","heartbeat_front_addr":"172.21.15.186:6813/3644445439","state":["exists","up"]},{"osd":6,"uuid":"2e8478cb-c8c9-47ce-a3a5-48ffc959ab24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6819","nonce":3068252891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6821","nonce":3068252891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6825","nonce":3068252891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":3068252891},{"type":"v1","addr":"172.21.15.6:6823","nonce":3068252891}]},"public_addr":"172.21.15.6:6819/3068252891","cluster_addr":"172.21.15.6:6821/3068252891","heartbeat_back_addr":"172.21.15.6:6825/3068252891","heartbeat_front_addr":"172.21.15.6:6823/3068252891","state":["exists","up"]},{"osd":7,"uuid":"0f432ee2-42a4-4efa-9233-8e5a2fd65d10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6817","nonce":1303584711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6819","nonce":1303584711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6823","nonce":1303584711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":1303584711},{"type":"v1","addr":"172.21.15.169:6821","nonce":1303584711}]},"public_addr":"172.21.15.169:6817/1303584711","cluster_addr":"172.21.15.169:6819/1303584711","heartbeat_back_addr":"172.21.15.169:6823/1303584711","heartbeat_front_addr":"172.21.15.169:6821/1303584711","state":["exists","up"]},{"osd":8,"uuid":"cf5dca81-131b-48e7-9417-d01d4be47316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6817","nonce":3475163771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6819","nonce":3475163771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6823","nonce":3475163771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3475163771},{"type":"v1","addr":"172.21.15.186:6821","nonce":3475163771}]},"public_addr":"172.21.15.186:6817/3475163771","cluster_addr":"172.21.15.186:6819/3475163771","heartbeat_back_addr":"172.21.15.186:6823/3475163771","heartbeat_front_addr":"172.21.15.186:6821/3475163771","state":["exists","up"]},{"osd":9,"uuid":"bca7bedd-1a57-4465-9033-064ca468add2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6827","nonce":3590990973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6829","nonce":3590990973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6833","nonce":3590990973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":3590990973},{"type":"v1","addr":"172.21.15.6:6831","nonce":3590990973}]},"public_addr":"172.21.15.6:6827/3590990973","cluster_addr":"172.21.15.6:6829/3590990973","heartbeat_back_addr":"172.21.15.6:6833/3590990973","heartbeat_front_addr":"172.21.15.6:6831/3590990973","state":["exists","up"]},{"osd":10,"uuid":"97b0e058-64f8-42ee-9b99-3ef8e39c8a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6825","nonce":61007341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6827","nonce":61007341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6831","nonce":61007341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":61007341},{"type":"v1","addr":"172.21.15.169:6829","nonce":61007341}]},"public_addr":"172.21.15.169:6825/61007341","cluster_addr":"172.21.15.169:6827/61007341","heartbeat_back_addr":"172.21.15.169:6831/61007341","heartbeat_front_addr":"172.21.15.169:6829/61007341","state":["exists","up"]},{"osd":11,"uuid":"7f2220b0-24cf-48b2-95ae-1d66e00e57fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6825","nonce":1138940441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6827","nonce":1138940441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6831","nonce":1138940441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1138940441},{"type":"v1","addr":"172.21.15.186:6829","nonce":1138940441}]},"public_addr":"172.21.15.186:6825/1138940441","cluster_addr":"172.21.15.186:6827/1138940441","heartbeat_back_addr":"172.21.15.186:6831/1138940441","heartbeat_front_addr":"172.21.15.186:6829/1138940441","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:53.766295+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:20.195243+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:44.153812+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:20.472086+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:48.415371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:13.795415+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:50.236398+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:18.066623+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:43.781009+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:47:20.084719+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:47:48.219901+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:48:14.965536+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/3742411767":"2024-05-10T04:40:55.357057+0000","172.21.15.6:0/2410771702":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6801/937713898":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6800/937713898":"2024-05-10T04:40:55.357057+0000","172.21.15.6:6801/3021503778":"2024-05-10T04:40:19.639151+0000","172.21.15.6:6800/3021503778":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/2559510850":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/3810474743":"2024-05-10T04:40:19.639151+0000","172.21.15.6:0/4133970555":"2024-05-10T04:40:55.357057+0000","172.21.15.6:0/664817743":"2024-05-10T04:40:19.639151+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:59:21.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.824+0000 7fd856400700 1 -- 172.21.15.6:0/1354089552 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd8340b2880 msgr2=0x7fd8340b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:21.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.824+0000 7fd856400700 1 --2- 172.21.15.6:0/1354089552 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd8340b2880 0x7fd8340b4d90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd8501079e0 tx=0x7fd840009260 comp rx=0 tx=0).stop 2024-05-09T04:59:21.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.824+0000 7fd856400700 1 -- 172.21.15.6:0/1354089552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd850106ec0 msgr2=0x7fd85019d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:21.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.824+0000 7fd856400700 1 --2- 172.21.15.6:0/1354089552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd850106ec0 0x7fd85019d550 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00ebb0 tx=0x7fd83c00eec0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.824+0000 7fd856400700 1 -- 172.21.15.6:0/1354089552 shutdown_connections 2024-05-09T04:59:21.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.824+0000 7fd856400700 1 --2- 172.21.15.6:0/1354089552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd85010c5b0 0x7fd8501a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.824+0000 7fd856400700 1 --2- 172.21.15.6:0/1354089552 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fd8340b2880 0x7fd8340b4d90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.824+0000 7fd856400700 1 --2- 172.21.15.6:0/1354089552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd850106570 0x7fd85019cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.824+0000 7fd856400700 1 --2- 172.21.15.6:0/1354089552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd850106ec0 0x7fd85019d550 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.824+0000 7fd856400700 1 -- 172.21.15.6:0/1354089552 >> 172.21.15.6:0/1354089552 conn(0x7fd850102050 msgr2=0x7fd8501035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:21.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.824+0000 7fd856400700 1 -- 172.21.15.6:0/1354089552 shutdown_connections 2024-05-09T04:59:21.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:21.824+0000 7fd856400700 1 -- 172.21.15.6:0/1354089552 wait complete. 2024-05-09T04:59:22.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:21 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1354089552' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:59:22.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:21 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/1354089552' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:59:22.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:21 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/1354089552' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:59:22.833 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs fail cephfs 2024-05-09T04:59:23.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:22 smithi006 ceph-mon[29835]: pgmap v632: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-09T04:59:23.268 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:22 smithi169 ceph-mon[34075]: pgmap v632: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-09T04:59:23.295 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:59:23.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:22 smithi186 ceph-mon[34668]: pgmap v632: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-09T04:59:25.065 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:24 smithi006 ceph-mon[29835]: pgmap v633: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-05-09T04:59:25.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:24 smithi186 ceph-mon[34668]: pgmap v633: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-05-09T04:59:25.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:24 smithi169 ceph-mon[34075]: pgmap v633: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-05-09T04:59:25.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.638+0000 7f5373200700 1 -- 172.21.15.6:0/2810179995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f536c1045f0 msgr2=0x7f536c104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:25.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.638+0000 7f5373200700 1 --2- 172.21.15.6:0/2810179995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f536c1045f0 0x7f536c104a00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f535c009990 tx=0x7f535c009ca0 comp rx=0 tx=0).stop 2024-05-09T04:59:25.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.638+0000 7f5373200700 1 -- 172.21.15.6:0/2810179995 shutdown_connections 2024-05-09T04:59:25.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.638+0000 7f5373200700 1 --2- 172.21.15.6:0/2810179995 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f536c109870 0x7f536c1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:25.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.638+0000 7f5373200700 1 --2- 172.21.15.6:0/2810179995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f536c1045f0 0x7f536c104a00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:25.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.638+0000 7f5373200700 1 --2- 172.21.15.6:0/2810179995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f536c108f20 0x7f536c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:25.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.638+0000 7f5373200700 1 -- 172.21.15.6:0/2810179995 >> 172.21.15.6:0/2810179995 conn(0x7f536c100040 msgr2=0x7f536c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:25.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.638+0000 7f5373200700 1 -- 172.21.15.6:0/2810179995 shutdown_connections 2024-05-09T04:59:25.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.638+0000 7f5373200700 1 -- 172.21.15.6:0/2810179995 wait complete. 2024-05-09T04:59:25.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.640+0000 7f5373200700 1 Processor -- start 2024-05-09T04:59:25.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.640+0000 7f5373200700 1 -- start start 2024-05-09T04:59:25.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.640+0000 7f5373200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f536c1045f0 0x7f536c19d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:25.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.640+0000 7f5373200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f536c108f20 0x7f536c19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:25.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.640+0000 7f5373200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f536c109870 0x7f536c1a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:25.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.640+0000 7f5373200700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f536c1144a0 con 0x7f536c108f20 2024-05-09T04:59:25.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.640+0000 7f5373200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f536c114320 con 0x7f536c109870 2024-05-09T04:59:25.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.640+0000 7f5373200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f536c114620 con 0x7f536c1045f0 2024-05-09T04:59:25.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.641+0000 7f536be00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f536c1045f0 0x7f536c19d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:25.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.641+0000 7f536be00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f536c1045f0 0x7f536c19d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:60438/0 (socket says 172.21.15.6:60438) 2024-05-09T04:59:25.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.641+0000 7f536be00700 1 -- 172.21.15.6:0/3228031474 learned_addr learned my addr 172.21.15.6:0/3228031474 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:59:25.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.641+0000 7f5363e00700 1 --2- 172.21.15.6:0/3228031474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f536c108f20 0x7f536c19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:25.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.641+0000 7f5371200700 1 --2- 172.21.15.6:0/3228031474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f536c109870 0x7f536c1a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:25.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.641+0000 7f536be00700 1 -- 172.21.15.6:0/3228031474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f536c109870 msgr2=0x7f536c1a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:25.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.641+0000 7f536be00700 1 --2- 172.21.15.6:0/3228031474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f536c109870 0x7f536c1a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:25.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.641+0000 7f536be00700 1 -- 172.21.15.6:0/3228031474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f536c108f20 msgr2=0x7f536c19d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:25.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.641+0000 7f536be00700 1 --2- 172.21.15.6:0/3228031474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f536c108f20 0x7f536c19d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:25.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.641+0000 7f536be00700 1 -- 172.21.15.6:0/3228031474 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f536c1a1f80 con 0x7f536c1045f0 2024-05-09T04:59:25.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.641+0000 7f5371200700 1 --2- 172.21.15.6:0/3228031474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f536c109870 0x7f536c1a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:59:25.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.641+0000 7f536be00700 1 --2- 172.21.15.6:0/3228031474 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f536c1045f0 0x7f536c19d0b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f535c0098e0 tx=0x7f535c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:25.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.642+0000 7f5369600700 1 -- 172.21.15.6:0/3228031474 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c027070 con 0x7f536c1045f0 2024-05-09T04:59:25.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.642+0000 7f5373200700 1 -- 172.21.15.6:0/3228031474 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f536c1a2210 con 0x7f536c1045f0 2024-05-09T04:59:25.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.642+0000 7f5369600700 1 -- 172.21.15.6:0/3228031474 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f535c022470 con 0x7f536c1045f0 2024-05-09T04:59:25.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.642+0000 7f5369600700 1 -- 172.21.15.6:0/3228031474 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c017650 con 0x7f536c1045f0 2024-05-09T04:59:25.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.642+0000 7f5373200700 1 -- 172.21.15.6:0/3228031474 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f536c1a26f0 con 0x7f536c1045f0 2024-05-09T04:59:25.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.643+0000 7f5373200700 1 -- 172.21.15.6:0/3228031474 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f536c0512c0 con 0x7f536c1045f0 2024-05-09T04:59:25.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.646+0000 7f5369600700 1 -- 172.21.15.6:0/3228031474 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f535c02a070 con 0x7f536c1045f0 2024-05-09T04:59:25.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.647+0000 7f5369600700 1 --2- 172.21.15.6:0/3228031474 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f53500b2880 0x7f53500b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:25.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.648+0000 7f5369600700 1 -- 172.21.15.6:0/3228031474 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f535c0e35d0 con 0x7f536c1045f0 2024-05-09T04:59:25.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.648+0000 7f5363e00700 1 --2- 172.21.15.6:0/3228031474 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f53500b2880 0x7f53500b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:25.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.649+0000 7f5363e00700 1 --2- 172.21.15.6:0/3228031474 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f53500b2880 0x7f53500b4d90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f536c107510 tx=0x7f5354009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:25.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.649+0000 7f5369600700 1 -- 172.21.15.6:0/3228031474 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f535c013070 con 0x7f536c1045f0 2024-05-09T04:59:25.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.939+0000 7f5373200700 1 -- 172.21.15.6:0/3228031474 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f536c1a9b90 con 0x7f536c1045f0 2024-05-09T04:59:25.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.952+0000 7f5369600700 1 -- 172.21.15.6:0/3228031474 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) ==== 161+0+0 (secure 0 0 0) 0x7f535c0aafa0 con 0x7f536c1045f0 2024-05-09T04:59:25.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.956+0000 7f5373200700 1 -- 172.21.15.6:0/3228031474 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f53500b2880 msgr2=0x7f53500b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:25.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.956+0000 7f5373200700 1 --2- 172.21.15.6:0/3228031474 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f53500b2880 0x7f53500b4d90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f536c107510 tx=0x7f5354009260 comp rx=0 tx=0).stop 2024-05-09T04:59:25.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.956+0000 7f5373200700 1 -- 172.21.15.6:0/3228031474 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f536c1045f0 msgr2=0x7f536c19d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:25.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.956+0000 7f5373200700 1 --2- 172.21.15.6:0/3228031474 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f536c1045f0 0x7f536c19d0b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f535c0098e0 tx=0x7f535c0047b0 comp rx=0 tx=0).stop 2024-05-09T04:59:25.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.956+0000 7f5373200700 1 -- 172.21.15.6:0/3228031474 shutdown_connections 2024-05-09T04:59:25.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.956+0000 7f5373200700 1 --2- 172.21.15.6:0/3228031474 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f53500b2880 0x7f53500b4d90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:25.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.956+0000 7f5373200700 1 --2- 172.21.15.6:0/3228031474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f536c109870 0x7f536c1a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:25.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.956+0000 7f5373200700 1 --2- 172.21.15.6:0/3228031474 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f536c1045f0 0x7f536c19d0b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:25.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.956+0000 7f5373200700 1 --2- 172.21.15.6:0/3228031474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f536c108f20 0x7f536c19d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:25.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.957+0000 7f5373200700 1 -- 172.21.15.6:0/3228031474 >> 172.21.15.6:0/3228031474 conn(0x7f536c100040 msgr2=0x7f536c100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:25.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.957+0000 7f5373200700 1 -- 172.21.15.6:0/3228031474 shutdown_connections 2024-05-09T04:59:25.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:25.957+0000 7f5373200700 1 -- 172.21.15.6:0/3228031474 wait complete. 2024-05-09T04:59:25.959 INFO:teuthology.orchestra.run.smithi006.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T04:59:26.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:25 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mon-a[29831]: 2024-05-09T04:59:25.941+0000 7efff0c00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T04:59:26.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:25 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/3228031474' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T04:59:26.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:25 smithi006 ceph-mon[29835]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T04:59:26.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:25 smithi006 ceph-mon[29835]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T04:59:26.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:25 smithi006 ceph-mon[29835]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T04:59:26.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:25 smithi006 ceph-mon[29835]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-09T04:59:26.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:25 smithi006 ceph-mon[29835]: osdmap e90: 12 total, 12 up, 12 in 2024-05-09T04:59:26.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:25 smithi006 ceph-mon[29835]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-09T04:59:26.199 INFO:journalctl@ceph.mds.d.smithi006.stdout:May 09 04:59:26 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-d[67968]: ignoring --setuser ceph since I am not root 2024-05-09T04:59:26.199 INFO:journalctl@ceph.mds.d.smithi006.stdout:May 09 04:59:26 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-d[67968]: ignoring --setgroup ceph since I am not root 2024-05-09T04:59:26.199 INFO:journalctl@ceph.mds.d.smithi006.stdout:May 09 04:59:26 smithi006 ceph-mds[67972]: -- 172.21.15.6:0/3727239584 <== mon.2 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561462ad7a40 con 0x561462b1f600 2024-05-09T04:59:26.199 INFO:journalctl@ceph.mds.d.smithi006.stdout:May 09 04:59:26 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-d[67968]: starting mds.d at 2024-05-09T04:59:26.199 INFO:journalctl@ceph.mds.g.smithi006.stdout:May 09 04:59:26 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-g[68748]: ignoring --setuser ceph since I am not root 2024-05-09T04:59:26.199 INFO:journalctl@ceph.mds.g.smithi006.stdout:May 09 04:59:26 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-g[68748]: ignoring --setgroup ceph since I am not root 2024-05-09T04:59:26.199 INFO:journalctl@ceph.mds.g.smithi006.stdout:May 09 04:59:26 smithi006 ceph-mds[68756]: -- 172.21.15.6:0/1593702715 <== mon.2 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dc60a7fa40 con 0x55dc60ac7a80 2024-05-09T04:59:26.199 INFO:journalctl@ceph.mds.g.smithi006.stdout:May 09 04:59:26 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-g[68748]: starting mds.g at 2024-05-09T04:59:26.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:25 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/3228031474' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T04:59:26.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:25 smithi186 ceph-mon[34668]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T04:59:26.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:25 smithi186 ceph-mon[34668]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T04:59:26.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:25 smithi186 ceph-mon[34668]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T04:59:26.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:25 smithi186 ceph-mon[34668]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-09T04:59:26.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:25 smithi186 ceph-mon[34668]: osdmap e90: 12 total, 12 up, 12 in 2024-05-09T04:59:26.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:25 smithi186 ceph-mon[34668]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-09T04:59:26.352 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 09 04:59:26 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-c[65558]: ignoring --setuser ceph since I am not root 2024-05-09T04:59:26.352 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 09 04:59:26 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-c[65558]: ignoring --setgroup ceph since I am not root 2024-05-09T04:59:26.352 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 09 04:59:26 smithi186 ceph-mds[65562]: -- 172.21.15.186:0/1750369828 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562bd42104e0 con 0x562bd425ba80 2024-05-09T04:59:26.352 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 09 04:59:26 smithi186 ceph-mds[65562]: -- 172.21.15.186:0/1750369828 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562bd4213a40 con 0x562bd425ba80 2024-05-09T04:59:26.352 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 09 04:59:26 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-c[65558]: starting mds.c at 2024-05-09T04:59:26.353 INFO:journalctl@ceph.mds.f.smithi186.stdout:May 09 04:59:26 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-f[65250]: ignoring --setuser ceph since I am not root 2024-05-09T04:59:26.353 INFO:journalctl@ceph.mds.f.smithi186.stdout:May 09 04:59:26 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-f[65250]: ignoring --setgroup ceph since I am not root 2024-05-09T04:59:26.353 INFO:journalctl@ceph.mds.f.smithi186.stdout:May 09 04:59:26 smithi186 ceph-mds[65254]: -- 172.21.15.186:0/734070392 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f1e4d25a40 con 0x55f1e4d6d180 2024-05-09T04:59:26.353 INFO:journalctl@ceph.mds.f.smithi186.stdout:May 09 04:59:26 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-f[65250]: starting mds.f at 2024-05-09T04:59:26.353 INFO:journalctl@ceph.mds.i.smithi186.stdout:May 09 04:59:26 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-i[65862]: ignoring --setuser ceph since I am not root 2024-05-09T04:59:26.353 INFO:journalctl@ceph.mds.i.smithi186.stdout:May 09 04:59:26 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-i[65862]: ignoring --setgroup ceph since I am not root 2024-05-09T04:59:26.353 INFO:journalctl@ceph.mds.i.smithi186.stdout:May 09 04:59:26 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-i[65862]: starting mds.i at 2024-05-09T04:59:26.354 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 09 04:59:26 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-l[64938]: ignoring --setuser ceph since I am not root 2024-05-09T04:59:26.354 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 09 04:59:26 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-l[64938]: ignoring --setgroup ceph since I am not root 2024-05-09T04:59:26.354 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 09 04:59:26 smithi186 ceph-mds[64942]: -- 172.21.15.186:0/3189871938 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a82715da40 con 0x55a8271a5a80 2024-05-09T04:59:26.354 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 09 04:59:26 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-l[64938]: starting mds.l at 2024-05-09T04:59:26.367 INFO:journalctl@ceph.mds.h.smithi169.stdout:May 09 04:59:26 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-h[64624]: ignoring --setuser ceph since I am not root 2024-05-09T04:59:26.367 INFO:journalctl@ceph.mds.h.smithi169.stdout:May 09 04:59:26 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-h[64624]: ignoring --setgroup ceph since I am not root 2024-05-09T04:59:26.367 INFO:journalctl@ceph.mds.h.smithi169.stdout:May 09 04:59:26 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-h[64624]: starting mds.h at 2024-05-09T04:59:26.367 INFO:journalctl@ceph.mds.b.smithi169.stdout:May 09 04:59:26 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-b[64939]: ignoring --setuser ceph since I am not root 2024-05-09T04:59:26.367 INFO:journalctl@ceph.mds.b.smithi169.stdout:May 09 04:59:26 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-b[64939]: ignoring --setgroup ceph since I am not root 2024-05-09T04:59:26.367 INFO:journalctl@ceph.mds.b.smithi169.stdout:May 09 04:59:26 smithi169 ceph-mds[64943]: -- 172.21.15.169:0/976162929 <== mon.2 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557a4f177a40 con 0x557a4f1bf600 2024-05-09T04:59:26.368 INFO:journalctl@ceph.mds.b.smithi169.stdout:May 09 04:59:26 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-b[64939]: starting mds.b at 2024-05-09T04:59:26.368 INFO:journalctl@ceph.mds.k.smithi169.stdout:May 09 04:59:26 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-k[63987]: ignoring --setuser ceph since I am not root 2024-05-09T04:59:26.368 INFO:journalctl@ceph.mds.k.smithi169.stdout:May 09 04:59:26 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-k[63987]: ignoring --setgroup ceph since I am not root 2024-05-09T04:59:26.368 INFO:journalctl@ceph.mds.k.smithi169.stdout:May 09 04:59:26 smithi169 ceph-mds[63991]: -- 172.21.15.169:0/269673046 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557dba375a40 con 0x557dba3bd600 2024-05-09T04:59:26.368 INFO:journalctl@ceph.mds.k.smithi169.stdout:May 09 04:59:26 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-k[63987]: starting mds.k at 2024-05-09T04:59:26.368 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:25 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/3228031474' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T04:59:26.368 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:25 smithi169 ceph-mon[34075]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T04:59:26.369 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:25 smithi169 ceph-mon[34075]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T04:59:26.369 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:25 smithi169 ceph-mon[34075]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T04:59:26.369 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:25 smithi169 ceph-mon[34075]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-09T04:59:26.369 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:25 smithi169 ceph-mon[34075]: osdmap e90: 12 total, 12 up, 12 in 2024-05-09T04:59:26.369 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:25 smithi169 ceph-mon[34075]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-09T04:59:26.369 INFO:journalctl@ceph.mds.e.smithi169.stdout:May 09 04:59:26 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-e[64307]: ignoring --setuser ceph since I am not root 2024-05-09T04:59:26.369 INFO:journalctl@ceph.mds.e.smithi169.stdout:May 09 04:59:26 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-e[64307]: ignoring --setgroup ceph since I am not root 2024-05-09T04:59:26.369 INFO:journalctl@ceph.mds.e.smithi169.stdout:May 09 04:59:26 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-e[64307]: starting mds.e at 2024-05-09T04:59:26.964 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-05-09T04:59:27.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:26 smithi006 ceph-mon[29835]: pgmap v634: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-05-09T04:59:27.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:26 smithi186 ceph-mon[34668]: pgmap v634: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-05-09T04:59:27.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:26 smithi169 ceph-mon[34075]: pgmap v634: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-05-09T04:59:27.427 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:59:28.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:27 smithi186 ceph-mon[34668]: mds.? [v2:172.21.15.169:6833/3648551190,v1:172.21.15.169:6835/3648551190] up:boot 2024-05-09T04:59:28.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:27 smithi186 ceph-mon[34668]: mds.? [v2:172.21.15.169:6832/593809479,v1:172.21.15.169:6834/593809479] up:boot 2024-05-09T04:59:28.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:27 smithi186 ceph-mon[34668]: mds.? [v2:172.21.15.186:6832/1391829290,v1:172.21.15.186:6833/1391829290] up:boot 2024-05-09T04:59:28.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:27 smithi186 ceph-mon[34668]: mds.? [v2:172.21.15.186:6834/2712905503,v1:172.21.15.186:6835/2712905503] up:boot 2024-05-09T04:59:28.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:27 smithi186 ceph-mon[34668]: mds.? [v2:172.21.15.169:6837/2683312927,v1:172.21.15.169:6839/2683312927] up:boot 2024-05-09T04:59:28.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:27 smithi186 ceph-mon[34668]: mds.? [v2:172.21.15.169:6836/1831635402,v1:172.21.15.169:6838/1831635402] up:boot 2024-05-09T04:59:28.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:27 smithi186 ceph-mon[34668]: mds.? [v2:172.21.15.186:6836/575376126,v1:172.21.15.186:6837/575376126] up:boot 2024-05-09T04:59:28.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:27 smithi186 ceph-mon[34668]: mds.? [v2:172.21.15.186:6838/1036203452,v1:172.21.15.186:6839/1036203452] up:boot 2024-05-09T04:59:28.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:27 smithi186 ceph-mon[34668]: mds.? [v2:172.21.15.6:6836/298533882,v1:172.21.15.6:6837/298533882] up:boot 2024-05-09T04:59:28.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:27 smithi186 ceph-mon[34668]: mds.? [v2:172.21.15.6:6838/976553318,v1:172.21.15.6:6839/976553318] up:boot 2024-05-09T04:59:28.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:27 smithi186 ceph-mon[34668]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-09T04:59:28.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:27 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T04:59:28.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:27 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T04:59:28.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:27 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T04:59:28.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:27 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T04:59:28.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:27 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T04:59:28.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:27 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T04:59:28.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:27 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T04:59:28.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:27 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T04:59:28.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:27 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T04:59:28.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:27 smithi186 ceph-mon[34668]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T04:59:28.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:27 smithi169 ceph-mon[34075]: mds.? [v2:172.21.15.169:6833/3648551190,v1:172.21.15.169:6835/3648551190] up:boot 2024-05-09T04:59:28.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:27 smithi169 ceph-mon[34075]: mds.? [v2:172.21.15.169:6832/593809479,v1:172.21.15.169:6834/593809479] up:boot 2024-05-09T04:59:28.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:27 smithi169 ceph-mon[34075]: mds.? [v2:172.21.15.186:6832/1391829290,v1:172.21.15.186:6833/1391829290] up:boot 2024-05-09T04:59:28.368 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:27 smithi169 ceph-mon[34075]: mds.? [v2:172.21.15.186:6834/2712905503,v1:172.21.15.186:6835/2712905503] up:boot 2024-05-09T04:59:28.368 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:27 smithi169 ceph-mon[34075]: mds.? [v2:172.21.15.169:6837/2683312927,v1:172.21.15.169:6839/2683312927] up:boot 2024-05-09T04:59:28.368 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:27 smithi169 ceph-mon[34075]: mds.? [v2:172.21.15.169:6836/1831635402,v1:172.21.15.169:6838/1831635402] up:boot 2024-05-09T04:59:28.368 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:27 smithi169 ceph-mon[34075]: mds.? [v2:172.21.15.186:6836/575376126,v1:172.21.15.186:6837/575376126] up:boot 2024-05-09T04:59:28.368 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:27 smithi169 ceph-mon[34075]: mds.? [v2:172.21.15.186:6838/1036203452,v1:172.21.15.186:6839/1036203452] up:boot 2024-05-09T04:59:28.368 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:27 smithi169 ceph-mon[34075]: mds.? [v2:172.21.15.6:6836/298533882,v1:172.21.15.6:6837/298533882] up:boot 2024-05-09T04:59:28.368 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:27 smithi169 ceph-mon[34075]: mds.? [v2:172.21.15.6:6838/976553318,v1:172.21.15.6:6839/976553318] up:boot 2024-05-09T04:59:28.368 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:27 smithi169 ceph-mon[34075]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-09T04:59:28.368 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:27 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T04:59:28.368 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:27 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T04:59:28.368 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:27 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T04:59:28.368 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:27 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T04:59:28.368 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:27 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T04:59:28.368 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:27 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T04:59:28.368 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:27 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T04:59:28.368 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:27 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T04:59:28.368 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:27 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T04:59:28.369 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:27 smithi169 ceph-mon[34075]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T04:59:28.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:27 smithi006 ceph-mon[29835]: mds.? [v2:172.21.15.169:6833/3648551190,v1:172.21.15.169:6835/3648551190] up:boot 2024-05-09T04:59:28.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:27 smithi006 ceph-mon[29835]: mds.? [v2:172.21.15.169:6832/593809479,v1:172.21.15.169:6834/593809479] up:boot 2024-05-09T04:59:28.450 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:27 smithi006 ceph-mon[29835]: mds.? [v2:172.21.15.186:6832/1391829290,v1:172.21.15.186:6833/1391829290] up:boot 2024-05-09T04:59:28.450 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:27 smithi006 ceph-mon[29835]: mds.? [v2:172.21.15.186:6834/2712905503,v1:172.21.15.186:6835/2712905503] up:boot 2024-05-09T04:59:28.450 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:27 smithi006 ceph-mon[29835]: mds.? [v2:172.21.15.169:6837/2683312927,v1:172.21.15.169:6839/2683312927] up:boot 2024-05-09T04:59:28.450 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:27 smithi006 ceph-mon[29835]: mds.? [v2:172.21.15.169:6836/1831635402,v1:172.21.15.169:6838/1831635402] up:boot 2024-05-09T04:59:28.450 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:27 smithi006 ceph-mon[29835]: mds.? [v2:172.21.15.186:6836/575376126,v1:172.21.15.186:6837/575376126] up:boot 2024-05-09T04:59:28.450 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:27 smithi006 ceph-mon[29835]: mds.? [v2:172.21.15.186:6838/1036203452,v1:172.21.15.186:6839/1036203452] up:boot 2024-05-09T04:59:28.450 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:27 smithi006 ceph-mon[29835]: mds.? [v2:172.21.15.6:6836/298533882,v1:172.21.15.6:6837/298533882] up:boot 2024-05-09T04:59:28.450 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:27 smithi006 ceph-mon[29835]: mds.? [v2:172.21.15.6:6838/976553318,v1:172.21.15.6:6839/976553318] up:boot 2024-05-09T04:59:28.450 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:27 smithi006 ceph-mon[29835]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-09T04:59:28.450 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:27 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T04:59:28.450 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:27 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T04:59:28.450 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:27 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T04:59:28.450 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:27 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T04:59:28.450 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:27 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T04:59:28.451 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:27 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T04:59:28.451 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:27 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T04:59:28.451 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:27 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T04:59:28.451 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:27 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T04:59:28.451 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:27 smithi006 ceph-mon[29835]: from='mgr.14152 172.21.15.6:0/4267434870' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T04:59:29.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:29 smithi006 ceph-mon[29835]: pgmap v636: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:59:29.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:29 smithi186 ceph-mon[34668]: pgmap v636: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:59:29.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:29 smithi169 ceph-mon[34075]: pgmap v636: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T04:59:29.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.852+0000 7f173b000700 1 -- 172.21.15.6:0/4254662117 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1734106620 msgr2=0x7f1734106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:29.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.852+0000 7f173b000700 1 --2- 172.21.15.6:0/4254662117 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1734106620 0x7f1734106a30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1728009920 tx=0x7f1728009c30 comp rx=0 tx=0).stop 2024-05-09T04:59:29.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.852+0000 7f173b000700 1 -- 172.21.15.6:0/4254662117 shutdown_connections 2024-05-09T04:59:29.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.852+0000 7f173b000700 1 --2- 172.21.15.6:0/4254662117 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f173410b8a0 0x7f1734113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:29.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.852+0000 7f173b000700 1 --2- 172.21.15.6:0/4254662117 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1734106620 0x7f1734106a30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:29.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.852+0000 7f173b000700 1 --2- 172.21.15.6:0/4254662117 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f173410af50 0x7f173410b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:29.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.852+0000 7f173b000700 1 -- 172.21.15.6:0/4254662117 >> 172.21.15.6:0/4254662117 conn(0x7f1734102030 msgr2=0x7f1734104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:29.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.852+0000 7f173b000700 1 -- 172.21.15.6:0/4254662117 shutdown_connections 2024-05-09T04:59:29.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.853+0000 7f173b000700 1 -- 172.21.15.6:0/4254662117 wait complete. 2024-05-09T04:59:29.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.854+0000 7f173b000700 1 Processor -- start 2024-05-09T04:59:29.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.854+0000 7f173b000700 1 -- start start 2024-05-09T04:59:29.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.855+0000 7f173b000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1734106620 0x7f173419cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:29.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.855+0000 7f173b000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f173410af50 0x7f173419d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:29.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.855+0000 7f173b000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f173410b8a0 0x7f17341a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:29.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.855+0000 7f173b000700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1734116540 con 0x7f173410af50 2024-05-09T04:59:29.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.855+0000 7f173b000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f17341163c0 con 0x7f173410b8a0 2024-05-09T04:59:29.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.855+0000 7f173b000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f17341166c0 con 0x7f1734106620 2024-05-09T04:59:29.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.855+0000 7f1739c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1734106620 0x7f173419cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:29.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.855+0000 7f1739200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f173410af50 0x7f173419d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:29.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.855+0000 7f1739c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1734106620 0x7f173419cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:55634/0 (socket says 172.21.15.6:55634) 2024-05-09T04:59:29.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.855+0000 7f1739c00700 1 -- 172.21.15.6:0/1030730296 learned_addr learned my addr 172.21.15.6:0/1030730296 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:59:29.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.856+0000 7f173a600700 1 --2- 172.21.15.6:0/1030730296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f173410b8a0 0x7f17341a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:29.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.856+0000 7f1739200700 1 -- 172.21.15.6:0/1030730296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1734106620 msgr2=0x7f173419cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:29.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.856+0000 7f1739200700 1 --2- 172.21.15.6:0/1030730296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1734106620 0x7f173419cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:29.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.856+0000 7f1739200700 1 -- 172.21.15.6:0/1030730296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f173410b8a0 msgr2=0x7f17341a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:29.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.856+0000 7f1739200700 1 --2- 172.21.15.6:0/1030730296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f173410b8a0 0x7f17341a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:29.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.856+0000 7f1739200700 1 -- 172.21.15.6:0/1030730296 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17341a1e20 con 0x7f173410af50 2024-05-09T04:59:29.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.856+0000 7f1739200700 1 --2- 172.21.15.6:0/1030730296 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f173410af50 0x7f173419d4f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f172000e9c0 tx=0x7f172000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:29.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.857+0000 7f1732000700 1 -- 172.21.15.6:0/1030730296 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1720004220 con 0x7f173410af50 2024-05-09T04:59:29.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.857+0000 7f1732000700 1 -- 172.21.15.6:0/1030730296 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17200043c0 con 0x7f173410af50 2024-05-09T04:59:29.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.857+0000 7f1732000700 1 -- 172.21.15.6:0/1030730296 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1720005020 con 0x7f173410af50 2024-05-09T04:59:29.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.857+0000 7f173b000700 1 -- 172.21.15.6:0/1030730296 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17341a2110 con 0x7f173410af50 2024-05-09T04:59:29.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.857+0000 7f173b000700 1 -- 172.21.15.6:0/1030730296 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17341a9a30 con 0x7f173410af50 2024-05-09T04:59:29.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.859+0000 7f173b000700 1 -- 172.21.15.6:0/1030730296 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17340512c0 con 0x7f173410af50 2024-05-09T04:59:29.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.861+0000 7f1732000700 1 -- 172.21.15.6:0/1030730296 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f1720004750 con 0x7f173410af50 2024-05-09T04:59:29.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.863+0000 7f1732000700 1 --2- 172.21.15.6:0/1030730296 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f171c0b2880 0x7f171c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:29.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.863+0000 7f1732000700 1 -- 172.21.15.6:0/1030730296 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9559+0+0 (secure 0 0 0) 0x7f17200e26e0 con 0x7f173410af50 2024-05-09T04:59:29.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.864+0000 7f1739c00700 1 --2- 172.21.15.6:0/1030730296 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f171c0b2880 0x7f171c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:29.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.864+0000 7f1739c00700 1 --2- 172.21.15.6:0/1030730296 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f171c0b2880 0x7f171c0b4d90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f17280098f0 tx=0x7f1728019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:29.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:29.869+0000 7f1732000700 1 -- 172.21.15.6:0/1030730296 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f17200a9e70 con 0x7f173410af50 2024-05-09T04:59:30.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:30.163+0000 7f173b000700 1 -- 172.21.15.6:0/1030730296 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f173419e2b0 con 0x7f173410af50 2024-05-09T04:59:31.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:31.013+0000 7f1732000700 1 -- 172.21.15.6:0/1030730296 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v33) ==== 104+0+0 (secure 0 0 0) 0x7f17200a9770 con 0x7f173410af50 2024-05-09T04:59:31.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:31.018+0000 7f173b000700 1 -- 172.21.15.6:0/1030730296 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f171c0b2880 msgr2=0x7f171c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:31.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:31.018+0000 7f173b000700 1 --2- 172.21.15.6:0/1030730296 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f171c0b2880 0x7f171c0b4d90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f17280098f0 tx=0x7f1728019040 comp rx=0 tx=0).stop 2024-05-09T04:59:31.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:31.018+0000 7f173b000700 1 -- 172.21.15.6:0/1030730296 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f173410af50 msgr2=0x7f173419d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:31.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:31.018+0000 7f173b000700 1 --2- 172.21.15.6:0/1030730296 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f173410af50 0x7f173419d4f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f172000e9c0 tx=0x7f172000ecd0 comp rx=0 tx=0).stop 2024-05-09T04:59:31.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:31.018+0000 7f173b000700 1 -- 172.21.15.6:0/1030730296 shutdown_connections 2024-05-09T04:59:31.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:31.018+0000 7f173b000700 1 --2- 172.21.15.6:0/1030730296 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f171c0b2880 0x7f171c0b4d90 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:31.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:31.018+0000 7f173b000700 1 --2- 172.21.15.6:0/1030730296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f173410b8a0 0x7f17341a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:31.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:31.018+0000 7f173b000700 1 --2- 172.21.15.6:0/1030730296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1734106620 0x7f173419cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:31.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:31.018+0000 7f173b000700 1 --2- 172.21.15.6:0/1030730296 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f173410af50 0x7f173419d4f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:31.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:31.018+0000 7f173b000700 1 -- 172.21.15.6:0/1030730296 >> 172.21.15.6:0/1030730296 conn(0x7f1734102030 msgr2=0x7f173410a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:31.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:31.019+0000 7f173b000700 1 -- 172.21.15.6:0/1030730296 shutdown_connections 2024-05-09T04:59:31.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:31.019+0000 7f173b000700 1 -- 172.21.15.6:0/1030730296 wait complete. 2024-05-09T04:59:31.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:31 smithi186 ceph-mon[34668]: pgmap v637: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T04:59:31.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:31 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/1030730296' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T04:59:31.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:31 smithi169 ceph-mon[34075]: pgmap v637: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T04:59:31.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:31 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/1030730296' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T04:59:31.425 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:31 smithi006 ceph-mon[29835]: pgmap v637: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T04:59:31.425 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:31 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1030730296' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T04:59:31.889 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-09T04:59:32.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:32 smithi006 ceph-mon[29835]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T04:59:32.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:32 smithi006 ceph-mon[29835]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T04:59:32.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:32 smithi006 ceph-mon[29835]: Cluster is now healthy 2024-05-09T04:59:32.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:32 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1030730296' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-09T04:59:32.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:32 smithi006 ceph-mon[29835]: fsmap 12 up:standby 2024-05-09T04:59:32.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:32 smithi186 ceph-mon[34668]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T04:59:32.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:32 smithi186 ceph-mon[34668]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T04:59:32.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:32 smithi186 ceph-mon[34668]: Cluster is now healthy 2024-05-09T04:59:32.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:32 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/1030730296' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-09T04:59:32.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:32 smithi186 ceph-mon[34668]: fsmap 12 up:standby 2024-05-09T04:59:32.363 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:59:32.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:32 smithi169 ceph-mon[34075]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T04:59:32.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:32 smithi169 ceph-mon[34075]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T04:59:32.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:32 smithi169 ceph-mon[34075]: Cluster is now healthy 2024-05-09T04:59:32.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:32 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/1030730296' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-09T04:59:32.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:32 smithi169 ceph-mon[34075]: fsmap 12 up:standby 2024-05-09T04:59:33.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:33 smithi186 ceph-mon[34668]: pgmap v638: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-09T04:59:33.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:33 smithi169 ceph-mon[34075]: pgmap v638: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-09T04:59:33.442 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:33 smithi006 ceph-mon[29835]: pgmap v638: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-09T04:59:34.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.699+0000 7f49be400700 1 -- 172.21.15.6:0/2006014269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49b810af60 msgr2=0x7f49b810b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:34.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.699+0000 7f49be400700 1 --2- 172.21.15.6:0/2006014269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49b810af60 0x7f49b810b370 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f49a40098f0 tx=0x7f49a4009c00 comp rx=0 tx=0).stop 2024-05-09T04:59:34.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.699+0000 7f49be400700 1 -- 172.21.15.6:0/2006014269 shutdown_connections 2024-05-09T04:59:34.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.699+0000 7f49be400700 1 --2- 172.21.15.6:0/2006014269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49b810b8b0 0x7f49b8113210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:34.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.699+0000 7f49be400700 1 --2- 172.21.15.6:0/2006014269 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49b8106630 0x7f49b8106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:34.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.699+0000 7f49be400700 1 --2- 172.21.15.6:0/2006014269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49b810af60 0x7f49b810b370 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:34.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.699+0000 7f49be400700 1 -- 172.21.15.6:0/2006014269 >> 172.21.15.6:0/2006014269 conn(0x7f49b8101ed0 msgr2=0x7f49b81042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:34.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.699+0000 7f49be400700 1 -- 172.21.15.6:0/2006014269 shutdown_connections 2024-05-09T04:59:34.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.699+0000 7f49be400700 1 -- 172.21.15.6:0/2006014269 wait complete. 2024-05-09T04:59:34.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.701+0000 7f49be400700 1 Processor -- start 2024-05-09T04:59:34.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.701+0000 7f49be400700 1 -- start start 2024-05-09T04:59:34.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.701+0000 7f49be400700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49b8106630 0x7f49b819cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:34.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.701+0000 7f49be400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49b810af60 0x7f49b819d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:34.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.701+0000 7f49be400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49b810b8b0 0x7f49b81a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:34.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.701+0000 7f49be400700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7f49b8116510 con 0x7f49b8106630 2024-05-09T04:59:34.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.701+0000 7f49be400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f49b8116390 con 0x7f49b810b8b0 2024-05-09T04:59:34.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.701+0000 7f49be400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f49b8116690 con 0x7f49b810af60 2024-05-09T04:59:34.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.702+0000 7f49bd000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49b8106630 0x7f49b819cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:34.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.702+0000 7f49bd000700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49b8106630 0x7f49b819cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58154/0 (socket says 172.21.15.6:58154) 2024-05-09T04:59:34.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.702+0000 7f49bd000700 1 -- 172.21.15.6:0/3696891429 learned_addr learned my addr 172.21.15.6:0/3696891429 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:59:34.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.702+0000 7f49b7e00700 1 --2- 172.21.15.6:0/3696891429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49b810af60 0x7f49b819d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:34.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.702+0000 7f49bda00700 1 --2- 172.21.15.6:0/3696891429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49b810b8b0 0x7f49b81a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:34.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.702+0000 7f49b7e00700 1 -- 172.21.15.6:0/3696891429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49b810b8b0 msgr2=0x7f49b81a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:34.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.702+0000 7f49b7e00700 1 --2- 172.21.15.6:0/3696891429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49b810b8b0 0x7f49b81a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:34.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.702+0000 7f49b7e00700 1 -- 172.21.15.6:0/3696891429 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49b8106630 msgr2=0x7f49b819cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:34.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.702+0000 7f49b7e00700 1 --2- 172.21.15.6:0/3696891429 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49b8106630 0x7f49b819cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:34.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.702+0000 7f49b7e00700 1 -- 172.21.15.6:0/3696891429 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b81a1db0 con 0x7f49b810af60 2024-05-09T04:59:34.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.703+0000 7f49b7e00700 1 --2- 172.21.15.6:0/3696891429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49b810af60 0x7f49b819d480 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f49a4000c00 tx=0x7f49a400fa20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:34.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.703+0000 7f49b5600700 1 -- 172.21.15.6:0/3696891429 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a40050c0 con 0x7f49b810af60 2024-05-09T04:59:34.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.703+0000 7f49be400700 1 -- 172.21.15.6:0/3696891429 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49b81a2040 con 0x7f49b810af60 2024-05-09T04:59:34.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.703+0000 7f49be400700 1 -- 172.21.15.6:0/3696891429 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49b81a2540 con 0x7f49b810af60 2024-05-09T04:59:34.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.703+0000 7f49b5600700 1 -- 172.21.15.6:0/3696891429 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49a400b560 con 0x7f49b810af60 2024-05-09T04:59:34.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.703+0000 7f49b5600700 1 -- 172.21.15.6:0/3696891429 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a40038c0 con 0x7f49b810af60 2024-05-09T04:59:34.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.704+0000 7f49be400700 1 -- 172.21.15.6:0/3696891429 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49b80512c0 con 0x7f49b810af60 2024-05-09T04:59:34.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.707+0000 7f49b5600700 1 -- 172.21.15.6:0/3696891429 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f49a400b9a0 con 0x7f49b810af60 2024-05-09T04:59:34.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.708+0000 7f49b5600700 1 --2- 172.21.15.6:0/3696891429 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f49a00b2900 0x7f49a00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:34.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.709+0000 7f49b5600700 1 -- 172.21.15.6:0/3696891429 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9559+0+0 (secure 0 0 0) 0x7f49a40e3a60 con 0x7f49b810af60 2024-05-09T04:59:34.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.709+0000 7f49bd000700 1 --2- 172.21.15.6:0/3696891429 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f49a00b2900 0x7f49a00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:34.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.710+0000 7f49bd000700 1 --2- 172.21.15.6:0/3696891429 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f49a00b2900 0x7f49a00b4e10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f49ac005950 tx=0x7f49ac009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:34.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.711+0000 7f49b5600700 1 -- 172.21.15.6:0/3696891429 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f49b81a2540 con 0x7f49b810af60 2024-05-09T04:59:34.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:34.948+0000 7f49be400700 1 -- 172.21.15.6:0/3696891429 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f49b802e5f0 con 0x7f49b810af60 2024-05-09T04:59:35.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:35.056+0000 7f49b5600700 1 -- 172.21.15.6:0/3696891429 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v91) ==== 181+0+0 (secure 0 0 0) 0x7f49a4013030 con 0x7f49b810af60 2024-05-09T04:59:35.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:35.062+0000 7f49be400700 1 -- 172.21.15.6:0/3696891429 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f49a00b2900 msgr2=0x7f49a00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:35.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:35.062+0000 7f49be400700 1 --2- 172.21.15.6:0/3696891429 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f49a00b2900 0x7f49a00b4e10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f49ac005950 tx=0x7f49ac009310 comp rx=0 tx=0).stop 2024-05-09T04:59:35.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:35.062+0000 7f49be400700 1 -- 172.21.15.6:0/3696891429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49b810af60 msgr2=0x7f49b819d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:35.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:35.062+0000 7f49be400700 1 --2- 172.21.15.6:0/3696891429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49b810af60 0x7f49b819d480 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f49a4000c00 tx=0x7f49a400fa20 comp rx=0 tx=0).stop 2024-05-09T04:59:35.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:35.063+0000 7f49be400700 1 -- 172.21.15.6:0/3696891429 shutdown_connections 2024-05-09T04:59:35.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:35.063+0000 7f49be400700 1 --2- 172.21.15.6:0/3696891429 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7f49a00b2900 0x7f49a00b4e10 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:35.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:35.063+0000 7f49be400700 1 --2- 172.21.15.6:0/3696891429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49b810b8b0 0x7f49b81a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:35.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:35.063+0000 7f49be400700 1 --2- 172.21.15.6:0/3696891429 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f49b8106630 0x7f49b819cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:35.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:35.063+0000 7f49be400700 1 --2- 172.21.15.6:0/3696891429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49b810af60 0x7f49b819d480 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:35.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:35.063+0000 7f49be400700 1 -- 172.21.15.6:0/3696891429 >> 172.21.15.6:0/3696891429 conn(0x7f49b8101ed0 msgr2=0x7f49b81025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:35.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:35.064+0000 7f49be400700 1 -- 172.21.15.6:0/3696891429 shutdown_connections 2024-05-09T04:59:35.065 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:35.064+0000 7f49be400700 1 -- 172.21.15.6:0/3696891429 wait complete. 2024-05-09T04:59:35.067 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_metadata' removed 2024-05-09T04:59:35.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:35 smithi186 ceph-mon[34668]: pgmap v639: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-09T04:59:35.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:35 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/3696891429' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T04:59:35.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:35 smithi186 ceph-mon[34668]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T04:59:35.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:35 smithi169 ceph-mon[34075]: pgmap v639: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-09T04:59:35.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:35 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/3696891429' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T04:59:35.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:35 smithi169 ceph-mon[34075]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T04:59:35.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:35 smithi006 ceph-mon[29835]: pgmap v639: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-09T04:59:35.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:35 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/3696891429' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T04:59:35.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:35 smithi006 ceph-mon[29835]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T04:59:35.948 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-09T04:59:36.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:36 smithi006 ceph-mon[29835]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T04:59:36.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:36 smithi006 ceph-mon[29835]: osdmap e91: 12 total, 12 up, 12 in 2024-05-09T04:59:36.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:36 smithi006 ceph-mon[29835]: pgmap v641: 97 pgs: 97 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:59:36.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:36 smithi186 ceph-mon[34668]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T04:59:36.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:36 smithi186 ceph-mon[34668]: osdmap e91: 12 total, 12 up, 12 in 2024-05-09T04:59:36.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:36 smithi186 ceph-mon[34668]: pgmap v641: 97 pgs: 97 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:59:36.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:36 smithi169 ceph-mon[34075]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T04:59:36.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:36 smithi169 ceph-mon[34075]: osdmap e91: 12 total, 12 up, 12 in 2024-05-09T04:59:36.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:36 smithi169 ceph-mon[34075]: pgmap v641: 97 pgs: 97 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:59:36.418 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:59:38.594 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:38 smithi006 ceph-mon[29835]: pgmap v642: 97 pgs: 97 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:59:38.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:38 smithi186 ceph-mon[34668]: pgmap v642: 97 pgs: 97 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:59:38.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:38 smithi169 ceph-mon[34075]: pgmap v642: 97 pgs: 97 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:59:39.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.007+0000 7ff288800700 1 -- 172.21.15.6:0/1625815325 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2801034c0 msgr2=0x7ff2801038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:39.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.007+0000 7ff288800700 1 --2- 172.21.15.6:0/1625815325 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2801034c0 0x7ff2801038d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff2740099c0 tx=0x7ff274009cd0 comp rx=0 tx=0).stop 2024-05-09T04:59:39.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.007+0000 7ff288800700 1 -- 172.21.15.6:0/1625815325 shutdown_connections 2024-05-09T04:59:39.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.007+0000 7ff288800700 1 --2- 172.21.15.6:0/1625815325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff280108740 0x7ff2801100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:39.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.007+0000 7ff288800700 1 --2- 172.21.15.6:0/1625815325 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2801034c0 0x7ff2801038d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:39.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.007+0000 7ff288800700 1 --2- 172.21.15.6:0/1625815325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff280107df0 0x7ff280108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:39.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.007+0000 7ff288800700 1 -- 172.21.15.6:0/1625815325 >> 172.21.15.6:0/1625815325 conn(0x7ff2800fedc0 msgr2=0x7ff2801011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:39.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.008+0000 7ff288800700 1 -- 172.21.15.6:0/1625815325 shutdown_connections 2024-05-09T04:59:39.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.008+0000 7ff288800700 1 -- 172.21.15.6:0/1625815325 wait complete. 2024-05-09T04:59:39.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.009+0000 7ff288800700 1 Processor -- start 2024-05-09T04:59:39.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.009+0000 7ff288800700 1 -- start start 2024-05-09T04:59:39.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.010+0000 7ff288800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2801034c0 0x7ff2800ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:39.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.010+0000 7ff288800700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff280107df0 0x7ff2800ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:39.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.010+0000 7ff288800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff280108740 0x7ff2800ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:39.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.010+0000 7ff288800700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801133c0 con 0x7ff280107df0 2024-05-09T04:59:39.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.010+0000 7ff288800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff280113240 con 0x7ff280108740 2024-05-09T04:59:39.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.010+0000 7ff288800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff280113540 con 0x7ff2801034c0 2024-05-09T04:59:39.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.010+0000 7ff285e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2801034c0 0x7ff2800ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:39.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.010+0000 7ff285e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2801034c0 0x7ff2800ff2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:58002/0 (socket says 172.21.15.6:58002) 2024-05-09T04:59:39.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.010+0000 7ff285e00700 1 -- 172.21.15.6:0/1927953167 learned_addr learned my addr 172.21.15.6:0/1927953167 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:59:39.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.010+0000 7ff285400700 1 --2- 172.21.15.6:0/1927953167 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff280107df0 0x7ff2800ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:39.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.010+0000 7ff286800700 1 --2- 172.21.15.6:0/1927953167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff280108740 0x7ff2800ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:39.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.011+0000 7ff285e00700 1 -- 172.21.15.6:0/1927953167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff280108740 msgr2=0x7ff2800ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:39.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.011+0000 7ff285e00700 1 --2- 172.21.15.6:0/1927953167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff280108740 0x7ff2800ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:39.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.011+0000 7ff285e00700 1 -- 172.21.15.6:0/1927953167 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff280107df0 msgr2=0x7ff2800ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:39.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.011+0000 7ff285e00700 1 --2- 172.21.15.6:0/1927953167 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff280107df0 0x7ff2800ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:39.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.011+0000 7ff285e00700 1 -- 172.21.15.6:0/1927953167 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2801a91e0 con 0x7ff2801034c0 2024-05-09T04:59:39.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.011+0000 7ff285400700 1 --2- 172.21.15.6:0/1927953167 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff280107df0 0x7ff2800ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:59:39.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.011+0000 7ff285e00700 1 --2- 172.21.15.6:0/1927953167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2801034c0 0x7ff2800ff2a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff274006010 tx=0x7ff274005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:39.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.012+0000 7ff27aa00700 1 -- 172.21.15.6:0/1927953167 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff274026070 con 0x7ff2801034c0 2024-05-09T04:59:39.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.012+0000 7ff27aa00700 1 -- 172.21.15.6:0/1927953167 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff274005680 con 0x7ff2801034c0 2024-05-09T04:59:39.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.012+0000 7ff27aa00700 1 -- 172.21.15.6:0/1927953167 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27400fb20 con 0x7ff2801034c0 2024-05-09T04:59:39.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.012+0000 7ff288800700 1 -- 172.21.15.6:0/1927953167 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2801a9470 con 0x7ff2801034c0 2024-05-09T04:59:39.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.012+0000 7ff288800700 1 -- 172.21.15.6:0/1927953167 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2801a9870 con 0x7ff2801034c0 2024-05-09T04:59:39.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.013+0000 7ff288800700 1 -- 172.21.15.6:0/1927953167 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2800512c0 con 0x7ff2801034c0 2024-05-09T04:59:39.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.016+0000 7ff27aa00700 1 -- 172.21.15.6:0/1927953167 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7ff274029070 con 0x7ff2801034c0 2024-05-09T04:59:39.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.018+0000 7ff27aa00700 1 --2- 172.21.15.6:0/1927953167 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff2680b2900 0x7ff2680b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:39.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.018+0000 7ff27aa00700 1 -- 172.21.15.6:0/1927953167 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9026+0+0 (secure 0 0 0) 0x7ff2740e3830 con 0x7ff2801034c0 2024-05-09T04:59:39.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.018+0000 7ff285400700 1 --2- 172.21.15.6:0/1927953167 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff2680b2900 0x7ff2680b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:39.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.019+0000 7ff285400700 1 --2- 172.21.15.6:0/1927953167 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff2680b2900 0x7ff2680b4e10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff280109a80 tx=0x7ff270009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:39.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.020+0000 7ff27aa00700 1 -- 172.21.15.6:0/1927953167 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff274013070 con 0x7ff2801034c0 2024-05-09T04:59:39.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.257+0000 7ff288800700 1 -- 172.21.15.6:0/1927953167 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff28002e5f0 con 0x7ff2801034c0 2024-05-09T04:59:39.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.280+0000 7ff27aa00700 1 -- 172.21.15.6:0/1927953167 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v92) ==== 169+0+0 (secure 0 0 0) 0x7ff2740ab260 con 0x7ff2801034c0 2024-05-09T04:59:39.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.284+0000 7ff288800700 1 -- 172.21.15.6:0/1927953167 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff2680b2900 msgr2=0x7ff2680b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:39.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.284+0000 7ff288800700 1 --2- 172.21.15.6:0/1927953167 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff2680b2900 0x7ff2680b4e10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff280109a80 tx=0x7ff270009260 comp rx=0 tx=0).stop 2024-05-09T04:59:39.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.284+0000 7ff288800700 1 -- 172.21.15.6:0/1927953167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2801034c0 msgr2=0x7ff2800ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:39.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.284+0000 7ff288800700 1 --2- 172.21.15.6:0/1927953167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2801034c0 0x7ff2800ff2a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff274006010 tx=0x7ff274005280 comp rx=0 tx=0).stop 2024-05-09T04:59:39.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.285+0000 7ff288800700 1 -- 172.21.15.6:0/1927953167 shutdown_connections 2024-05-09T04:59:39.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.285+0000 7ff288800700 1 --2- 172.21.15.6:0/1927953167 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7ff2680b2900 0x7ff2680b4e10 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:39.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.285+0000 7ff288800700 1 --2- 172.21.15.6:0/1927953167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff280108740 0x7ff2800ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:39.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.285+0000 7ff288800700 1 --2- 172.21.15.6:0/1927953167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2801034c0 0x7ff2800ff2a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:39.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.285+0000 7ff288800700 1 --2- 172.21.15.6:0/1927953167 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff280107df0 0x7ff2800ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:39.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.285+0000 7ff288800700 1 -- 172.21.15.6:0/1927953167 >> 172.21.15.6:0/1927953167 conn(0x7ff2800fedc0 msgr2=0x7ff28006b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:39.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.285+0000 7ff288800700 1 -- 172.21.15.6:0/1927953167 shutdown_connections 2024-05-09T04:59:39.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:39.285+0000 7ff288800700 1 -- 172.21.15.6:0/1927953167 wait complete. 2024-05-09T04:59:39.287 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_data' removed 2024-05-09T04:59:39.599 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:39 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/1927953167' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T04:59:39.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:39 smithi186 ceph-mon[34668]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T04:59:39.616 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:39 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/1927953167' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T04:59:39.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:39 smithi169 ceph-mon[34075]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T04:59:39.678 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:39 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/1927953167' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T04:59:39.678 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:39 smithi006 ceph-mon[29835]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T04:59:40.195 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-09T04:59:40.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:40 smithi186 ceph-mon[34668]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T04:59:40.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:40 smithi186 ceph-mon[34668]: osdmap e92: 12 total, 12 up, 12 in 2024-05-09T04:59:40.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:40 smithi186 ceph-mon[34668]: pgmap v644: 33 pgs: 33 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:59:40.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:40 smithi169 ceph-mon[34075]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T04:59:40.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:40 smithi169 ceph-mon[34075]: osdmap e92: 12 total, 12 up, 12 in 2024-05-09T04:59:40.617 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:40 smithi169 ceph-mon[34075]: pgmap v644: 33 pgs: 33 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:59:40.657 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:59:40.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:40 smithi006 ceph-mon[29835]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T04:59:40.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:40 smithi006 ceph-mon[29835]: osdmap e92: 12 total, 12 up, 12 in 2024-05-09T04:59:40.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:40 smithi006 ceph-mon[29835]: pgmap v644: 33 pgs: 33 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:59:42.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.934+0000 7fae5fc00700 1 -- 172.21.15.6:0/2519408957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae58110c00 msgr2=0x7fae58113020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:42.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.934+0000 7fae5fc00700 1 --2- 172.21.15.6:0/2519408957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae58110c00 0x7fae58113020 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fae5000b0a0 tx=0x7fae5000b3b0 comp rx=0 tx=0).stop 2024-05-09T04:59:42.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.934+0000 7fae5fc00700 1 -- 172.21.15.6:0/2519408957 shutdown_connections 2024-05-09T04:59:42.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.934+0000 7fae5fc00700 1 --2- 172.21.15.6:0/2519408957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae58110c00 0x7fae58113020 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:42.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.934+0000 7fae5fc00700 1 --2- 172.21.15.6:0/2519408957 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae58075b00 0x7fae58075f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:42.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.934+0000 7fae5fc00700 1 --2- 172.21.15.6:0/2519408957 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae58076450 0x7fae581106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:42.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.934+0000 7fae5fc00700 1 -- 172.21.15.6:0/2519408957 >> 172.21.15.6:0/2519408957 conn(0x7fae58100040 msgr2=0x7fae58102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:42.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.935+0000 7fae5fc00700 1 -- 172.21.15.6:0/2519408957 shutdown_connections 2024-05-09T04:59:42.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.935+0000 7fae5fc00700 1 -- 172.21.15.6:0/2519408957 wait complete. 2024-05-09T04:59:42.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.936+0000 7fae5fc00700 1 Processor -- start 2024-05-09T04:59:42.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.936+0000 7fae5fc00700 1 -- start start 2024-05-09T04:59:42.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.936+0000 7fae5fc00700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae58075b00 0x7fae5819d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:42.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.936+0000 7fae5fc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae58076450 0x7fae5819d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:42.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.937+0000 7fae5fc00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae58110c00 0x7fae581a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:42.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.937+0000 7fae5fc00700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 -- 0x7fae58116710 con 0x7fae58075b00 2024-05-09T04:59:42.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.937+0000 7fae5fc00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fae58116590 con 0x7fae58110c00 2024-05-09T04:59:42.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.937+0000 7fae5fc00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fae58116890 con 0x7fae58076450 2024-05-09T04:59:42.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.937+0000 7fae5d200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae58075b00 0x7fae5819d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:42.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.937+0000 7fae5d200700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae58075b00 0x7fae5819d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52470/0 (socket says 172.21.15.6:52470) 2024-05-09T04:59:42.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.937+0000 7fae57e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae58076450 0x7fae5819d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:42.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.937+0000 7fae5d200700 1 -- 172.21.15.6:0/3067082634 learned_addr learned my addr 172.21.15.6:0/3067082634 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-05-09T04:59:42.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.937+0000 7fae5dc00700 1 --2- 172.21.15.6:0/3067082634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae58110c00 0x7fae581a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:42.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.937+0000 7fae5d200700 1 -- 172.21.15.6:0/3067082634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae58076450 msgr2=0x7fae5819d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:42.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.937+0000 7fae5d200700 1 --2- 172.21.15.6:0/3067082634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae58076450 0x7fae5819d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:42.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.937+0000 7fae5d200700 1 -- 172.21.15.6:0/3067082634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae58110c00 msgr2=0x7fae581a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:42.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.937+0000 7fae5d200700 1 --2- 172.21.15.6:0/3067082634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae58110c00 0x7fae581a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:42.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.937+0000 7fae5d200700 1 -- 172.21.15.6:0/3067082634 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae581a1f30 con 0x7fae58075b00 2024-05-09T04:59:42.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.938+0000 7fae57e00700 1 --2- 172.21.15.6:0/3067082634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae58076450 0x7fae5819d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:59:42.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.938+0000 7fae5dc00700 1 --2- 172.21.15.6:0/3067082634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae58110c00 0x7fae581a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:59:42.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.938+0000 7fae5d200700 1 --2- 172.21.15.6:0/3067082634 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae58075b00 0x7fae5819d060 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fae4c00e9a0 tx=0x7fae4c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:42.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.938+0000 7fae55600700 1 -- 172.21.15.6:0/3067082634 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c00ca10 con 0x7fae58075b00 2024-05-09T04:59:42.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.938+0000 7fae5fc00700 1 -- 172.21.15.6:0/3067082634 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae581a2220 con 0x7fae58075b00 2024-05-09T04:59:42.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.938+0000 7fae55600700 1 -- 172.21.15.6:0/3067082634 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae4c00cbb0 con 0x7fae58075b00 2024-05-09T04:59:42.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.938+0000 7fae55600700 1 -- 172.21.15.6:0/3067082634 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c018680 con 0x7fae58075b00 2024-05-09T04:59:42.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.938+0000 7fae5fc00700 1 -- 172.21.15.6:0/3067082634 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae581a9b40 con 0x7fae58075b00 2024-05-09T04:59:42.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.940+0000 7fae5fc00700 1 -- 172.21.15.6:0/3067082634 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae580512c0 con 0x7fae58075b00 2024-05-09T04:59:42.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.942+0000 7fae55600700 1 -- 172.21.15.6:0/3067082634 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7fae4c010430 con 0x7fae58075b00 2024-05-09T04:59:42.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.944+0000 7fae55600700 1 --2- 172.21.15.6:0/3067082634 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fae400b2880 0x7fae400b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:42.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.944+0000 7fae55600700 1 -- 172.21.15.6:0/3067082634 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8496+0+0 (secure 0 0 0) 0x7fae4c0e2800 con 0x7fae58075b00 2024-05-09T04:59:42.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.944+0000 7fae57e00700 1 --2- 172.21.15.6:0/3067082634 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fae400b2880 0x7fae400b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:42.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.945+0000 7fae57e00700 1 --2- 172.21.15.6:0/3067082634 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fae400b2880 0x7fae400b4d90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fae581033f0 tx=0x7fae44009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:42.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:42.947+0000 7fae55600700 1 -- 172.21.15.6:0/3067082634 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fae4c0aa3e0 con 0x7fae58075b00 2024-05-09T04:59:43.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:43.189+0000 7fae5fc00700 1 -- 172.21.15.6:0/3067082634 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fae5802e5f0 con 0x7fae58075b00 2024-05-09T04:59:43.190 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:42 smithi006 ceph-mon[29835]: pgmap v645: 33 pgs: 33 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:59:43.349 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:42 smithi186 ceph-mon[34668]: pgmap v645: 33 pgs: 33 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:59:43.366 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:42 smithi169 ceph-mon[34075]: pgmap v645: 33 pgs: 33 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:59:43.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:43.940+0000 7fae55600700 1 -- 172.21.15.6:0/3067082634 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v93) ==== 178+0+0 (secure 0 0 0) 0x7fae4c0a9ce0 con 0x7fae58075b00 2024-05-09T04:59:43.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:43.947+0000 7fae5fc00700 1 -- 172.21.15.6:0/3067082634 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fae400b2880 msgr2=0x7fae400b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:43.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:43.947+0000 7fae5fc00700 1 --2- 172.21.15.6:0/3067082634 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fae400b2880 0x7fae400b4d90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fae581033f0 tx=0x7fae44009260 comp rx=0 tx=0).stop 2024-05-09T04:59:43.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:43.947+0000 7fae5fc00700 1 -- 172.21.15.6:0/3067082634 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae58075b00 msgr2=0x7fae5819d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:43.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:43.947+0000 7fae5fc00700 1 --2- 172.21.15.6:0/3067082634 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae58075b00 0x7fae5819d060 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fae4c00e9a0 tx=0x7fae4c00ecb0 comp rx=0 tx=0).stop 2024-05-09T04:59:43.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:43.948+0000 7fae5fc00700 1 -- 172.21.15.6:0/3067082634 shutdown_connections 2024-05-09T04:59:43.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:43.948+0000 7fae5fc00700 1 --2- 172.21.15.6:0/3067082634 >> [v2:172.21.15.6:6800/2997423326,v1:172.21.15.6:6801/2997423326] conn(0x7fae400b2880 0x7fae400b4d90 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:43.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:43.948+0000 7fae5fc00700 1 --2- 172.21.15.6:0/3067082634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae58110c00 0x7fae581a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:43.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:43.948+0000 7fae5fc00700 1 --2- 172.21.15.6:0/3067082634 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae58075b00 0x7fae5819d060 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:43.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:43.948+0000 7fae5fc00700 1 --2- 172.21.15.6:0/3067082634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae58076450 0x7fae5819d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:43.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:43.948+0000 7fae5fc00700 1 -- 172.21.15.6:0/3067082634 >> 172.21.15.6:0/3067082634 conn(0x7fae58100040 msgr2=0x7fae58100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:43.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:43.948+0000 7fae5fc00700 1 -- 172.21.15.6:0/3067082634 shutdown_connections 2024-05-09T04:59:43.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-09T04:59:43.948+0000 7fae5fc00700 1 -- 172.21.15.6:0/3067082634 wait complete. 2024-05-09T04:59:43.950 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_data_ec' removed 2024-05-09T04:59:44.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:43 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/3067082634' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T04:59:44.350 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:43 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/3067082634' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T04:59:44.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:43 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/3067082634' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T04:59:44.928 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T04:59:45.023 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T04:59:45.057 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T04:59:45.089 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-09T04:59:45.090 DEBUG:teuthology.orchestra.run.smithi006:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T04:59:45.150 DEBUG:teuthology.orchestra.run.smithi169:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T04:59:45.167 DEBUG:teuthology.orchestra.run.smithi186:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T04:59:45.184 INFO:tasks.cephadm:Stopping all daemons... 2024-05-09T04:59:45.185 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-09T04:59:45.185 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mon.a 2024-05-09T04:59:45.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:44 smithi006 ceph-mon[29835]: pgmap v646: 33 pgs: 33 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:59:45.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:44 smithi006 ceph-mon[29835]: from='client.? 172.21.15.6:0/3067082634' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T04:59:45.197 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:44 smithi006 ceph-mon[29835]: osdmap e93: 12 total, 12 up, 12 in 2024-05-09T04:59:45.251 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:44 smithi186 ceph-mon[34668]: pgmap v646: 33 pgs: 33 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:59:45.251 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:44 smithi186 ceph-mon[34668]: from='client.? 172.21.15.6:0/3067082634' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T04:59:45.251 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:44 smithi186 ceph-mon[34668]: osdmap e93: 12 total, 12 up, 12 in 2024-05-09T04:59:45.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:44 smithi169 ceph-mon[34075]: pgmap v646: 33 pgs: 33 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:59:45.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:44 smithi169 ceph-mon[34075]: from='client.? 172.21.15.6:0/3067082634' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T04:59:45.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:44 smithi169 ceph-mon[34075]: osdmap e93: 12 total, 12 up, 12 in 2024-05-09T04:59:45.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:45 smithi006 systemd[1]: Stopping Ceph mon.a for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T04:59:46.050 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:45 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mon-a[29831]: 2024-05-09T04:59:45.771+0000 7efff7a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T04:59:46.051 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:45 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mon-a[29831]: 2024-05-09T04:59:45.771+0000 7efff7a00700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-09T04:59:46.379 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:46 smithi006 podman[93982]: 2024-05-09 04:59:46.050619831 +0000 UTC m=+0.422418090 container died 34c0cb98e2cc80a5e82399c5c05feb7cf9f03b2681647ba61167705b326893ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T04:59:46.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:46 smithi006 podman[93982]: 2024-05-09 04:59:46.604326981 +0000 UTC m=+0.976125276 container cleanup 34c0cb98e2cc80a5e82399c5c05feb7cf9f03b2681647ba61167705b326893ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mon-a, org.label-schema.build-date=20240507, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5) 2024-05-09T04:59:46.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:46 smithi006 bash[93982]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mon-a 2024-05-09T04:59:47.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:May 09 04:59:47 smithi006 podman[93996]: 2024-05-09 04:59:47.022553573 +0000 UTC m=+0.954298404 container remove 34c0cb98e2cc80a5e82399c5c05feb7cf9f03b2681647ba61167705b326893ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T04:59:47.598 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mon.a.service' 2024-05-09T04:59:47.780 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T04:59:47.781 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-09T04:59:47.781 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-09T04:59:47.781 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mon.b 2024-05-09T04:59:48.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:47 smithi169 systemd[1]: Stopping Ceph mon.b for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T04:59:48.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:48 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mon-b[34070]: 2024-05-09T04:59:48.098+0000 7fbc57d0e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T04:59:48.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:48 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mon-b[34070]: 2024-05-09T04:59:48.098+0000 7fbc57d0e700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-09T04:59:48.367 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:48 smithi169 podman[68603]: 2024-05-09 04:59:48.149880031 +0000 UTC m=+0.171238015 container died 1ca93128349d94aaf4848614aad9fac3d2fdfaa260dd2809c70d0eb4e380f5c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mon-b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T04:59:48.826 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:48 smithi169 podman[68603]: 2024-05-09 04:59:48.549648535 +0000 UTC m=+0.571006529 container cleanup 1ca93128349d94aaf4848614aad9fac3d2fdfaa260dd2809c70d0eb4e380f5c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mon-b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-09T04:59:48.826 INFO:journalctl@ceph.mon.b.smithi169.stdout:May 09 04:59:48 smithi169 bash[68603]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mon-b 2024-05-09T04:59:48.958 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mon.b.service' 2024-05-09T04:59:49.013 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T04:59:49.014 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-09T04:59:49.014 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-09T04:59:49.014 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mon.c 2024-05-09T04:59:49.313 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:49 smithi186 systemd[1]: Stopping Ceph mon.c for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T04:59:49.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:49 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mon-c[34662]: 2024-05-09T04:59:49.312+0000 7f76b5472700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T04:59:49.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:49 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mon-c[34662]: 2024-05-09T04:59:49.312+0000 7f76b5472700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-09T04:59:49.600 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:49 smithi186 podman[69554]: 2024-05-09 04:59:49.445728526 +0000 UTC m=+0.239993739 container died 1bc9bdc0322d2ae782fb57c58adf9bbe5b6115d7cbd28cafe6c2665a6659f7e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mon-c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-09T04:59:50.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:49 smithi186 podman[69554]: 2024-05-09 04:59:49.838782278 +0000 UTC m=+0.633047489 container cleanup 1bc9bdc0322d2ae782fb57c58adf9bbe5b6115d7cbd28cafe6c2665a6659f7e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-05-09T04:59:50.101 INFO:journalctl@ceph.mon.c.smithi186.stdout:May 09 04:59:49 smithi186 bash[69554]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mon-c 2024-05-09T04:59:50.273 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mon.c.service' 2024-05-09T04:59:50.315 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T04:59:50.315 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-09T04:59:50.315 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-09T04:59:50.316 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mgr.x 2024-05-09T04:59:50.654 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:59:50 smithi006 systemd[1]: Stopping Ceph mgr.x for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T04:59:51.198 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:59:50 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:59:50.932+0000 7f9436800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T04:59:51.198 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:59:50 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x[30063]: 2024-05-09T04:59:50.932+0000 7f9436800700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-05-09T04:59:51.198 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:59:51 smithi006 podman[94101]: 2024-05-09 04:59:51.056974561 +0000 UTC m=+0.283129931 container died ede2170d178c1d12a413da01b2c3882aa847430f4e564d614a89488d7ab06074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-09T04:59:51.922 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:59:51 smithi006 podman[94101]: 2024-05-09 04:59:51.668005409 +0000 UTC m=+0.894160811 container cleanup ede2170d178c1d12a413da01b2c3882aa847430f4e564d614a89488d7ab06074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T04:59:51.922 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:59:51 smithi006 bash[94101]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x 2024-05-09T04:59:52.197 INFO:journalctl@ceph.mgr.x.smithi006.stdout:May 09 04:59:51 smithi006 podman[94115]: 2024-05-09 04:59:51.91973695 +0000 UTC m=+0.851750909 container remove ede2170d178c1d12a413da01b2c3882aa847430f4e564d614a89488d7ab06074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-09T04:59:52.508 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mgr.x.service' 2024-05-09T04:59:52.622 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T04:59:52.622 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-09T04:59:52.623 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-09T04:59:52.623 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mgr.y 2024-05-09T04:59:52.974 INFO:journalctl@ceph.mgr.y.smithi169.stdout:May 09 04:59:52 smithi169 systemd[1]: Stopping Ceph mgr.y for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T04:59:53.280 INFO:journalctl@ceph.mgr.y.smithi169.stdout:May 09 04:59:52 smithi169 podman[68726]: 2024-05-09 04:59:52.975504298 +0000 UTC m=+0.163633403 container died 2029de5ec1e1eafcde799ab2f3d379ed9c8b081d04d59e3625aa788aa3be482b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-09T04:59:53.617 INFO:journalctl@ceph.mgr.y.smithi169.stdout:May 09 04:59:53 smithi169 podman[68726]: 2024-05-09 04:59:53.38080391 +0000 UTC m=+0.568933030 container cleanup 2029de5ec1e1eafcde799ab2f3d379ed9c8b081d04d59e3625aa788aa3be482b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-09T04:59:53.617 INFO:journalctl@ceph.mgr.y.smithi169.stdout:May 09 04:59:53 smithi169 bash[68726]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-y 2024-05-09T04:59:53.762 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mgr.y.service' 2024-05-09T04:59:53.796 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T04:59:53.796 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-09T04:59:53.796 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-09T04:59:53.796 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mgr.z 2024-05-09T04:59:54.100 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:59:53 smithi186 systemd[1]: Stopping Ceph mgr.z for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T04:59:54.462 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:59:54 smithi186 podman[69676]: 2024-05-09 04:59:54.14029512 +0000 UTC m=+0.146332136 container died 2681ae679c05200f527fc4a7aa2b9c66ca6055e77311b07abc1b30aa0ed4e6b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-z, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-09T04:59:54.796 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:59:54 smithi186 podman[69676]: 2024-05-09 04:59:54.561753057 +0000 UTC m=+0.567790062 container cleanup 2681ae679c05200f527fc4a7aa2b9c66ca6055e77311b07abc1b30aa0ed4e6b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True) 2024-05-09T04:59:54.796 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:59:54 smithi186 bash[69676]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-z 2024-05-09T04:59:55.040 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mgr.z.service' 2024-05-09T04:59:55.061 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:59:54 smithi186 podman[69690]: 2024-05-09 04:59:54.796829721 +0000 UTC m=+0.652194103 container remove 2681ae679c05200f527fc4a7aa2b9c66ca6055e77311b07abc1b30aa0ed4e6b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True) 2024-05-09T04:59:55.061 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:59:54 smithi186 systemd[1]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-05-09T04:59:55.061 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:59:55 smithi186 systemd[1]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mgr.z.service: Failed with result 'exit-code'. 2024-05-09T04:59:55.061 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:59:55 smithi186 systemd[1]: Stopped Ceph mgr.z for f74d813c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:59:55.061 INFO:journalctl@ceph.mgr.z.smithi186.stdout:May 09 04:59:55 smithi186 systemd[1]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mgr.z.service: Consumed 12.903s CPU time. 2024-05-09T04:59:55.074 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T04:59:55.074 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-09T04:59:55.074 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-09T04:59:55.074 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.0 2024-05-09T04:59:55.447 INFO:journalctl@ceph.osd.0.smithi006.stdout:May 09 04:59:55 smithi006 systemd[1]: Stopping Ceph osd.0 for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T04:59:56.198 INFO:journalctl@ceph.osd.0.smithi006.stdout:May 09 04:59:55 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-0[42091]: 2024-05-09T04:59:55.720+0000 7f16c2000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T04:59:56.198 INFO:journalctl@ceph.osd.0.smithi006.stdout:May 09 04:59:55 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-0[42091]: 2024-05-09T04:59:55.720+0000 7f16c2000700 -1 osd.0 93 *** Got signal Terminated *** 2024-05-09T04:59:56.198 INFO:journalctl@ceph.osd.0.smithi006.stdout:May 09 04:59:55 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-0[42091]: 2024-05-09T04:59:55.720+0000 7f16c2000700 -1 osd.0 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T05:00:01.446 INFO:journalctl@ceph.osd.0.smithi006.stdout:May 09 05:00:01 smithi006 podman[94221]: 2024-05-09 05:00:01.109811092 +0000 UTC m=+5.536243843 container died 4bc898547b8010586a5c5897e1673da9ba8029b70ba43a8c040a943ac95a40b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-09T05:00:01.697 INFO:journalctl@ceph.osd.0.smithi006.stdout:May 09 05:00:01 smithi006 podman[94221]: 2024-05-09 05:00:01.657500379 +0000 UTC m=+6.083933162 container cleanup 4bc898547b8010586a5c5897e1673da9ba8029b70ba43a8c040a943ac95a40b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T05:00:01.697 INFO:journalctl@ceph.osd.0.smithi006.stdout:May 09 05:00:01 smithi006 bash[94221]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-0 2024-05-09T05:00:02.197 INFO:journalctl@ceph.osd.0.smithi006.stdout:May 09 05:00:01 smithi006 podman[94241]: 2024-05-09 05:00:01.908269656 +0000 UTC m=+0.781035484 container remove 4bc898547b8010586a5c5897e1673da9ba8029b70ba43a8c040a943ac95a40b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-09T05:00:03.147 INFO:journalctl@ceph.osd.0.smithi006.stdout:May 09 05:00:02 smithi006 podman[94330]: 2024-05-09 05:00:02.719930694 +0000 UTC m=+0.043125249 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:00:03.147 INFO:journalctl@ceph.osd.0.smithi006.stdout:May 09 05:00:02 smithi006 podman[94330]: 2024-05-09 05:00:02.85083251 +0000 UTC m=+0.174027051 container create 7b9a6eea4059bbcba587f1183eb4266636b050d93bdcb6fa426ebcdf5eccd773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-05-09T05:00:03.448 INFO:journalctl@ceph.osd.0.smithi006.stdout:May 09 05:00:03 smithi006 podman[94330]: 2024-05-09 05:00:03.14680161 +0000 UTC m=+0.469996235 container init 7b9a6eea4059bbcba587f1183eb4266636b050d93bdcb6fa426ebcdf5eccd773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T05:00:03.448 INFO:journalctl@ceph.osd.0.smithi006.stdout:May 09 05:00:03 smithi006 podman[94330]: 2024-05-09 05:00:03.158214494 +0000 UTC m=+0.481409105 container start 7b9a6eea4059bbcba587f1183eb4266636b050d93bdcb6fa426ebcdf5eccd773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-0-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-09T05:00:03.448 INFO:journalctl@ceph.osd.0.smithi006.stdout:May 09 05:00:03 smithi006 podman[94330]: 2024-05-09 05:00:03.245339802 +0000 UTC m=+0.568534426 container attach 7b9a6eea4059bbcba587f1183eb4266636b050d93bdcb6fa426ebcdf5eccd773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , ceph=True) 2024-05-09T05:00:03.947 INFO:journalctl@ceph.osd.0.smithi006.stdout:May 09 05:00:03 smithi006 podman[94330]: 2024-05-09 05:00:03.634559608 +0000 UTC m=+0.957754181 container died 7b9a6eea4059bbcba587f1183eb4266636b050d93bdcb6fa426ebcdf5eccd773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-09T05:00:04.495 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.0.service' 2024-05-09T05:00:04.612 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:00:04.613 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-09T05:00:04.613 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-09T05:00:04.613 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.1 2024-05-09T05:00:04.910 INFO:journalctl@ceph.osd.1.smithi169.stdout:May 09 05:00:04 smithi169 systemd[1]: Stopping Ceph osd.1 for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:00:05.367 INFO:journalctl@ceph.osd.1.smithi169.stdout:May 09 05:00:04 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-1[38703]: 2024-05-09T05:00:04.909+0000 7f2ec6674700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:00:05.367 INFO:journalctl@ceph.osd.1.smithi169.stdout:May 09 05:00:04 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-1[38703]: 2024-05-09T05:00:04.909+0000 7f2ec6674700 -1 osd.1 93 *** Got signal Terminated *** 2024-05-09T05:00:05.367 INFO:journalctl@ceph.osd.1.smithi169.stdout:May 09 05:00:04 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-1[38703]: 2024-05-09T05:00:04.909+0000 7f2ec6674700 -1 osd.1 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T05:00:10.617 INFO:journalctl@ceph.osd.1.smithi169.stdout:May 09 05:00:10 smithi169 podman[68864]: 2024-05-09 05:00:10.353814315 +0000 UTC m=+5.544909958 container died 980ad70d4ffece83add864984eaf801910adf27ce8f74ab604ad3c949d59b2d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-1, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-09T05:00:11.012 INFO:journalctl@ceph.osd.1.smithi169.stdout:May 09 05:00:10 smithi169 podman[68864]: 2024-05-09 05:00:10.752416263 +0000 UTC m=+5.943511911 container cleanup 980ad70d4ffece83add864984eaf801910adf27ce8f74ab604ad3c949d59b2d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-1, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux ) 2024-05-09T05:00:11.012 INFO:journalctl@ceph.osd.1.smithi169.stdout:May 09 05:00:10 smithi169 bash[68864]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-1 2024-05-09T05:00:11.325 INFO:journalctl@ceph.osd.1.smithi169.stdout:May 09 05:00:11 smithi169 podman[68882]: 2024-05-09 05:00:11.012286796 +0000 UTC m=+0.650715512 container remove 980ad70d4ffece83add864984eaf801910adf27ce8f74ab604ad3c949d59b2d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T05:00:11.617 INFO:journalctl@ceph.osd.1.smithi169.stdout:May 09 05:00:11 smithi169 podman[68976]: 2024-05-09 05:00:11.228680118 +0000 UTC m=+0.017843366 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:00:11.617 INFO:journalctl@ceph.osd.1.smithi169.stdout:May 09 05:00:11 smithi169 podman[68976]: 2024-05-09 05:00:11.371634631 +0000 UTC m=+0.160797872 container create fc26a49a2b84ac12d39c9ed0058bc620081da935ad8a2f69cc68de9541480be4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-1-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-09T05:00:11.933 INFO:journalctl@ceph.osd.1.smithi169.stdout:May 09 05:00:11 smithi169 podman[68976]: 2024-05-09 05:00:11.633033516 +0000 UTC m=+0.422196755 container init fc26a49a2b84ac12d39c9ed0058bc620081da935ad8a2f69cc68de9541480be4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-09T05:00:11.933 INFO:journalctl@ceph.osd.1.smithi169.stdout:May 09 05:00:11 smithi169 podman[68976]: 2024-05-09 05:00:11.636267501 +0000 UTC m=+0.425430742 container start fc26a49a2b84ac12d39c9ed0058bc620081da935ad8a2f69cc68de9541480be4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-09T05:00:11.933 INFO:journalctl@ceph.osd.1.smithi169.stdout:May 09 05:00:11 smithi169 podman[68976]: 2024-05-09 05:00:11.727024746 +0000 UTC m=+0.516187988 container attach fc26a49a2b84ac12d39c9ed0058bc620081da935ad8a2f69cc68de9541480be4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-09T05:00:12.239 INFO:journalctl@ceph.osd.1.smithi169.stdout:May 09 05:00:11 smithi169 podman[68976]: 2024-05-09 05:00:11.933605362 +0000 UTC m=+0.722768601 container died fc26a49a2b84ac12d39c9ed0058bc620081da935ad8a2f69cc68de9541480be4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-09T05:00:12.571 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.1.service' 2024-05-09T05:00:12.609 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:00:12.610 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-09T05:00:12.610 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-09T05:00:12.610 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.2 2024-05-09T05:00:12.933 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 09 05:00:12 smithi186 systemd[1]: Stopping Ceph osd.2 for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:00:13.350 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 09 05:00:12 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-2[39854]: 2024-05-09T05:00:12.932+0000 7f8b7747c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:00:13.350 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 09 05:00:12 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-2[39854]: 2024-05-09T05:00:12.932+0000 7f8b7747c700 -1 osd.2 93 *** Got signal Terminated *** 2024-05-09T05:00:13.350 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 09 05:00:12 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-2[39854]: 2024-05-09T05:00:12.932+0000 7f8b7747c700 -1 osd.2 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T05:00:18.751 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 09 05:00:18 smithi186 podman[69801]: 2024-05-09 05:00:18.447185526 +0000 UTC m=+5.639287700 container died be5c698b34c6d0a057292a8a0b8b0dea4669015d7125f4c37fa5377a535ff0d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD) 2024-05-09T05:00:19.092 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 09 05:00:18 smithi186 podman[69801]: 2024-05-09 05:00:18.866444781 +0000 UTC m=+6.058546949 container cleanup be5c698b34c6d0a057292a8a0b8b0dea4669015d7125f4c37fa5377a535ff0d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-09T05:00:19.092 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 09 05:00:18 smithi186 bash[69801]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-2 2024-05-09T05:00:19.349 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 09 05:00:19 smithi186 podman[69818]: 2024-05-09 05:00:19.092992155 +0000 UTC m=+0.642605842 container remove be5c698b34c6d0a057292a8a0b8b0dea4669015d7125f4c37fa5377a535ff0d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T05:00:19.814 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 09 05:00:19 smithi186 podman[69909]: 2024-05-09 05:00:19.428460979 +0000 UTC m=+0.017654378 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:00:19.814 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 09 05:00:19 smithi186 podman[69909]: 2024-05-09 05:00:19.560769922 +0000 UTC m=+0.149963304 container create a82480fdd02aeb7a60da44ab3f787a3bbe5fdaae18f91e6718eab14e2abfb531 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T05:00:19.814 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 09 05:00:19 smithi186 podman[69909]: 2024-05-09 05:00:19.814746598 +0000 UTC m=+0.403939993 container init a82480fdd02aeb7a60da44ab3f787a3bbe5fdaae18f91e6718eab14e2abfb531 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, ceph=True) 2024-05-09T05:00:20.100 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 09 05:00:19 smithi186 podman[69909]: 2024-05-09 05:00:19.818392373 +0000 UTC m=+0.407585759 container start a82480fdd02aeb7a60da44ab3f787a3bbe5fdaae18f91e6718eab14e2abfb531 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-09T05:00:20.100 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 09 05:00:19 smithi186 podman[69909]: 2024-05-09 05:00:19.897381001 +0000 UTC m=+0.486574387 container attach a82480fdd02aeb7a60da44ab3f787a3bbe5fdaae18f91e6718eab14e2abfb531 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T05:00:20.397 INFO:journalctl@ceph.osd.2.smithi186.stdout:May 09 05:00:20 smithi186 podman[69909]: 2024-05-09 05:00:20.117201488 +0000 UTC m=+0.706394866 container died a82480fdd02aeb7a60da44ab3f787a3bbe5fdaae18f91e6718eab14e2abfb531 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507) 2024-05-09T05:00:20.770 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.2.service' 2024-05-09T05:00:20.808 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:00:20.808 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-09T05:00:20.808 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-09T05:00:20.809 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.3 2024-05-09T05:00:21.166 INFO:journalctl@ceph.osd.3.smithi006.stdout:May 09 05:00:20 smithi006 systemd[1]: Stopping Ceph osd.3 for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:00:21.448 INFO:journalctl@ceph.osd.3.smithi006.stdout:May 09 05:00:21 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-3[48741]: 2024-05-09T05:00:21.349+0000 7f0b69800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:00:21.448 INFO:journalctl@ceph.osd.3.smithi006.stdout:May 09 05:00:21 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-3[48741]: 2024-05-09T05:00:21.349+0000 7f0b69800700 -1 osd.3 93 *** Got signal Terminated *** 2024-05-09T05:00:21.448 INFO:journalctl@ceph.osd.3.smithi006.stdout:May 09 05:00:21 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-3[48741]: 2024-05-09T05:00:21.349+0000 7f0b69800700 -1 osd.3 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T05:00:21.599 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:21 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:21.139+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:59.437940+0000 front 2024-05-09T04:59:59.438149+0000 (oldest deadline 2024-05-09T05:00:21.137741+0000) 2024-05-09T05:00:22.600 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:22 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:22.104+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:59.437940+0000 front 2024-05-09T04:59:59.438149+0000 (oldest deadline 2024-05-09T05:00:21.137741+0000) 2024-05-09T05:00:22.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:22 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:22.165+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:23.198 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:22 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:22.876+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:23.349 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:23 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:23.067+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:59.437940+0000 front 2024-05-09T04:59:59.438149+0000 (oldest deadline 2024-05-09T05:00:21.137741+0000) 2024-05-09T05:00:23.518 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:23 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:23.205+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:24.197 INFO:journalctl@ceph.osd.3.smithi006.stdout:May 09 05:00:24 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-3[48741]: 2024-05-09T05:00:24.033+0000 7f0b60000700 -1 osd.3 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:56.229334+0000 front 2024-05-09T04:59:56.230053+0000 (oldest deadline 2024-05-09T05:00:23.230861+0000) 2024-05-09T05:00:24.198 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:23 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:23.890+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:24.349 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:24 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:24.071+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:59.437940+0000 front 2024-05-09T04:59:59.438149+0000 (oldest deadline 2024-05-09T05:00:21.137741+0000) 2024-05-09T05:00:24.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:24 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:24.238+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:25.100 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:24 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:24.760+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:25.197 INFO:journalctl@ceph.osd.3.smithi006.stdout:May 09 05:00:25 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-3[48741]: 2024-05-09T05:00:25.051+0000 7f0b60000700 -1 osd.3 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:56.229334+0000 front 2024-05-09T04:59:56.230053+0000 (oldest deadline 2024-05-09T05:00:23.230861+0000) 2024-05-09T05:00:25.198 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:24 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:24.904+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:25.599 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:25 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:25.113+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:59.437940+0000 front 2024-05-09T04:59:59.438149+0000 (oldest deadline 2024-05-09T05:00:21.137741+0000) 2024-05-09T05:00:25.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:25 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:25.262+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:26.100 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:25 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:25.755+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:26.447 INFO:journalctl@ceph.osd.3.smithi006.stdout:May 09 05:00:26 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-3[48741]: 2024-05-09T05:00:26.049+0000 7f0b60000700 -1 osd.3 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:56.229334+0000 front 2024-05-09T04:59:56.230053+0000 (oldest deadline 2024-05-09T05:00:23.230861+0000) 2024-05-09T05:00:26.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:25 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:25.950+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:26.599 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:26 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:26.122+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:59.437940+0000 front 2024-05-09T04:59:59.438149+0000 (oldest deadline 2024-05-09T05:00:21.137741+0000) 2024-05-09T05:00:26.616 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:26 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:26.255+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:27.099 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:26 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:26.796+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:27.192 INFO:journalctl@ceph.osd.3.smithi006.stdout:May 09 05:00:26 smithi006 podman[94435]: 2024-05-09 05:00:26.833103078 +0000 UTC m=+5.625740399 container died 3c1301617b185680a7cd5d20ff473586d34c8aa1a3ceb1e2715e7ab9c05fc35c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-3, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T05:00:27.193 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:26 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:26.962+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:27.444 INFO:journalctl@ceph.osd.3.smithi006.stdout:May 09 05:00:27 smithi006 podman[94435]: 2024-05-09 05:00:27.418397597 +0000 UTC m=+6.211034883 container cleanup 3c1301617b185680a7cd5d20ff473586d34c8aa1a3ceb1e2715e7ab9c05fc35c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-3, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T05:00:27.445 INFO:journalctl@ceph.osd.3.smithi006.stdout:May 09 05:00:27 smithi006 bash[94435]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-3 2024-05-09T05:00:27.599 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:27 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:27.125+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:59.437940+0000 front 2024-05-09T04:59:59.438149+0000 (oldest deadline 2024-05-09T05:00:21.137741+0000) 2024-05-09T05:00:27.616 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:27 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:27.211+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:27.697 INFO:journalctl@ceph.osd.3.smithi006.stdout:May 09 05:00:27 smithi006 podman[94466]: 2024-05-09 05:00:27.662116354 +0000 UTC m=+0.817546974 container remove 3c1301617b185680a7cd5d20ff473586d34c8aa1a3ceb1e2715e7ab9c05fc35c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-3, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-09T05:00:28.099 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:27 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:27.755+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:28.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:27 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:27.949+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:28.599 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:28 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:28.107+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:59.437940+0000 front 2024-05-09T04:59:59.438149+0000 (oldest deadline 2024-05-09T05:00:21.137741+0000) 2024-05-09T05:00:28.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:28 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:28.179+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:28.826 INFO:journalctl@ceph.osd.3.smithi006.stdout:May 09 05:00:28 smithi006 podman[94556]: 2024-05-09 05:00:28.432102365 +0000 UTC m=+0.041185533 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:00:28.826 INFO:journalctl@ceph.osd.3.smithi006.stdout:May 09 05:00:28 smithi006 podman[94556]: 2024-05-09 05:00:28.5547784 +0000 UTC m=+0.163861591 container create fb402459eb015056418e1e0b320d4b057976c1e569a71ea06ff31f6ca5047d3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-09T05:00:28.826 INFO:journalctl@ceph.osd.3.smithi006.stdout:May 09 05:00:28 smithi006 podman[94556]: 2024-05-09 05:00:28.82510357 +0000 UTC m=+0.434186768 container init fb402459eb015056418e1e0b320d4b057976c1e569a71ea06ff31f6ca5047d3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-09T05:00:29.074 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:28 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:28.801+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:29.197 INFO:journalctl@ceph.osd.3.smithi006.stdout:May 09 05:00:28 smithi006 podman[94556]: 2024-05-09 05:00:28.837331338 +0000 UTC m=+0.446414525 container start fb402459eb015056418e1e0b320d4b057976c1e569a71ea06ff31f6ca5047d3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T05:00:29.197 INFO:journalctl@ceph.osd.3.smithi006.stdout:May 09 05:00:28 smithi006 podman[94556]: 2024-05-09 05:00:28.965874116 +0000 UTC m=+0.574957360 container attach fb402459eb015056418e1e0b320d4b057976c1e569a71ea06ff31f6ca5047d3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T05:00:29.198 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:28 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:28.932+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:29.349 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:29 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:29.074+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:59.437940+0000 front 2024-05-09T04:59:59.438149+0000 (oldest deadline 2024-05-09T05:00:21.137741+0000) 2024-05-09T05:00:29.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:29 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:29.178+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:29.698 INFO:journalctl@ceph.osd.3.smithi006.stdout:May 09 05:00:29 smithi006 podman[94556]: 2024-05-09 05:00:29.314037815 +0000 UTC m=+0.923121000 container died fb402459eb015056418e1e0b320d4b057976c1e569a71ea06ff31f6ca5047d3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-09T05:00:30.083 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:29 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:29.787+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:30.187 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:29 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:29.892+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:30.238 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.3.service' 2024-05-09T05:00:30.349 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:30 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:30.083+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:59.437940+0000 front 2024-05-09T04:59:59.438149+0000 (oldest deadline 2024-05-09T05:00:21.137741+0000) 2024-05-09T05:00:30.353 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:00:30.353 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-09T05:00:30.354 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-09T05:00:30.354 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.4 2024-05-09T05:00:30.408 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:30 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:30.139+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:30.675 INFO:journalctl@ceph.osd.4.smithi169.stdout:May 09 05:00:30 smithi169 systemd[1]: Stopping Ceph osd.4 for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:00:31.086 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:30 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:30.827+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:31.117 INFO:journalctl@ceph.osd.4.smithi169.stdout:May 09 05:00:30 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-4[45231]: 2024-05-09T05:00:30.674+0000 7f4d652ba700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:00:31.117 INFO:journalctl@ceph.osd.4.smithi169.stdout:May 09 05:00:30 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-4[45231]: 2024-05-09T05:00:30.674+0000 7f4d652ba700 -1 osd.4 93 *** Got signal Terminated *** 2024-05-09T05:00:31.117 INFO:journalctl@ceph.osd.4.smithi169.stdout:May 09 05:00:30 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-4[45231]: 2024-05-09T05:00:30.674+0000 7f4d652ba700 -1 osd.4 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T05:00:31.198 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:30 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:30.929+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:31.350 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:31 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:31.086+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:59.437940+0000 front 2024-05-09T04:59:59.438149+0000 (oldest deadline 2024-05-09T05:00:21.137741+0000) 2024-05-09T05:00:31.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:31 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:31.220+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:00:31.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:31 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:31.135+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:32.197 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:31 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:31.897+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:32.252 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:32 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:32.084+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:59.437940+0000 front 2024-05-09T04:59:59.438149+0000 (oldest deadline 2024-05-09T05:00:21.137741+0000) 2024-05-09T05:00:32.252 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:31 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:31.875+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:32.545 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:32 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:32.116+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:32.600 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:32 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:32.252+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:00:32.867 INFO:journalctl@ceph.osd.4.smithi169.stdout:May 09 05:00:32 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-4[45231]: 2024-05-09T05:00:32.545+0000 7f4d5d868700 -1 osd.4 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:06.517906+0000 front 2024-05-09T05:00:06.518048+0000 (oldest deadline 2024-05-09T05:00:32.418016+0000) 2024-05-09T05:00:33.197 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:32 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:32.854+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:33.244 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:32 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:32.895+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:33.244 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:33 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:33.065+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:59.437940+0000 front 2024-05-09T04:59:59.438149+0000 (oldest deadline 2024-05-09T05:00:21.137741+0000) 2024-05-09T05:00:33.367 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:33 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:33.090+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:33.367 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:33 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:33.090+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:00:33.600 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:33 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:33.243+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:00:33.820 INFO:journalctl@ceph.osd.4.smithi169.stdout:May 09 05:00:33 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-4[45231]: 2024-05-09T05:00:33.513+0000 7f4d5d868700 -1 osd.4 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:06.517906+0000 front 2024-05-09T05:00:06.518048+0000 (oldest deadline 2024-05-09T05:00:32.418016+0000) 2024-05-09T05:00:34.197 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:33 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:33.888+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:34.198 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:33 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:33.888+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:00:34.213 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:33 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:33.881+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:34.213 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:34 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:34.031+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:59.437940+0000 front 2024-05-09T04:59:59.438149+0000 (oldest deadline 2024-05-09T05:00:21.137741+0000) 2024-05-09T05:00:34.213 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:34 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:34.032+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.840005+0000 front 2024-05-09T05:00:09.839947+0000 (oldest deadline 2024-05-09T05:00:33.840559+0000) 2024-05-09T05:00:34.345 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:34 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:34.091+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:34.345 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:34 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:34.091+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:00:34.599 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:34 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:34.213+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:00:34.617 INFO:journalctl@ceph.osd.4.smithi169.stdout:May 09 05:00:34 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-4[45231]: 2024-05-09T05:00:34.513+0000 7f4d5d868700 -1 osd.4 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:06.517906+0000 front 2024-05-09T05:00:06.518048+0000 (oldest deadline 2024-05-09T05:00:32.418016+0000) 2024-05-09T05:00:34.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:34 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:34.344+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.771587+0000 front 2024-05-09T05:00:09.771610+0000 (oldest deadline 2024-05-09T05:00:33.772656+0000) 2024-05-09T05:00:35.197 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:34 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:34.903+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:35.197 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:34 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:34.903+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:00:35.257 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:35 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:35.065+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:59.437940+0000 front 2024-05-09T04:59:59.438149+0000 (oldest deadline 2024-05-09T05:00:21.137741+0000) 2024-05-09T05:00:35.257 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:35 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:35.065+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.840005+0000 front 2024-05-09T05:00:09.839947+0000 (oldest deadline 2024-05-09T05:00:33.840559+0000) 2024-05-09T05:00:35.257 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:34 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:34.927+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:35.511 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:35 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:35.343+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.771587+0000 front 2024-05-09T05:00:09.771610+0000 (oldest deadline 2024-05-09T05:00:33.772656+0000) 2024-05-09T05:00:35.511 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:35 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:35.120+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:35.511 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:35 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:35.120+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:00:35.599 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:35 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:35.256+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:00:35.867 INFO:journalctl@ceph.osd.4.smithi169.stdout:May 09 05:00:35 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-4[45231]: 2024-05-09T05:00:35.511+0000 7f4d5d868700 -1 osd.4 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:06.517906+0000 front 2024-05-09T05:00:06.518048+0000 (oldest deadline 2024-05-09T05:00:32.418016+0000) 2024-05-09T05:00:36.251 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:36 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:36.056+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:59.437940+0000 front 2024-05-09T04:59:59.438149+0000 (oldest deadline 2024-05-09T05:00:21.137741+0000) 2024-05-09T05:00:36.252 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:36 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:36.056+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.840005+0000 front 2024-05-09T05:00:09.839947+0000 (oldest deadline 2024-05-09T05:00:33.840559+0000) 2024-05-09T05:00:36.252 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:35 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:35.926+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:36.409 INFO:journalctl@ceph.osd.4.smithi169.stdout:May 09 05:00:36 smithi169 podman[69098]: 2024-05-09 05:00:36.120225342 +0000 UTC m=+5.566519065 container died ce788644115f3661cc731151379a36fa3dfbad9c1b5d0392eceb17087297017b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True) 2024-05-09T05:00:36.409 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:36 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:36.300+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.771587+0000 front 2024-05-09T05:00:09.771610+0000 (oldest deadline 2024-05-09T05:00:33.772656+0000) 2024-05-09T05:00:36.410 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:36 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:36.128+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:36.410 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:36 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:36.128+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:00:36.447 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:35 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:35.946+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:36.447 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:35 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:35.946+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:00:36.599 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:36 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:36.251+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:00:36.767 INFO:journalctl@ceph.osd.4.smithi169.stdout:May 09 05:00:36 smithi169 podman[69098]: 2024-05-09 05:00:36.51693422 +0000 UTC m=+5.963227932 container cleanup ce788644115f3661cc731151379a36fa3dfbad9c1b5d0392eceb17087297017b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-4, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-09T05:00:36.768 INFO:journalctl@ceph.osd.4.smithi169.stdout:May 09 05:00:36 smithi169 bash[69098]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-4 2024-05-09T05:00:37.088 INFO:journalctl@ceph.osd.4.smithi169.stdout:May 09 05:00:36 smithi169 podman[69116]: 2024-05-09 05:00:36.768154614 +0000 UTC m=+0.649219995 container remove ce788644115f3661cc731151379a36fa3dfbad9c1b5d0392eceb17087297017b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T05:00:37.197 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:36 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:36.941+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:37.197 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:36 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:36.941+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:00:37.198 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:36 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6[55562]: 2024-05-09T05:00:36.722+0000 7fdc6bc00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:04.396883+0000 front 2024-05-09T05:00:04.396963+0000 (oldest deadline 2024-05-09T05:00:36.198640+0000) 2024-05-09T05:00:37.285 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:36 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:36.950+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:37.285 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:37 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:37.093+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:59.437940+0000 front 2024-05-09T04:59:59.438149+0000 (oldest deadline 2024-05-09T05:00:21.137741+0000) 2024-05-09T05:00:37.285 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:37 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:37.093+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.840005+0000 front 2024-05-09T05:00:09.839947+0000 (oldest deadline 2024-05-09T05:00:33.840559+0000) 2024-05-09T05:00:37.346 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:37 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:37.287+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.771587+0000 front 2024-05-09T05:00:09.771610+0000 (oldest deadline 2024-05-09T05:00:33.772656+0000) 2024-05-09T05:00:37.346 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:37 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:37.103+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:37.346 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:37 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:37.103+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:00:37.347 INFO:journalctl@ceph.osd.4.smithi169.stdout:May 09 05:00:37 smithi169 podman[69210]: 2024-05-09 05:00:36.992143281 +0000 UTC m=+0.018113408 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:00:37.347 INFO:journalctl@ceph.osd.4.smithi169.stdout:May 09 05:00:37 smithi169 podman[69210]: 2024-05-09 05:00:37.094201849 +0000 UTC m=+0.120172007 container create 6e60a179e76bd7bec4064d4502f46239fd82b982b2445d1202e51848f037b7a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD) 2024-05-09T05:00:37.347 INFO:journalctl@ceph.osd.4.smithi169.stdout:May 09 05:00:37 smithi169 podman[69210]: 2024-05-09 05:00:37.346602986 +0000 UTC m=+0.372573092 container init 6e60a179e76bd7bec4064d4502f46239fd82b982b2445d1202e51848f037b7a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507) 2024-05-09T05:00:37.600 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:37 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:37.285+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:00:37.617 INFO:journalctl@ceph.osd.4.smithi169.stdout:May 09 05:00:37 smithi169 podman[69210]: 2024-05-09 05:00:37.350152166 +0000 UTC m=+0.376122275 container start 6e60a179e76bd7bec4064d4502f46239fd82b982b2445d1202e51848f037b7a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-09T05:00:37.617 INFO:journalctl@ceph.osd.4.smithi169.stdout:May 09 05:00:37 smithi169 podman[69210]: 2024-05-09 05:00:37.44678397 +0000 UTC m=+0.472754100 container attach 6e60a179e76bd7bec4064d4502f46239fd82b982b2445d1202e51848f037b7a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507) 2024-05-09T05:00:37.941 INFO:journalctl@ceph.osd.4.smithi169.stdout:May 09 05:00:37 smithi169 podman[69210]: 2024-05-09 05:00:37.643140168 +0000 UTC m=+0.669110283 container died 6e60a179e76bd7bec4064d4502f46239fd82b982b2445d1202e51848f037b7a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507) 2024-05-09T05:00:37.947 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:37 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6[55562]: 2024-05-09T05:00:37.686+0000 7fdc6bc00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:04.396883+0000 front 2024-05-09T05:00:04.396963+0000 (oldest deadline 2024-05-09T05:00:36.198640+0000) 2024-05-09T05:00:37.948 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:37 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:37.914+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:37.948 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:37 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:37.914+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:00:38.250 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:38 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:38.142+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:38.251 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:38 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:38.142+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:00:38.274 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.4.service' 2024-05-09T05:00:38.281 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:37 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:37.961+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:38.281 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:38 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:38.143+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:59.437940+0000 front 2024-05-09T04:59:59.438149+0000 (oldest deadline 2024-05-09T05:00:21.137741+0000) 2024-05-09T05:00:38.281 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:38 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:38.143+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.840005+0000 front 2024-05-09T05:00:09.839947+0000 (oldest deadline 2024-05-09T05:00:33.840559+0000) 2024-05-09T05:00:38.315 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:00:38.315 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-09T05:00:38.315 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-09T05:00:38.315 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.5 2024-05-09T05:00:38.600 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:38 smithi186 systemd[1]: Stopping Ceph osd.5 for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:00:38.600 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:38 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:38.281+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:00:38.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:38 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:38.318+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.771587+0000 front 2024-05-09T05:00:09.771610+0000 (oldest deadline 2024-05-09T05:00:33.772656+0000) 2024-05-09T05:00:38.988 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:38 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:38.646+0000 7fce48d1a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:00:38.989 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:38 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:38.646+0000 7fce48d1a700 -1 osd.5 93 *** Got signal Terminated *** 2024-05-09T05:00:38.989 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:38 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:38.646+0000 7fce48d1a700 -1 osd.5 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T05:00:39.197 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:38 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6[55562]: 2024-05-09T05:00:38.708+0000 7fdc6bc00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:04.396883+0000 front 2024-05-09T05:00:04.396963+0000 (oldest deadline 2024-05-09T05:00:36.198640+0000) 2024-05-09T05:00:39.198 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:38 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:38.894+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:39.198 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:38 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:38.894+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:00:39.326 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:39 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:39.191+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:59.437940+0000 front 2024-05-09T04:59:59.438149+0000 (oldest deadline 2024-05-09T05:00:21.137741+0000) 2024-05-09T05:00:39.326 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:39 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:39.191+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.840005+0000 front 2024-05-09T05:00:09.839947+0000 (oldest deadline 2024-05-09T05:00:33.840559+0000) 2024-05-09T05:00:39.326 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:39 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:39.191+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.141625+0000 front 2024-05-09T05:00:16.141553+0000 (oldest deadline 2024-05-09T05:00:39.041782+0000) 2024-05-09T05:00:39.326 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:38 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:38.988+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:39.326 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:38 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:38.988+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:00:39.599 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:39 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:39.326+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:00:39.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:39 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:39.284+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.771587+0000 front 2024-05-09T05:00:09.771610+0000 (oldest deadline 2024-05-09T05:00:33.772656+0000) 2024-05-09T05:00:39.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:39 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:39.171+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:39.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:39 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:39.171+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:00:40.197 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:39 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6[55562]: 2024-05-09T05:00:39.717+0000 7fdc6bc00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:04.396883+0000 front 2024-05-09T05:00:04.396963+0000 (oldest deadline 2024-05-09T05:00:36.198640+0000) 2024-05-09T05:00:40.198 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:39 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:39.856+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:40.198 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:39 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:39.856+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:00:40.350 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:40 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:40.159+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:59.437940+0000 front 2024-05-09T04:59:59.438149+0000 (oldest deadline 2024-05-09T05:00:21.137741+0000) 2024-05-09T05:00:40.350 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:40 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:40.159+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.840005+0000 front 2024-05-09T05:00:09.839947+0000 (oldest deadline 2024-05-09T05:00:33.840559+0000) 2024-05-09T05:00:40.350 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:40 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:40.159+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.141625+0000 front 2024-05-09T05:00:16.141553+0000 (oldest deadline 2024-05-09T05:00:39.041782+0000) 2024-05-09T05:00:40.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:39 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:39.964+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:40.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:39 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:39.964+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:00:40.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:40 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:40.282+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.771587+0000 front 2024-05-09T05:00:09.771610+0000 (oldest deadline 2024-05-09T05:00:33.772656+0000) 2024-05-09T05:00:40.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:40 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:40.199+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:40.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:40 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:40.199+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:00:40.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:40 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:40.199+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:00:40.849 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:40 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:40.361+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:00:40.850 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:40 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:40.361+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.223703+0000 front 2024-05-09T05:00:15.223669+0000 (oldest deadline 2024-05-09T05:00:39.923801+0000) 2024-05-09T05:00:41.197 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:40 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6[55562]: 2024-05-09T05:00:40.700+0000 7fdc6bc00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:04.396883+0000 front 2024-05-09T05:00:04.396963+0000 (oldest deadline 2024-05-09T05:00:36.198640+0000) 2024-05-09T05:00:41.198 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:40 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:40.882+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:41.198 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:40 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:40.882+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:00:41.350 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:41 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:41.180+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:59.437940+0000 front 2024-05-09T04:59:59.438149+0000 (oldest deadline 2024-05-09T05:00:21.137741+0000) 2024-05-09T05:00:41.350 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:41 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:41.180+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.840005+0000 front 2024-05-09T05:00:09.839947+0000 (oldest deadline 2024-05-09T05:00:33.840559+0000) 2024-05-09T05:00:41.350 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:41 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:41.180+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.141625+0000 front 2024-05-09T05:00:16.141553+0000 (oldest deadline 2024-05-09T05:00:39.041782+0000) 2024-05-09T05:00:41.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:40 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:40.937+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:41.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:40 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:40.937+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:00:41.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:41 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:41.316+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.771587+0000 front 2024-05-09T05:00:09.771610+0000 (oldest deadline 2024-05-09T05:00:33.772656+0000) 2024-05-09T05:00:41.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:41 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:41.316+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.073917+0000 front 2024-05-09T05:00:16.073768+0000 (oldest deadline 2024-05-09T05:00:40.774002+0000) 2024-05-09T05:00:41.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:41 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:41.178+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:41.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:41 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:41.178+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:00:41.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:41 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:41.178+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:00:41.850 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:41 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:41.377+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:00:41.850 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:41 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:41.377+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.223703+0000 front 2024-05-09T05:00:15.223669+0000 (oldest deadline 2024-05-09T05:00:39.923801+0000) 2024-05-09T05:00:42.194 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:41 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:41.922+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:42.195 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:41 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:41.922+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:00:42.197 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:41 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6[55562]: 2024-05-09T05:00:41.744+0000 7fdc6bc00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:04.396883+0000 front 2024-05-09T05:00:04.396963+0000 (oldest deadline 2024-05-09T05:00:36.198640+0000) 2024-05-09T05:00:42.197 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:41 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6[55562]: 2024-05-09T05:00:41.744+0000 7fdc6bc00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.199563+0000 front 2024-05-09T05:00:16.199511+0000 (oldest deadline 2024-05-09T05:00:41.499771+0000) 2024-05-09T05:00:42.198 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:41 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:41.905+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:42.198 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:41 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:41.905+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:00:42.198 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:41 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:41.905+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:00:42.600 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:42 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:42.342+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:00:42.600 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:42 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:42.342+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.223703+0000 front 2024-05-09T05:00:15.223669+0000 (oldest deadline 2024-05-09T05:00:39.923801+0000) 2024-05-09T05:00:42.600 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:42 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:42.194+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:59.437940+0000 front 2024-05-09T04:59:59.438149+0000 (oldest deadline 2024-05-09T05:00:21.137741+0000) 2024-05-09T05:00:42.600 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:42 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:42.194+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.840005+0000 front 2024-05-09T05:00:09.839947+0000 (oldest deadline 2024-05-09T05:00:33.840559+0000) 2024-05-09T05:00:42.600 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:42 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:42.194+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.141625+0000 front 2024-05-09T05:00:16.141553+0000 (oldest deadline 2024-05-09T05:00:39.041782+0000) 2024-05-09T05:00:42.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:42 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:42.285+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.771587+0000 front 2024-05-09T05:00:09.771610+0000 (oldest deadline 2024-05-09T05:00:33.772656+0000) 2024-05-09T05:00:42.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:42 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:42.285+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.073917+0000 front 2024-05-09T05:00:16.073768+0000 (oldest deadline 2024-05-09T05:00:40.774002+0000) 2024-05-09T05:00:42.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:42 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:42.227+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:42.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:42 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:42.227+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:00:42.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:42 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:42.227+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:00:43.197 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:42 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6[55562]: 2024-05-09T05:00:42.698+0000 7fdc6bc00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:04.396883+0000 front 2024-05-09T05:00:04.396963+0000 (oldest deadline 2024-05-09T05:00:36.198640+0000) 2024-05-09T05:00:43.197 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:42 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6[55562]: 2024-05-09T05:00:42.698+0000 7fdc6bc00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.199563+0000 front 2024-05-09T05:00:16.199511+0000 (oldest deadline 2024-05-09T05:00:41.499771+0000) 2024-05-09T05:00:43.198 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:42 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:42.913+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:43.198 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:42 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:42.913+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:00:43.198 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:42 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:42.913+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:00:43.346 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:42 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:42.970+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:43.347 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:42 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:42.970+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:00:43.347 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:43 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:43.202+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:59.437940+0000 front 2024-05-09T04:59:59.438149+0000 (oldest deadline 2024-05-09T05:00:21.137741+0000) 2024-05-09T05:00:43.347 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:43 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:43.202+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.840005+0000 front 2024-05-09T05:00:09.839947+0000 (oldest deadline 2024-05-09T05:00:33.840559+0000) 2024-05-09T05:00:43.347 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:43 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5[46666]: 2024-05-09T05:00:43.202+0000 7fce452d0700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.141625+0000 front 2024-05-09T05:00:16.141553+0000 (oldest deadline 2024-05-09T05:00:39.041782+0000) 2024-05-09T05:00:43.600 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:43 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:43.346+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:00:43.600 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:43 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:43.346+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.223703+0000 front 2024-05-09T05:00:15.223669+0000 (oldest deadline 2024-05-09T05:00:39.923801+0000) 2024-05-09T05:00:43.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:43 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:43.249+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.771587+0000 front 2024-05-09T05:00:09.771610+0000 (oldest deadline 2024-05-09T05:00:33.772656+0000) 2024-05-09T05:00:43.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:43 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:43.249+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.073917+0000 front 2024-05-09T05:00:16.073768+0000 (oldest deadline 2024-05-09T05:00:40.774002+0000) 2024-05-09T05:00:43.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:43 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:43.263+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:43.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:43 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:43.263+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:00:43.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:43 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:43.263+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:00:43.942 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:43 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6[55562]: 2024-05-09T05:00:43.666+0000 7fdc6bc00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:04.396883+0000 front 2024-05-09T05:00:04.396963+0000 (oldest deadline 2024-05-09T05:00:36.198640+0000) 2024-05-09T05:00:43.942 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:43 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6[55562]: 2024-05-09T05:00:43.666+0000 7fdc6bc00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.199563+0000 front 2024-05-09T05:00:16.199511+0000 (oldest deadline 2024-05-09T05:00:41.499771+0000) 2024-05-09T05:00:44.197 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:43 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:43.941+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:44.198 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:43 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:43.941+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:00:44.198 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:43 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:43.941+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:00:44.328 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:44 smithi186 podman[70028]: 2024-05-09 05:00:44.106497829 +0000 UTC m=+5.589746528 container died 2e63acb9f07483bc08c4870162b4651b3a1abe221ce36e47f7296304e14d4927 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T05:00:44.329 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:44 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:44.014+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:44.329 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:44 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:44.014+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:00:44.329 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:44 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:44.328+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:00:44.329 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:44 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:44.328+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.223703+0000 front 2024-05-09T05:00:15.223669+0000 (oldest deadline 2024-05-09T05:00:39.923801+0000) 2024-05-09T05:00:44.600 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:44 smithi186 podman[70028]: 2024-05-09 05:00:44.504922345 +0000 UTC m=+5.988171087 container cleanup 2e63acb9f07483bc08c4870162b4651b3a1abe221ce36e47f7296304e14d4927 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T05:00:44.600 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:44 smithi186 bash[70028]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5 2024-05-09T05:00:44.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:44 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:44.258+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.771587+0000 front 2024-05-09T05:00:09.771610+0000 (oldest deadline 2024-05-09T05:00:33.772656+0000) 2024-05-09T05:00:44.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:44 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:44.258+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.073917+0000 front 2024-05-09T05:00:16.073768+0000 (oldest deadline 2024-05-09T05:00:40.774002+0000) 2024-05-09T05:00:44.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:44 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:44.264+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:44.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:44 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:44.264+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:00:44.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:44 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:44.264+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:00:44.947 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:44 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6[55562]: 2024-05-09T05:00:44.688+0000 7fdc6bc00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:04.396883+0000 front 2024-05-09T05:00:04.396963+0000 (oldest deadline 2024-05-09T05:00:36.198640+0000) 2024-05-09T05:00:44.947 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:44 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6[55562]: 2024-05-09T05:00:44.688+0000 7fdc6bc00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.199563+0000 front 2024-05-09T05:00:16.199511+0000 (oldest deadline 2024-05-09T05:00:41.499771+0000) 2024-05-09T05:00:45.050 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:44 smithi186 podman[70045]: 2024-05-09 05:00:44.739528797 +0000 UTC m=+0.631322069 container remove 2e63acb9f07483bc08c4870162b4651b3a1abe221ce36e47f7296304e14d4927 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T05:00:45.333 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:45 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:45.049+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:45.333 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:45 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:45.049+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:00:45.333 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:45 smithi186 podman[70132]: 2024-05-09 05:00:44.973311473 +0000 UTC m=+0.018240852 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:00:45.333 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:45 smithi186 podman[70132]: 2024-05-09 05:00:45.140609215 +0000 UTC m=+0.185538570 container create 706281ae64027552015741edf79c491e6bcfc525f4ea118e9f881f80b5a1e121 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-09T05:00:45.447 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:44 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:44.964+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:45.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:44 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:44.964+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:00:45.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:44 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:44.964+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:00:45.600 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:45 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:45.332+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:00:45.600 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:45 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:45.332+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.223703+0000 front 2024-05-09T05:00:15.223669+0000 (oldest deadline 2024-05-09T05:00:39.923801+0000) 2024-05-09T05:00:45.600 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:45 smithi186 podman[70132]: 2024-05-09 05:00:45.385332938 +0000 UTC m=+0.430262292 container init 706281ae64027552015741edf79c491e6bcfc525f4ea118e9f881f80b5a1e121 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True) 2024-05-09T05:00:45.600 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:45 smithi186 podman[70132]: 2024-05-09 05:00:45.388374552 +0000 UTC m=+0.433303915 container start 706281ae64027552015741edf79c491e6bcfc525f4ea118e9f881f80b5a1e121 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-09T05:00:45.600 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:45 smithi186 podman[70132]: 2024-05-09 05:00:45.49358774 +0000 UTC m=+0.538517104 container attach 706281ae64027552015741edf79c491e6bcfc525f4ea118e9f881f80b5a1e121 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-09T05:00:45.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:45 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:45.272+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.771587+0000 front 2024-05-09T05:00:09.771610+0000 (oldest deadline 2024-05-09T05:00:33.772656+0000) 2024-05-09T05:00:45.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:45 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:45.272+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.073917+0000 front 2024-05-09T05:00:16.073768+0000 (oldest deadline 2024-05-09T05:00:40.774002+0000) 2024-05-09T05:00:45.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:45 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:45.272+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:45.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:45 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:45.272+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:00:45.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:45 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:45.272+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:00:45.947 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:45 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6[55562]: 2024-05-09T05:00:45.688+0000 7fdc6bc00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:04.396883+0000 front 2024-05-09T05:00:04.396963+0000 (oldest deadline 2024-05-09T05:00:36.198640+0000) 2024-05-09T05:00:45.947 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:45 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6[55562]: 2024-05-09T05:00:45.688+0000 7fdc6bc00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.199563+0000 front 2024-05-09T05:00:16.199511+0000 (oldest deadline 2024-05-09T05:00:41.499771+0000) 2024-05-09T05:00:45.948 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:45 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:45.924+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:45.948 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:45 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:45.924+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:00:45.948 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:45 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:45.924+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:00:45.988 INFO:journalctl@ceph.osd.5.smithi186.stdout:May 09 05:00:45 smithi186 podman[70132]: 2024-05-09 05:00:45.682513658 +0000 UTC m=+0.727443026 container died 706281ae64027552015741edf79c491e6bcfc525f4ea118e9f881f80b5a1e121 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-09T05:00:46.286 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:46 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:46.041+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:46.286 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:46 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:46.041+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:00:46.313 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.5.service' 2024-05-09T05:00:46.350 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:00:46.351 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-09T05:00:46.351 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-09T05:00:46.351 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.6 2024-05-09T05:00:46.599 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:46 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:46.285+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:00:46.600 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:46 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:46.285+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.223703+0000 front 2024-05-09T05:00:15.223669+0000 (oldest deadline 2024-05-09T05:00:39.923801+0000) 2024-05-09T05:00:46.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:46 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:46.226+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.771587+0000 front 2024-05-09T05:00:09.771610+0000 (oldest deadline 2024-05-09T05:00:33.772656+0000) 2024-05-09T05:00:46.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:46 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:46.226+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.073917+0000 front 2024-05-09T05:00:16.073768+0000 (oldest deadline 2024-05-09T05:00:40.774002+0000) 2024-05-09T05:00:46.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:46 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:46.268+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:46.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:46 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:46.268+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:00:46.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:46 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:46.268+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:00:46.697 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:46 smithi006 systemd[1]: Stopping Ceph osd.6 for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:00:46.697 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:46 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6[55562]: 2024-05-09T05:00:46.650+0000 7fdc6bc00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:04.396883+0000 front 2024-05-09T05:00:04.396963+0000 (oldest deadline 2024-05-09T05:00:36.198640+0000) 2024-05-09T05:00:46.698 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:46 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6[55562]: 2024-05-09T05:00:46.650+0000 7fdc6bc00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.199563+0000 front 2024-05-09T05:00:16.199511+0000 (oldest deadline 2024-05-09T05:00:41.499771+0000) 2024-05-09T05:00:47.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:47 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:47.264+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:00:47.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:47 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:47.264+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.223703+0000 front 2024-05-09T05:00:15.223669+0000 (oldest deadline 2024-05-09T05:00:39.923801+0000) 2024-05-09T05:00:47.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:47 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:47.026+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:47.351 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:47 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:47.026+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:00:47.447 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:46 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6[55562]: 2024-05-09T05:00:46.996+0000 7fdc70400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:00:47.447 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:46 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6[55562]: 2024-05-09T05:00:46.996+0000 7fdc70400700 -1 osd.6 93 *** Got signal Terminated *** 2024-05-09T05:00:47.448 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:46 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6[55562]: 2024-05-09T05:00:46.996+0000 7fdc70400700 -1 osd.6 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T05:00:47.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:46 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:46.964+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:47.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:46 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:46.964+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:00:47.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:46 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:46.964+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:00:47.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:47 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:47.190+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.771587+0000 front 2024-05-09T05:00:09.771610+0000 (oldest deadline 2024-05-09T05:00:33.772656+0000) 2024-05-09T05:00:47.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:47 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:47.190+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.073917+0000 front 2024-05-09T05:00:16.073768+0000 (oldest deadline 2024-05-09T05:00:40.774002+0000) 2024-05-09T05:00:47.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:47 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:47.290+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:47.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:47 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:47.290+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:00:47.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:47 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:47.290+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:00:47.917 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:47 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6[55562]: 2024-05-09T05:00:47.664+0000 7fdc6bc00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:04.396883+0000 front 2024-05-09T05:00:04.396963+0000 (oldest deadline 2024-05-09T05:00:36.198640+0000) 2024-05-09T05:00:47.917 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:47 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6[55562]: 2024-05-09T05:00:47.664+0000 7fdc6bc00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.199563+0000 front 2024-05-09T05:00:16.199511+0000 (oldest deadline 2024-05-09T05:00:41.499771+0000) 2024-05-09T05:00:48.198 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:47 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:47.916+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:48.198 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:47 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:47.916+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:00:48.198 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:47 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:47.916+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:00:48.198 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:47 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:47.916+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.999302+0000 front 2024-05-09T05:00:25.999270+0000 (oldest deadline 2024-05-09T05:00:47.099558+0000) 2024-05-09T05:00:48.269 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:48 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:48.012+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:48.270 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:48 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:48.012+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:00:48.270 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:48 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:48.236+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:00:48.270 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:48 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:48.236+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.223703+0000 front 2024-05-09T05:00:15.223669+0000 (oldest deadline 2024-05-09T05:00:39.923801+0000) 2024-05-09T05:00:48.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:48 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:48.148+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.771587+0000 front 2024-05-09T05:00:09.771610+0000 (oldest deadline 2024-05-09T05:00:33.772656+0000) 2024-05-09T05:00:48.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:48 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:48.148+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.073917+0000 front 2024-05-09T05:00:16.073768+0000 (oldest deadline 2024-05-09T05:00:40.774002+0000) 2024-05-09T05:00:48.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:48 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:48.148+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.276519+0000 front 2024-05-09T05:00:25.276423+0000 (oldest deadline 2024-05-09T05:00:47.576312+0000) 2024-05-09T05:00:48.618 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:48 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:48.279+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:48.618 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:48 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:48.279+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:00:48.618 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:48 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:48.279+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:00:48.947 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:48 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6[55562]: 2024-05-09T05:00:48.685+0000 7fdc6bc00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:04.396883+0000 front 2024-05-09T05:00:04.396963+0000 (oldest deadline 2024-05-09T05:00:36.198640+0000) 2024-05-09T05:00:48.948 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:48 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6[55562]: 2024-05-09T05:00:48.685+0000 7fdc6bc00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.199563+0000 front 2024-05-09T05:00:16.199511+0000 (oldest deadline 2024-05-09T05:00:41.499771+0000) 2024-05-09T05:00:48.948 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:48 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:48.919+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:48.948 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:48 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:48.919+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:00:48.948 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:48 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:48.919+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:00:48.948 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:48 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:48.919+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.999302+0000 front 2024-05-09T05:00:25.999270+0000 (oldest deadline 2024-05-09T05:00:47.099558+0000) 2024-05-09T05:00:49.349 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:49 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:49.187+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:00:49.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:49 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:49.187+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.223703+0000 front 2024-05-09T05:00:15.223669+0000 (oldest deadline 2024-05-09T05:00:39.923801+0000) 2024-05-09T05:00:49.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:49 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:48.998+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:49.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:49 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:48.998+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:00:49.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:49 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:49.174+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.771587+0000 front 2024-05-09T05:00:09.771610+0000 (oldest deadline 2024-05-09T05:00:33.772656+0000) 2024-05-09T05:00:49.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:49 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:49.174+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.073917+0000 front 2024-05-09T05:00:16.073768+0000 (oldest deadline 2024-05-09T05:00:40.774002+0000) 2024-05-09T05:00:49.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:49 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:49.174+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.276519+0000 front 2024-05-09T05:00:25.276423+0000 (oldest deadline 2024-05-09T05:00:47.576312+0000) 2024-05-09T05:00:49.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:49 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:49.250+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:49.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:49 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:49.250+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:00:49.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:49 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:49.250+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:00:49.947 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:49 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6[55562]: 2024-05-09T05:00:49.645+0000 7fdc6bc00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:04.396883+0000 front 2024-05-09T05:00:04.396963+0000 (oldest deadline 2024-05-09T05:00:36.198640+0000) 2024-05-09T05:00:49.947 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:49 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6[55562]: 2024-05-09T05:00:49.645+0000 7fdc6bc00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.199563+0000 front 2024-05-09T05:00:16.199511+0000 (oldest deadline 2024-05-09T05:00:41.499771+0000) 2024-05-09T05:00:49.948 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:49 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:49.895+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:49.948 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:49 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:49.895+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:00:49.948 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:49 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:49.895+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:00:49.948 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:49 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:49.895+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.999302+0000 front 2024-05-09T05:00:25.999270+0000 (oldest deadline 2024-05-09T05:00:47.099558+0000) 2024-05-09T05:00:50.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:50 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:50.192+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:00:50.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:50 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:50.192+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.223703+0000 front 2024-05-09T05:00:15.223669+0000 (oldest deadline 2024-05-09T05:00:39.923801+0000) 2024-05-09T05:00:50.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:49 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:49.952+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:50.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:49 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:49.952+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:00:50.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:50 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:50.186+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.771587+0000 front 2024-05-09T05:00:09.771610+0000 (oldest deadline 2024-05-09T05:00:33.772656+0000) 2024-05-09T05:00:50.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:50 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:50.186+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.073917+0000 front 2024-05-09T05:00:16.073768+0000 (oldest deadline 2024-05-09T05:00:40.774002+0000) 2024-05-09T05:00:50.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:50 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:50.186+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.276519+0000 front 2024-05-09T05:00:25.276423+0000 (oldest deadline 2024-05-09T05:00:47.576312+0000) 2024-05-09T05:00:50.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:50 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:50.271+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:50.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:50 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:50.271+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:00:50.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:50 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:50.271+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:00:50.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:50 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:50.271+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:00:50.916 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:50 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6[55562]: 2024-05-09T05:00:50.664+0000 7fdc6bc00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:04.396883+0000 front 2024-05-09T05:00:04.396963+0000 (oldest deadline 2024-05-09T05:00:36.198640+0000) 2024-05-09T05:00:50.916 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:50 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6[55562]: 2024-05-09T05:00:50.664+0000 7fdc6bc00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.199563+0000 front 2024-05-09T05:00:16.199511+0000 (oldest deadline 2024-05-09T05:00:41.499771+0000) 2024-05-09T05:00:50.916 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:50 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6[55562]: 2024-05-09T05:00:50.664+0000 7fdc6bc00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:24.402610+0000 front 2024-05-09T05:00:24.403413+0000 (oldest deadline 2024-05-09T05:00:50.302719+0000) 2024-05-09T05:00:51.197 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:50 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:50.915+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:51.198 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:50 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:50.915+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:00:51.198 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:50 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:50.915+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:00:51.198 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:50 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:50.915+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.999302+0000 front 2024-05-09T05:00:25.999270+0000 (oldest deadline 2024-05-09T05:00:47.099558+0000) 2024-05-09T05:00:51.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:50 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:50.997+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:51.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:50 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:50.997+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:00:51.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:51 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:51.193+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:00:51.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:51 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:51.193+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.223703+0000 front 2024-05-09T05:00:15.223669+0000 (oldest deadline 2024-05-09T05:00:39.923801+0000) 2024-05-09T05:00:51.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:51 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:51.193+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.125777+0000 front 2024-05-09T05:00:25.125791+0000 (oldest deadline 2024-05-09T05:00:51.025568+0000) 2024-05-09T05:00:51.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:51 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:51.228+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.771587+0000 front 2024-05-09T05:00:09.771610+0000 (oldest deadline 2024-05-09T05:00:33.772656+0000) 2024-05-09T05:00:51.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:51 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:51.228+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.073917+0000 front 2024-05-09T05:00:16.073768+0000 (oldest deadline 2024-05-09T05:00:40.774002+0000) 2024-05-09T05:00:51.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:51 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:51.228+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.276519+0000 front 2024-05-09T05:00:25.276423+0000 (oldest deadline 2024-05-09T05:00:47.576312+0000) 2024-05-09T05:00:51.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:51 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:51.316+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:51.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:51 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:51.316+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:00:51.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:51 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:51.316+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:00:51.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:51 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:51.316+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:00:51.947 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:51 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6[55562]: 2024-05-09T05:00:51.680+0000 7fdc6bc00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:04.396883+0000 front 2024-05-09T05:00:04.396963+0000 (oldest deadline 2024-05-09T05:00:36.198640+0000) 2024-05-09T05:00:51.948 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:51 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6[55562]: 2024-05-09T05:00:51.680+0000 7fdc6bc00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.199563+0000 front 2024-05-09T05:00:16.199511+0000 (oldest deadline 2024-05-09T05:00:41.499771+0000) 2024-05-09T05:00:51.948 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:51 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6[55562]: 2024-05-09T05:00:51.680+0000 7fdc6bc00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:24.402610+0000 front 2024-05-09T05:00:24.403413+0000 (oldest deadline 2024-05-09T05:00:50.302719+0000) 2024-05-09T05:00:52.232 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:51 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:51.959+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:52.232 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:51 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:51.959+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:00:52.232 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:51 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:51.959+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:00:52.232 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:51 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:51.959+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.999302+0000 front 2024-05-09T05:00:25.999270+0000 (oldest deadline 2024-05-09T05:00:47.099558+0000) 2024-05-09T05:00:52.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:52 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:52.005+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:52.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:52 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:52.005+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:00:52.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:52 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:52.241+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:00:52.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:52 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:52.241+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.223703+0000 front 2024-05-09T05:00:15.223669+0000 (oldest deadline 2024-05-09T05:00:39.923801+0000) 2024-05-09T05:00:52.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:52 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:52.241+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.125777+0000 front 2024-05-09T05:00:25.125791+0000 (oldest deadline 2024-05-09T05:00:51.025568+0000) 2024-05-09T05:00:52.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:52 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:52.206+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.771587+0000 front 2024-05-09T05:00:09.771610+0000 (oldest deadline 2024-05-09T05:00:33.772656+0000) 2024-05-09T05:00:52.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:52 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:52.206+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.073917+0000 front 2024-05-09T05:00:16.073768+0000 (oldest deadline 2024-05-09T05:00:40.774002+0000) 2024-05-09T05:00:52.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:52 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:52.206+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.276519+0000 front 2024-05-09T05:00:25.276423+0000 (oldest deadline 2024-05-09T05:00:47.576312+0000) 2024-05-09T05:00:52.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:52 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:52.360+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:52.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:52 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:52.360+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:00:52.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:52 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:52.360+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:00:52.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:52 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:52.360+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:00:52.622 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:52 smithi006 podman[94661]: 2024-05-09 05:00:52.482024663 +0000 UTC m=+5.628903421 container died 007f7da53666beb05915bea637dfab860f73ac2beb3a343b7b46e01224b909c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-05-09T05:00:53.198 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:53 smithi006 podman[94661]: 2024-05-09 05:00:53.064762557 +0000 UTC m=+6.211641315 container cleanup 007f7da53666beb05915bea637dfab860f73ac2beb3a343b7b46e01224b909c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-09T05:00:53.198 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:53 smithi006 bash[94661]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6 2024-05-09T05:00:53.198 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:53 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:52.999+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:53.198 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:53 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:52.999+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:00:53.198 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:53 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:52.999+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:00:53.198 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:53 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:52.999+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.999302+0000 front 2024-05-09T05:00:25.999270+0000 (oldest deadline 2024-05-09T05:00:47.099558+0000) 2024-05-09T05:00:53.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:53 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:53.219+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:00:53.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:53 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:53.219+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.223703+0000 front 2024-05-09T05:00:15.223669+0000 (oldest deadline 2024-05-09T05:00:39.923801+0000) 2024-05-09T05:00:53.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:53 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:53.219+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.125777+0000 front 2024-05-09T05:00:25.125791+0000 (oldest deadline 2024-05-09T05:00:51.025568+0000) 2024-05-09T05:00:53.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:53 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:53.035+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:53.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:53 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:53.035+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:00:53.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:53 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:53.035+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:00:53.518 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:53 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:53.212+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.771587+0000 front 2024-05-09T05:00:09.771610+0000 (oldest deadline 2024-05-09T05:00:33.772656+0000) 2024-05-09T05:00:53.518 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:53 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:53.212+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.073917+0000 front 2024-05-09T05:00:16.073768+0000 (oldest deadline 2024-05-09T05:00:40.774002+0000) 2024-05-09T05:00:53.519 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:53 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:53.212+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.276519+0000 front 2024-05-09T05:00:25.276423+0000 (oldest deadline 2024-05-09T05:00:47.576312+0000) 2024-05-09T05:00:53.519 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:53 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:53.386+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:53.519 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:53 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:53.386+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:00:53.519 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:53 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:53.386+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:00:53.519 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:53 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:53.386+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:00:53.697 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:53 smithi006 podman[94680]: 2024-05-09 05:00:53.32466389 +0000 UTC m=+0.834768661 container remove 007f7da53666beb05915bea637dfab860f73ac2beb3a343b7b46e01224b909c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS) 2024-05-09T05:00:54.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:54 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:54.246+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:00:54.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:54 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:54.246+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.223703+0000 front 2024-05-09T05:00:15.223669+0000 (oldest deadline 2024-05-09T05:00:39.923801+0000) 2024-05-09T05:00:54.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:54 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:54.246+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.125777+0000 front 2024-05-09T05:00:25.125791+0000 (oldest deadline 2024-05-09T05:00:51.025568+0000) 2024-05-09T05:00:54.351 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:54 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:54.054+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:54.351 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:54 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:54.054+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:00:54.351 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:54 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:54.054+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:00:54.395 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:54 smithi006 podman[94780]: 2024-05-09 05:00:54.078349273 +0000 UTC m=+0.038661363 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:00:54.395 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:54 smithi006 podman[94780]: 2024-05-09 05:00:54.22582737 +0000 UTC m=+0.186139437 container create 8293b940acc034647ba0f91d5e9db44a5f522dca33b2561701f0df92d2be6bd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5) 2024-05-09T05:00:54.395 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:54 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:54.038+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:54.395 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:54 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:54.038+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:00:54.395 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:54 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:54.038+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:00:54.395 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:54 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:54.038+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.999302+0000 front 2024-05-09T05:00:25.999270+0000 (oldest deadline 2024-05-09T05:00:47.099558+0000) 2024-05-09T05:00:54.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:54 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:54.188+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.771587+0000 front 2024-05-09T05:00:09.771610+0000 (oldest deadline 2024-05-09T05:00:33.772656+0000) 2024-05-09T05:00:54.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:54 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:54.188+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.073917+0000 front 2024-05-09T05:00:16.073768+0000 (oldest deadline 2024-05-09T05:00:40.774002+0000) 2024-05-09T05:00:54.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:54 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:54.188+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.276519+0000 front 2024-05-09T05:00:25.276423+0000 (oldest deadline 2024-05-09T05:00:47.576312+0000) 2024-05-09T05:00:54.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:54 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:54.363+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:54.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:54 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:54.363+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:00:54.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:54 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:54.363+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:00:54.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:54 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:54.363+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:00:54.698 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:54 smithi006 podman[94780]: 2024-05-09 05:00:54.529845481 +0000 UTC m=+0.490157588 container init 8293b940acc034647ba0f91d5e9db44a5f522dca33b2561701f0df92d2be6bd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T05:00:54.698 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:54 smithi006 podman[94780]: 2024-05-09 05:00:54.542245887 +0000 UTC m=+0.502557934 container start 8293b940acc034647ba0f91d5e9db44a5f522dca33b2561701f0df92d2be6bd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-09T05:00:54.698 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:54 smithi006 podman[94780]: 2024-05-09 05:00:54.636756137 +0000 UTC m=+0.597068261 container attach 8293b940acc034647ba0f91d5e9db44a5f522dca33b2561701f0df92d2be6bd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6-deactivate, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-09T05:00:55.343 INFO:journalctl@ceph.osd.6.smithi006.stdout:May 09 05:00:55 smithi006 podman[94780]: 2024-05-09 05:00:55.006783017 +0000 UTC m=+0.967095094 container died 8293b940acc034647ba0f91d5e9db44a5f522dca33b2561701f0df92d2be6bd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-6-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2) 2024-05-09T05:00:55.344 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:55 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:55.069+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:55.344 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:55 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:55.069+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:00:55.344 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:55 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:55.069+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:00:55.344 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:55 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:55.069+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.999302+0000 front 2024-05-09T05:00:25.999270+0000 (oldest deadline 2024-05-09T05:00:47.099558+0000) 2024-05-09T05:00:55.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:55 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:55.196+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:00:55.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:55 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:55.197+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.223703+0000 front 2024-05-09T05:00:15.223669+0000 (oldest deadline 2024-05-09T05:00:39.923801+0000) 2024-05-09T05:00:55.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:55 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:55.197+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.125777+0000 front 2024-05-09T05:00:25.125791+0000 (oldest deadline 2024-05-09T05:00:51.025568+0000) 2024-05-09T05:00:55.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:55 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:55.038+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:55.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:55 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:55.038+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:00:55.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:55 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:55.038+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:00:55.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:55 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:55.175+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.771587+0000 front 2024-05-09T05:00:09.771610+0000 (oldest deadline 2024-05-09T05:00:33.772656+0000) 2024-05-09T05:00:55.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:55 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:55.175+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.073917+0000 front 2024-05-09T05:00:16.073768+0000 (oldest deadline 2024-05-09T05:00:40.774002+0000) 2024-05-09T05:00:55.617 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:55 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:55.175+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.276519+0000 front 2024-05-09T05:00:25.276423+0000 (oldest deadline 2024-05-09T05:00:47.576312+0000) 2024-05-09T05:00:55.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:55 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:55.373+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:55.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:55 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:55.373+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:00:55.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:55 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:55.373+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:00:55.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:55 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:55.373+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:00:55.755 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.6.service' 2024-05-09T05:00:55.870 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:00:55.870 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-09T05:00:55.871 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-09T05:00:55.871 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.7 2024-05-09T05:00:56.181 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:55 smithi169 systemd[1]: Stopping Ceph osd.7 for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:00:56.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:56 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:56.155+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:00:56.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:56 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:56.155+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.223703+0000 front 2024-05-09T05:00:15.223669+0000 (oldest deadline 2024-05-09T05:00:39.923801+0000) 2024-05-09T05:00:56.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:56 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:56.155+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.125777+0000 front 2024-05-09T05:00:25.125791+0000 (oldest deadline 2024-05-09T05:00:51.025568+0000) 2024-05-09T05:00:56.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:56 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:56.016+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:56.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:56 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:56.017+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:00:56.351 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:56 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:56.017+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:00:56.447 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:56 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:56.085+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:56.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:56 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:56.085+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:00:56.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:56 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:56.085+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:00:56.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:56 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:56.085+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.999302+0000 front 2024-05-09T05:00:25.999270+0000 (oldest deadline 2024-05-09T05:00:47.099558+0000) 2024-05-09T05:00:56.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:56 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:56.421+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:56.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:56 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:56.421+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:00:56.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:56 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:56.421+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:00:56.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:56 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:56.421+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:00:56.618 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:56 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:56.181+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.771587+0000 front 2024-05-09T05:00:09.771610+0000 (oldest deadline 2024-05-09T05:00:33.772656+0000) 2024-05-09T05:00:56.618 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:56 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:56.181+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.073917+0000 front 2024-05-09T05:00:16.073768+0000 (oldest deadline 2024-05-09T05:00:40.774002+0000) 2024-05-09T05:00:56.618 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:56 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:56.181+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.276519+0000 front 2024-05-09T05:00:25.276423+0000 (oldest deadline 2024-05-09T05:00:47.576312+0000) 2024-05-09T05:00:56.618 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:56 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:56.194+0000 7f0a46331700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:00:56.618 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:56 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:56.194+0000 7f0a46331700 -1 osd.7 93 *** Got signal Terminated *** 2024-05-09T05:00:56.618 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:56 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:56.194+0000 7f0a46331700 -1 osd.7 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T05:00:57.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:57 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:57.147+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:00:57.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:57 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:57.147+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.223703+0000 front 2024-05-09T05:00:15.223669+0000 (oldest deadline 2024-05-09T05:00:39.923801+0000) 2024-05-09T05:00:57.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:57 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:57.147+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.125777+0000 front 2024-05-09T05:00:25.125791+0000 (oldest deadline 2024-05-09T05:00:51.025568+0000) 2024-05-09T05:00:57.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:56 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:56.971+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:57.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:56 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:56.971+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:00:57.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:56 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:56.971+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:00:57.441 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:57 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:57.149+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.771587+0000 front 2024-05-09T05:00:09.771610+0000 (oldest deadline 2024-05-09T05:00:33.772656+0000) 2024-05-09T05:00:57.441 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:57 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:57.149+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.073917+0000 front 2024-05-09T05:00:16.073768+0000 (oldest deadline 2024-05-09T05:00:40.774002+0000) 2024-05-09T05:00:57.442 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:57 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:57.149+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.276519+0000 front 2024-05-09T05:00:25.276423+0000 (oldest deadline 2024-05-09T05:00:47.576312+0000) 2024-05-09T05:00:57.447 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:57 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:57.070+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:57.447 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:57 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:57.070+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:00:57.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:57 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:57.070+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:00:57.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:57 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:57.070+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.999302+0000 front 2024-05-09T05:00:25.999270+0000 (oldest deadline 2024-05-09T05:00:47.099558+0000) 2024-05-09T05:00:57.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:57 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:57.070+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:32.303524+0000 front 2024-05-09T05:00:32.303793+0000 (oldest deadline 2024-05-09T05:00:56.403805+0000) 2024-05-09T05:00:57.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:57 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:57.441+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:57.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:57 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:57.441+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:00:57.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:57 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:57.441+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:00:57.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:57 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:57.441+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:00:58.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:57 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:57.969+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:58.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:57 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:57.969+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:00:58.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:57 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:57.969+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:00:58.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:58 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:58.117+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:00:58.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:58 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:58.117+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.223703+0000 front 2024-05-09T05:00:15.223669+0000 (oldest deadline 2024-05-09T05:00:39.923801+0000) 2024-05-09T05:00:58.351 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:58 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:58.117+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.125777+0000 front 2024-05-09T05:00:25.125791+0000 (oldest deadline 2024-05-09T05:00:51.025568+0000) 2024-05-09T05:00:58.351 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:58 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:58.117+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:32.726856+0000 front 2024-05-09T05:00:32.726755+0000 (oldest deadline 2024-05-09T05:00:57.426618+0000) 2024-05-09T05:00:58.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:58 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:58.078+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:58.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:58 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:58.078+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:00:58.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:58 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:58.078+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:00:58.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:58 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:58.078+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.999302+0000 front 2024-05-09T05:00:25.999270+0000 (oldest deadline 2024-05-09T05:00:47.099558+0000) 2024-05-09T05:00:58.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:58 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:58.079+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:32.303524+0000 front 2024-05-09T05:00:32.303793+0000 (oldest deadline 2024-05-09T05:00:56.403805+0000) 2024-05-09T05:00:58.454 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:58 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:58.194+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.771587+0000 front 2024-05-09T05:00:09.771610+0000 (oldest deadline 2024-05-09T05:00:33.772656+0000) 2024-05-09T05:00:58.454 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:58 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:58.194+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.073917+0000 front 2024-05-09T05:00:16.073768+0000 (oldest deadline 2024-05-09T05:00:40.774002+0000) 2024-05-09T05:00:58.454 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:58 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:58.194+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.276519+0000 front 2024-05-09T05:00:25.276423+0000 (oldest deadline 2024-05-09T05:00:47.576312+0000) 2024-05-09T05:00:58.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:58 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:58.453+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:58.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:58 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:58.453+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:00:58.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:58 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:58.453+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:00:58.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:58 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:58.453+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:00:59.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:58 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:58.951+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:00:59.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:58 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:58.951+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:00:59.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:58 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:58.951+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:00:59.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:58 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:58.951+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:00:59.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:59 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:59.131+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:00:59.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:59 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:59.131+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.223703+0000 front 2024-05-09T05:00:15.223669+0000 (oldest deadline 2024-05-09T05:00:39.923801+0000) 2024-05-09T05:00:59.351 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:59 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:59.131+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.125777+0000 front 2024-05-09T05:00:25.125791+0000 (oldest deadline 2024-05-09T05:00:51.025568+0000) 2024-05-09T05:00:59.351 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:00:59 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:00:59.131+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:32.726856+0000 front 2024-05-09T05:00:32.726755+0000 (oldest deadline 2024-05-09T05:00:57.426618+0000) 2024-05-09T05:00:59.447 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:59 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:59.125+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:00:59.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:59 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:59.125+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:00:59.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:59 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:59.125+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:00:59.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:59 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:59.125+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.999302+0000 front 2024-05-09T05:00:25.999270+0000 (oldest deadline 2024-05-09T05:00:47.099558+0000) 2024-05-09T05:00:59.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:00:59 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:00:59.125+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:32.303524+0000 front 2024-05-09T05:00:32.303793+0000 (oldest deadline 2024-05-09T05:00:56.403805+0000) 2024-05-09T05:00:59.497 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:59 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:59.207+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.771587+0000 front 2024-05-09T05:00:09.771610+0000 (oldest deadline 2024-05-09T05:00:33.772656+0000) 2024-05-09T05:00:59.497 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:59 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:59.207+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.073917+0000 front 2024-05-09T05:00:16.073768+0000 (oldest deadline 2024-05-09T05:00:40.774002+0000) 2024-05-09T05:00:59.497 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:59 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:59.207+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.276519+0000 front 2024-05-09T05:00:25.276423+0000 (oldest deadline 2024-05-09T05:00:47.576312+0000) 2024-05-09T05:00:59.497 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:00:59 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:00:59.207+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:34.577853+0000 front 2024-05-09T05:00:34.577901+0000 (oldest deadline 2024-05-09T05:00:58.677977+0000) 2024-05-09T05:00:59.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:59 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:59.496+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:00:59.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:59 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:59.496+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:00:59.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:59 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:59.496+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:00:59.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:00:59 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:00:59.496+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:01:00.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:00 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:00.154+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:01:00.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:00 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:00.154+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.223703+0000 front 2024-05-09T05:00:15.223669+0000 (oldest deadline 2024-05-09T05:00:39.923801+0000) 2024-05-09T05:01:00.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:00 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:00.154+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.125777+0000 front 2024-05-09T05:00:25.125791+0000 (oldest deadline 2024-05-09T05:00:51.025568+0000) 2024-05-09T05:01:00.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:00 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:00.154+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:32.726856+0000 front 2024-05-09T05:00:32.726755+0000 (oldest deadline 2024-05-09T05:00:57.426618+0000) 2024-05-09T05:01:00.351 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:59 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:59.940+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:00.351 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:59 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:59.940+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:00.351 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:59 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:59.940+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:00.351 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:00:59 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:00:59.940+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:00.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:00 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:00.160+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:01:00.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:00 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:00.160+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:01:00.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:00 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:00.160+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:01:00.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:00 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:00.160+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.999302+0000 front 2024-05-09T05:00:25.999270+0000 (oldest deadline 2024-05-09T05:00:47.099558+0000) 2024-05-09T05:01:00.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:00 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:00.161+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:32.303524+0000 front 2024-05-09T05:00:32.303793+0000 (oldest deadline 2024-05-09T05:00:56.403805+0000) 2024-05-09T05:01:00.483 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:01:00 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:01:00.173+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.771587+0000 front 2024-05-09T05:00:09.771610+0000 (oldest deadline 2024-05-09T05:00:33.772656+0000) 2024-05-09T05:01:00.483 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:01:00 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:01:00.173+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.073917+0000 front 2024-05-09T05:00:16.073768+0000 (oldest deadline 2024-05-09T05:00:40.774002+0000) 2024-05-09T05:01:00.483 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:01:00 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:01:00.173+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.276519+0000 front 2024-05-09T05:00:25.276423+0000 (oldest deadline 2024-05-09T05:00:47.576312+0000) 2024-05-09T05:01:00.483 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:01:00 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:01:00.173+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:34.577853+0000 front 2024-05-09T05:00:34.577901+0000 (oldest deadline 2024-05-09T05:00:58.677977+0000) 2024-05-09T05:01:00.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:00 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:00.482+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:01:00.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:00 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:00.482+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:01:00.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:00 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:00.482+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:01:00.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:00 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:00.482+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:01:01.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:01 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:01.160+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:01:01.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:01 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:01.160+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.223703+0000 front 2024-05-09T05:00:15.223669+0000 (oldest deadline 2024-05-09T05:00:39.923801+0000) 2024-05-09T05:01:01.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:01 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:01.160+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.125777+0000 front 2024-05-09T05:00:25.125791+0000 (oldest deadline 2024-05-09T05:00:51.025568+0000) 2024-05-09T05:01:01.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:01 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:01.160+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:32.726856+0000 front 2024-05-09T05:00:32.726755+0000 (oldest deadline 2024-05-09T05:00:57.426618+0000) 2024-05-09T05:01:01.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:00 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:00.954+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:01.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:00 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:00.954+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:01.351 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:00 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:00.954+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:01.351 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:00 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:00.954+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:01.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:01 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:01.133+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:01:01.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:01 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:01.133+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:01:01.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:01 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:01.133+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:01:01.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:01 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:01.133+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.999302+0000 front 2024-05-09T05:00:25.999270+0000 (oldest deadline 2024-05-09T05:00:47.099558+0000) 2024-05-09T05:01:01.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:01 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:01.133+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:32.303524+0000 front 2024-05-09T05:00:32.303793+0000 (oldest deadline 2024-05-09T05:00:56.403805+0000) 2024-05-09T05:01:01.448 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:01:01 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:01:01.171+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:09.771587+0000 front 2024-05-09T05:00:09.771610+0000 (oldest deadline 2024-05-09T05:00:33.772656+0000) 2024-05-09T05:01:01.449 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:01:01 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:01:01.171+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.073917+0000 front 2024-05-09T05:00:16.073768+0000 (oldest deadline 2024-05-09T05:00:40.774002+0000) 2024-05-09T05:01:01.449 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:01:01 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:01:01.171+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.276519+0000 front 2024-05-09T05:00:25.276423+0000 (oldest deadline 2024-05-09T05:00:47.576312+0000) 2024-05-09T05:01:01.449 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:01:01 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7[52344]: 2024-05-09T05:01:01.171+0000 7f0a428e7700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:34.577853+0000 front 2024-05-09T05:00:34.577901+0000 (oldest deadline 2024-05-09T05:00:58.677977+0000) 2024-05-09T05:01:01.803 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:01:01 smithi169 podman[69329]: 2024-05-09 05:01:01.521640147 +0000 UTC m=+5.447427137 container died e67e07953657c1d541e0d83b408e9648d338e76140ea608d1ed9b5c3eb856bd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T05:01:01.803 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:01 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:01.448+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:01:01.803 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:01 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:01.448+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:01:01.803 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:01 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:01.448+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:01:01.803 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:01 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:01.448+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:01:01.804 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:01 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:01.448+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:35.512200+0000 front 2024-05-09T05:00:35.512344+0000 (oldest deadline 2024-05-09T05:01:00.812363+0000) 2024-05-09T05:01:02.117 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:01:01 smithi169 podman[69329]: 2024-05-09 05:01:01.911560753 +0000 UTC m=+5.837347742 container cleanup e67e07953657c1d541e0d83b408e9648d338e76140ea608d1ed9b5c3eb856bd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-09T05:01:02.117 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:01:01 smithi169 bash[69329]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7 2024-05-09T05:01:02.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:02 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:02.147+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:01:02.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:02 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:02.147+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.223703+0000 front 2024-05-09T05:00:15.223669+0000 (oldest deadline 2024-05-09T05:00:39.923801+0000) 2024-05-09T05:01:02.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:02 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:02.147+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.125777+0000 front 2024-05-09T05:00:25.125791+0000 (oldest deadline 2024-05-09T05:00:51.025568+0000) 2024-05-09T05:01:02.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:02 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:02.147+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:32.726856+0000 front 2024-05-09T05:00:32.726755+0000 (oldest deadline 2024-05-09T05:00:57.426618+0000) 2024-05-09T05:01:02.351 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:01 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:01.965+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:02.351 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:01 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:01.965+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:02.351 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:01 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:01.965+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:02.351 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:01 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:01.965+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:02.438 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:01:02 smithi169 podman[69348]: 2024-05-09 05:01:02.163005325 +0000 UTC m=+0.638645503 container remove e67e07953657c1d541e0d83b408e9648d338e76140ea608d1ed9b5c3eb856bd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-09T05:01:02.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:02 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:02.156+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:01:02.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:02 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:02.156+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:01:02.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:02 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:02.156+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:01:02.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:02 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:02.156+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.999302+0000 front 2024-05-09T05:00:25.999270+0000 (oldest deadline 2024-05-09T05:00:47.099558+0000) 2024-05-09T05:01:02.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:02 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:02.156+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:32.303524+0000 front 2024-05-09T05:00:32.303793+0000 (oldest deadline 2024-05-09T05:00:56.403805+0000) 2024-05-09T05:01:02.741 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:01:02 smithi169 podman[69456]: 2024-05-09 05:01:02.388545263 +0000 UTC m=+0.018104494 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:01:02.741 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:01:02 smithi169 podman[69456]: 2024-05-09 05:01:02.497485053 +0000 UTC m=+0.127044326 container create a95b2e8953edda5dcc67bbd6b65d0265d487d2984e6ff685537549ddc3f34aba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=) 2024-05-09T05:01:02.741 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:01:02 smithi169 podman[69456]: 2024-05-09 05:01:02.741876334 +0000 UTC m=+0.371435565 container init a95b2e8953edda5dcc67bbd6b65d0265d487d2984e6ff685537549ddc3f34aba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T05:01:02.742 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:02 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:02.437+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:01:02.742 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:02 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:02.437+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:01:02.742 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:02 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:02.437+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:01:02.742 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:02 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:02.437+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:01:02.742 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:02 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:02.437+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:35.512200+0000 front 2024-05-09T05:00:35.512344+0000 (oldest deadline 2024-05-09T05:01:00.812363+0000) 2024-05-09T05:01:03.035 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:01:02 smithi169 podman[69456]: 2024-05-09 05:01:02.74510348 +0000 UTC m=+0.374662721 container start a95b2e8953edda5dcc67bbd6b65d0265d487d2984e6ff685537549ddc3f34aba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-09T05:01:03.035 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:01:02 smithi169 podman[69456]: 2024-05-09 05:01:02.833493273 +0000 UTC m=+0.463052507 container attach a95b2e8953edda5dcc67bbd6b65d0265d487d2984e6ff685537549ddc3f34aba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-09T05:01:03.336 INFO:journalctl@ceph.osd.7.smithi169.stdout:May 09 05:01:03 smithi169 podman[69456]: 2024-05-09 05:01:03.036015079 +0000 UTC m=+0.665574308 container died a95b2e8953edda5dcc67bbd6b65d0265d487d2984e6ff685537549ddc3f34aba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T05:01:03.349 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:03 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:03.177+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:01:03.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:03 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:03.177+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.223703+0000 front 2024-05-09T05:00:15.223669+0000 (oldest deadline 2024-05-09T05:00:39.923801+0000) 2024-05-09T05:01:03.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:03 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:03.177+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.125777+0000 front 2024-05-09T05:00:25.125791+0000 (oldest deadline 2024-05-09T05:00:51.025568+0000) 2024-05-09T05:01:03.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:03 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:03.177+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:32.726856+0000 front 2024-05-09T05:00:32.726755+0000 (oldest deadline 2024-05-09T05:00:57.426618+0000) 2024-05-09T05:01:03.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:02 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:02.948+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:03.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:02 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:02.948+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:03.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:02 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:02.948+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:03.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:02 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:02.948+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:03.447 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:03 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:03.135+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:01:03.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:03 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:03.135+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:01:03.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:03 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:03.135+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:01:03.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:03 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:03.135+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.999302+0000 front 2024-05-09T05:00:25.999270+0000 (oldest deadline 2024-05-09T05:00:47.099558+0000) 2024-05-09T05:01:03.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:03 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:03.136+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:32.303524+0000 front 2024-05-09T05:00:32.303793+0000 (oldest deadline 2024-05-09T05:00:56.403805+0000) 2024-05-09T05:01:03.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:03 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:03.435+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:01:03.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:03 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:03.435+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:01:03.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:03 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:03.435+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:01:03.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:03 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:03.435+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:01:03.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:03 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:03.435+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:35.512200+0000 front 2024-05-09T05:00:35.512344+0000 (oldest deadline 2024-05-09T05:01:00.812363+0000) 2024-05-09T05:01:03.673 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.7.service' 2024-05-09T05:01:03.712 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:01:03.712 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-09T05:01:03.712 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-09T05:01:03.712 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.8 2024-05-09T05:01:04.042 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:03 smithi186 systemd[1]: Stopping Ceph osd.8 for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:01:04.043 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:04 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:04.042+0000 7fc4f9bbf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:01:04.043 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:03 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:03.960+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:04.043 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:03 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:03.960+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:04.043 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:03 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:03.960+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:04.043 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:03 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:03.960+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:04.349 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:04 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:04.042+0000 7fc4f9bbf700 -1 osd.8 93 *** Got signal Terminated *** 2024-05-09T05:01:04.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:04 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:04.042+0000 7fc4f9bbf700 -1 osd.8 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T05:01:04.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:04 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:04.194+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:01:04.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:04 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:04.194+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.223703+0000 front 2024-05-09T05:00:15.223669+0000 (oldest deadline 2024-05-09T05:00:39.923801+0000) 2024-05-09T05:01:04.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:04 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:04.194+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.125777+0000 front 2024-05-09T05:00:25.125791+0000 (oldest deadline 2024-05-09T05:00:51.025568+0000) 2024-05-09T05:01:04.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:04 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:04.194+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:32.726856+0000 front 2024-05-09T05:00:32.726755+0000 (oldest deadline 2024-05-09T05:00:57.426618+0000) 2024-05-09T05:01:04.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:04 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:04.112+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:01:04.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:04 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:04.112+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:01:04.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:04 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:04.112+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:01:04.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:04 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:04.112+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.999302+0000 front 2024-05-09T05:00:25.999270+0000 (oldest deadline 2024-05-09T05:00:47.099558+0000) 2024-05-09T05:01:04.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:04 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:04.112+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:32.303524+0000 front 2024-05-09T05:00:32.303793+0000 (oldest deadline 2024-05-09T05:00:56.403805+0000) 2024-05-09T05:01:04.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:04 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:04.437+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:01:04.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:04 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:04.437+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:01:04.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:04 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:04.437+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:01:04.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:04 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:04.437+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:01:04.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:04 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:04.437+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:35.512200+0000 front 2024-05-09T05:00:35.512344+0000 (oldest deadline 2024-05-09T05:01:00.812363+0000) 2024-05-09T05:01:05.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:05 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:04.999+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:05.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:05 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:04.999+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:05.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:05 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:04.999+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:05.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:05 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:04.999+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:05.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:05 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:05.212+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:01:05.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:05 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:05.212+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.223703+0000 front 2024-05-09T05:00:15.223669+0000 (oldest deadline 2024-05-09T05:00:39.923801+0000) 2024-05-09T05:01:05.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:05 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:05.212+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.125777+0000 front 2024-05-09T05:00:25.125791+0000 (oldest deadline 2024-05-09T05:00:51.025568+0000) 2024-05-09T05:01:05.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:05 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:05.212+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:32.726856+0000 front 2024-05-09T05:00:32.726755+0000 (oldest deadline 2024-05-09T05:00:57.426618+0000) 2024-05-09T05:01:05.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:05 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:05.212+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:41.527464+0000 front 2024-05-09T05:00:41.527511+0000 (oldest deadline 2024-05-09T05:01:04.427634+0000) 2024-05-09T05:01:05.447 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:05 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:05.153+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:01:05.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:05 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:05.153+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:01:05.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:05 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:05.153+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:01:05.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:05 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:05.153+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.999302+0000 front 2024-05-09T05:00:25.999270+0000 (oldest deadline 2024-05-09T05:00:47.099558+0000) 2024-05-09T05:01:05.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:05 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:05.153+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:32.303524+0000 front 2024-05-09T05:00:32.303793+0000 (oldest deadline 2024-05-09T05:00:56.403805+0000) 2024-05-09T05:01:05.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:05 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:05.449+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:01:05.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:05 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:05.449+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:01:05.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:05 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:05.449+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:01:05.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:05 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:05.449+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:01:05.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:05 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:05.449+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:35.512200+0000 front 2024-05-09T05:00:35.512344+0000 (oldest deadline 2024-05-09T05:01:00.812363+0000) 2024-05-09T05:01:06.219 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:05 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:05.957+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:06.219 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:05 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:05.957+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:06.220 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:05 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:05.957+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:06.220 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:05 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:05.957+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:06.220 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:05 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:05.957+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:42.943560+0000 front 2024-05-09T05:00:42.943457+0000 (oldest deadline 2024-05-09T05:01:05.243517+0000) 2024-05-09T05:01:06.447 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:06 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:06.184+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:01:06.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:06 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:06.184+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:01:06.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:06 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:06.184+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:01:06.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:06 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:06.184+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.999302+0000 front 2024-05-09T05:00:25.999270+0000 (oldest deadline 2024-05-09T05:00:47.099558+0000) 2024-05-09T05:01:06.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:06 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:06.184+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:32.303524+0000 front 2024-05-09T05:00:32.303793+0000 (oldest deadline 2024-05-09T05:00:56.403805+0000) 2024-05-09T05:01:06.600 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:06 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:06.219+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:01:06.600 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:06 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:06.219+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.223703+0000 front 2024-05-09T05:00:15.223669+0000 (oldest deadline 2024-05-09T05:00:39.923801+0000) 2024-05-09T05:01:06.600 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:06 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:06.219+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.125777+0000 front 2024-05-09T05:00:25.125791+0000 (oldest deadline 2024-05-09T05:00:51.025568+0000) 2024-05-09T05:01:06.600 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:06 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:06.219+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:32.726856+0000 front 2024-05-09T05:00:32.726755+0000 (oldest deadline 2024-05-09T05:00:57.426618+0000) 2024-05-09T05:01:06.600 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:06 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:06.219+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:41.527464+0000 front 2024-05-09T05:00:41.527511+0000 (oldest deadline 2024-05-09T05:01:04.427634+0000) 2024-05-09T05:01:06.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:06 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:06.454+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:01:06.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:06 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:06.455+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:01:06.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:06 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:06.455+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:01:06.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:06 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:06.455+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:01:06.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:06 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:06.455+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:35.512200+0000 front 2024-05-09T05:00:35.512344+0000 (oldest deadline 2024-05-09T05:01:00.812363+0000) 2024-05-09T05:01:06.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:06 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:06.455+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:40.812868+0000 front 2024-05-09T05:00:40.812823+0000 (oldest deadline 2024-05-09T05:01:05.512865+0000) 2024-05-09T05:01:07.190 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:06 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:06.938+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:07.190 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:06 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:06.938+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:07.190 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:06 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:06.938+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:07.190 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:06 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:06.938+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:07.190 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:06 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:06.938+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:42.943560+0000 front 2024-05-09T05:00:42.943457+0000 (oldest deadline 2024-05-09T05:01:05.243517+0000) 2024-05-09T05:01:07.447 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:07 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:07.181+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:01:07.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:07 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:07.181+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:01:07.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:07 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:07.181+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:01:07.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:07 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:07.181+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.999302+0000 front 2024-05-09T05:00:25.999270+0000 (oldest deadline 2024-05-09T05:00:47.099558+0000) 2024-05-09T05:01:07.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:07 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:07.181+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:32.303524+0000 front 2024-05-09T05:00:32.303793+0000 (oldest deadline 2024-05-09T05:00:56.403805+0000) 2024-05-09T05:01:07.600 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:07 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:07.189+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:01:07.600 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:07 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:07.189+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.223703+0000 front 2024-05-09T05:00:15.223669+0000 (oldest deadline 2024-05-09T05:00:39.923801+0000) 2024-05-09T05:01:07.600 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:07 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:07.189+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.125777+0000 front 2024-05-09T05:00:25.125791+0000 (oldest deadline 2024-05-09T05:00:51.025568+0000) 2024-05-09T05:01:07.600 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:07 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:07.189+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:32.726856+0000 front 2024-05-09T05:00:32.726755+0000 (oldest deadline 2024-05-09T05:00:57.426618+0000) 2024-05-09T05:01:07.600 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:07 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:07.189+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:41.527464+0000 front 2024-05-09T05:00:41.527511+0000 (oldest deadline 2024-05-09T05:01:04.427634+0000) 2024-05-09T05:01:07.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:07 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:07.407+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:01:07.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:07 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:07.407+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:01:07.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:07 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:07.407+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:01:07.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:07 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:07.407+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:01:07.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:07 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:07.407+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:35.512200+0000 front 2024-05-09T05:00:35.512344+0000 (oldest deadline 2024-05-09T05:01:00.812363+0000) 2024-05-09T05:01:07.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:07 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:07.407+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:40.812868+0000 front 2024-05-09T05:00:40.812823+0000 (oldest deadline 2024-05-09T05:01:05.512865+0000) 2024-05-09T05:01:08.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:08 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:08.227+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:05.222634+0000 front 2024-05-09T05:00:05.222598+0000 (oldest deadline 2024-05-09T05:00:30.522637+0000) 2024-05-09T05:01:08.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:08 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:08.227+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.223703+0000 front 2024-05-09T05:00:15.223669+0000 (oldest deadline 2024-05-09T05:00:39.923801+0000) 2024-05-09T05:01:08.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:08 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:08.227+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.125777+0000 front 2024-05-09T05:00:25.125791+0000 (oldest deadline 2024-05-09T05:00:51.025568+0000) 2024-05-09T05:01:08.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:08 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:08.227+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:32.726856+0000 front 2024-05-09T05:00:32.726755+0000 (oldest deadline 2024-05-09T05:00:57.426618+0000) 2024-05-09T05:01:08.350 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:08 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8[53684]: 2024-05-09T05:01:08.227+0000 7fc4f216d700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:41.527464+0000 front 2024-05-09T05:00:41.527511+0000 (oldest deadline 2024-05-09T05:01:04.427634+0000) 2024-05-09T05:01:08.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:07 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:07.977+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:08.351 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:07 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:07.977+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:08.351 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:07 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:07.977+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:08.351 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:07 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:07.977+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:08.351 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:07 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:07.977+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:42.943560+0000 front 2024-05-09T05:00:42.943457+0000 (oldest deadline 2024-05-09T05:01:05.243517+0000) 2024-05-09T05:01:08.447 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:08 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:08.190+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:01:08.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:08 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:08.190+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:01:08.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:08 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:08.190+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:01:08.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:08 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:08.190+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.999302+0000 front 2024-05-09T05:00:25.999270+0000 (oldest deadline 2024-05-09T05:00:47.099558+0000) 2024-05-09T05:01:08.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:08 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:08.190+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:32.303524+0000 front 2024-05-09T05:00:32.303793+0000 (oldest deadline 2024-05-09T05:00:56.403805+0000) 2024-05-09T05:01:08.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:08 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:08.391+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:01:08.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:08 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:08.391+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:01:08.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:08 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:08.391+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:01:08.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:08 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:08.391+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:01:08.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:08 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:08.391+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:35.512200+0000 front 2024-05-09T05:00:35.512344+0000 (oldest deadline 2024-05-09T05:01:00.812363+0000) 2024-05-09T05:01:08.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:08 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:08.391+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:40.812868+0000 front 2024-05-09T05:00:40.812823+0000 (oldest deadline 2024-05-09T05:01:05.512865+0000) 2024-05-09T05:01:09.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:08 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:08.942+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:09.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:08 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:08.942+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:09.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:08 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:08.942+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:09.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:08 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:08.942+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:09.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:08 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:08.942+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:42.943560+0000 front 2024-05-09T05:00:42.943457+0000 (oldest deadline 2024-05-09T05:01:05.243517+0000) 2024-05-09T05:01:09.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:09 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:09.352+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:01:09.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:09 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:09.352+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:01:09.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:09 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:09.352+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:01:09.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:09 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:09.352+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:01:09.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:09 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:09.352+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:35.512200+0000 front 2024-05-09T05:00:35.512344+0000 (oldest deadline 2024-05-09T05:01:00.812363+0000) 2024-05-09T05:01:09.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:09 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:09.352+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:40.812868+0000 front 2024-05-09T05:00:40.812823+0000 (oldest deadline 2024-05-09T05:01:05.512865+0000) 2024-05-09T05:01:09.697 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:09 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:09.214+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:01:09.698 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:09 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:09.214+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:01:09.698 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:09 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:09.214+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:01:09.698 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:09 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:09.214+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.999302+0000 front 2024-05-09T05:00:25.999270+0000 (oldest deadline 2024-05-09T05:00:47.099558+0000) 2024-05-09T05:01:09.698 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:09 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:09.214+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:32.303524+0000 front 2024-05-09T05:00:32.303793+0000 (oldest deadline 2024-05-09T05:00:56.403805+0000) 2024-05-09T05:01:09.849 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:09 smithi186 podman[70263]: 2024-05-09 05:01:09.579812115 +0000 UTC m=+5.668693744 container died 0af8d4b19537bab2a237124fa7c1dc10803341fdfe857b3f8d1ba5174ef8abf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T05:01:10.210 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:09 smithi186 podman[70263]: 2024-05-09 05:01:09.950705085 +0000 UTC m=+6.039586713 container cleanup 0af8d4b19537bab2a237124fa7c1dc10803341fdfe857b3f8d1ba5174ef8abf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T05:01:10.210 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:09 smithi186 bash[70263]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8 2024-05-09T05:01:10.210 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:09 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:09.983+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:10.211 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:09 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:09.983+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:10.211 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:09 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:09.983+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:10.211 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:09 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:09.983+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:10.211 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:09 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:09.983+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:42.943560+0000 front 2024-05-09T05:00:42.943457+0000 (oldest deadline 2024-05-09T05:01:05.243517+0000) 2024-05-09T05:01:10.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:10 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:10.177+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:01:10.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:10 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:10.177+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:01:10.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:10 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:10.177+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:01:10.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:10 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:10.177+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.999302+0000 front 2024-05-09T05:00:25.999270+0000 (oldest deadline 2024-05-09T05:00:47.099558+0000) 2024-05-09T05:01:10.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:10 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:10.177+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:32.303524+0000 front 2024-05-09T05:00:32.303793+0000 (oldest deadline 2024-05-09T05:00:56.403805+0000) 2024-05-09T05:01:10.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:10 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:10.177+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:40.505925+0000 front 2024-05-09T05:00:40.505672+0000 (oldest deadline 2024-05-09T05:01:09.306605+0000) 2024-05-09T05:01:10.569 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:10 smithi186 podman[70280]: 2024-05-09 05:01:10.210264572 +0000 UTC m=+0.630781885 container remove 0af8d4b19537bab2a237124fa7c1dc10803341fdfe857b3f8d1ba5174ef8abf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T05:01:10.850 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:10 smithi186 podman[70369]: 2024-05-09 05:01:10.472389126 +0000 UTC m=+0.017780515 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:01:10.850 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:10 smithi186 podman[70369]: 2024-05-09 05:01:10.577943606 +0000 UTC m=+0.123334979 container create 801cc5189b516098383ae5f76090bfaf6ba285eabcb5a45f4c3822f8a87ff5f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux ) 2024-05-09T05:01:10.850 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:10 smithi186 podman[70369]: 2024-05-09 05:01:10.80599387 +0000 UTC m=+0.351385260 container init 801cc5189b516098383ae5f76090bfaf6ba285eabcb5a45f4c3822f8a87ff5f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240507) 2024-05-09T05:01:10.850 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:10 smithi186 podman[70369]: 2024-05-09 05:01:10.809497679 +0000 UTC m=+0.354889053 container start 801cc5189b516098383ae5f76090bfaf6ba285eabcb5a45f4c3822f8a87ff5f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T05:01:10.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:10 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:10.389+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:01:10.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:10 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:10.389+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:01:10.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:10 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:10.389+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:01:10.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:10 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:10.389+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:01:10.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:10 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:10.389+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:35.512200+0000 front 2024-05-09T05:00:35.512344+0000 (oldest deadline 2024-05-09T05:01:00.812363+0000) 2024-05-09T05:01:10.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:10 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:10.389+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:40.812868+0000 front 2024-05-09T05:00:40.812823+0000 (oldest deadline 2024-05-09T05:01:05.512865+0000) 2024-05-09T05:01:11.103 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:10 smithi186 podman[70369]: 2024-05-09 05:01:10.905905149 +0000 UTC m=+0.451296540 container attach 801cc5189b516098383ae5f76090bfaf6ba285eabcb5a45f4c3822f8a87ff5f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-09T05:01:11.103 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:10 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:10.978+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:11.104 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:10 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:10.978+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:11.104 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:10 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:10.978+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:11.104 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:10 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:10.978+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:11.104 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:10 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:10.978+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:42.943560+0000 front 2024-05-09T05:00:42.943457+0000 (oldest deadline 2024-05-09T05:01:05.243517+0000) 2024-05-09T05:01:11.382 INFO:journalctl@ceph.osd.8.smithi186.stdout:May 09 05:01:11 smithi186 podman[70369]: 2024-05-09 05:01:11.103973397 +0000 UTC m=+0.649364771 container died 801cc5189b516098383ae5f76090bfaf6ba285eabcb5a45f4c3822f8a87ff5f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-09T05:01:11.447 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:11 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:11.142+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:01:11.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:11 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:11.142+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:01:11.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:11 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:11.142+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:01:11.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:11 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:11.142+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.999302+0000 front 2024-05-09T05:00:25.999270+0000 (oldest deadline 2024-05-09T05:00:47.099558+0000) 2024-05-09T05:01:11.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:11 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:11.142+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:32.303524+0000 front 2024-05-09T05:00:32.303793+0000 (oldest deadline 2024-05-09T05:00:56.403805+0000) 2024-05-09T05:01:11.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:11 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:11.142+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:40.505925+0000 front 2024-05-09T05:00:40.505672+0000 (oldest deadline 2024-05-09T05:01:09.306605+0000) 2024-05-09T05:01:11.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:11 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:11.361+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:01:11.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:11 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:11.361+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:01:11.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:11 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:11.361+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:01:11.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:11 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:11.361+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:01:11.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:11 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:11.361+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:35.512200+0000 front 2024-05-09T05:00:35.512344+0000 (oldest deadline 2024-05-09T05:01:00.812363+0000) 2024-05-09T05:01:11.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:11 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:11.361+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:40.812868+0000 front 2024-05-09T05:00:40.812823+0000 (oldest deadline 2024-05-09T05:01:05.512865+0000) 2024-05-09T05:01:11.794 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.8.service' 2024-05-09T05:01:11.831 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:01:11.831 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-09T05:01:11.831 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-09T05:01:11.831 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.9 2024-05-09T05:01:12.102 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:11 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:11.937+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:12.102 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:11 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:11.937+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:12.102 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:11 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:11.937+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:12.102 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:11 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:11.937+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:12.102 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:11 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:11.937+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:42.943560+0000 front 2024-05-09T05:00:42.943457+0000 (oldest deadline 2024-05-09T05:01:05.243517+0000) 2024-05-09T05:01:12.198 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:11 smithi006 systemd[1]: Stopping Ceph osd.9 for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:01:12.198 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:12 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:12.137+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:01:12.199 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:12 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:12.137+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:01:12.199 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:12 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:12.137+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:01:12.199 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:12 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:12.137+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.999302+0000 front 2024-05-09T05:00:25.999270+0000 (oldest deadline 2024-05-09T05:00:47.099558+0000) 2024-05-09T05:01:12.199 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:12 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:12.137+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:32.303524+0000 front 2024-05-09T05:00:32.303793+0000 (oldest deadline 2024-05-09T05:00:56.403805+0000) 2024-05-09T05:01:12.199 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:12 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:12.137+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:40.505925+0000 front 2024-05-09T05:00:40.505672+0000 (oldest deadline 2024-05-09T05:01:09.306605+0000) 2024-05-09T05:01:12.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:12 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:12.381+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:01:12.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:12 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:12.381+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:01:12.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:12 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:12.381+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:01:12.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:12 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:12.381+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:01:12.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:12 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:12.381+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:35.512200+0000 front 2024-05-09T05:00:35.512344+0000 (oldest deadline 2024-05-09T05:01:00.812363+0000) 2024-05-09T05:01:12.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:12 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:12.381+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:40.812868+0000 front 2024-05-09T05:00:40.812823+0000 (oldest deadline 2024-05-09T05:01:05.512865+0000) 2024-05-09T05:01:12.947 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:12 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:12.484+0000 7f9794800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:01:12.947 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:12 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:12.484+0000 7f9794800700 -1 osd.9 93 *** Got signal Terminated *** 2024-05-09T05:01:12.948 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:12 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:12.484+0000 7f9794800700 -1 osd.9 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T05:01:13.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:12 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:12.987+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:13.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:12 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:12.987+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:13.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:12 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:12.987+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:13.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:12 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:12.987+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:13.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:12 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:12.987+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:42.943560+0000 front 2024-05-09T05:00:42.943457+0000 (oldest deadline 2024-05-09T05:01:05.243517+0000) 2024-05-09T05:01:13.367 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:13 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:13.343+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:01:13.367 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:13 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:13.343+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:01:13.367 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:13 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:13.343+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:01:13.367 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:13 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:13.343+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:01:13.367 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:13 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:13.343+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:35.512200+0000 front 2024-05-09T05:00:35.512344+0000 (oldest deadline 2024-05-09T05:01:00.812363+0000) 2024-05-09T05:01:13.367 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:13 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:13.343+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:40.812868+0000 front 2024-05-09T05:00:40.812823+0000 (oldest deadline 2024-05-09T05:01:05.512865+0000) 2024-05-09T05:01:13.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:13 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:13.115+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:01:13.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:13 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:13.115+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:01:13.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:13 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:13.115+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:01:13.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:13 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:13.115+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.999302+0000 front 2024-05-09T05:00:25.999270+0000 (oldest deadline 2024-05-09T05:00:47.099558+0000) 2024-05-09T05:01:13.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:13 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:13.115+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:32.303524+0000 front 2024-05-09T05:00:32.303793+0000 (oldest deadline 2024-05-09T05:00:56.403805+0000) 2024-05-09T05:01:13.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:13 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:13.115+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:40.505925+0000 front 2024-05-09T05:00:40.505672+0000 (oldest deadline 2024-05-09T05:01:09.306605+0000) 2024-05-09T05:01:14.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:14 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:14.025+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:14.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:14 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:14.025+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:14.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:14 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:14.025+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:14.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:14 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:14.025+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:14.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:14 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:14.025+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:42.943560+0000 front 2024-05-09T05:00:42.943457+0000 (oldest deadline 2024-05-09T05:01:05.243517+0000) 2024-05-09T05:01:14.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:14 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:14.076+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:01:14.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:14 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:14.076+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:01:14.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:14 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:14.076+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:01:14.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:14 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:14.076+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.999302+0000 front 2024-05-09T05:00:25.999270+0000 (oldest deadline 2024-05-09T05:00:47.099558+0000) 2024-05-09T05:01:14.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:14 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:14.076+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:32.303524+0000 front 2024-05-09T05:00:32.303793+0000 (oldest deadline 2024-05-09T05:00:56.403805+0000) 2024-05-09T05:01:14.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:14 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:14.076+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:40.505925+0000 front 2024-05-09T05:00:40.505672+0000 (oldest deadline 2024-05-09T05:01:09.306605+0000) 2024-05-09T05:01:14.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:14 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:14.329+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:01:14.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:14 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:14.329+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:01:14.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:14 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:14.329+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:01:14.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:14 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:14.329+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:01:14.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:14 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:14.329+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:35.512200+0000 front 2024-05-09T05:00:35.512344+0000 (oldest deadline 2024-05-09T05:01:00.812363+0000) 2024-05-09T05:01:14.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:14 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:14.329+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:40.812868+0000 front 2024-05-09T05:00:40.812823+0000 (oldest deadline 2024-05-09T05:01:05.512865+0000) 2024-05-09T05:01:15.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:15 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:15.002+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:15.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:15 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:15.002+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:15.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:15 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:15.002+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:15.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:15 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:15.002+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:15.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:15 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:15.002+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:42.943560+0000 front 2024-05-09T05:00:42.943457+0000 (oldest deadline 2024-05-09T05:01:05.243517+0000) 2024-05-09T05:01:15.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:15 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:15.091+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:01:15.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:15 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:15.091+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:01:15.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:15 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:15.091+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:01:15.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:15 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:15.091+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.999302+0000 front 2024-05-09T05:00:25.999270+0000 (oldest deadline 2024-05-09T05:00:47.099558+0000) 2024-05-09T05:01:15.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:15 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:15.091+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:32.303524+0000 front 2024-05-09T05:00:32.303793+0000 (oldest deadline 2024-05-09T05:00:56.403805+0000) 2024-05-09T05:01:15.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:15 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:15.091+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:40.505925+0000 front 2024-05-09T05:00:40.505672+0000 (oldest deadline 2024-05-09T05:01:09.306605+0000) 2024-05-09T05:01:15.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:15 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:15.336+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:01:15.618 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:15 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:15.336+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:01:15.618 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:15 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:15.336+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:01:15.618 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:15 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:15.336+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:01:15.618 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:15 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:15.336+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:35.512200+0000 front 2024-05-09T05:00:35.512344+0000 (oldest deadline 2024-05-09T05:01:00.812363+0000) 2024-05-09T05:01:15.618 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:15 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:15.336+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:40.812868+0000 front 2024-05-09T05:00:40.812823+0000 (oldest deadline 2024-05-09T05:01:05.512865+0000) 2024-05-09T05:01:15.618 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:15 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:15.336+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-05-09T05:00:50.213996+0000 front 2024-05-09T05:00:50.214122+0000 (oldest deadline 2024-05-09T05:01:14.913762+0000) 2024-05-09T05:01:16.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:16 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:16.016+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:16.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:16 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:16.016+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:16.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:16 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:16.016+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:16.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:16 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:16.016+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:16.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:16 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:16.016+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:42.943560+0000 front 2024-05-09T05:00:42.943457+0000 (oldest deadline 2024-05-09T05:01:05.243517+0000) 2024-05-09T05:01:16.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:16 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:16.016+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-05-09T05:00:51.045316+0000 front 2024-05-09T05:00:51.045520+0000 (oldest deadline 2024-05-09T05:01:15.145110+0000) 2024-05-09T05:01:16.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:16 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:16.048+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:01:16.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:16 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:16.048+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:01:16.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:16 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:16.048+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:01:16.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:16 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:16.048+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.999302+0000 front 2024-05-09T05:00:25.999270+0000 (oldest deadline 2024-05-09T05:00:47.099558+0000) 2024-05-09T05:01:16.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:16 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:16.048+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:32.303524+0000 front 2024-05-09T05:00:32.303793+0000 (oldest deadline 2024-05-09T05:00:56.403805+0000) 2024-05-09T05:01:16.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:16 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:16.048+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:40.505925+0000 front 2024-05-09T05:00:40.505672+0000 (oldest deadline 2024-05-09T05:01:09.306605+0000) 2024-05-09T05:01:16.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:16 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:16.325+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:01:16.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:16 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:16.325+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:01:16.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:16 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:16.325+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:01:16.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:16 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:16.325+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:01:16.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:16 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:16.325+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:35.512200+0000 front 2024-05-09T05:00:35.512344+0000 (oldest deadline 2024-05-09T05:01:00.812363+0000) 2024-05-09T05:01:16.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:16 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:16.325+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:40.812868+0000 front 2024-05-09T05:00:40.812823+0000 (oldest deadline 2024-05-09T05:01:05.512865+0000) 2024-05-09T05:01:16.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:16 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:16.325+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-05-09T05:00:50.213996+0000 front 2024-05-09T05:00:50.214122+0000 (oldest deadline 2024-05-09T05:01:14.913762+0000) 2024-05-09T05:01:17.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:17 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:17.017+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:17.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:17 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:17.017+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:17.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:17 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:17.017+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:17.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:17 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:17.017+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:17.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:17 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:17.017+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:42.943560+0000 front 2024-05-09T05:00:42.943457+0000 (oldest deadline 2024-05-09T05:01:05.243517+0000) 2024-05-09T05:01:17.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:17 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:17.017+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-05-09T05:00:51.045316+0000 front 2024-05-09T05:00:51.045520+0000 (oldest deadline 2024-05-09T05:01:15.145110+0000) 2024-05-09T05:01:17.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:17 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:17.006+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.087951+0000 front 2024-05-09T04:59:58.088272+0000 (oldest deadline 2024-05-09T05:00:22.689793+0000) 2024-05-09T05:01:17.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:17 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:17.006+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:07.194381+0000 front 2024-05-09T05:00:07.194458+0000 (oldest deadline 2024-05-09T05:00:33.094525+0000) 2024-05-09T05:01:17.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:17 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:17.006+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:17.196830+0000 front 2024-05-09T05:00:17.196873+0000 (oldest deadline 2024-05-09T05:00:41.296908+0000) 2024-05-09T05:01:17.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:17 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:17.006+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:25.999302+0000 front 2024-05-09T05:00:25.999270+0000 (oldest deadline 2024-05-09T05:00:47.099558+0000) 2024-05-09T05:01:17.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:17 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:17.006+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:32.303524+0000 front 2024-05-09T05:00:32.303793+0000 (oldest deadline 2024-05-09T05:00:56.403805+0000) 2024-05-09T05:01:17.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:17 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9[62320]: 2024-05-09T05:01:17.006+0000 7f9790000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:40.505925+0000 front 2024-05-09T05:00:40.505672+0000 (oldest deadline 2024-05-09T05:01:09.306605+0000) 2024-05-09T05:01:17.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:17 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:17.354+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:01:17.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:17 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:17.354+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:01:17.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:17 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:17.354+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:01:17.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:17 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:17.354+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:01:17.618 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:17 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:17.354+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:35.512200+0000 front 2024-05-09T05:00:35.512344+0000 (oldest deadline 2024-05-09T05:01:00.812363+0000) 2024-05-09T05:01:17.618 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:17 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:17.354+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:40.812868+0000 front 2024-05-09T05:00:40.812823+0000 (oldest deadline 2024-05-09T05:01:05.512865+0000) 2024-05-09T05:01:17.618 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:17 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:17.354+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-05-09T05:00:50.213996+0000 front 2024-05-09T05:00:50.214122+0000 (oldest deadline 2024-05-09T05:01:14.913762+0000) 2024-05-09T05:01:18.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:18 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:18.052+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:18.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:18 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:18.052+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:18.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:18 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:18.052+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:18.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:18 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:18.052+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:18.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:18 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:18.052+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:42.943560+0000 front 2024-05-09T05:00:42.943457+0000 (oldest deadline 2024-05-09T05:01:05.243517+0000) 2024-05-09T05:01:18.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:18 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:18.052+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-05-09T05:00:51.045316+0000 front 2024-05-09T05:00:51.045520+0000 (oldest deadline 2024-05-09T05:01:15.145110+0000) 2024-05-09T05:01:18.372 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:18 smithi006 podman[94899]: 2024-05-09 05:01:18.06057765 +0000 UTC m=+5.736799427 container died 2a522e6a0ace05bf4a3f7736ffd99e1cd11400c532288000cdd700775df852a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T05:01:18.697 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:18 smithi006 podman[94899]: 2024-05-09 05:01:18.567954659 +0000 UTC m=+6.244176464 container cleanup 2a522e6a0ace05bf4a3f7736ffd99e1cd11400c532288000cdd700775df852a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T05:01:18.698 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:18 smithi006 bash[94899]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9 2024-05-09T05:01:18.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:18 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:18.370+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:01:18.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:18 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:18.370+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:01:18.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:18 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:18.370+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:01:18.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:18 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:18.370+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:01:18.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:18 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:18.370+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:35.512200+0000 front 2024-05-09T05:00:35.512344+0000 (oldest deadline 2024-05-09T05:01:00.812363+0000) 2024-05-09T05:01:18.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:18 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:18.370+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:40.812868+0000 front 2024-05-09T05:00:40.812823+0000 (oldest deadline 2024-05-09T05:01:05.512865+0000) 2024-05-09T05:01:18.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:18 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:18.370+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-05-09T05:00:50.213996+0000 front 2024-05-09T05:00:50.214122+0000 (oldest deadline 2024-05-09T05:01:14.913762+0000) 2024-05-09T05:01:19.197 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:18 smithi006 podman[94920]: 2024-05-09 05:01:18.886350005 +0000 UTC m=+0.810446626 container remove 2a522e6a0ace05bf4a3f7736ffd99e1cd11400c532288000cdd700775df852a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-09T05:01:19.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:19 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:19.037+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:19.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:19 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:19.037+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:19.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:19 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:19.037+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:19.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:19 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:19.037+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:19.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:19 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:19.037+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:42.943560+0000 front 2024-05-09T05:00:42.943457+0000 (oldest deadline 2024-05-09T05:01:05.243517+0000) 2024-05-09T05:01:19.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:19 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:19.037+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-05-09T05:00:51.045316+0000 front 2024-05-09T05:00:51.045520+0000 (oldest deadline 2024-05-09T05:01:15.145110+0000) 2024-05-09T05:01:19.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:19 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:19.363+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:01:19.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:19 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:19.363+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:01:19.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:19 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:19.363+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:01:19.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:19 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:19.363+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:01:19.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:19 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:19.363+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:35.512200+0000 front 2024-05-09T05:00:35.512344+0000 (oldest deadline 2024-05-09T05:01:00.812363+0000) 2024-05-09T05:01:19.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:19 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:19.363+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:40.812868+0000 front 2024-05-09T05:00:40.812823+0000 (oldest deadline 2024-05-09T05:01:05.512865+0000) 2024-05-09T05:01:19.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:19 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:19.363+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-05-09T05:00:50.213996+0000 front 2024-05-09T05:00:50.214122+0000 (oldest deadline 2024-05-09T05:01:14.913762+0000) 2024-05-09T05:01:20.117 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:19 smithi006 podman[95007]: 2024-05-09 05:01:19.733512261 +0000 UTC m=+0.038732003 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:01:20.117 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:19 smithi006 podman[95007]: 2024-05-09 05:01:19.828588046 +0000 UTC m=+0.133807821 container create 63adfb284b4d8015eb99027fc7304ffa81de34a2e9b4c4f5f17d7aca08b2e591 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T05:01:20.117 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:20 smithi006 podman[95007]: 2024-05-09 05:01:20.115692942 +0000 UTC m=+0.420912724 container init 63adfb284b4d8015eb99027fc7304ffa81de34a2e9b4c4f5f17d7aca08b2e591 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-09T05:01:20.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:20 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:20.065+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:20.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:20 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:20.065+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:20.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:20 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:20.065+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:20.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:20 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:20.065+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:20.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:20 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:20.065+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:42.943560+0000 front 2024-05-09T05:00:42.943457+0000 (oldest deadline 2024-05-09T05:01:05.243517+0000) 2024-05-09T05:01:20.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:20 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:20.065+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-05-09T05:00:51.045316+0000 front 2024-05-09T05:00:51.045520+0000 (oldest deadline 2024-05-09T05:01:15.145110+0000) 2024-05-09T05:01:20.447 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:20 smithi006 podman[95007]: 2024-05-09 05:01:20.12789081 +0000 UTC m=+0.433110567 container start 63adfb284b4d8015eb99027fc7304ffa81de34a2e9b4c4f5f17d7aca08b2e591 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T05:01:20.448 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:20 smithi006 podman[95007]: 2024-05-09 05:01:20.222874948 +0000 UTC m=+0.528094760 container attach 63adfb284b4d8015eb99027fc7304ffa81de34a2e9b4c4f5f17d7aca08b2e591 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T05:01:20.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:20 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:20.363+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:01:20.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:20 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:20.363+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:01:20.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:20 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:20.363+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:01:20.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:20 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:20.363+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:01:20.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:20 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:20.363+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:35.512200+0000 front 2024-05-09T05:00:35.512344+0000 (oldest deadline 2024-05-09T05:01:00.812363+0000) 2024-05-09T05:01:20.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:20 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:20.363+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:40.812868+0000 front 2024-05-09T05:00:40.812823+0000 (oldest deadline 2024-05-09T05:01:05.512865+0000) 2024-05-09T05:01:20.618 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:20 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:20.363+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-05-09T05:00:50.213996+0000 front 2024-05-09T05:00:50.214122+0000 (oldest deadline 2024-05-09T05:01:14.913762+0000) 2024-05-09T05:01:20.913 INFO:journalctl@ceph.osd.9.smithi006.stdout:May 09 05:01:20 smithi006 podman[95007]: 2024-05-09 05:01:20.601235976 +0000 UTC m=+0.906455758 container died 63adfb284b4d8015eb99027fc7304ffa81de34a2e9b4c4f5f17d7aca08b2e591 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-9-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-09T05:01:21.263 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.9.service' 2024-05-09T05:01:21.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:21 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:21.094+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:21.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:21 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:21.094+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:21.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:21 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:21.094+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:21.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:21 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:21.094+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:21.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:21 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:21.094+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:42.943560+0000 front 2024-05-09T05:00:42.943457+0000 (oldest deadline 2024-05-09T05:01:05.243517+0000) 2024-05-09T05:01:21.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:21 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:21.094+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-05-09T05:00:51.045316+0000 front 2024-05-09T05:00:51.045520+0000 (oldest deadline 2024-05-09T05:01:15.145110+0000) 2024-05-09T05:01:21.370 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:01:21.371 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-09T05:01:21.371 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-09T05:01:21.371 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.10 2024-05-09T05:01:21.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:21 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:21.353+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:01:21.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:21 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:21.353+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:01:21.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:21 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:21.353+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:01:21.618 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:21 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:21.353+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:01:21.618 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:21 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:21.353+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:35.512200+0000 front 2024-05-09T05:00:35.512344+0000 (oldest deadline 2024-05-09T05:01:00.812363+0000) 2024-05-09T05:01:21.618 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:21 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:21.353+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:40.812868+0000 front 2024-05-09T05:00:40.812823+0000 (oldest deadline 2024-05-09T05:01:05.512865+0000) 2024-05-09T05:01:21.618 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:21 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:21.353+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-05-09T05:00:50.213996+0000 front 2024-05-09T05:00:50.214122+0000 (oldest deadline 2024-05-09T05:01:14.913762+0000) 2024-05-09T05:01:21.618 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:21 smithi169 systemd[1]: Stopping Ceph osd.10 for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:01:22.117 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:21 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:21.664+0000 7fcefba7a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:01:22.117 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:21 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:21.664+0000 7fcefba7a700 -1 osd.10 93 *** Got signal Terminated *** 2024-05-09T05:01:22.117 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:21 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:21.664+0000 7fcefba7a700 -1 osd.10 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T05:01:22.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:22 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:22.048+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:22.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:22 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:22.048+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:22.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:22 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:22.048+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:22.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:22 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:22.048+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:22.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:22 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:22.048+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:42.943560+0000 front 2024-05-09T05:00:42.943457+0000 (oldest deadline 2024-05-09T05:01:05.243517+0000) 2024-05-09T05:01:22.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:22 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:22.048+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-05-09T05:00:51.045316+0000 front 2024-05-09T05:00:51.045520+0000 (oldest deadline 2024-05-09T05:01:15.145110+0000) 2024-05-09T05:01:22.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:22 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:22.356+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:01:22.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:22 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:22.356+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:01:22.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:22 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:22.356+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:01:22.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:22 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:22.356+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:01:22.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:22 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:22.356+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:35.512200+0000 front 2024-05-09T05:00:35.512344+0000 (oldest deadline 2024-05-09T05:01:00.812363+0000) 2024-05-09T05:01:22.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:22 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:22.356+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:40.812868+0000 front 2024-05-09T05:00:40.812823+0000 (oldest deadline 2024-05-09T05:01:05.512865+0000) 2024-05-09T05:01:22.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:22 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:22.356+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-05-09T05:00:50.213996+0000 front 2024-05-09T05:00:50.214122+0000 (oldest deadline 2024-05-09T05:01:14.913762+0000) 2024-05-09T05:01:23.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:23 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:23.084+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:23.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:23 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:23.084+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:23.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:23 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:23.084+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:23.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:23 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:23.084+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:23.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:23 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:23.084+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:42.943560+0000 front 2024-05-09T05:00:42.943457+0000 (oldest deadline 2024-05-09T05:01:05.243517+0000) 2024-05-09T05:01:23.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:23 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:23.084+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-05-09T05:00:51.045316+0000 front 2024-05-09T05:00:51.045520+0000 (oldest deadline 2024-05-09T05:01:15.145110+0000) 2024-05-09T05:01:23.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:23 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:23.373+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:01:23.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:23 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:23.373+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:01:23.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:23 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:23.373+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:01:23.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:23 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:23.373+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:01:23.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:23 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:23.373+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:35.512200+0000 front 2024-05-09T05:00:35.512344+0000 (oldest deadline 2024-05-09T05:01:00.812363+0000) 2024-05-09T05:01:23.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:23 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:23.373+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:40.812868+0000 front 2024-05-09T05:00:40.812823+0000 (oldest deadline 2024-05-09T05:01:05.512865+0000) 2024-05-09T05:01:23.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:23 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:23.373+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-05-09T05:00:50.213996+0000 front 2024-05-09T05:00:50.214122+0000 (oldest deadline 2024-05-09T05:01:14.913762+0000) 2024-05-09T05:01:24.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:24 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:24.101+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:24.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:24 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:24.101+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:24.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:24 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:24.101+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:24.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:24 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:24.101+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:24.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:24 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:24.101+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:42.943560+0000 front 2024-05-09T05:00:42.943457+0000 (oldest deadline 2024-05-09T05:01:05.243517+0000) 2024-05-09T05:01:24.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:24 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:24.101+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-05-09T05:00:51.045316+0000 front 2024-05-09T05:00:51.045520+0000 (oldest deadline 2024-05-09T05:01:15.145110+0000) 2024-05-09T05:01:24.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:24 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:24.358+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:01:24.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:24 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:24.358+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:01:24.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:24 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:24.358+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:01:24.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:24 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:24.358+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:01:24.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:24 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:24.358+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:35.512200+0000 front 2024-05-09T05:00:35.512344+0000 (oldest deadline 2024-05-09T05:01:00.812363+0000) 2024-05-09T05:01:24.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:24 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:24.358+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:40.812868+0000 front 2024-05-09T05:00:40.812823+0000 (oldest deadline 2024-05-09T05:01:05.512865+0000) 2024-05-09T05:01:24.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:24 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:24.358+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-05-09T05:00:50.213996+0000 front 2024-05-09T05:00:50.214122+0000 (oldest deadline 2024-05-09T05:01:14.913762+0000) 2024-05-09T05:01:25.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:25 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:25.061+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:25.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:25 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:25.061+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:25.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:25 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:25.061+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:25.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:25 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:25.061+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:25.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:25 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:25.061+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:42.943560+0000 front 2024-05-09T05:00:42.943457+0000 (oldest deadline 2024-05-09T05:01:05.243517+0000) 2024-05-09T05:01:25.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:25 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:25.061+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-05-09T05:00:51.045316+0000 front 2024-05-09T05:00:51.045520+0000 (oldest deadline 2024-05-09T05:01:15.145110+0000) 2024-05-09T05:01:25.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:25 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:25.061+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6820 osd.7 since back 2024-05-09T05:01:00.446390+0000 front 2024-05-09T05:01:00.446454+0000 (oldest deadline 2024-05-09T05:01:24.546257+0000) 2024-05-09T05:01:25.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:25 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:25.408+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:01:25.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:25 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:25.408+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:01:25.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:25 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:25.408+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:01:25.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:25 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:25.408+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:01:25.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:25 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:25.408+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:35.512200+0000 front 2024-05-09T05:00:35.512344+0000 (oldest deadline 2024-05-09T05:01:00.812363+0000) 2024-05-09T05:01:25.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:25 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:25.408+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:40.812868+0000 front 2024-05-09T05:00:40.812823+0000 (oldest deadline 2024-05-09T05:01:05.512865+0000) 2024-05-09T05:01:25.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:25 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:25.408+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-05-09T05:00:50.213996+0000 front 2024-05-09T05:00:50.214122+0000 (oldest deadline 2024-05-09T05:01:14.913762+0000) 2024-05-09T05:01:26.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:26 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:26.107+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:26.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:26 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:26.107+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:26.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:26 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:26.107+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:26.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:26 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:26.107+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:26.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:26 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:26.107+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:42.943560+0000 front 2024-05-09T05:00:42.943457+0000 (oldest deadline 2024-05-09T05:01:05.243517+0000) 2024-05-09T05:01:26.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:26 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:26.107+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-05-09T05:00:51.045316+0000 front 2024-05-09T05:00:51.045520+0000 (oldest deadline 2024-05-09T05:01:15.145110+0000) 2024-05-09T05:01:26.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:26 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:26.107+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6820 osd.7 since back 2024-05-09T05:01:00.446390+0000 front 2024-05-09T05:01:00.446454+0000 (oldest deadline 2024-05-09T05:01:24.546257+0000) 2024-05-09T05:01:26.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:26 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:26.393+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T05:00:00.606582+0000 front 2024-05-09T05:00:00.606433+0000 (oldest deadline 2024-05-09T05:00:21.706139+0000) 2024-05-09T05:01:26.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:26 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:26.393+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-05-09T05:00:08.107665+0000 front 2024-05-09T05:00:08.107692+0000 (oldest deadline 2024-05-09T05:00:32.807894+0000) 2024-05-09T05:01:26.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:26 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:26.393+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:15.609364+0000 front 2024-05-09T05:00:15.609437+0000 (oldest deadline 2024-05-09T05:00:39.709476+0000) 2024-05-09T05:01:26.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:26 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:26.393+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:23.810848+0000 front 2024-05-09T05:00:23.810406+0000 (oldest deadline 2024-05-09T05:00:49.710416+0000) 2024-05-09T05:01:26.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:26 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:26.393+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:35.512200+0000 front 2024-05-09T05:00:35.512344+0000 (oldest deadline 2024-05-09T05:01:00.812363+0000) 2024-05-09T05:01:26.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:26 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:26.393+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:40.812868+0000 front 2024-05-09T05:00:40.812823+0000 (oldest deadline 2024-05-09T05:01:05.512865+0000) 2024-05-09T05:01:26.867 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:26 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10[59381]: 2024-05-09T05:01:26.393+0000 7fcef4028700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-05-09T05:00:50.213996+0000 front 2024-05-09T05:00:50.214122+0000 (oldest deadline 2024-05-09T05:01:14.913762+0000) 2024-05-09T05:01:27.440 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:27 smithi169 podman[69561]: 2024-05-09 05:01:27.132251302 +0000 UTC m=+5.566514726 container died 311ce3fec2fa09c8f7c8cb0caaeae525a909a10a4b3f01c24f02c38daf8ecfa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-09T05:01:27.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:27 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:27.115+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:27.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:27 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:27.115+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:27.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:27 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:27.115+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:27.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:27 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:27.115+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:27.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:27 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:27.115+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:42.943560+0000 front 2024-05-09T05:00:42.943457+0000 (oldest deadline 2024-05-09T05:01:05.243517+0000) 2024-05-09T05:01:27.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:27 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:27.115+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-05-09T05:00:51.045316+0000 front 2024-05-09T05:00:51.045520+0000 (oldest deadline 2024-05-09T05:01:15.145110+0000) 2024-05-09T05:01:27.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:27 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:27.115+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6820 osd.7 since back 2024-05-09T05:01:00.446390+0000 front 2024-05-09T05:01:00.446454+0000 (oldest deadline 2024-05-09T05:01:24.546257+0000) 2024-05-09T05:01:27.740 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:27 smithi169 podman[69561]: 2024-05-09 05:01:27.531467737 +0000 UTC m=+5.965731155 container cleanup 311ce3fec2fa09c8f7c8cb0caaeae525a909a10a4b3f01c24f02c38daf8ecfa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-09T05:01:27.741 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:27 smithi169 bash[69561]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10 2024-05-09T05:01:28.074 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:27 smithi169 podman[69578]: 2024-05-09 05:01:27.740956724 +0000 UTC m=+0.598817428 container remove 311ce3fec2fa09c8f7c8cb0caaeae525a909a10a4b3f01c24f02c38daf8ecfa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-09T05:01:28.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:28 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:28.086+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:28.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:28 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:28.086+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:28.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:28 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:28.086+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:28.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:28 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:28.086+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:28.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:28 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:28.086+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:42.943560+0000 front 2024-05-09T05:00:42.943457+0000 (oldest deadline 2024-05-09T05:01:05.243517+0000) 2024-05-09T05:01:28.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:28 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:28.086+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-05-09T05:00:51.045316+0000 front 2024-05-09T05:00:51.045520+0000 (oldest deadline 2024-05-09T05:01:15.145110+0000) 2024-05-09T05:01:28.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:28 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:28.086+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6820 osd.7 since back 2024-05-09T05:01:00.446390+0000 front 2024-05-09T05:01:00.446454+0000 (oldest deadline 2024-05-09T05:01:24.546257+0000) 2024-05-09T05:01:28.360 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:28 smithi169 podman[69667]: 2024-05-09 05:01:27.977755615 +0000 UTC m=+0.018146280 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:01:28.360 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:28 smithi169 podman[69667]: 2024-05-09 05:01:28.075035627 +0000 UTC m=+0.115426294 container create e04cff57070b0198a17b4fc25cb1ea75c971fdaf8f372f7c020e59de9eae5fd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-09T05:01:28.360 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:28 smithi169 podman[69667]: 2024-05-09 05:01:28.277359929 +0000 UTC m=+0.317750591 container init e04cff57070b0198a17b4fc25cb1ea75c971fdaf8f372f7c020e59de9eae5fd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0) 2024-05-09T05:01:28.360 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:28 smithi169 podman[69667]: 2024-05-09 05:01:28.280784152 +0000 UTC m=+0.321174814 container start e04cff57070b0198a17b4fc25cb1ea75c971fdaf8f372f7c020e59de9eae5fd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T05:01:28.360 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:28 smithi169 podman[69667]: 2024-05-09 05:01:28.360498379 +0000 UTC m=+0.400889054 container attach e04cff57070b0198a17b4fc25cb1ea75c971fdaf8f372f7c020e59de9eae5fd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-09T05:01:28.617 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:28 smithi169 podman[69667]: 2024-05-09 05:01:28.571510974 +0000 UTC m=+0.611901651 container died e04cff57070b0198a17b4fc25cb1ea75c971fdaf8f372f7c020e59de9eae5fd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-09T05:01:29.132 INFO:journalctl@ceph.osd.10.smithi169.stdout:May 09 05:01:29 smithi169 podman[69693]: 2024-05-09 05:01:29.113522279 +0000 UTC m=+0.535933636 container remove e04cff57070b0198a17b4fc25cb1ea75c971fdaf8f372f7c020e59de9eae5fd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True) 2024-05-09T05:01:29.137 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.10.service' 2024-05-09T05:01:29.175 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:01:29.175 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-09T05:01:29.176 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-09T05:01:29.176 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.11 2024-05-09T05:01:29.335 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:29 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:29.083+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:29.335 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:29 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:29.083+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:29.335 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:29 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:29.083+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:29.335 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:29 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:29.083+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:29.335 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:29 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:29.083+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:42.943560+0000 front 2024-05-09T05:00:42.943457+0000 (oldest deadline 2024-05-09T05:01:05.243517+0000) 2024-05-09T05:01:29.336 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:29 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:29.083+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-05-09T05:00:51.045316+0000 front 2024-05-09T05:00:51.045520+0000 (oldest deadline 2024-05-09T05:01:15.145110+0000) 2024-05-09T05:01:29.336 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:29 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:29.083+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6820 osd.7 since back 2024-05-09T05:01:00.446390+0000 front 2024-05-09T05:01:00.446454+0000 (oldest deadline 2024-05-09T05:01:24.546257+0000) 2024-05-09T05:01:29.336 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:29 smithi186 systemd[1]: Stopping Ceph osd.11 for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:01:29.599 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:29 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:29.503+0000 7f063013e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:01:29.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:29 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:29.503+0000 7f063013e700 -1 osd.11 93 *** Got signal Terminated *** 2024-05-09T05:01:29.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:29 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:29.503+0000 7f063013e700 -1 osd.11 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T05:01:30.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:30 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:30.125+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:30.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:30 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:30.125+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:30.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:30 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:30.125+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:30.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:30 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:30.125+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:30.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:30 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:30.125+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:42.943560+0000 front 2024-05-09T05:00:42.943457+0000 (oldest deadline 2024-05-09T05:01:05.243517+0000) 2024-05-09T05:01:30.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:30 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:30.125+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-05-09T05:00:51.045316+0000 front 2024-05-09T05:00:51.045520+0000 (oldest deadline 2024-05-09T05:01:15.145110+0000) 2024-05-09T05:01:30.601 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:30 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:30.125+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6820 osd.7 since back 2024-05-09T05:01:00.446390+0000 front 2024-05-09T05:01:00.446454+0000 (oldest deadline 2024-05-09T05:01:24.546257+0000) 2024-05-09T05:01:31.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:31 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:31.164+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:31.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:31 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:31.164+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:31.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:31 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:31.164+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:31.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:31 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:31.164+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:31.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:31 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:31.164+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:42.943560+0000 front 2024-05-09T05:00:42.943457+0000 (oldest deadline 2024-05-09T05:01:05.243517+0000) 2024-05-09T05:01:31.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:31 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:31.164+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-05-09T05:00:51.045316+0000 front 2024-05-09T05:00:51.045520+0000 (oldest deadline 2024-05-09T05:01:15.145110+0000) 2024-05-09T05:01:31.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:31 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:31.164+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6820 osd.7 since back 2024-05-09T05:01:00.446390+0000 front 2024-05-09T05:01:00.446454+0000 (oldest deadline 2024-05-09T05:01:24.546257+0000) 2024-05-09T05:01:32.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:32 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:32.121+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:32.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:32 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:32.121+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:32.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:32 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:32.121+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:32.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:32 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:32.121+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:32.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:32 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:32.121+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:42.943560+0000 front 2024-05-09T05:00:42.943457+0000 (oldest deadline 2024-05-09T05:01:05.243517+0000) 2024-05-09T05:01:32.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:32 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:32.121+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-05-09T05:00:51.045316+0000 front 2024-05-09T05:00:51.045520+0000 (oldest deadline 2024-05-09T05:01:15.145110+0000) 2024-05-09T05:01:32.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:32 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:32.121+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6820 osd.7 since back 2024-05-09T05:01:00.446390+0000 front 2024-05-09T05:01:00.446454+0000 (oldest deadline 2024-05-09T05:01:24.546257+0000) 2024-05-09T05:01:32.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:32 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:32.121+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6820 osd.8 since back 2024-05-09T05:01:06.847220+0000 front 2024-05-09T05:01:06.847259+0000 (oldest deadline 2024-05-09T05:01:31.547289+0000) 2024-05-09T05:01:33.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:33 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:33.130+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:33.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:33 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:33.130+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:33.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:33 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:33.130+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:33.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:33 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:33.130+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:33.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:33 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:33.130+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:42.943560+0000 front 2024-05-09T05:00:42.943457+0000 (oldest deadline 2024-05-09T05:01:05.243517+0000) 2024-05-09T05:01:33.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:33 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:33.130+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-05-09T05:00:51.045316+0000 front 2024-05-09T05:00:51.045520+0000 (oldest deadline 2024-05-09T05:01:15.145110+0000) 2024-05-09T05:01:33.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:33 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:33.130+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6820 osd.7 since back 2024-05-09T05:01:00.446390+0000 front 2024-05-09T05:01:00.446454+0000 (oldest deadline 2024-05-09T05:01:24.546257+0000) 2024-05-09T05:01:33.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:33 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:33.130+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6820 osd.8 since back 2024-05-09T05:01:06.847220+0000 front 2024-05-09T05:01:06.847259+0000 (oldest deadline 2024-05-09T05:01:31.547289+0000) 2024-05-09T05:01:34.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:34 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:34.160+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-05-09T04:59:58.735770+0000 front 2024-05-09T04:59:58.735965+0000 (oldest deadline 2024-05-09T05:00:24.635483+0000) 2024-05-09T05:01:34.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:34 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:34.160+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-05-09T05:00:16.838245+0000 front 2024-05-09T05:00:16.838279+0000 (oldest deadline 2024-05-09T05:00:38.538471+0000) 2024-05-09T05:01:34.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:34 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:34.160+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2024-05-09T05:00:26.140500+0000 front 2024-05-09T05:00:26.140638+0000 (oldest deadline 2024-05-09T05:00:52.040315+0000) 2024-05-09T05:01:34.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:34 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:34.160+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-05-09T05:00:33.141184+0000 front 2024-05-09T05:00:33.141231+0000 (oldest deadline 2024-05-09T05:00:58.341714+0000) 2024-05-09T05:01:34.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:34 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:34.160+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2024-05-09T05:00:42.943560+0000 front 2024-05-09T05:00:42.943457+0000 (oldest deadline 2024-05-09T05:01:05.243517+0000) 2024-05-09T05:01:34.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:34 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:34.160+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2024-05-09T05:00:51.045316+0000 front 2024-05-09T05:00:51.045520+0000 (oldest deadline 2024-05-09T05:01:15.145110+0000) 2024-05-09T05:01:34.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:34 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:34.160+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.169:6820 osd.7 since back 2024-05-09T05:01:00.446390+0000 front 2024-05-09T05:01:00.446454+0000 (oldest deadline 2024-05-09T05:01:24.546257+0000) 2024-05-09T05:01:34.600 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:34 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11[60296]: 2024-05-09T05:01:34.160+0000 7f062c6f4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6820 osd.8 since back 2024-05-09T05:01:06.847220+0000 front 2024-05-09T05:01:06.847259+0000 (oldest deadline 2024-05-09T05:01:31.547289+0000) 2024-05-09T05:01:35.088 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:34 smithi186 podman[70488]: 2024-05-09 05:01:34.837077708 +0000 UTC m=+5.457066194 container died 9f23ba9a373bbced3cb5acfc6d3fcf17552638f4d77b31bfd94fa38482014517 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-09T05:01:35.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:35 smithi186 podman[70488]: 2024-05-09 05:01:35.187402125 +0000 UTC m=+5.807390652 container cleanup 9f23ba9a373bbced3cb5acfc6d3fcf17552638f4d77b31bfd94fa38482014517 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2) 2024-05-09T05:01:35.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:35 smithi186 bash[70488]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11 2024-05-09T05:01:35.789 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:35 smithi186 podman[70506]: 2024-05-09 05:01:35.355119859 +0000 UTC m=+0.517964013 container remove 9f23ba9a373bbced3cb5acfc6d3fcf17552638f4d77b31bfd94fa38482014517 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-09T05:01:36.091 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:35 smithi186 podman[70594]: 2024-05-09 05:01:35.789377464 +0000 UTC m=+0.110877567 container create 9dcbcb5c78d269e7c9263303772e070e3e7434b31827c2997bfa3f5e288d0be7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T05:01:36.091 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:35 smithi186 podman[70594]: 2024-05-09 05:01:35.697299895 +0000 UTC m=+0.018799956 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:01:36.091 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:36 smithi186 podman[70594]: 2024-05-09 05:01:36.016959575 +0000 UTC m=+0.338459627 container init 9dcbcb5c78d269e7c9263303772e070e3e7434b31827c2997bfa3f5e288d0be7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T05:01:36.091 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:36 smithi186 podman[70594]: 2024-05-09 05:01:36.020557121 +0000 UTC m=+0.342057159 container start 9dcbcb5c78d269e7c9263303772e070e3e7434b31827c2997bfa3f5e288d0be7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2) 2024-05-09T05:01:36.092 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:36 smithi186 podman[70594]: 2024-05-09 05:01:36.091565782 +0000 UTC m=+0.413065819 container attach 9dcbcb5c78d269e7c9263303772e070e3e7434b31827c2997bfa3f5e288d0be7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2) 2024-05-09T05:01:36.350 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:36 smithi186 podman[70594]: 2024-05-09 05:01:36.311461944 +0000 UTC m=+0.632961992 container died 9dcbcb5c78d269e7c9263303772e070e3e7434b31827c2997bfa3f5e288d0be7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-09T05:01:36.846 INFO:journalctl@ceph.osd.11.smithi186.stdout:May 09 05:01:36 smithi186 podman[70620]: 2024-05-09 05:01:36.828462864 +0000 UTC m=+0.511548102 container remove 9dcbcb5c78d269e7c9263303772e070e3e7434b31827c2997bfa3f5e288d0be7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux ) 2024-05-09T05:01:36.852 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@osd.11.service' 2024-05-09T05:01:36.891 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:01:36.891 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-09T05:01:36.891 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-09T05:01:36.892 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.a 2024-05-09T05:01:37.447 INFO:journalctl@ceph.mds.a.smithi006.stdout:May 09 05:01:37 smithi006 systemd[1]: Stopping Ceph mds.a for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:01:37.947 INFO:journalctl@ceph.mds.a.smithi006.stdout:May 09 05:01:37 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-a[69196]: 2024-05-09T05:01:37.509+0000 7feb87000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:01:37.948 INFO:journalctl@ceph.mds.a.smithi006.stdout:May 09 05:01:37 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-a[69196]: 2024-05-09T05:01:37.509+0000 7feb87000700 -1 mds.a *** got signal Terminated *** 2024-05-09T05:01:41.300 INFO:journalctl@ceph.mds.a.smithi006.stdout:May 09 05:01:40 smithi006 podman[95123]: 2024-05-09 05:01:40.967014666 +0000 UTC m=+3.580315844 container died 1a921a25b562d644a9a12a0414bc0692449e91e82b86ba061d5889097ddb893a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-09T05:01:41.697 INFO:journalctl@ceph.mds.a.smithi006.stdout:May 09 05:01:41 smithi006 podman[95123]: 2024-05-09 05:01:41.494228384 +0000 UTC m=+4.107529555 container cleanup 1a921a25b562d644a9a12a0414bc0692449e91e82b86ba061d5889097ddb893a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-a, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240507, ceph=True) 2024-05-09T05:01:41.697 INFO:journalctl@ceph.mds.a.smithi006.stdout:May 09 05:01:41 smithi006 bash[95123]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-a 2024-05-09T05:01:42.197 INFO:journalctl@ceph.mds.a.smithi006.stdout:May 09 05:01:41 smithi006 podman[95137]: 2024-05-09 05:01:41.75430015 +0000 UTC m=+0.767798025 container remove 1a921a25b562d644a9a12a0414bc0692449e91e82b86ba061d5889097ddb893a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T05:01:42.292 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.a.service' 2024-05-09T05:01:42.408 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:01:42.408 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-09T05:01:42.409 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-09T05:01:42.409 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.d 2024-05-09T05:01:42.947 INFO:journalctl@ceph.mds.d.smithi006.stdout:May 09 05:01:42 smithi006 systemd[1]: Stopping Ceph mds.d for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:01:43.447 INFO:journalctl@ceph.mds.d.smithi006.stdout:May 09 05:01:42 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-d[67968]: 2024-05-09T05:01:42.954+0000 7f649e200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:01:43.447 INFO:journalctl@ceph.mds.d.smithi006.stdout:May 09 05:01:42 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-d[67968]: 2024-05-09T05:01:42.954+0000 7f649e200700 -1 mds.d *** got signal Terminated *** 2024-05-09T05:01:46.447 INFO:journalctl@ceph.mds.d.smithi006.stdout:May 09 05:01:46 smithi006 podman[95242]: 2024-05-09 05:01:46.186213894 +0000 UTC m=+3.350215650 container died 507be399f6da9b932396846fd7b80be24df57057a4c0b09639e9cc3fdda09903 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-d, org.label-schema.build-date=20240507, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-09T05:01:46.947 INFO:journalctl@ceph.mds.d.smithi006.stdout:May 09 05:01:46 smithi006 podman[95242]: 2024-05-09 05:01:46.714956913 +0000 UTC m=+3.878958662 container cleanup 507be399f6da9b932396846fd7b80be24df57057a4c0b09639e9cc3fdda09903 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-d, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507) 2024-05-09T05:01:46.947 INFO:journalctl@ceph.mds.d.smithi006.stdout:May 09 05:01:46 smithi006 bash[95242]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-d 2024-05-09T05:01:47.447 INFO:journalctl@ceph.mds.d.smithi006.stdout:May 09 05:01:47 smithi006 podman[95256]: 2024-05-09 05:01:47.016505142 +0000 UTC m=+0.808825098 container remove 507be399f6da9b932396846fd7b80be24df57057a4c0b09639e9cc3fdda09903 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-d, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T05:01:47.541 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.d.service' 2024-05-09T05:01:47.650 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:01:47.650 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-09T05:01:47.650 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-09T05:01:47.650 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.g 2024-05-09T05:01:48.197 INFO:journalctl@ceph.mds.g.smithi006.stdout:May 09 05:01:47 smithi006 systemd[1]: Stopping Ceph mds.g for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:01:48.697 INFO:journalctl@ceph.mds.g.smithi006.stdout:May 09 05:01:48 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-g[68748]: 2024-05-09T05:01:48.316+0000 7f7c24400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:01:48.697 INFO:journalctl@ceph.mds.g.smithi006.stdout:May 09 05:01:48 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-g[68748]: 2024-05-09T05:01:48.316+0000 7f7c24400700 -1 mds.g *** got signal Terminated *** 2024-05-09T05:01:50.448 INFO:journalctl@ceph.mds.g.smithi006.stdout:May 09 05:01:50 smithi006 podman[95361]: 2024-05-09 05:01:50.149994519 +0000 UTC m=+1.995144207 container died 1e1230aa6c00ecb35b0ff366f3ff58c2eef2396131e3fb18993ed145db5d3bcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-g, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=) 2024-05-09T05:01:50.848 INFO:journalctl@ceph.mds.g.smithi006.stdout:May 09 05:01:50 smithi006 podman[95361]: 2024-05-09 05:01:50.569065049 +0000 UTC m=+2.414214750 container cleanup 1e1230aa6c00ecb35b0ff366f3ff58c2eef2396131e3fb18993ed145db5d3bcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-g, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD) 2024-05-09T05:01:50.848 INFO:journalctl@ceph.mds.g.smithi006.stdout:May 09 05:01:50 smithi006 bash[95361]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-g 2024-05-09T05:01:51.197 INFO:journalctl@ceph.mds.g.smithi006.stdout:May 09 05:01:50 smithi006 podman[95375]: 2024-05-09 05:01:50.847346593 +0000 UTC m=+0.673711675 container remove 1e1230aa6c00ecb35b0ff366f3ff58c2eef2396131e3fb18993ed145db5d3bcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-g, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-09T05:01:51.382 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.g.service' 2024-05-09T05:01:51.463 INFO:journalctl@ceph.mds.g.smithi006.stdout:May 09 05:01:51 smithi006 systemd[1]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.g.service: Deactivated successfully. 2024-05-09T05:01:51.463 INFO:journalctl@ceph.mds.g.smithi006.stdout:May 09 05:01:51 smithi006 systemd[1]: Stopped Ceph mds.g for f74d813c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T05:01:51.463 INFO:journalctl@ceph.mds.g.smithi006.stdout:May 09 05:01:51 smithi006 systemd[1]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.g.service: Consumed 4.094s CPU time. 2024-05-09T05:01:51.497 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:01:51.497 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-09T05:01:51.498 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-09T05:01:51.498 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.j 2024-05-09T05:01:51.947 INFO:journalctl@ceph.mds.j.smithi006.stdout:May 09 05:01:51 smithi006 systemd[1]: Stopping Ceph mds.j for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:01:52.448 INFO:journalctl@ceph.mds.j.smithi006.stdout:May 09 05:01:52 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-j[67555]: 2024-05-09T05:01:52.172+0000 7f3ed1000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:01:52.448 INFO:journalctl@ceph.mds.j.smithi006.stdout:May 09 05:01:52 smithi006 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-j[67555]: 2024-05-09T05:01:52.172+0000 7f3ed1000700 -1 mds.j *** got signal Terminated *** 2024-05-09T05:01:53.947 INFO:journalctl@ceph.mds.j.smithi006.stdout:May 09 05:01:53 smithi006 podman[95481]: 2024-05-09 05:01:53.666670921 +0000 UTC m=+1.671113562 container died 443ae74a980ba5014215ebc561068c6a81c08e127d73668c6f64fd387522638a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-j, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-09T05:01:54.306 INFO:journalctl@ceph.mds.j.smithi006.stdout:May 09 05:01:54 smithi006 podman[95481]: 2024-05-09 05:01:54.141769151 +0000 UTC m=+2.146211786 container cleanup 443ae74a980ba5014215ebc561068c6a81c08e127d73668c6f64fd387522638a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-j, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240507) 2024-05-09T05:01:54.306 INFO:journalctl@ceph.mds.j.smithi006.stdout:May 09 05:01:54 smithi006 bash[95481]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-j 2024-05-09T05:01:54.697 INFO:journalctl@ceph.mds.j.smithi006.stdout:May 09 05:01:54 smithi006 podman[95504]: 2024-05-09 05:01:54.351124329 +0000 UTC m=+0.664523374 container remove 443ae74a980ba5014215ebc561068c6a81c08e127d73668c6f64fd387522638a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-j, org.label-schema.build-date=20240507, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T05:01:54.815 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.j.service' 2024-05-09T05:01:54.930 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:01:54.931 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-09T05:01:54.931 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-09T05:01:54.931 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.b 2024-05-09T05:01:55.311 INFO:journalctl@ceph.mds.b.smithi169.stdout:May 09 05:01:54 smithi169 systemd[1]: Stopping Ceph mds.b for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:01:55.574 INFO:journalctl@ceph.mds.b.smithi169.stdout:May 09 05:01:55 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-b[64939]: 2024-05-09T05:01:55.310+0000 7f8d6ce5b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:01:55.574 INFO:journalctl@ceph.mds.b.smithi169.stdout:May 09 05:01:55 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-b[64939]: 2024-05-09T05:01:55.310+0000 7f8d6ce5b700 -1 mds.b *** got signal Terminated *** 2024-05-09T05:01:58.331 INFO:journalctl@ceph.mds.b.smithi169.stdout:May 09 05:01:58 smithi169 podman[69772]: 2024-05-09 05:01:58.038504536 +0000 UTC m=+2.916910094 container died a0c62ed0cc4211281ba66c5ad0884d34ab3623019c4a485d6ba2e49626f24168 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-09T05:01:58.617 INFO:journalctl@ceph.mds.b.smithi169.stdout:May 09 05:01:58 smithi169 podman[69772]: 2024-05-09 05:01:58.430876646 +0000 UTC m=+3.309282213 container cleanup a0c62ed0cc4211281ba66c5ad0884d34ab3623019c4a485d6ba2e49626f24168 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-b, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-09T05:01:58.617 INFO:journalctl@ceph.mds.b.smithi169.stdout:May 09 05:01:58 smithi169 bash[69772]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-b 2024-05-09T05:01:58.859 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.b.service' 2024-05-09T05:01:58.893 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:01:58.893 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-09T05:01:58.893 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-09T05:01:58.893 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.e 2024-05-09T05:01:59.218 INFO:journalctl@ceph.mds.e.smithi169.stdout:May 09 05:01:58 smithi169 systemd[1]: Stopping Ceph mds.e for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:01:59.617 INFO:journalctl@ceph.mds.e.smithi169.stdout:May 09 05:01:59 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-e[64307]: 2024-05-09T05:01:59.218+0000 7f7ca792d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:01:59.617 INFO:journalctl@ceph.mds.e.smithi169.stdout:May 09 05:01:59 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-e[64307]: 2024-05-09T05:01:59.218+0000 7f7ca792d700 -1 mds.e *** got signal Terminated *** 2024-05-09T05:02:02.311 INFO:journalctl@ceph.mds.e.smithi169.stdout:May 09 05:02:02 smithi169 podman[69905]: 2024-05-09 05:02:02.053928373 +0000 UTC m=+2.947275716 container died 67aa99d0163a6f1b85c5651a87547231a4461e9b79c5103d8f368068c9b16510 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T05:02:02.617 INFO:journalctl@ceph.mds.e.smithi169.stdout:May 09 05:02:02 smithi169 podman[69905]: 2024-05-09 05:02:02.411476116 +0000 UTC m=+3.304823465 container cleanup 67aa99d0163a6f1b85c5651a87547231a4461e9b79c5103d8f368068c9b16510 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-09T05:02:02.617 INFO:journalctl@ceph.mds.e.smithi169.stdout:May 09 05:02:02 smithi169 bash[69905]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-e 2024-05-09T05:02:02.838 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.e.service' 2024-05-09T05:02:02.870 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:02:02.871 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-09T05:02:02.871 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-09T05:02:02.871 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.h 2024-05-09T05:02:03.199 INFO:journalctl@ceph.mds.h.smithi169.stdout:May 09 05:02:02 smithi169 systemd[1]: Stopping Ceph mds.h for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:02:03.617 INFO:journalctl@ceph.mds.h.smithi169.stdout:May 09 05:02:03 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-h[64624]: 2024-05-09T05:02:03.198+0000 7f818086c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:02:03.617 INFO:journalctl@ceph.mds.h.smithi169.stdout:May 09 05:02:03 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-h[64624]: 2024-05-09T05:02:03.198+0000 7f818086c700 -1 mds.h *** got signal Terminated *** 2024-05-09T05:02:06.324 INFO:journalctl@ceph.mds.h.smithi169.stdout:May 09 05:02:06 smithi169 podman[70027]: 2024-05-09 05:02:06.049682543 +0000 UTC m=+2.957951608 container died 03ff7e9c27ce244b5fee5d19ff1cd4cf60d110016c546945b8cccfe50fac0a7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-h, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-09T05:02:06.617 INFO:journalctl@ceph.mds.h.smithi169.stdout:May 09 05:02:06 smithi169 podman[70027]: 2024-05-09 05:02:06.407194069 +0000 UTC m=+3.315463133 container cleanup 03ff7e9c27ce244b5fee5d19ff1cd4cf60d110016c546945b8cccfe50fac0a7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-h, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-09T05:02:06.617 INFO:journalctl@ceph.mds.h.smithi169.stdout:May 09 05:02:06 smithi169 bash[70027]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-h 2024-05-09T05:02:06.871 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.h.service' 2024-05-09T05:02:06.902 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:02:06.903 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-09T05:02:06.903 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-09T05:02:06.903 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.k 2024-05-09T05:02:07.228 INFO:journalctl@ceph.mds.k.smithi169.stdout:May 09 05:02:06 smithi169 systemd[1]: Stopping Ceph mds.k for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:02:07.617 INFO:journalctl@ceph.mds.k.smithi169.stdout:May 09 05:02:07 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-k[63987]: 2024-05-09T05:02:07.228+0000 7f1b47f3a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:02:07.617 INFO:journalctl@ceph.mds.k.smithi169.stdout:May 09 05:02:07 smithi169 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-k[63987]: 2024-05-09T05:02:07.228+0000 7f1b47f3a700 -1 mds.k *** got signal Terminated *** 2024-05-09T05:02:10.337 INFO:journalctl@ceph.mds.k.smithi169.stdout:May 09 05:02:10 smithi169 podman[70147]: 2024-05-09 05:02:10.045858037 +0000 UTC m=+2.922620927 container died 92ceeac7bec680bffe8909eff262ed4a3d8d3aebf19b8b8c8dcc9dbdc378a4d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-k, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-09T05:02:10.617 INFO:journalctl@ceph.mds.k.smithi169.stdout:May 09 05:02:10 smithi169 podman[70147]: 2024-05-09 05:02:10.420287364 +0000 UTC m=+3.297050252 container cleanup 92ceeac7bec680bffe8909eff262ed4a3d8d3aebf19b8b8c8dcc9dbdc378a4d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T05:02:10.617 INFO:journalctl@ceph.mds.k.smithi169.stdout:May 09 05:02:10 smithi169 bash[70147]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-k 2024-05-09T05:02:10.749 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.k.service' 2024-05-09T05:02:10.779 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:02:10.780 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-09T05:02:10.780 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-09T05:02:10.780 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.c 2024-05-09T05:02:11.100 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 09 05:02:10 smithi186 systemd[1]: Stopping Ceph mds.c for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:02:11.599 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 09 05:02:11 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-c[65558]: 2024-05-09T05:02:11.119+0000 7f4a86a43700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:02:11.600 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 09 05:02:11 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-c[65558]: 2024-05-09T05:02:11.119+0000 7f4a86a43700 -1 mds.c *** got signal Terminated *** 2024-05-09T05:02:14.350 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 09 05:02:14 smithi186 podman[70698]: 2024-05-09 05:02:14.066655723 +0000 UTC m=+3.093216323 container died c5741dc9744be66f4b72d6096f10095bb938a33e5890f76b3dfc8f2e1d186052 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-09T05:02:14.695 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 09 05:02:14 smithi186 podman[70698]: 2024-05-09 05:02:14.502424162 +0000 UTC m=+3.528984774 container cleanup c5741dc9744be66f4b72d6096f10095bb938a33e5890f76b3dfc8f2e1d186052 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T05:02:14.695 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 09 05:02:14 smithi186 bash[70698]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-c 2024-05-09T05:02:14.928 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.c.service' 2024-05-09T05:02:14.949 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 09 05:02:14 smithi186 podman[70724]: 2024-05-09 05:02:14.695038099 +0000 UTC m=+0.632084440 container remove c5741dc9744be66f4b72d6096f10095bb938a33e5890f76b3dfc8f2e1d186052 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-c, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=) 2024-05-09T05:02:14.949 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 09 05:02:14 smithi186 systemd[1]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.c.service: Deactivated successfully. 2024-05-09T05:02:14.949 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 09 05:02:14 smithi186 systemd[1]: Stopped Ceph mds.c for f74d813c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T05:02:14.949 INFO:journalctl@ceph.mds.c.smithi186.stdout:May 09 05:02:14 smithi186 systemd[1]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.c.service: Consumed 1.696s CPU time. 2024-05-09T05:02:14.962 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:02:14.963 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-09T05:02:14.963 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-09T05:02:14.963 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.f 2024-05-09T05:02:15.340 INFO:journalctl@ceph.mds.f.smithi186.stdout:May 09 05:02:15 smithi186 systemd[1]: Stopping Ceph mds.f for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:02:15.599 INFO:journalctl@ceph.mds.f.smithi186.stdout:May 09 05:02:15 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-f[65250]: 2024-05-09T05:02:15.339+0000 7f8f54918700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:02:15.599 INFO:journalctl@ceph.mds.f.smithi186.stdout:May 09 05:02:15 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-f[65250]: 2024-05-09T05:02:15.339+0000 7f8f54918700 -1 mds.f *** got signal Terminated *** 2024-05-09T05:02:18.349 INFO:journalctl@ceph.mds.f.smithi186.stdout:May 09 05:02:18 smithi186 podman[70830]: 2024-05-09 05:02:18.065240225 +0000 UTC m=+2.882441202 container died 6d28b7e38c06ba1773736d8ee72bd76b1139cf9b7ed2f3372a4455edaf67961a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507) 2024-05-09T05:02:18.699 INFO:journalctl@ceph.mds.f.smithi186.stdout:May 09 05:02:18 smithi186 podman[70830]: 2024-05-09 05:02:18.473423831 +0000 UTC m=+3.290624817 container cleanup 6d28b7e38c06ba1773736d8ee72bd76b1139cf9b7ed2f3372a4455edaf67961a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-f, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-09T05:02:18.699 INFO:journalctl@ceph.mds.f.smithi186.stdout:May 09 05:02:18 smithi186 bash[70830]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-f 2024-05-09T05:02:18.847 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.f.service' 2024-05-09T05:02:18.880 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:02:18.880 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-09T05:02:18.880 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-09T05:02:18.880 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.i 2024-05-09T05:02:19.219 INFO:journalctl@ceph.mds.i.smithi186.stdout:May 09 05:02:18 smithi186 systemd[1]: Stopping Ceph mds.i for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:02:19.599 INFO:journalctl@ceph.mds.i.smithi186.stdout:May 09 05:02:19 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-i[65862]: 2024-05-09T05:02:19.218+0000 7f586e04c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:02:19.600 INFO:journalctl@ceph.mds.i.smithi186.stdout:May 09 05:02:19 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-i[65862]: 2024-05-09T05:02:19.218+0000 7f586e04c700 -1 mds.i *** got signal Terminated *** 2024-05-09T05:02:22.335 INFO:journalctl@ceph.mds.i.smithi186.stdout:May 09 05:02:22 smithi186 podman[70952]: 2024-05-09 05:02:22.055040364 +0000 UTC m=+2.959752720 container died 4e2839b809cc7e0eb94f27cff72206490eb070c57e62331819c33b1594867916 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T05:02:22.600 INFO:journalctl@ceph.mds.i.smithi186.stdout:May 09 05:02:22 smithi186 podman[70952]: 2024-05-09 05:02:22.418591269 +0000 UTC m=+3.323303633 container cleanup 4e2839b809cc7e0eb94f27cff72206490eb070c57e62331819c33b1594867916 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-i, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2) 2024-05-09T05:02:22.600 INFO:journalctl@ceph.mds.i.smithi186.stdout:May 09 05:02:22 smithi186 bash[70952]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-i 2024-05-09T05:02:22.835 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.i.service' 2024-05-09T05:02:22.867 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:02:22.868 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-09T05:02:22.868 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-09T05:02:22.868 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.l 2024-05-09T05:02:23.099 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 09 05:02:22 smithi186 systemd[1]: Stopping Ceph mds.l for f74d813c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:02:23.599 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 09 05:02:23 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-l[64938]: 2024-05-09T05:02:23.206+0000 7fa941005700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:02:23.600 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 09 05:02:23 smithi186 ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-l[64938]: 2024-05-09T05:02:23.206+0000 7fa941005700 -1 mds.l *** got signal Terminated *** 2024-05-09T05:02:26.298 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 09 05:02:26 smithi186 podman[71073]: 2024-05-09 05:02:26.043337677 +0000 UTC m=+2.963224484 container died 1123264bf8e8e376547c91d8ef07d5dbe1c7c5680c7da11ea977381c1f2a8a90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-l, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-09T05:02:26.573 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 09 05:02:26 smithi186 podman[71073]: 2024-05-09 05:02:26.389461487 +0000 UTC m=+3.309348295 container cleanup 1123264bf8e8e376547c91d8ef07d5dbe1c7c5680c7da11ea977381c1f2a8a90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-l, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240507, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-09T05:02:26.573 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 09 05:02:26 smithi186 bash[71073]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-l 2024-05-09T05:02:26.812 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.l.service' 2024-05-09T05:02:26.832 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 09 05:02:26 smithi186 podman[71087]: 2024-05-09 05:02:26.573836956 +0000 UTC m=+0.525446766 container remove 1123264bf8e8e376547c91d8ef07d5dbe1c7c5680c7da11ea977381c1f2a8a90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-f74d813c-0dbd-11ef-bc97-c7b262605968-mds-l, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2) 2024-05-09T05:02:26.832 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 09 05:02:26 smithi186 systemd[1]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.l.service: Deactivated successfully. 2024-05-09T05:02:26.832 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 09 05:02:26 smithi186 systemd[1]: Stopped Ceph mds.l for f74d813c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T05:02:26.832 INFO:journalctl@ceph.mds.l.smithi186.stdout:May 09 05:02:26 smithi186 systemd[1]: ceph-f74d813c-0dbd-11ef-bc97-c7b262605968@mds.l.service: Consumed 1.853s CPU time. 2024-05-09T05:02:26.843 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:02:26.843 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-09T05:02:26.843 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-09T05:02:27.178 INFO:teuthology.orchestra.run.smithi006.stdout:Deleting cluster with fsid: f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T05:02:32.587 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-09T05:02:32.780 INFO:teuthology.orchestra.run.smithi169.stdout:Deleting cluster with fsid: f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T05:02:35.219 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-09T05:02:35.409 INFO:teuthology.orchestra.run.smithi186.stdout:Deleting cluster with fsid: f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T05:02:37.717 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T05:02:37.816 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T05:02:37.848 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T05:02:37.873 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-09T05:02:37.875 DEBUG:teuthology.misc:Transferring archived files from smithi006:/var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699065/remote/smithi006/crash 2024-05-09T05:02:37.876 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar c -f - -C /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/crash -- . 2024-05-09T05:02:37.998 INFO:teuthology.orchestra.run.smithi006.stderr:tar: /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T05:02:37.999 INFO:teuthology.orchestra.run.smithi006.stderr:tar: Error is not recoverable: exiting now 2024-05-09T05:02:38.002 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699065/remote/smithi169/crash 2024-05-09T05:02:38.007 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/crash -- . 2024-05-09T05:02:38.045 INFO:teuthology.orchestra.run.smithi169.stderr:tar: /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T05:02:38.045 INFO:teuthology.orchestra.run.smithi169.stderr:tar: Error is not recoverable: exiting now 2024-05-09T05:02:38.046 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699065/remote/smithi186/crash 2024-05-09T05:02:38.047 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/crash -- . 2024-05-09T05:02:38.081 INFO:teuthology.orchestra.run.smithi186.stderr:tar: /var/lib/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T05:02:38.081 INFO:teuthology.orchestra.run.smithi186.stderr:tar: Error is not recoverable: exiting now 2024-05-09T05:02:38.082 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-09T05:02:38.083 DEBUG:teuthology.orchestra.run.smithi006:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-05-09T05:02:38.192 INFO:tasks.cephadm:Compressing logs... 2024-05-09T05:02:38.192 DEBUG:teuthology.orchestra.run.smithi006:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T05:02:38.197 DEBUG:teuthology.orchestra.run.smithi169:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T05:02:38.200 DEBUG:teuthology.orchestra.run.smithi186:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T05:02:38.223 INFO:teuthology.orchestra.run.smithi186.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph.log 2024-05-09T05:02:38.224 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-05-09T05:02:38.224 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.2.log 2024-05-09T05:02:38.224 INFO:teuthology.orchestra.run.smithi186.stderr:gzip/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph.log: -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.f.log 2024-05-09T05:02:38.224 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-09T05:02:38.224 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.5.log 2024-05-09T05:02:38.226 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.c.log 2024-05-09T05:02:38.226 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph.audit.log: /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mon.c.log 2024-05-09T05:02:38.227 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mgr.z.log 2024-05-09T05:02:38.227 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.i.log 2024-05-09T05:02:38.227 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mgr.z.log: 87.3% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-09T05:02:38.227 INFO:teuthology.orchestra.run.smithi186.stderr: 90.2% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-09T05:02:38.231 INFO:teuthology.orchestra.run.smithi169.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph.log 2024-05-09T05:02:38.231 INFO:teuthology.orchestra.run.smithi169.stderr:: No such file or directory 2024-05-09T05:02:38.231 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-09T05:02:38.232 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.7.log 2024-05-09T05:02:38.232 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph.log: /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.b.log 2024-05-09T05:02:38.232 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.4.log 2024-05-09T05:02:38.232 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.1.log 2024-05-09T05:02:38.232 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.b.log: /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mgr.y.log 2024-05-09T05:02:38.232 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.h.log 2024-05-09T05:02:38.232 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mgr.y.log: 87.3%gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.10.log 2024-05-09T05:02:38.232 INFO:teuthology.orchestra.run.smithi169.stderr: -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-09T05:02:38.232 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.h.log: 90.2% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-09T05:02:38.232 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-09T05:02:38.232 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.e.log 2024-05-09T05:02:38.232 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph.cephadm.log: 82.4% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-09T05:02:38.232 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-09T05:02:38.234 INFO:teuthology.orchestra.run.smithi186.stderr: 92.7% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mgr.z.log.gz 2024-05-09T05:02:38.234 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-09T05:02:38.234 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.11.log 2024-05-09T05:02:38.234 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.8.log 2024-05-09T05:02:38.234 INFO:teuthology.orchestra.run.smithi186.stderr: 82.4% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-09T05:02:38.236 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.e.log: 92.6% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mgr.y.log.gz 2024-05-09T05:02:38.236 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mon.b.log 2024-05-09T05:02:38.247 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-09T05:02:38.249 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-volume.log: 91.0% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-09T05:02:38.249 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.k.log 2024-05-09T05:02:38.249 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-09T05:02:38.255 INFO:teuthology.orchestra.run.smithi006.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-09T05:02:38.258 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph.log 2024-05-09T05:02:38.259 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-09T05:02:38.260 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.l.log 2024-05-09T05:02:38.261 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.g.log 2024-05-09T05:02:38.261 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.j.log 2024-05-09T05:02:38.262 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.3.log 2024-05-09T05:02:38.262 INFO:teuthology.orchestra.run.smithi006.stderr: 87.2% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-09T05:02:38.263 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.0.log 2024-05-09T05:02:38.264 INFO:teuthology.orchestra.run.smithi006.stderr: 90.0% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-09T05:02:38.264 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mgr.x.log 2024-05-09T05:02:38.266 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mon.a.log 2024-05-09T05:02:38.267 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mgr.x.log: gzip -5 94.5% --verbose -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.j.log.gz -- 2024-05-09T05:02:38.267 INFO:teuthology.orchestra.run.smithi006.stderr: /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.6.log 2024-05-09T05:02:38.268 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-09T05:02:38.270 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.a.log 2024-05-09T05:02:38.270 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-09T05:02:38.272 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph.cephadm.log: 82.7% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-09T05:02:38.272 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.l.log: 94.3% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.f.log.gz 2024-05-09T05:02:38.273 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-09T05:02:38.274 INFO:teuthology.orchestra.run.smithi186.stderr: 93.1% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.c.log.gz 2024-05-09T05:02:38.275 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.d.log 2024-05-09T05:02:38.276 INFO:teuthology.orchestra.run.smithi186.stderr: 91.0% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-09T05:02:38.277 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.9.log 2024-05-09T05:02:38.280 INFO:teuthology.orchestra.run.smithi186.stderr: 92.7% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.i.log.gz 2024-05-09T05:02:38.283 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.9.log: /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-09T05:02:38.283 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-09T05:02:38.284 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mon.b.log: /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-09T05:02:38.284 INFO:teuthology.orchestra.run.smithi006.stderr: 94.3% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.a.log.gz 2024-05-09T05:02:38.285 INFO:teuthology.orchestra.run.smithi169.stderr: 92.8% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.h.log.gz 2024-05-09T05:02:38.286 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-05-09T05:02:38.292 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.admin.log: 90.9% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-09T05:02:38.294 INFO:teuthology.orchestra.run.smithi006.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-05-09T05:02:38.299 INFO:teuthology.orchestra.run.smithi169.stderr: 94.2% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.k.log.gz 2024-05-09T05:02:38.303 INFO:teuthology.orchestra.run.smithi006.stderr: 93.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-09T05:02:38.316 INFO:teuthology.orchestra.run.smithi186.stderr: 94.2% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.l.log.gz 2024-05-09T05:02:38.321 INFO:teuthology.orchestra.run.smithi006.stderr: 92.7% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.g.log.gz 2024-05-09T05:02:38.327 INFO:teuthology.orchestra.run.smithi169.stderr: 93.7% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.e.log.gz 2024-05-09T05:02:38.346 INFO:teuthology.orchestra.run.smithi006.stderr: 94.1% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.d.log.gz 2024-05-09T05:02:38.556 INFO:teuthology.orchestra.run.smithi006.stderr: 90.1% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mgr.x.log.gz 2024-05-09T05:02:38.557 INFO:teuthology.orchestra.run.smithi169.stderr: 93.0% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mds.b.log.gz 2024-05-09T05:02:38.633 INFO:teuthology.orchestra.run.smithi186.stderr: 91.7% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mon.c.log.gz 2024-05-09T05:02:38.646 INFO:teuthology.orchestra.run.smithi169.stderr: 91.9% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mon.b.log.gz 2024-05-09T05:02:38.862 INFO:teuthology.orchestra.run.smithi186.stderr: 92.7% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.11.log.gz 2024-05-09T05:02:39.065 INFO:teuthology.orchestra.run.smithi169.stderr: 92.6% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.7.log.gz 2024-05-09T05:02:39.155 INFO:teuthology.orchestra.run.smithi006.stderr: 92.4% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.6.log.gz 2024-05-09T05:02:39.181 INFO:teuthology.orchestra.run.smithi186.stderr: 92.6% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.5.log.gz 2024-05-09T05:02:39.196 INFO:teuthology.orchestra.run.smithi006.stderr: 90.4% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-mon.a.log.gz 2024-05-09T05:02:39.206 INFO:teuthology.orchestra.run.smithi169.stderr: 92.3% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.1.log.gz 2024-05-09T05:02:39.298 INFO:teuthology.orchestra.run.smithi006.stderr: 92.5% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.0.log.gz 2024-05-09T05:02:39.348 INFO:teuthology.orchestra.run.smithi186.stderr: 93.4% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.8.log.gz 2024-05-09T05:02:39.554 INFO:teuthology.orchestra.run.smithi169.stderr: 93.3% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.4.log.gz 2024-05-09T05:02:39.569 INFO:teuthology.orchestra.run.smithi006.stderr: 92.4% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.9.log.gz 2024-05-09T05:02:39.645 INFO:teuthology.orchestra.run.smithi186.stderr: 93.1% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.2.log.gz 2024-05-09T05:02:39.647 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-05-09T05:02:39.647 INFO:teuthology.orchestra.run.smithi186.stderr:real 0m1.433s 2024-05-09T05:02:39.647 INFO:teuthology.orchestra.run.smithi186.stderr:user 0m4.563s 2024-05-09T05:02:39.647 INFO:teuthology.orchestra.run.smithi186.stderr:sys 0m0.178s 2024-05-09T05:02:39.704 INFO:teuthology.orchestra.run.smithi169.stderr: 93.2% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.10.log.gz 2024-05-09T05:02:39.706 INFO:teuthology.orchestra.run.smithi169.stderr: 2024-05-09T05:02:39.706 INFO:teuthology.orchestra.run.smithi169.stderr:real 0m1.492s 2024-05-09T05:02:39.706 INFO:teuthology.orchestra.run.smithi169.stderr:user 0m5.360s 2024-05-09T05:02:39.706 INFO:teuthology.orchestra.run.smithi169.stderr:sys 0m0.202s 2024-05-09T05:02:39.958 INFO:teuthology.orchestra.run.smithi006.stderr: 93.1% -- replaced with /var/log/ceph/f74d813c-0dbd-11ef-bc97-c7b262605968/ceph-osd.3.log.gz 2024-05-09T05:02:39.962 INFO:teuthology.orchestra.run.smithi006.stderr: 2024-05-09T05:02:39.962 INFO:teuthology.orchestra.run.smithi006.stderr:real 0m1.732s 2024-05-09T05:02:39.962 INFO:teuthology.orchestra.run.smithi006.stderr:user 0m5.801s 2024-05-09T05:02:39.962 INFO:teuthology.orchestra.run.smithi006.stderr:sys 0m0.545s 2024-05-09T05:02:39.963 INFO:tasks.cephadm:Archiving logs... 2024-05-09T05:02:39.964 DEBUG:teuthology.misc:Transferring archived files from smithi006:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699065/remote/smithi006/log 2024-05-09T05:02:39.965 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T05:02:40.818 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699065/remote/smithi169/log 2024-05-09T05:02:40.819 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T05:02:41.654 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699065/remote/smithi186/log 2024-05-09T05:02:41.655 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T05:02:42.382 INFO:tasks.cephadm:Removing cluster... 2024-05-09T05:02:42.383 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 --force 2024-05-09T05:02:42.706 INFO:teuthology.orchestra.run.smithi006.stdout:Deleting cluster with fsid: f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T05:02:43.389 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 --force 2024-05-09T05:02:43.581 INFO:teuthology.orchestra.run.smithi169.stdout:Deleting cluster with fsid: f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T05:02:43.838 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f74d813c-0dbd-11ef-bc97-c7b262605968 --force 2024-05-09T05:02:44.016 INFO:teuthology.orchestra.run.smithi186.stdout:Deleting cluster with fsid: f74d813c-0dbd-11ef-bc97-c7b262605968 2024-05-09T05:02:44.270 INFO:tasks.cephadm:Removing cephadm ... 2024-05-09T05:02:44.270 DEBUG:teuthology.orchestra.run.smithi006:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T05:02:44.330 DEBUG:teuthology.orchestra.run.smithi169:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T05:02:44.350 DEBUG:teuthology.orchestra.run.smithi186:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T05:02:44.366 INFO:tasks.cephadm:Teardown complete 2024-05-09T05:02:44.366 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-09T05:02:44.378 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-09T05:02:44.379 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T05:02:44.387 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T05:02:44.392 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T05:02:44.546 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-09T05:02:44.559 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-09T05:02:44.559 DEBUG:teuthology.orchestra.run.smithi006:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T05:02:44.567 DEBUG:teuthology.orchestra.run.smithi169:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T05:02:44.570 DEBUG:teuthology.orchestra.run.smithi186:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T05:02:44.582 INFO:teuthology.orchestra.run.smithi169.stderr:bash: line 1: ntpq: command not found 2024-05-09T05:02:44.584 INFO:teuthology.orchestra.run.smithi186.stderr:bash: line 1: ntpq: command not found 2024-05-09T05:02:44.586 INFO:teuthology.orchestra.run.smithi169.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T05:02:44.586 INFO:teuthology.orchestra.run.smithi169.stdout:=============================================================================== 2024-05-09T05:02:44.586 INFO:teuthology.orchestra.run.smithi169.stdout:^* hv01.front.sepia.ceph.com 3 6 377 56 +2651us[+2660us] +/- 41ms 2024-05-09T05:02:44.586 INFO:teuthology.orchestra.run.smithi169.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 54 -2463us[-2463us] +/- 79ms 2024-05-09T05:02:44.586 INFO:teuthology.orchestra.run.smithi169.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 -2228us[-2228us] +/- 67ms 2024-05-09T05:02:44.586 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T05:02:44.589 INFO:teuthology.orchestra.run.smithi186.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T05:02:44.590 INFO:teuthology.orchestra.run.smithi186.stdout:=============================================================================== 2024-05-09T05:02:44.590 INFO:teuthology.orchestra.run.smithi186.stdout:^* hv01.front.sepia.ceph.com 3 6 377 54 +2655us[+2624us] +/- 41ms 2024-05-09T05:02:44.590 INFO:teuthology.orchestra.run.smithi186.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 54 -2453us[-2483us] +/- 79ms 2024-05-09T05:02:44.590 INFO:teuthology.orchestra.run.smithi186.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 53 -2213us[-2213us] +/- 67ms 2024-05-09T05:02:44.590 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T05:02:44.619 INFO:teuthology.orchestra.run.smithi006.stderr:bash: line 1: ntpq: command not found 2024-05-09T05:02:44.630 INFO:teuthology.orchestra.run.smithi006.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T05:02:44.630 INFO:teuthology.orchestra.run.smithi006.stdout:=============================================================================== 2024-05-09T05:02:44.630 INFO:teuthology.orchestra.run.smithi006.stdout:^* hv01.front.sepia.ceph.com 3 7 377 120 +2734us[+2727us] +/- 40ms 2024-05-09T05:02:44.630 INFO:teuthology.orchestra.run.smithi006.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 121 -2383us[-2390us] +/- 78ms 2024-05-09T05:02:44.630 INFO:teuthology.orchestra.run.smithi006.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 54 -2221us[-2221us] +/- 67ms 2024-05-09T05:02:44.630 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T05:02:44.632 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-09T05:02:44.643 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-09T05:02:44.644 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-09T05:02:44.690 DEBUG:teuthology.orchestra.run.smithi006:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T05:02:44.919 DEBUG:teuthology.orchestra.run.smithi169:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T05:02:44.983 DEBUG:teuthology.orchestra.run.smithi186:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T05:02:45.048 DEBUG:teuthology.orchestra.run.smithi006:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T05:02:45.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T05:02:45.149 DEBUG:teuthology.orchestra.run.smithi169:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T05:02:45.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T05:02:45.176 DEBUG:teuthology.orchestra.run.smithi186:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T05:02:45.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T05:02:45.203 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-09T05:02:45.214 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-09T05:02:45.249 INFO:teuthology.task.internal:Duration was 1968.960631 seconds 2024-05-09T05:02:45.249 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-09T05:02:45.299 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-09T05:02:45.299 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T05:02:45.309 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T05:02:45.311 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T05:02:45.346 INFO:teuthology.orchestra.run.smithi169.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T05:02:45.348 INFO:teuthology.orchestra.run.smithi186.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T05:02:45.455 INFO:teuthology.orchestra.run.smithi006.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T05:02:45.800 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-09T05:02:45.801 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi006.front.sepia.ceph.com 2024-05-09T05:02:45.801 DEBUG:teuthology.orchestra.run.smithi006:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T05:02:45.872 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi169.front.sepia.ceph.com 2024-05-09T05:02:45.872 DEBUG:teuthology.orchestra.run.smithi169:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T05:02:45.894 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi186.front.sepia.ceph.com 2024-05-09T05:02:45.894 DEBUG:teuthology.orchestra.run.smithi186:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T05:02:45.917 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-09T05:02:45.917 DEBUG:teuthology.orchestra.run.smithi006:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T05:02:45.926 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T05:02:45.936 DEBUG:teuthology.orchestra.run.smithi186:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T05:02:46.014 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-09T05:02:46.015 DEBUG:teuthology.orchestra.run.smithi006:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T05:02:46.020 DEBUG:teuthology.orchestra.run.smithi169:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T05:02:46.023 DEBUG:teuthology.orchestra.run.smithi186:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T05:02:46.852 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-09T05:02:46.872 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-09T05:02:46.872 DEBUG:teuthology.orchestra.run.smithi006:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T05:02:46.955 DEBUG:teuthology.orchestra.run.smithi169:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T05:02:46.982 DEBUG:teuthology.orchestra.run.smithi186:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T05:02:47.009 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-09T05:02:47.021 DEBUG:teuthology.orchestra.run.smithi006:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T05:02:47.029 DEBUG:teuthology.orchestra.run.smithi169:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T05:02:47.031 DEBUG:teuthology.orchestra.run.smithi186:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T05:02:47.053 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = core 2024-05-09T05:02:47.073 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern = core 2024-05-09T05:02:47.122 INFO:teuthology.orchestra.run.smithi006.stdout:kernel.core_pattern = core 2024-05-09T05:02:47.167 DEBUG:teuthology.orchestra.run.smithi006:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T05:02:47.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T05:02:47.217 DEBUG:teuthology.orchestra.run.smithi169:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T05:02:47.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T05:02:47.232 DEBUG:teuthology.orchestra.run.smithi186:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T05:02:47.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T05:02:47.247 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-09T05:02:47.259 INFO:teuthology.task.internal:Transferring archived files... 2024-05-09T05:02:47.260 DEBUG:teuthology.misc:Transferring archived files from smithi006:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699065/remote/smithi006 2024-05-09T05:02:47.260 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T05:02:47.372 DEBUG:teuthology.misc:Transferring archived files from smithi169:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699065/remote/smithi169 2024-05-09T05:02:47.373 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T05:02:47.414 DEBUG:teuthology.misc:Transferring archived files from smithi186:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699065/remote/smithi186 2024-05-09T05:02:47.415 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T05:02:47.453 INFO:teuthology.task.internal:Removing archive directory... 2024-05-09T05:02:47.453 DEBUG:teuthology.orchestra.run.smithi006:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T05:02:47.461 DEBUG:teuthology.orchestra.run.smithi169:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T05:02:47.464 DEBUG:teuthology.orchestra.run.smithi186:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T05:02:47.515 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-09T05:02:47.533 INFO:teuthology.task.internal:Not uploading archives. 2024-05-09T05:02:47.533 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-09T05:02:47.565 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-09T05:02:47.565 DEBUG:teuthology.orchestra.run.smithi006:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T05:02:47.573 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T05:02:47.576 DEBUG:teuthology.orchestra.run.smithi186:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T05:02:47.590 INFO:teuthology.orchestra.run.smithi169.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 05:02 /home/ubuntu/cephtest 2024-05-09T05:02:47.592 INFO:teuthology.orchestra.run.smithi186.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 05:02 /home/ubuntu/cephtest 2024-05-09T05:02:47.627 INFO:teuthology.orchestra.run.smithi006.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 05:02 /home/ubuntu/cephtest 2024-05-09T05:02:47.630 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-09T05:02:47.643 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-09T05:02:47.753 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/fsync-tester}} duration: 1968.9606311321259 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-05-09T05:02:47.753 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T05:02:47.838 INFO:teuthology.run:pass